The intent of this literature reappraisal is to understand how the operational hazards are managed, prevented and how much it cost by non-managing and pull offing it for fiscal constitution. To analyze the operational hazard direction of Bankss and the impact of the non-management of operational hazards for a Gallic fiscal sector, our research will peculiarly concentrate under the certain fiscal ordinance like Basel III and scientific articles about the direction of operational hazards. It will allow to hold a clear thought of the places of those companies about it. First, we are traveling to hold an overview of hazards in general so operational hazards for different types of concern companies to so germinate on world-wide ordinance towards the latest one Basel III. The literature reappraisal is traveling to be based on scientific reappraisals and the Bank for International Settlement studies for analyzing place of regulators and Bankss about operational hazards.
To better cognize of what the subject is all about, it is mandatory to explicate how are define the hazards for Basel commission and so travel further with the operational hazards.
The BIS ( Bank for International Settlement ) has created in 1974 the Basel Committee on Banking Supervision ( BCBS ) in Basel in Switzerland. The purpose is to make a worldwide banking ordinance by printing a set of minimal capital demands for Bankss nevertheless the Committee can non compel states to follow those regulations. The representative ‘s members are from cardinal Bankss and regulative governments chiefly from leader states i.e. G 20 states and they use to run into 4 times per twelvemonth. It has been created because of the systemic impact of the bankruptcy of Herstatt Bank in Germany. Basel I in 1988 focused on the capital measuring system with the COOKE ratio where 8 % of a recognition has to be financed by bank equity. Then Basel II in 2004 has every bit end to make an international criterion for banking regulators to command how much capital Bankss need to set aside to guard against the types of fiscal and operational hazards Bankss ( and the whole economic system ) thanks to the Mac Donough ratio. For Jayamaha ( 2005 ) , Basel II needs a separation between the capital allotments for operational hazard. This ratio does non merely take into history the recognition hazard but besides the operational hazard and the market hazard ( cf bcbs 196 ) . In response to crises and the bankruptcy of Lehman Brothers, Bankss have from 2013 to 2018 to implement the new regulations adopted of Basel III.
For Basel commission ( bcbs 222 ) , the hazard direction will be portion of the politic of Bankss and that determinations “ in footings of hazard coverage, analysis and reading, scalability and comparison across group establishments ” will hold be take to as a precedence i.e. “ A bank should find hazard coverage demands to outdo suit its ain concern theoretical accounts and hazard profiles. For illustration, an aggregative hazard study should include, but non be limited to, the undermentioned information: capital adequateness, regulative capital, capital and liquidness ratio projections, recognition hazard, market hazard, operational hazard, liquidness hazard, stress proving consequences, inter- and intra-risk concentrations, and funding places and programs “
As the ordinance advice, pull offing all the hazards ( specific, systemic, liquidness, counterparty, involvement rate, FOREX, operationalaˆ¦ ) are the most of import to non lose money for companies. This question is really of import for the fiscal sector because it is one of the chief grounds where most of the jobs come from taking a loss of money and so clients. Almost all of the Bankss and large agents have a hazard direction section but it is still rather new in this sector.
In this respect, the BIS took into history legion factors that have an influence over the value alteration: volatility, recognition evaluation, correlativity with hazardous assets, investors ‘ assurance ( the flight to quality ) , and the liquidness of these assets ( that is, the velocity at which they can be traded without a loss of value ) . It has been found that these factors have a important impact on Bankss ‘ profitableness and solvency, and that Bankss should hold a tighter hazard direction policy sing the counterparty hazard ( turning away of exposure above 1 % of the entire balance sheet ) and the quality of assets ( they should non be pledged, nor collateralized ) . This survey is a good start to analyse the Basel III ratio as it establishes the variables that have an influence over Bankss ‘ fiscal places.
Therefore, operational hazards began to be managed from Basel Agreement II in order to apportion an sum of equity. Before those companies was non pull offing those hazards and realized that after dirts, frauds and crises that it was compulsory to take into history the operational hazard in the direction of the Bankss. This subject is existent because it has been highlighted thanks to the different crises, fraud and events that occurred this last 4 old ages. These events have showed that pull offing hazard particularly runing hazards is indispensable from different point of positions. Besides, operational hazards are linked to people, to the direction, the intra and inter-communication. A good direction can avoid disbursement money for errors that you be averted and maintain a good repute. Thus it will travel through a immense betterment for Bankss and immense development for agents these following few old ages.
Thereby this graduating undertaking will be based on being efficient in instance of operating jobs and control at the maximal operational hazards.
First of wholly, an operational hazard is defined for the Basel commission as the “ hazard of loss ensuing from inadequate or failed due to processs, human factors and systems or from external causes. Legal hazards are non included but strategic hazards are ” ( BCBS, 2003 ) . However, this definition evolved during the creative activity of Basel II because at the get downing the Basel commission did n’t specify it like that. However non everybody is agreed on this definition, Vanini ( 2004 ) criticized it and Wild West Semantico defined it as “ any hazard other than recognition hazard and market ” .
To come back to Basel II understanding, it structured the operational hazards in 7 types: internal and external frauds, employment relation and safety at work, client, concern and merchandise dealingss, harm to physical assets, concern break and system failures and executing of operations, bringings and procedure which allow us to analyze on different instances sing on what face more a agent. Jobst ( 2007 ) agrees on the definition below by dividing it in two categorizes: internal and external operational hazard. He spilt internal hazards in assorted hazards as people hazards by imputing loss exposure to the potency for failure of people “ from direction failure, organisational construction or other human failures, which may be exacerbated by hapless preparation, unequal controls, hapless staffing resources, or other factors ” , so the procedure hazard, “ with the dislocation of established procedures, failure of following procedures or unequal procedure function within concern lines and eventually engineering or system hazard, “ which reflects the operational exposure to breaks and outright system failure in both internal and outsourced operations ” . These hazards are applicable in the class of regular concern operations, such as breaches in internal controls and monitoring, internal and external fraud, legal claims or concern breaks and improper concern patterns ” but besides goes more farther by specifying it “ as the hazard of some inauspicious result ensuing from Acts of the Apostless undertaken ( or neglected ) in transporting out concern activities, inadequate or failed internal procedures and information systems, misconduct by people or from external events. This definition includes legal hazard from the failure to follow with Torahs every bit good as prudent ethical criterions and contractual duties, but excludes strategic and reputational hazard ” . On the side of external hazard, Jobs ( 2007 ) spread out his point of position based on “ environmental factors, such as a new rival that changes the concern paradigm, a major political and regulative government alteration, unanticipated ( natural ) catastrophes, terrorist act, hooliganism, and other such factors that are outside the control of the house ” .
For Jarrow ( 2008 ) , he defines the operational hazards into two types which create loss procedures. The first 1 is “ the hazard of a loss due to the house ‘s runing engineering ” and the 2nd one “ the hazard of a loss due to bureau costs ” . He argues that the “ current methodological analysis for the finding of economic capital for operational hazard is overstated. It is biased high because the calculation omits the bank ‘s net present value ( NPV ) bring forthing procedure. Although it is conceptually possible to gauge the operational hazard procedures ‘ parametric quantities utilizing lone market monetary values, the non-observability of the house ‘s value makes this an improbable possibility, except in rare instances. Alternatively, we argue that informations internal to the house, in concurrence with standard jeopardy rate appraisal processs, provides a more fruitful alternate ” .
So the appraisal and the direction of operational hazards are different sing activities in the fiscal sector emmet states. We besides have to accommodate the appraisal of operational hazards sing the type of companies i.e. Bankss, recognition constitution, agents and besides on the states where the concerns are based Spain, France, England, China, India, United Statesaˆ¦that ‘s why we are traveling to make.
Operationss hazards are linked to internal control and analysis of internal losingss of companies. To take into history the per centum of those hazards, a hazard function has been done and imposed for and by Bankss. For Bon Michel ( XXXX 71930056 ) “ This attack is portion of an attempt to streamline the complexness in an unsure universe ” with the crisis, we had the presentation in the fiscal sector where figures mean everything and nil at the same clip. So this function hazard was created to reassure directors, regulatorsaˆ¦ . But for Bon Michel ( XXXX 71930056 ) “ can we truly trust the consequences of the function? ” Taleb ( 2009 ) thinks that: “ Rationality itself could be an obstruction to the designation of a hazard frequently characterized by the mutuality of causes because of the complexness ” . Operational hazards, as we saw on Basel, are linked to people so that hazard by nature is strongly linked to the human constituent ( Power, 2005 ) . “ It is so in a uninterrupted procedure of interaction that the building of intelligence hazards can bring forth a alteration in the perceptual experience of hazard by the same persons and therefore of their actions ” . Bon Michel
But we besides have to accommodate the appraisal of operational hazards sing the type of companies i.e. Bankss, recognition constitution, agents and besides on the states where the concerns are based Spain, France, England, China, India, United Statesaˆ¦that ‘s why we are traveling to make.
To get down with Hedge Funds, the definition of operational hazards is somewhat different. For ( CAPCO 2005 ) it includes the operating environment of the fund i.e. “ in-between and back office maps such as trade processing, accounting, disposal, rating and coverage ” . This combination of “ Multiple Risks ” spans these classs.
The grounds of non pull offing hazards has been highlighted particularly for Hedge Funds From a study[ 1 ], it resulted that over 10 old ages ( 1996-2006 ) half of the hedge financess went to bankruptcy because of an deficient control of operational hazard merely ( 43843882 ) . For most of them, fraud is the chief ground i.e. peculation or false ratings of capital. ( 43443142 deja ref en Ba ) as the graph below shows ( CAPCO 2005 ) :
Figure 1: graphs demoing the different types of hazard failure and the item of operational hazards
Breakdown of Fund Failures Attributed to Operational Risk Merely by Operational Issue
The opacity of the direction of operational hazard exist because those companies are non obliged to supply information to investors about the direction procedure and hazards linked to operational hazards. Besides, Hedge financess have a specific construction, normally off shore companies merchandising on OTC ( Over The Counter ) in illiquid merchandises market or by making arbitrage so it result that operational hazards are specific therefore it is less easy to forestall it. Regulations tried to implement for each complex merchandise a prospectus. However, all the operations hazards can non be mentioned on it and defined due to a deficiency of adulthood of the industry and its opacity so these prospectus qui vive investors to be cognizant of the hazards known.
“ Hazards related to the operational failings of hedge financess significantly outweigh the degrees of fiscal hazard, which are normally the focal point of the directors ‘ attending and investors ‘ concerns. ” If we assume that the existence of investable financess is 2,500, the failure rate can be estimated at 0.6 % , which represents a really high chance of default in the context of financess of hedge financess that invest in 10 to 25 vehicles EDHEC
Une meilleure prise en compte diethylstilbestrols risques operationnels sera necessaire pour retrouver la confiance diethylstilbestrols investisseurs notamment a la suite des scandales recents
Sol: managing operational hazards – working paper EDHEC
Recognition constitutions have to follow Basel regulations ( BCBC, 2006 ) and ordinances. For illustration, Power ( 2005 ) established that the “ operational hazard appraisal differs from traditional hazards of the bank including the absence of known outstanding. It is diffuse, multifaceted and equivocal ” . That is all the complexness of the appraisal of operational hazards. It is based on the chance of a hazard occur. And the hazards as we saw before are assorted so even more hard to pull off and forestall.
Banks are split in different activities ; traditionally it is composed of retail banking, corporate and investing banking. However, these last few old ages ‘ Bankss developed a new service, the online banking specialized in stock exchange for single investors. This was truly an outlook from clients so most of the Gallic Bankss have their ain online agents under a different trade name name. They are making advertisement to convey the clients of the retail bank but besides clients from others rivals. Those agents had a immense development and did n’t concentrate on hazard direction. Normally the Director of Operation besides deals and solves the hazards jobs in these little entities. All the Bankss for traditional activities and others agents in France or abroad peculiarly in the FOREX market have a hazard direction section because they understood it was aboriginal. Thereby we will concentrate more on what has been done and developed in those companies and how it has been treated on the research side.
The operation hazards are those that use the most of equities after recognition hazards. To work out the operating losingss, three schemes have been implemented in the banking system: the internal controls inside the section i.e. the back office and the direction control the operations, the internal audit where a specialised section checks the operations to guarantee that policies and processs are followed and external audits where it is an external and independent establishment who checks the fiscal statements and more. Those three actions permit to cut down operational hazards and its cost.
To concentrate on internal controls, for Arnold, Larsen, Hollinger, O’Doherty, and Milne ( 2008 ) , the job of Bankss is that they do n’t care about it and are done by a “ spawned allegations of moral jeopardy ” and for Anna Chernobai, Philippe Jorion, and Fan Yu ( 2011 ) this is the “ effects of a weak internal control environment ” so it makes loss an tremendous sum of money ” . In the same article, they illustrate it by taking the illustration of the Trader Jerome Kerviel in Societe Generale in 2008 that made a loss of $ 7.2 billion “ due to unauthorised trading ” and that the direction did nil by allowing him trade. However, for Alberto Balestra ( 2006 ) , there is “ bing internal control processs ” but because of a “ deficiency of conformity ” companies makes large losingss.
For the internal and external audit, the attack is rather different, the first 1 has to be lasting and the 2nd one is normally periodic. The Basel commission defines what the Bankss have to make to pull off its internal audit of its operational hazard like create an operational hazard direction map “ responsible for codifying firm-level policies and processs refering operational hazard direction and risk-reporting system for operational hazard ; and for developing schemes to place ” . And for the GCAP ( group of Assistance to hapless ) the internal control is define by look intoing all the undermentioned points: “ The fiscal information and operating informations are accurate and dependable, the policies and internal processs are followed, the concern hazards of the establishment are identified and minimized, resources are used expeditiously and economically, the establishment ‘s aims are being achieved and the external ordinances are met ” . J. Cernes agreed to state that “ Internal audit sees his expanded function, including the control of new accomplishments related to this development ” . Although for Balestra ( 2006 ) , the operational hazard director can hold troubles to make it so it has to “ inquire aid from rely besides on reengineers for this operational auditing ” .
From the external point of position of an independent hearer, C. Cox ( 2008 ) manager of the Securities and Exchange Commission ( SEC ) agreed that “ his bureau failed to move for about a decennary on believable allegations about Bernard Madoff, whose deceitful internal controls allowed a Ponzi strategy that cost hedge fund investors every bit much as $ 50 billion ” The end of these hearers is to look into the fiscal statements, records, minutess, and minutess of a construction by an external party to show an sentiment on the fiscal statements of the IMF.
However, Markus Leippold ( 2003 ) uncertainties on the impression of pull offing hazard and more peculiarly on “ happening a sensible impression of variegation ” but besides on the rating and quantification of the benefits from changing value ironss and work flow constructions ” For George S. Oldfield and Anthony M. Santomero ( 1997 ) “ the degree of attempt focused on cut downing these hazards can be communicated to stockholders and cost-justified ” .
To reason, for Leippold and Vanini ( 2003 ) , the operation hazard occurs at low frequence but it has a high impact. Usually it comes from of human behaviours. In 2001, the Basel commission realized a study on 89 international Bankss, “ a bank has been affected by an norm of 528 cases of operational hazards, chiefly on retail banking, bring forthing an mean loss per unit norm of 10 000 a‚¬ . It follows a gross loss norm of 90 million a‚¬ per establishment ” . For Jeffery Atik ( 2010 ) “ Not all hazards can be anticipated. Indeed, it is hard to pull off hazards that can non be imagined ( those that are outside of experience ) although experience Teachs that unanticipated and undreamed events do happen ” in mention to those crises. So we can see the troubles to understand and restrict the impact of the operational hazards. Tonss of things have been done from the past events on how to cipher the operational hazards with the Value At Risk ( VaR ) but about nil has been done yet to restrict the impact of operational hazards on the fiscal sector after the recent past events of these last four old ages.
From what we saw, hazard direction has been surveies on its definition or on the computation on the VaR few old ages ago. However, the hazard direction is still rather new on the fiscal sector and even more for operating hazards on the manner how to pull off it. Thankss to the yesteryear, it has to be re-studies to better its direction to avoid to reiterate error and to happen of the possible hazards that it could happen. From few illustrations, when one job occurs it costs a batch for the bank but it is rather surprising that it can still go on presents. With the crises and recent frauds, it was obvious that nil has been done on this subject.
The Basel commission is working on Basel III therefore the operation hazards will be more controlled and highlighted. Like that, the Bankss would less suffer of the impact of a bad direction of operating hazards. Besides, after all the crisis and dirts like as seen earlier for Societe Generale ‘s instance in 2008 or even for Denudations in 1995, Calyon in 2006 or celebrated internal frauds like JP Morgan Chase in 2006, Bank of China in 2007 or commercial judicial proceeding in 2006 with the Deutsche Bank or fire in the headquarter of the Credit Lyonnais in 1996, these dirts do non give a positive image of the bank. The imperativeness uses to relay it in the headline for few yearss or months. Clients, clients read it and after the word of oral cavity Begin. This external communicating can non be controlled by the bank. By and large, this is the worse for a company. The employees and directors suffer besides of that so preparation has to be done to excel the impact of the event and be able to give an reply to all the onslaught they have from the household and clients. This impact of the direction of operation hazards has ne’er been studied in term of figures for the bank and from the human side.
The intangible assets are a really of import impression than operating hazards affect for old ages like the trade name name, the image, the repute. Even external events affect Bankss like onslaughts because citizens do non swear in establishments, fear appearaˆ¦ During the crises, the insolvent of a bank or a dirt like Madoff ‘s system was able to impact the full sector and by definition all the others Bankss, which lead that clients wanted to retreat their economic systems. So it would be of import to see what the impact of the direction is in all those instances of systemic crisis.
Ethic is a bad word in finance so it has to alter by re-establishing a good image of the finance sector, in what manner a new civilization. The alteration has to be done from the top of the direction i.e. operational hazards could be avoided if a good direction, a twenty-four hours to twenty-four hours direction for illustration, would be implementing and if the employees would care about their Bankss. In this globalized universe, people have to be reassured ; the assurance is one of cardinal word and of import informations that everybody forgot because the end was to do money without any regard and trust toward the counterparty.
The purpose of the direction of operational hazards is to reenforce transparence and restore assurance. That end ever existed but events, dirts have updated it.
Operating jobs bulk come from an mistake from the machines or an mistake from the human. Therefore to avoid it, it could be possible to implement really rigorous schemes to extinguish it by capitalising the errors whether voluntary or non. The Bankss should, as what has been done before control and audit from a restrictive point of position but besides train employees. A fillip could be given if the employee reacts in a good manner, do non seek to fraud and advance its company merely is a “ good employee ” . In my old company i.e. a agent, there were a hazard director and all the employees received at the get downing a preparation to face to possible jobs. This should be compulsory because it permits to travel through all the jobs with clients but besides in intern with the clang of the trading platformaˆ¦ In France, it is coming but non in this manner, they oblige the employees to go through an internal test to be disposed to confront to it, some companies ask to go through the AMF ( Gallic Market Authorities ) enfranchisement. However it has to go compulsory to everybody, like that when employees know what they can or can non make for an internal or external state of affairs, they wo n’t seek to rip off, conceal or fraud because they will be cognizant of the harm that it can be for them, for the section, for the company and at the terminal for the clients.
Besides, different sections are involved in the direction of operating hazards, like the general review, IT, back office, communicating, client services, investors relationsaˆ¦so all those employees has to be trained to face to it hereby one of the solutions is to reenforce the corporate civilization, like that, employees will be involved in their occupation but besides in the betterment of their company. Sometimes, board of managers and directors are non besides in the best place for that, because they can hold unusual behaviours and reactions i.e. terror of the direction of Kerviel instance. They can bury the common sense and act as single careerists. It would be interesting to analyze the impact of preparation in clip of crisis and how to respond to those instances, like that companies would command the state of affairs.
However to reconstruct a good relationship with the clients, the large celebrated Bankss created on-line bank with less fees and more attractive contracts, merchandises and new names than their female parent societies. Those agents were non impacted by the crises and by the bad image of the banker. The ground why, is because there are new companies with no background like the major Bankss where most of population believe that they are non clear, corrupted and usage to do a batch of guess. Clients built this bad image on operating hazards that occurs for Bankss.
The directors did n’t acquire the of import of those hazards so it would be nice to analyze it by acquiring information from clients, employees and directors to understand the importance of the operating hazard, the direction that they do, how they prevent, avoid and communicate on it from an old and traditional but besides from a agent point of position.
To reason, the impact of a non-management has to be measured late, the BIS wrote in 1998 a papers on the direction of operational hazards. It has non been done since the crises and the internal events, fraud that occurred these last 5 old ages. Clients, clients, chances, employees, managers are enduring of the non-management of operations hazards and it has to be studied with the aid of the stakeholders to happen a via media and get down to work out it.
71930056 p327 Si l’evaluation quantitative du risque a ete lupus erythematosus Prime Minister exercice realise par les banques ( Lamarque et Maurer, 2009 ) , en temoignent les nombreuses recherches qui se sont orientees vers une approche quantitative ( Frachot et alii, 2003 ; Klugman et alii, 1998 ; King, 2001 ) , le role de la composante humaine dans le blue lui meme et dans son processus d’identification rip particulierement interessante toute approche de nature qualitative. Comme le soulignent Lamarque et Maurer ( 2009 ) , l’approche quantitative du risque operationnel, compte tenu de sa difficile previsibilite , est A« insuffisante pour maitriser Ces risques et la gravite des evenements exceptionnels est extremement difficile a evaluer A» . C’est generalement l’apparition meme du risque qui permet de le connaitre et donc de le gerer de maniere efficace, c’est-a-dire a posteriori. Si l’organisation se caracterise par SA diversite et SA complexite ( Weick, 1995 ) , celles-ci component un frein a l’identification precise du risque futur.
Beginnings:
hypertext transfer protocol: //www.bis.org/bcbs
Arnold M. , Larsen P. T. , Hollinger P. , O’Doherty J. and Milne R. ( 2008 ) “ The foreigner: How
Kerviel exposed slack controls at Societe Generale ” , In The Financial Times, February 7th,
London.
Atik, Jeffery, ( 2010 ) “ Basel II and Extreme Risk Analysis ” . Loyola-LA Legal Studies Paper No. 2010-40. Available at SSRN: hypertext transfer protocol: //ssrn.com/abstract=1677682
Balestra A. , ( 2006 ) “ Quantification of Operational Risk ” , Mohassaba S.A. , Geneva.
Basel commission ( 2006 ) , “ International Convergence of Capital Measurement and Capital
Standards ” , A Revised Framework, Comprehensive Version of the digest of the June 2004
Basel II Framework
Bon-Michel, B 2011, ‘La cartographie diethylstilbestrols risques: de la rationalization du futur a l’apprentissage du risque. Cas de l’identification du risque operationnel gold sein d’un etablissement de credit. ( Gallic ) ‘ , Revue Management Et Avenir, 48, pp. 326-341
Capco White Paper ( 2003 ) “ Understanding and Mitigating Operational Risk in Hedge Fund InvestmentsA A»
Carter L. F. ( 2011 ) “ MANAGING Hazard IN OPERATIONS ” Academy of Information and Management Sciences Journal, Volume 14, Number 2
Chernobai A. , Jorion P. , Yu F. ( 2011 ) “ The Determinants of Operational Risk in U.S. Financial Institutions ” Journal Of Financial And Quantitative Analysis Vol. 46, No. 6, Dec. 2011, pp. 1683- 1725
Control sur le comite de Bale ( 1998 ) , “ Gestion du risque operationnel ”
Cox C. ( 2008 ) Director of the Securities and Exchange Commission ( SEC ) “ SEC Press Release ” 2008-297
Giraud J.R ( 2004 ) “ The Management of Hedge Funds’Operational Risks ” EDHEC RISK AND ASSET MANAGEMENT RESEARCH CENTER
Jarrow, RA 2008, ‘Operational hazard ‘ , Journal Of Banking & A ; Finance, 32, 5, pp. 870-879, Business Source Complete, EBSCO.
Jobst, AA 2007, ‘The intervention of operational hazard under the New Basel model: Critical issues ‘ , Journal Of Banking Regulation, 8, 4, pp. 316-352, Business Source Complete, EBSCO.
Leippoldy M. , Vaniniz P. ( 2003 ) “ The Quantification of Operational Risk “
Vanini P. ( 2003 ) Operational Hazard: A Practitioner ‘s View Journal of Risk, Vol. 5, No. 3, pp. 1-16, 2003
A«A LES HEDGE FUNDS FACE AU RISQUE OPERATIONNELA A» 2009, Revue Banque/Banque Magazine, 714, pp. 71-73
Weiss A. , and Nikitin G. ( 2004 ) , “ Foreign Portfolio Investment Improves Performance: Evidence from the Czech Republic ” , Topics in Economic Analysis and Policy, 48, 817-38
Power M. ( 2005 ) , A« The innovation of operational hazard A» , Review of International Political Economy, 12 ( 4 ) , p. 577-599.
Taleb N.-N. ( 2009 ) , “ Mistakes, Robustness, and the Fourth Quadrant ” , International Journal of Forecasting theory and research, Englewood Cliffs, Prenctice-Hall, p. 10-37.