The Risk Enabled Enterprise® Enterprise-level stress testing

The Risk Enabled Enterprise®

Enterprise-level stress testing

ChartisResearch

Independent research by

In collaboration withMarch 2015

© Copyright Chartis Research Ltd 2015. All Rights Reserved 2

The Risk Enabled Enterprise® March 2015

About Chartis

Chartis is the leading provider of research and analysis covering the global market for risk management technology. Our goal is to support enterprises seeking to optimize business performance through better risk management, corporate governance and compliance. We help clients make informed technology and business decisions by providing in-depth analysis and actionable advice on the broad spectrum of risk and compliance technology offerings. Areas of expertise include:

• Credit risk• Operational risk and governance, risk and compliance (GRC)• Market risk• Asset and liability management (ALM) and liquidity risk• Energy and commodity trading risk• Financial crime including trader surveillance, anti-fraud and anti-money laundering• Insurance risk • Regulatory requirements including Basel 2, Basel 3, Dodd-Frank, EMIR and Solvency II

Chartis is solely focused on risk and compliance technology, giving it significant advantage over generic market analysts.

Chartis has brought together a leading team of analysts and advisors from the risk management and financial services industries. This team has hands-on experience of implementing and developing risk management systems and programs for Fortune 500 companies and leading consulting houses.

Chartis Research is authorized and regulated in the United Kingdom by the Financial Conduct Authority (FCA) to provide investment advice.

Join our global online community at www.risktech-forum.com.Visit www.chartis-research.com for more information.

© Copyright Chartis Research Ltd 2015. All Rights Reserved.



No part of this publication may be reproduced, adapted, stored in a retrieval system or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of Chartis Research Ltd.

The facts of this report are believed to be correct at the time of publication but cannot be guaranteed.

Please note that the findings, conclusions and recommendations that Chartis Research delivers will be based on information gathered in good faith, whose accuracy we cannot guarantee. Chartis Research accepts no liability whatever for actions taken based on any information that may subsequently prove to be incorrect or errors in our analysis. See Chartis “Terms of Use” on www.chartis-research.com.

RiskTech100®, RiskTech Quadrant® and The Risk Enabled Enterprise® are Registered Trade Marks of Chartis Research Limited.

Unauthorized use of Chartis’s name and trademarks is strictly prohibited and subject to legal penalties.

IBM and the IBM logo are trademarks of the International Business Machines Corp., registered in many jurisdictions worldwide.



Table of contents

1 Executive summary 6

2 Why is enterprise stress testing an enabler of ERM? 9

3 The data challenge 14

4 Broadening the stress testing horizons 17

5 Integration, collaboration and dialogue 22

6 Art and science 25

7 Practical implementation of best practice for enterprise stress testing 27

8 Conclusion 29

9 Appendix A: Survey demographics 32

10 Appendix B: Key survey results 34

11 How to use research and services from Chartis 37

12 Further reading 39



Table of figures

1 Roadmap to become risk-enabled 9

2 Variation of methodology across different jurisdictions 11

3 Post-crisis expansion in scope and methodology of stress testing 12

4 Seven pillars of enterprise-level stress tests 12

5 How would you characterize the key challenges in updating and managing stress tests? 14

6 How would you describe the integration of assumptions and data used in stress testing with other key business processes? 15

7 How important are the following groups/types of stress tests in your organization? 17

8 From an industry perspective how do you see the future evolution of the various stress tests? 20

9 Executives assuming primary responsibility for stress tests 23

10 How involved are the following groups in the stress testing process? 23

11 Which areas of your firm’s stress testing program need the greatest level of improvement? 26

12 Example framework for linking activities to stakeholders 28

13 Example framework depicting functional elements of a comprehensive enterprise-level stress testing architecture 29

Appendix A: Survey demographics 31

Appendix B: Key survey results 33



1

Executive summary

The Risk Enabled Enterprise® is a two-year research initiative from Chartis, in collaboration with IBM, which explores the enablers of enterprise risk management (ERM). Our research identified four strategic initiatives that firms can use to become “risk-enabled” across organizational structure and process, people and culture, and data and systems. This report, covering enterprise-level stress testing, is the second of four to examine these initiatives and identify best practice practical implementation.

Stress testing is essential for financial institutions having to respond to regulatory demand. At an enterprise-level, stress testing is more than regulatory compliance for its own sake. Enterprise-level stress testing is a rapidly evolving area within financial institutions, where the approaches and applications are expanding well beyond the risk measures demanded by regulators. With stress testing already established as a fundamental pillar of regulatory oversight, firms continue to invest in streamlining their operations to deliver more comprehensive reporting, and are applying new analytical infrastructures to develop a better understanding of their structural risks and tail risks. Firms are already incorporating stress testing results into operational and strategic decisions, and going forward many expect this analysis to be weighted with growing importance, and to be applied in more business planning decisions.

For this report, Chartis set out to understand the challenges financial institutions are facing in implementing enterprise-level stress testing frameworks and to what extent enterprise-level stress testing is applied in business decision making. Chartis takes into account both the current state of enterprise-level stress testing as well as expectations for future developments. Our research included a global survey of 68 professionals working in the financial services industry, as well as in-depth follow-up interviews with 22 experts from financial institutions and consulting firms. We also held discussions with subject matter experts from IBM. To learn more about IBM’s approach to Enterprise Stress Testing, visit ibm.com/SmarterRiskAnalytics.

The key findings from the research include the following:

• Data problems are a major impediment to effective stress testing. The data challenges encountered in the implementation of risk management systems are



particularly acute for stress testing at an enterprise level. Data quality is identified by more than 95% of surveyed firms as the toughest challenge facing their stress testing initiatives. Issues with timeliness of data availability and gaps are also stumbling blocks to aggregation and integrated analysis across risk types. Moreover, integration tends to be more visible within risk silos and not at an enterprise-wide level. Data silos must be broken down to achieve an integrated, enterprise-wide approach to stress testing.

• Stress testing is viewed as largely a compliance exercise. The majority of survey respondents that run CCAR and ECB/EBA stress tests are applying the results exclusively towards regulatory compliance, and are not integrating these results into broader business decisions. A small minority (on average 16%) have embedded these and other stress tests into all their business decision-making processes. This underscores the need to review the stress testing exercises demanded by regulators, and explore revisions to the requirements that would make the standardized results more applicable to the business decision-making of financial institutions.

• Smaller firms lag behind stress testing best practice. Most regulators are currently focusing their supervisory activities relating to stress testing requirements on the larger firms – our research shows that smaller institutions have not yet adopted more sophisticated approaches to stress testing. The current lack of “trickle-down” of best-practices to smaller institutions could, in the long-term, have a negative impact on overall confidence in the financial services industry.

• To increase utilization beyond compliance, the scope of stress testing needs to

be broadened. To date, most stress testing efforts have focused on a particular type of risk, such as market risk and credit risk. True enterprise-level stress testing needs to develop scenarios that include all risk types and their inter-dependencies, such as the compound effect of market risk and credit risk on each other, overlaps between market risk and liquidity risk, and overlaps between credit risk and operational risk. It is also clear that reverse stress testing is under-utilized, which could impair financial institutions’ ability to identify potential business vulnerabilities. Broadening the scope will make stress testing results more trustworthy, and applicable to more decision makers within the institution. Therefore, we propose that increasing breadth will increase usage of stress testing beyond compliance.

• Integration of different stress testing approaches is critical. Only 29% of respondents indicated stress testing is well integrated with business planning. On average, only 25% were confident that stress testing was integrated with other key business processes such as front office risk management and risk strategy. Careful consideration needs to be given to coordinating disparate stress testing activities and establishing the right level of interaction between top-down and bottom-up approaches. These are necessary steps along the way in establishing a truly integrated enterprise stress testing framework. Greater collaboration is also needed between the risk, finance,



capital management, treasury and business lines to help establish the risk appetite and effectively communicate stress testing results to the board.

• Skill shortages are constraining improvements in enterprise-level stress testing.

72% of survey respondents cited improvements in knowledge and skills as a medium to high priority. Assembling a team with the right composition of skills and experiences (i.e. quantitative, qualitative, business, financial and technology) is challenging for many firms. Furthermore, retaining such skills in the current market (high demand, scarce resource) is even more difficult and continuously pushes up costs.

• Stress testing is ultimately a blend of science and art. The quantitative and qualitative facets of stress testing are both important. Executive level interviews carried out for this report highlighted that as much as high-quality data and robust models are a precondition for effective stress testing, so too are the ability to make judgment calls, to facilitate collaboration between risk, finance and other functions, and to communicate test results to senior management and the board.



Why is enterprise stress testing an

enabler of ERM?

In the first The Risk Enabled Enterprise® report, Chartis reported that firms wanted to make the transformation to become risk-enabled, but lacked a realistic roadmap for this process. Chartis’s research showed that the way for firms to achieve their ERM goals is to use a series of strategic initiatives to drive improvements across organizational structure and process, people and culture, and data and systems. Figure 1, below, illustrates this process.

Figure 1: Roadmap to become risk-enabled

In order to drive transformation, strategic initiatives must:

• Have an impact across organizational structure and processes, people and culture, and data and technology;

• Require change throughout the enterprise and across all three lines of defense; and

• Have clear goals and compliance, financial and reputational benefits that will allow a business case to be made and make completion of the initiative a priority.

2

CURRENT STATE

Silo-based risk &

compliance management

Lack of timely risk-based

information

Mis-alignment of incentives

and rewards

Reactive reputation

management

Spiraling cost of

compliance

FUTURE STATE

Integrated risk &


Early-warning systems

Proactive reputation

management

Improved risk vs reward

decision making

Cost-effective risk &


FIRM-WIDECAPABILITIES

STRATEGIC INITIATIVES

Organizational structure

& processes

People & culture

Data & technology

Model risk

managem

ent

1

Ente

rprise-level

str

ess testing

2

Conduct risk

managem

ent

3

Ris

k-b

ased p

erf

orm

ance

managem

ent

4

The Risk Enabled Enterprise®

Maturity Model5



The second report, Model Risk Management, showed that a strategic approach to model risk management will involve all the areas of the enterprise – structure and process, people and culture, and data and systems – and will seek to improve risk awareness and risk-related behavior, hierarchies, escalation structures and links between different functions. In this third installment, Chartis looks at Enterprise-Level Stress Testing1.

Enterprise-level stress testing can help organizations to become risk-enabled. But to reap the benefits of stress testing and enhance ERM across the enterprise, financial institutions must ensure that they understand the full scope of enterprise-level stress testing and, most importantly, how to implement it.

What is enterprise-level stress testing?

Stress testing describes a range of techniques to assess the vulnerability of a firm’s business model to major changes in the macroeconomic environment, or to exceptional and plausible events or structural changes to the market microstructure.

There are different types of stress tests, with different uses. Some variants have been in use for a number of years. For example, financial institutions conduct their own internal stress tests to manage various kinds of risks. Their supervisors conduct stress tests as part of their ongoing oversight to ensure financial institutions can cope with market shocks, such as a sharp drop in housing prices or sudden rises in interest rates.

Despite these tactical uses, enterprise-level stress testing has yet to be fully utilized by firms in business decisions, as it necessarily involves bringing together information from different lines of business, asset classes, risk classes, functions and departments, which is a daunting task. If successful, however, this integration can help break down silos and build links between functions. Chartis believes that having a well-defined enterprise-level stress testing framework is a prerequisite for financial institutions to become risk-enabled enterprises.

What the regulators expect to achieve with stress testing?

Since the onset of the recent financial crisis, regulators across the world are encouraging financial institutions to use stress testing frameworks to make a quantitative, forward-looking assessment of capital adequacy. Basel 3 is the main driver of this, with regulators seeking to implement its capital requirements, led by stress tests, as part of Internal Capital Adequacy Assessment Process (ICAAP) under Basel 2.

1. Unless otherwise noted, we will use ¨stress testing¨ to refer to ¨enterprise-level stress testing¨ in this document.



The United States has adopted similar stress testing frameworks. US regulators have unveiled their latest requirements for stress testing under the Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank) as well as the Comprehensive Capital Analysis and Review (CCAR). As a consequence, the largest US banks with more than $50 billion in assets will have to run two internal stress tests each year, expanding the existing programs already in place.

The market disruptions of 2007-08 exposed most existing stress test models, whether quantitative or qualitative, as inadequate for capturing potential shifts in liquidity. As a result, stress tests have evolved considerably since then. Although stress testing now forms part of the banks’ standard regulatory and business landscape (for example, CCAR, EBA/ECB stress tests, stress tests of assets and supporting collateral), there is a variation in methodology across different jurisdictions, as shown in Figure 2.

Figure 2: Variation of methodology across different jurisdictions

European Union United States

Coverage Largest EU banks Bank holding companies

Frequency of stress tests Annual Annual - regulatory

Semi-Annual - Banks

Data requirements Private data Private (granular data at

account level)

Modeling Internal models Internal models

Micro-prudential models

Scenarios Regulatory Regulatory

Adverse

Severely adverse

Disclosure Detailed disclosure Individual institutions’ disclosure

Use Capital ratios

Regulatory reporting

Business strategy

Stress tests results feed into

capital planning process,

subject to regulatory approval

Moving forward, banks and other financial institutions such as insurers and central counterparties (CCPs), can expect regulators to impose more varied and frequent stress tests in order to cope with any future episodes of economic distress. There has been a significant expansion in stress testing scope and methodology following the financial crisis of 2007-08, as outlined in Figure 3 below. Furthermore, regulators across the world also want banks to envision more hypothetical economic and financial market scenarios, however unlikely they may be. As a result, financial institutions now have to find a way to incorporate such scenarios into their stress testing frameworks.



Figure 3: Post-crisis expansion in scope and methodology of stress testing

As regulators seek to expand rapidly the footprint of stress testing programs across the industry, financial institutions can expect the number of scenarios they have to incorporate into their stress testing process to increase. Additionally, financial institutions are expected to provide the results of the stress tests on a regular basis, in some cases for multiple regulatory bodies. The frequency of related reporting (which in some jurisdictions is currently only annual) appears very likely to increase given the increased urgency of regulatory initiatives. This means that financial institutions will need to establish a structured enterprise-level framework that enables increasingly rapid response (see Figure 4).

Figure 4: Seven pillars of enterprise-level stress tests

Viability of

banks’ business model

Liquidity and

funding gaps

Bank-wide P&L

and economic

capital

Bank-wide

P&L, business

line P&L and

regulatory

capital

Standard scenario

set

Ad-hoc scenario

set

Macroeconomic

set

Out of the box

scenario set

Pillar 2 stress testing

Pre-crisis stress testing

Post-crisis stress testing

Scope

Define scope of stress

testing

Boundaries

Market

structure

Scenarios

Define scenarios

Regulatory/

internal

Validation of

severity

Duration

Risk

transmission

mechanisms

Data and

infrastructure

Data sourcing

Data

compilation

and formatting

Data audit

Data

aggregation

Parameters

Calibration of

curves and

surfaces

Link macro-

economic

variables to

market micro

structure

Performance

indicators

Stress risk

measures

(SVaR)

Capital

adequacy

gaps

Funding gaps

Liquidity gaps

Reporting

Integration into

risk appetite,

credit limits,

liquidity

planning

and growth

adjustments

Strategic

integration

Integrating

into overall

risk appetite



About the survey

The analysis presented in the following chapters is based on an online survey on enterprise-level stress testing conducted by Chartis, in which 68 professionals from financial services and non-financial services industries took part. The respondents came from a range of job functions, locations and titles. About 75% of respondents hailed from North America and Europe, while the remainder came from Asia Pacific and the rest of the world. This representation is important given the fact that currently the strongest regulatory imperatives relevant to stress testing are in North America and Europe. The full results of the survey can be found in the Appendix.

In addition, Chartis carried out in-depth follow-up interviews with 22 industry practitioners to discuss their views on enterprise-level stress testing practices. Discussions were also undertaken with subject matter experts from IBM.



The data challenge

Poor data quality impedes effective stress testing

Among the various challenges facing firms in improving their stress testing practices, none are more difficult than issues with data quality. More than 95% of respondents consider data quality a significant challenge to stress testing. Modeling also figures as a major challenge for 83% of respondents, as do knowledge and skills (70%), and systems functionality and performance (55%). The responses are shown in Figure 5.

Figure 5: How would you characterize the key challenges in updating and managing

stress tests?

Over half of respondents – 56% – characterize data quality issues as an “important” challenge in stress testing, while 39% say this challenge is “critical”. This highlights the reality that, though firms are conducting stress tests, much needs to be done regarding the underlying data and the associated models being developed, as the outcomes of plausible stress scenarios depend on the reliable data and conceptual soundness of the model.

of the respondents

think data quality is

a critical challenge39% of the respondents

think data quality is

an important challenge56%

3

0% 20% 40% 60% 80% 100%

Not a challenge

Moderate challenge

Important challenge

Critical challenge

Systems functionalityand performance

Modeling

Knowledge and skills

Data quality 39% 56% 5%

30% 40% 28% 3%

20% 63% 15%

18% 38% 43%

3%

3%



The data challenge can be divided into two sub-challenges:

• Getting the right data to calculate good loss estimates for different risk types;• Aggregating this data to provide a firm-wide view of risks.

Data gaps remain a key problem. To overcome data challenges, firms need to have a structured process in place that incorporates well-defined processes for timely and accurate data. This process includes:

• Data scrubbing;• Data quality;• Periodically refreshing the data;• De-duplication;• Reconciliation; and • Revision

Data used in stress testing largely remains trapped in silos

Another facet of the data challenge is the ability to integrate data used in stress testing with other key business processes. Here, the performance of surveyed financial institutions is mixed. Nearly half (45%) of respondents have managed to effectively integrate the assumptions and data used in stress testing in credit portfolio management. Far fewer, however, have achieved the same when it comes to risk strategy (33%) and business strategy and planning (29%). In these and other areas (model validation, front office risk management and operations), partial integration is the norm. Moreover, integration tends to be more visible within risk silos and not at an enterprise-wide level.

Figure 6: How would you describe the integration of assumptions and data used in

stress testing with other key business processes?

0% 20% 40% 60% 80% 100%

No significantintegration

artiallyintegrated

ellintegrated

perations reference dataaluations etc

ront office risk management

Model alidation

usiness strategy planning

isk strategy

Credit portfolio management 45% 45% 11%

33% 51% 15%

29% 50% 21%

21% 45% 34%

13% 45% 42%

13% 62% 26%



It is fair to say that stress testing is a more visible and integral part of credit portfolio management, risk appetite and business planning. Integration with front office risk management and operations (clearly indicating market risk portfolios), on the other hand, appears to be relatively poor. Only 13% indicated front-office risk management and stress testing are “well integrated”. We believe this is consistent with the message received from interviewees that stress testing, at present, is undertaken less as business activity and more as a matter of compliance.

Furthermore, model validation, under new model risk management regulation, is an emerging topic in stress testing as firms are still developing their first-generation or champion models, while the industry has already begun talking about the challenger models. However, implementation and use across lines of business is still a limited practice, with 34% of respondents suggesting no such integration exists.At present, integration of risk data across business lines and risk types (credit, market, liquidity, operational) is rudimentary as noted, for example, in bank self-assessments of readiness to achieve BCBS 239 compliance.

The IBM view

IBM is seeing a scramble to find good data and questions about the provenance, validation and trustworthiness of that data. In the survey results, 94% said data quality was either a significant or important challenge. The issue in many cases is that when data is aggregated up to the enterprise level, differences arise in the way data is described. Many of the problems occur around the commonality of products and scenarios, and common data sets used for building the scenarios and the risk factors. IBM makes large investments in helping organizations normalize the data, ensuring consistent taxonomies, and ensuring that the data is in a fit state to be aggregated. This continuous improvement of data is a critical functionality.

The Chartis view

Enterprise-level stress testing faces the same challenges in terms of data quality, modeling, systems and knowledge and skills. It is important here to take into account regulatory initiatives, such as BCBS 239 (Risk Data Aggregation and Risk Reporting – applicable to G-SIBs from Jan 2016) and Model Risk Management (applicable to all models in US), which address the very same issues (data, IT systems, model risk etc) that the survey highlights as the key challenges.



Broadening the stress testing horizons

Stress testing is largely a compliance exercise

The survey results strongly suggest that financial institutions currently treat stress testing as a compliance-related activity. When asked to rate the importance of various types of tests, capital stress tests are being emphasized at the moment. FIs are primarily focusing on the Pillar 2 ICAAP (34% of respondents) of Basel 2, as well the Federal Reserve’s CCAR (24%). The relative importance to surveyed financial institutions of various types of stress tests are illustrated in Figure 7.

Figure 7: How important are the following groups/types of stress tests in your

organization?

Given the focus of these regulations (i.e., primarily market and credit risk) and the fact that most respondents see stress testing efforts as a compliance only exercise, we conclude that stress testing has not yet moved beyond a siloed approach. Interviewees have also pointed to the current lack of more comprehensive, cross risk-type, stress testing. Going forward, this means that, in order to achieve true enterprise-level stress testing, FIs will need to develop scenarios that include multiple risk types and their inter-dependencies, such as the compound effect of market risk and credit risk on each other, overlaps between market risk and liquidity risk, and overlaps between credit risk and operational risk.

The observation that financial institutions see these tests as primarily compliance activities is reinforced by the finding that only small numbers have embedded the tests into all their

4

0% 20% 40% 60% 80% 100%

Not at all

Minor importance

Somewhat important

ery important

C stress tests

ther internal

CC

ther regulatory

illar 2 IC 34% 26% 18%

25% 35% 13%

24% 21% 29%

20% 52% 12%

21%

28%

26%

2 %

15% 36%13% 36%



business decision making. Large majorities in North America and the European Union (EU) consider CCAR (53%) and ECB/EBA (52%) stress tests, respectively, as regulatory compliance activity only, with an additional 38% and 42%, respectively, confirming that these tests are not critical. It is the same with Pillar 2 ICAAP, which 40% consider as no more than a compliance activity, while another 31% consider these tests to be not critical beyond compliance.

This supports the view, held by many market participants, that compliance teams today spend a large amount of time simply identifying and analyzing the impact of various stress scenarios on the financial institution’s portfolio solely for the purpose of satisfying regulations. This is a necessary response to regulatory requirements embodied in, for example, CCAR, EU-wide stress testing and Pillar 2 ICAAP. Consequently, there is a need to review the actual adoption of stress testing exercises in the FIs’ business decision-making process and management actions.

Smaller FIs lag behind stress testing best practice

Financial services industry executives interviewed in the course of Chartis’ research pointed out that, given the current focus of regulators on larger, systemically important FIs there is a danger that smaller FIs will lag behind stress testing best practice. In the UK, for example this large-financial institution focus means only 30 of 361 institutions are immediately impacted by Bank of England (BoE) stress tests. It is worth noting that firms the size of Northern Rock (one of the five largest mortgage lenders at the time of the crisis) may be excluded. The story is similar in the US. With respect to stress testing, the US Dodd-Frank Act has focused on FIs with more than $10 billion in assets. The Federal Reserve will examine 30 FIs with assets of more than $50 billion. This is a relatively small number bearing in mind that there are approximately 6,800 FIs in the US (of which a very large portion are small community banks). It is also worth noting that in the US more than 60 FIs, with assets under the Dodd-Frank $10 billion limit, failed during the financial crisis. Similarly, in the EU relatively few FIs are (so far) impacted by regulatory stress testing. The ECB estimates that there are over 7,300 FIs in the EU. To put this in the perspective of stress testing, the EBA put 123 banks in 22 countries through the recently (Oct. 26, 2014) concluded stress test. It appears that 24 FIs failed the EBA test.

Interviewed executives point out that, as it currently stands, the lack of “trickle-down” of best-practices to smaller FIs could, in the long-term, have a negative impact on overall confidence in the financial service industry.

In Chartis’s view, regulatory focused stress testing will be extended over the mid- to long-term to include Tier-2 (assets from $10bn to $100bn) and eventually Tier-3 financial institutions (with assets under $10bn). Other financial institutions such as insurers and CCPs are also under scrutiny. From the point of view of best business practice, regardless of the regulatory compliance motivation, financial institutions should be taking steps now



to move toward enterprise-level stress testing.

Looking ahead, as technology and the capabilities of regulators and financial institutions to respond improve, Chartis expects stress testing to be expanded to cover financial institutions generally and carried out on an on-going basis. We expect also that the scope of stress testing (and reverse stress testing) will widen considerably over the next 2 to 3 years, eventually achieving enterprise-level stress testing. As with other risk management change, this progress will vary considerably from region to region. The inevitability of this trend has a profound importance for both financial institutions and the service and technology companies that will step up to the increased demand for systems and processes to enable stress testing. Advances in technology (e.g., “cloud-based” risk management services) and increases in capacity on the part of regulators will facilitate the ‘trickle-down’ of best practice stress testing from large financial institutions to small financial institutions. We expect Europe to experience this change in advance of other regions due to pervasiveness and early adoption of regulations.

Reverse stress testing is under-utilized

Another noteworthy finding is that none of the surveyed firms are embedding reverse stress testing results into business decision-making. All respondents (100%) indicated that reverse stress testing is only carried out for compliance reasons in their firms. Many (32%) indicated it is not critical. This seems to run counter to regulatory expectations that reverse testing should be more than merely a matter of compliance. The regulatory intent is for reverse stress tests to be conducted to identify potential business vulnerabilities by assessing scenarios that would render a firm’s business model unviable.

Financial institutions are also ignoring the merit of using reverse stress testing as a means to stimulate strategic risk discussions. The failure to do so could negatively impact on financial institutions’ ability to revise and enhance contingency plans, business volumes and capital planning processes.

The IBM view

In the survey results, respondents stated that the credit portfolio is well integrated into the business, and that they are proactively using the results of stress tests. Moving forward, IBM customers are looking to use the stress testing results in longer term capital planning and in bringing risk and finance together. In addition, they wish to be able to ask specific questions, such as: “What’s our business strategy and what’s the effect going to be of some macroeconomic event over a period of time”? IBM sees stress testing coming into the planning cycles of risk and finance integration and is helping its clients address this important need.



The Chartis view

The operating model and business use test consideration for stress testing has to change. In order to meet regulatory demands, financial institutions have to move away from regulatory stress testing or a standalone, siloed process to one that is more strongly integrated with business strategies, planning and execution. Financial institutions should employ reverse stress testing and develop processes to really embed the outcomes of stress testing into their decision-making through areas, such as allocation of capital to business lines, performance monitoring, setting the risk appetite and portfolio concentrations.

Regulatory pressure will remain unabated

As shown in Figure 8, the majority of respondents expect the requirements of the existing regulatory frameworks to grow in one way or another. The greatest changes are expected in relation to Pillar 2 ICAAP guidelines: 44% of respondents expect these to increase in both depth and scope; 31% expect the same of both ECB/EBA and CCAR stress testing. This points to a sense of uncertainty about the future outcomes of stress testing. In the face of this uncertainty financial institutions may be too slow in the implementation of comprehensive enterprise-level stress testing.

While the survey results did not show a strong consensus with respect to change in scope (on average 42% expect no change) Chartis is of the view that this is, in large part, due to the fact that current regulations are focused on larger (tier 1 and tier 2) banks. Despite the uncertainty, given the trends identified elsewhere in this report for increased regulatory scrutiny of financial institutions generally and best practice “trickle-down” effects, regulatory stress testing guidelines are likely to gain more depth and scope in the future.

Figure 8: From an industry perspective how do you see the future evolution of the

various stress tests?

0% 20% 40% 60% 80% 100%

No change orreduction inscope or depth

Increased depthand scope

Increased scope

Increased depth

e erse stress tests

ther regulatory

ther internal

CC

illar 2 IC

C stress tests 13% 9% 31%

12% 12% 44%

9% 9% 31%

6% 29% 32%

6% 30% 15%

28%3% 25%

4 %

32%

50%

32%

48%

44%



The Chartis View

Stress testing practice is more mature in the US than in Europe with the Fed having taken the lead by initiating the CCAR process back in 2010/11. While the EBA has started conducting EU-wide stress tests as of October 2014, this only underlines the fact that stress testing is being pursued on a much wider scale than before. Pillar 2 ICAAP has been implemented more closely in the EU and Asia Pacific where the methodologies, assessments and use within financial institutions’ internal business management and capital allocation processes is well documented and integrated. This will only be further enhanced, in terms of depth and scope going forward, when banks initiate or start adopting advanced methodologies for risk measurement.



Integration, collaboration and dialogue

Stress testing is not the monopoly of the risk management function: tests are conducted in different parts of the business and at many different levels. Careful consideration thus needs to be given to co-ordinating these disparate activities and establishing the right level of interaction between top-down and bottom-up stress testing. These are necessary steps along the way to establishing a truly integrated enterprise stress testing framework. Greater collaboration is also needed between the risk, finance, capital management, treasury and business lines to help establish the risk appetite and communicate stress testing results to the board.

There is need for greater integration of risk and finance

It is clear from the survey results (see Figure 9) that, where respondents were given the choice to identify one or more roles involved in stress testing, CROs and the risk management teams are primarily responsible (67%) in their organizations for developing the stress testing models, conducting analysis and reviewing results. Other key stakeholders such as business unit heads and CFOs (13% each) are also responsible for stress testing, albeit to a very low degree.

This highlights the need for greater integration between the CRO and CFO functions (i.e. risk and finance). It also underlines the importance of stress testing in determining capital adequacy at global FIs, where the business units are responsible for their own stress testing and the CFO and his team are responsible for ensuring adequate capital levels post-stress testing. This means that for financial institutions to become truly risk-enabled, the most useful aspect of stress testing may be the process of collaboration and involving stakeholders in a dialogue about scenarios and risks. This is arguably more important than the actual numbers that are generated.

5



Figure 9: Executives assuming primary responsibility for stress tests

Involvement of the board is minimal

Boards and executive committees are only partially involved in stress testing. While a large majority of respondents report that these bodies are involved to a “medium” degree, few describe their involvement as “high” (see Figure 10 – 18% say this of the executive committee and 12% say it of the board). The onus and responsibility of stress testing has largely been entrusted to senior management (mainly the risk committee) with the board of directors having oversight.

As discussed earlier, for most financial institutions stress testing is not treated as a strategic initiative but rather as one component of other risk management activities. This highlights the danger that financial institutions are not giving enough attention to the need for behavioral change and to the strategic thinking needed to fully realize enterprise-level stress testing. To become truly risk-enabled, financial institutions must stop looking at stress testing as largely a box-ticking exercise.

Figure 10: How involved are the following groups in the stress testing process?

0% 10% 20% 30% 40% 50% 60% 0% 80%

ther

C

usiness unit head

C

C 6 %

13%

13%

5%

3%

0% 20% 40% 60% 80% 100%

No in ol ement

ow

Medium

igh

oard

ecuti e committee

usiness lines

Management committee

isk committee 66% 24% 5%

22% 49% 2 %

9%

20% 45%

6%

68%18% 10%

3%12% 10%

5%

30% 5%

5%

5%

2%



The Chartis view

Current attitudes to stress testing as a non-strategic activity are likely to change as there is a strong regulatory push towards the second generation of regulatory stress tests that incorporate test strategies into business planning. As the supervisors continue to adopt a more pro-active engagement model to provide a robust challenge to end-to-end process management, senior executives and the board will be required to understand and be aware of the assumptions and scenarios being incorporated into stress testing models.

Lack of involvement from business lines and management committees

Clearly, the risk committee is the key body in driving the stress testing initiatives of financial institutions. Two-thirds of survey respondents (66%) report that their risk committee is highly involved in stress testing. This is not surprising, as the risk committee serves as a forum for challenging the methodologies, assumptions and results of the stress tests. The same point is highlighted with very few respondents (less than 10%) expressing the opinion that the risk committee has low or zero involvement in the stress testing process.

More significant is the fact that, according to respondents, business lines and management committees are relatively uninvolved. We would expect higher involvement if stress testing was embedded at an enterprise-level in day-to-day business strategies. The more limited involvement of the management committee and business lines raises questions about how stress test results are to be linked to strategic business initiatives, as these levels of management are where business plans are prepared and executed. They assist the risk management teams in preparing plausible stress scenarios and provide the data required.

To take full advantage of stress testing and to drive efficiencies, financial institutions should focus on integrating the stress testing process into their existing business planning, forecasting and business processes. For the risk-enabled enterprise, integration at an enterprise-level requires clear governance, active participation of all stakeholders, dedicated funds/resources and a focus on program management.

The Chartis view

With their increasing focus on stress testing guidelines, such as CCAR, DFAST, ECB/EBA, banks are adopting leading best practices whereby the CRO and risk committee are primarily responsible for stress testing. On the other hand, the board and senior management are engaged in results review, approval and action planning of stress tests, and the business lines are yet to be fully involved in the stress testing process. This means that they are likely to fall short of the increasing requirements for embedding stress testing in day-to-day business strategies and measuring their profitability.



Art and science

Stress testing is ultimately a coming together of art and science. The quantitative facets of effective stress testing are inherent in the requirements for high quality data and robust models. The qualitative facets, however, are just as important, as judgment calls regularly need to be made. Scenario selection, for example, is inherently judgmental. The collaboration required between risk, finance, capital management, treasury and business lines, as well as effective communication of testing results to senior management and the board, demand essentially qualitative skills.

The Art: Lack of knowledge and skills is a key challenge

Against this background, it is no surprise that that finding people with the right knowledge and skills to manage and implement stress testing is a major challenge for many financial institutions. While 40% of survey respondents consider it to be an “important” challenge, 30% characterize it as “critical”. This highlights the fact that it is ultimately the human element that plays the key role in identifying the appropriate assumptions, developing meaningful and relevant scenarios and models, developing and reviewing the results and presenting the facts and possible action plans.

of the respondents

think lack of skills and

knowledge is a very

important challenge

70%

Several of the executives interviewed for this report confirmed that assembling a team with the right composition of skills and expertise (i.e. quantitative, qualitative, business, financial and technology skills) is becoming increasingly difficult. And retaining the talent they do employ within the current market (high demand, scarce resources) will become tougher as costs rise and budgets are diminished. Increased regulatory demands going forward will exacerbate the problem.

6



The science: Modeling and data quality top the improvement agenda

Over one-fifth of survey respondents say that knowledge and skills development is a high priority on their improvement agenda for stress testing. Top of the list, however, when it comes to improving the stress testing approach is better modeling, cited as high priority by 45% of respondents. Another 42% ascribe high priority to improving data quality, and 38% say the same of improving their stress testing technology and systems. This supports our earlier observation that modeling (considering scenario development to be a modeling/business challenge) and data quality along with technology and knowledge are key areas of focus for financial institutions.

of respondents

emphasized the

need to improve the

modeling approach

45% of respondents

emphasized the need

to improve data quality42%

Figure 11: Which areas of your firm’s stress testing program need the greatest level of improvement?

On the other hand, it is noteworthy that almost 40% of respondents accord a low priority to improving governance and control for stress testing. This result on its own would be difficult to reconcile with the fact that regulators and best practices emphasize governance and control as an important element in the success of any program implementation such as stress testing and model risk. However, as noted elsewhere in this report, this may be a short- to mid-term effect given the current focus of regulations. Over the mid- to long-term the trend toward more pervasive regulatory enforcement of stress testing and the “trickle-down” effect of best practice enterprise-level stress testing, in Chartis’s view will alter this outlook. We expect that future surveys will demonstrate a closer alignment of responses to the regulatory mandate, and a higher priority accorded to improvement in governance and control.

0% 20% 40% 60% 80% 100%

ow priority

igh priority

Medium priority

Scenario de elopment

Knowledge skills

o ernance controls

eadership sponsorship

udgets

echnology systems

Data quality

Modeling approach

42% 39% 18%

38% 23% 38%

29% 32% 39%

28% 41% 31%

2 % 35% 38%

56%18% 26%

51%21% 28%

43%45% 13%



Practical implementation of best

practice for enterprise stress testing

Financial institutions have taken action to improve the effectiveness of risk management within their firms, and to advance their stress testing capabilities. The results from our survey, however, clearly demonstrate gaps in stress testing frameworks, which in turn have weakened senior management confidence that key decisions or preventative actions can be based on stress testing results.

A strategic approach first and foremost will mean bringing together disparate processes and structures into an end-to-end stress testing process and providing an overall governance umbrella. This will allow firms to make use of enterprise-level stress testing to facilitate the strategic realization of “The Risk Enabled Enterprise®”.

Enterprise-level stress testing must be seen as an activity that affects the decisions the business takes at a high level and on a day-to-day basis, instead of as an isolated compliance activity. Leadership from the top will be required to drive this forward. Thus, the first group of people that must be convinced of the strategic value of enterprise stress testing is senior management.

Figure 12 describes an example framework linking key activities to key stakeholders.

7



Figure 12: Example framework for linking activities to stakeholders

The model aligns stakeholder responsibilities with implementation imperatives. It should be stressed that integration with respect to data management, risk types (credit, market, liquidity, operational) and business lines is key.

The five steps are outlined below.

Step 1: Establish defined objectives and governance structure for enterprise stress testing

At present organizational silos, which dominate many financial institutions, are proving an ongoing challenge in establishing an efficient enterprise-level stress testing framework. To mitigate that challenge financial institutions need to establish dedicated teams and define stress testing objectives.

Secondly, communication with senior management is critical. With stress testing teams reporting into CROs or CFOs, the need to establish a channel of communication with the

• Board

• Risk committee

• Stress testing teams

• Business lines

• Senior management

Define objectives

and

governance

structure

• Quantitative teams

• Risk committees

• Business lines

Ensure

participation

• Data management and technology teams

• Business lines

• Second line of defense

Boost data

integration

• Board


Establish

KPIs

• Board


• Second and third lines of defense

Ensure

ongoing

monitoring

Key stakeholders

Fe

ed

ba

ck

Key activities



board as well as business line management is critical. Figure 13 below shows an example high-level functional framework to enable enterprise level stress testing with components needed to establish such a framework. Some of these components, for example ALM and market risk, are commonplace within business line silos. However, the stress testing components indicated are not currently fully employed at financial institutions.

Figure 13: Example framework depicting functional elements of a comprehensive

enterprise-level stress testing architecture

Step 2: Ensure participation when defining scenarios

Financial institutions should establish committees/departments with the sole purpose of developing and managing enterprise-level stress testing. These committees/departments should use external scenarios, such as macro-economic shocks, when benchmarking their scenarios developed specifically for internal purposes. Taking these steps would enable financial institutions to develop internal scenarios very specific to business lines. Again, this should take into account different risk types and both quantitative and qualitative measures as well as reverse stress testing.

Financial

systems

Market

data

Other source

systems

Back

office

Front

office

Co

mm

on

me

ssa

gin

g la

ye

r

Cas

h flo

w d

atab

ase

Da

ta in

teg

ratio

n la

ye

r

Rules engines

Reporting interface

Business

planning systems

Scenario variables/

market risk factors,

BI dashboards

ALM systems

RWA calculations,

credit, operational

and market risk

models

Risk

management

systems

Loan, counterparty,

collateral and trading data

Baseline variables

and other P&L items

Required capital,

scenario variables,

stressed parameters

Scenarios

Regulatory

reports

Stress

testing

reports

Management

reportsDa

ta in

teg

ratio

n

Stress testing

system

Finance

& risk

data mart

Stress

testing

Scenario

module

Stress

testing

work flow module

Regulatory

reporting

engine



Furthermore, financial institutions should find ways to encourage and facilitate collaboration so that all parties can have a dialogue about scenarios and risks. This is more important than the actual numbers generated.

Step 3: Break the data silos and boost integration

Financial institutions are confronting major challenges in terms of data quality, quantity and availability. These challenges are compounded by the fact that most of the data sits in various organizational silos and legacy infrastructure which are unable to aggregate data efficiently.

Therefore, financial institutions should invest in infrastructure changes to enhance the capability to efficiently integrate and aggregate data from across various silos. Furthermore, developing powerful computing systems, in-memory analytics and calculation engines, with sophisticated hardware accelerators such as GPUs/FPGAs, would enhance the performance levels to enable faster turnaround of results.

Step 4: Establish key performance indicators

Financial institutions should ensure that scenario development teams have a strong understanding of the key business drivers when developing macro-economic scenarios, as these have a strong impact on organizational cash flows. This is especially true when calculating quantitative measures, such as PDs, EADs and LGDs, which are directly linked to performance.

As a best practice, financial institutions should establish key performance indicators when developing internal models, as well as using external models provided by third-party vendors for benchmarking purposes.

Step 5: Ensure monitoring is an ongoing process

Monitoring sensitive limits provides useful insights to set the risk appetite and also helps financial institutions in their internal risk management activities and day-to-day business planning. Ultimately, the regular monitoring of stress testing processes would enable financial institutions to make adjustments to asset and liability management composition and reduce concentration risks, thus facilitating easier compliance with regulations and confident decision making.



Conclusion

Stress testing is a potent tool at the disposal of financial institutions to keep check of their financial health. For financial institutions to implement this as a strategic program, they should establish a concrete framework reinforced by strong governance.

However, strategic implementation of enterprise-level stress testing by financial institutions needs out-of-the-box thinking to address issues with data quality, shortage of skills and investments in technology and siloed risk management. This will yield benefits not only in terms of compliance but also enhanced performance management to steer strategic decisions.

Implementing enterprise-level stress testing as a strategic program can significantly improve risk awareness. This will further serve to improve risk-based behavior and enhance strategic business decision making. To implement any such framework, however, it is critical that financial institutions take the program beyond a regulatory compliance exercise to a comprehensive business embedded process, a necessary pre-requisite for The Risk Enabled Enterprise®.

8



Appendix A: Survey demographics 9

0% 5% 10% 15% 20%ther

inancial crimeusiness line

reasury managementechnology systems

CMarket risk

perational riskCompliance

Internal auditCredit risk

nterprise risk 19%

16%

13%

10%

10%

8%

5%

5%

5%

2%

2%

%

Job function

0% 10% 20% 30% 40% 50% 60%ensions

Clearing house change

nergy

egulator

Manufacturing

o ernment

Insurance ife

Insurance Non life

Capital markets Sell side

Capital markets uy side

anking 56%

12%

10%

10%

%

%

%

5%

3%

3%

2%

Primary business line



0% 5% 10% 15% 20% 25%

More than 30 n

10 n to 30 n

5 n to 10 n

1 n to 5 n

500m to 1 n

100m to 500m

ess than 100m 23%

16%

5%

9%

16%

16%

16%

Size of organization by revenues

0% 10% 20% 30% 40% 50% 60%

More than 200 000

100 000 to 200 000

50 000 to 100 000

10 000 to 50 000

5 000 to 10 000

ess than 5 000 52%

13%

18%

13%

4%

2%

Size of organization by employees

0% 10% 20% 30% 40% 50%

o

C

urope

North merica 44%

20%

9%

19%

Location



Appendix B: Key survey results10

0% 20% 40% 60% 80% 100%

Not at all

Minor importance

Somewhat important

ery important

C stress tests

ther internal

CC

ther regulatory

illar 2 IC 34% 26% 18%

25% 35% 13%

24% 21% 29%

20% 52% 12%

21%

28%

26%

2 %

15% 36%13% 36%

How important are the following groups/types of stress tests in your organization?

0% 10% 20% 30% 40% 50% 60% 0% 80%

ther

C

usiness unit head

C

C 6 %

13%

13%

5%

3%

Responsibility for stress tests



0% 20% 40% 60% 80% 100%

No in ol ement

ow

Medium

igh

oard

ecuti e committee

usiness lines

Management committee

isk committee 66% 24% 5%

22% 49% 2 %

9%

20% 45%

6%

68%18% 10%

3%12% 10%

5%

30% 5%

5%

5%

2%

How involved are the following groups in the stress testing process?

0% 20% 40% 60% 80% 100%

un forregulatorycompliance onlynot critical

un forregulatorycompliance only

m edded intoall usinessdecision making

e erse stress tests

C stress tests

CC

ther regulatory

ther internal

illar 2 IC 29% 40% 31%

23% 43% 34%

14% 49% 3 %

9% 53% 38%

6% 52% 42%

32% 68%

How integrated are different types of stress tests into other decision making

processes?

0% 20% 40% 60% 80% 100%

Not a challenge

Moderate challenge

Important challenge

Critical challenge

Systems functionalityand performance

Modeling

Knowledge and skills

Data quality 39% 56% 5%

30% 40% 28% 3%

20% 63% 15%

18% 38% 43%

3%

3%

How would you characterize the key challenges in updating and managing stress

tests?



0% 20% 40% 60% 80% 100%

No significantintegration

artiallyintegrated

ellintegrated

perations reference dataaluations etc

ront office risk management

Model alidation

usiness strategy planning

isk strategy

Credit portfolio management 45% 45% 11%

33% 51% 15%

29% 50% 21%

21% 45% 34%

13% 45% 42%

13% 62% 26%

How would you describe the integration of assumptions and data used in the stress

testing with other key business processes?

0% 20% 40% 60% 80% 100%

No change orreduction inscope or depth

Increased depthand scope

Increased scope

Increased depth

e erse stress tests

ther regulatory

ther internal

CC

illar 2 IC

C stress tests 13% 9% 31%

12% 12% 44%

9% 9% 31%

6% 29% 32%

6% 30% 15%

28%3% 25%

4 %

32%

50%

32%

48%

44%

From an industry perspective how do you see the future evolution of the various

stress tests?

0% 20% 40% 60% 80% 100%

ow priority

igh priority

Medium priority

Scenario de elopment

Knowledge skills

o ernance controls

eadership sponsorship

udgets

echnology systems

Data quality

Modeling approach

42% 39% 18%

38% 23% 38%

29% 32% 39%

28% 41% 31%

2 % 35% 38%

56%18% 26%

51%21% 28%

43%45% 13%

Which areas of your firm’s stress testing program need the greatest level of improvement?



11

How to use research and services

from Chartis

In addition to our flagship industry reports, Chartis also offers customized information and consulting services. Our in-depth knowledge of the risk technology market and best-practice allows us to provide high quality and cost-effective advice to our clients. If you found this report informative and useful, you may be interested in the following services from Chartis.

For risk technology buyers

If you are purchasing risk management software, Chartis’s vendor selection service is designed to help you find the most appropriate risk technology solution for your needs.

We monitor the market to identify the strengths and weaknesses of the different risk technology solutions, and track the post-sales performance of companies selling and implementing these systems. Our market intelligence includes key decision criteria such as TCO (total cost of ownership) comparisons and customer satisfaction ratings.

Our research and advisory services cover a range of risk and compliance management topics such as credit risk, market risk, operational risk, GRC, financial crime, liquidity risk, asset and liability management, collateral management, regulatory compliance, risk data aggregation, risk analytics and risk BI.

Our vendor selection services include:

• Buy vs. Build decision support• Business and functional requirements gathering• Identification of suitable risk and compliance implementation partners• Review of vendor proposals• Assessment of vendor presentations and demonstrations• Definition and execution of Proof-of-Concept (PoC) projects• Due diligence activities



For risk technology vendors

Strategy

Chartis can provide specific strategy advice for risk technology vendors and innovators, with a special focus on growth strategy, product direction, go-to-market plans, and more. Some of our specific offerings include:

• Market analysis, including market segmentation, market demands, buyer needs, and competitive forces

• Strategy sessions focused on aligning product and company direction based upon analyst data, research, and market intelligence

• Advice on go-to-market positioning, messaging, and lead generation• Advice on pricing strategy, alliance strategy, and licensing/pricing models

Thought leadership

Risk technology vendors can also engage Chartis to provide thought leadership on industry trends in the form of in-person speeches and webinars, as well as custom research and thought-leadership reports. Target audiences and objectives range from internal teams to customer and user conferences.

Some recent examples include:

• Participation on a “Panel of Experts” at a global user conference for a leading ERM (Enterprise Risk Management) software vendor

• Custom research and thought-leadership paper on Basel 3 and implications for risk technology

• Webinar on Financial crime risk management• Internal education of sales team on key regulatory and business trends and engaging

C-level decision makers



Further reading

• The Risk Enabled Enterprise® – Global Survey Results and Two-Year Research Agenda • The Risk Enabled Enterprise® – Model Risk Management• Model risk management solutions 2014• RiskTech100® 2015

For both of these reports see: www.chartis-research.com

12

The Risk Enabled Enterprise® Enterprise-level stress testing

Software

Transcript of The Risk Enabled Enterprise® Enterprise-level stress testing