Stress Testing EMEA 2014, London

Presenter: Anshuman Prasad, Director, Risk and Analytics

Practical Challenges in Building Effective

Models for Stress Testing

February 26, 2014

Executive Summary

Regulatory guidelines on stress testing require banks to build

models that have a greater linkage to macroeconomic variables

Macro-to-micro modeling is the preferred method for ensuring that

macroeconomic variables are incorporated into existing models

The logistics of conducting an enterprise-wide stress testing process

make it imperative to balance model complexity with model elegance

Stress scenario design has implications on variable selection and

additional variable needs while building models

In particular, banks are adopting a more granular variable selection

approach to:

– Enhance model performance across non-standard stress scenarios

– Adequately stress bank-specific/idiosyncratic risk exposures

2

Agenda

What Do Regulators Look For?

Macro-to-micro Models

Forecasting PPNR

Balancing Model Complexity with Model Elegance

Not All Recessions Are Created Equal – Building Flexible Models for

Atypical Stress Scenarios

3

What Do Regulators Look For? (1/2)

4

Stress testing guidelines issued by regulators globally (US, UK, EU)

have certain common themes

Quantitative models clearly conditioned on macroeconomic

scenarios

Consistent modeling and quantification techniques for both, “Base”

and “Stress” scenario forecasts

Modeling

Techniques

Scenario

Development

Intellectual

Rigor

Adequate variable coverage for robust model development

Tailored scenarios, sufficient to address all material risk exposures

and business activities

Sound theoretical basis for independent variable selection that is

clearly communicated

Demonstration of thought process and progression of model

development to justify the final approach

What Do Regulators Look For? (2/2)

5

A good model cannot operate in isolation

Integrate forecast models and processes with consistent

assumptions amongst related forecast components

Ability to incorporate business intelligence with stressed model

outcomes

Integration

Governance

Documen-

tation

Robust independent model review and validation

Evidence of effective review and challenge, including Board of

Directors

Often overlooked, but just as critical as the modeling itself

Clear narratives and transparent documentation of the model

development process and results

Rationale and process for quantifying qualitative adjustments

clearly explained

Macro-to-micro Models

6

Typically needed in the following two situations:

Variable expansion required to improve overall model fit and model performance

Non-standard risk parameters needed to adequately stress bank-specific/

idiosyncratic risk

Expanded variable coverage to include components of GDP

Forecasting macroeconomic variables by geography to account for geographic

concentration risk

Forecasting market indices specifically used to price bank assets subject to fair

market value accounting treatment

Examples

Why Are They Useful?

Used to form relationships between macroeconomic variables and

model risk parameters

Provision Expense PPNR

Forecasting PPNR

7

Stress Testing

New Business Volume Liquidity Stress Prepayment Model Non-accrual Loan Balance

Forecasted Stressed Balance Sheet

Credit Loss Forecast

Top-down Approach

Bottom-up Approach

PPNR

Net Interest Income Forecast

Non-interest Income Forecast

Non-interest Expense Forecast

Revenue sources need to be modeled at a granular level in order to establish

appropriate macroeconomic linkages to the models

Trading operations difficult to model due to a large number of granular market

shocks, nearly 3000 in CCAR 2014

Forecasting capital adequacy under stress must take into account

the balance sheet and the income statement

Forecasted Macroeconomic Scenarios

Advanced modeling techniques can provide unique advantages to the

practitioner, however, these advantages must be weighed-in by the

cost of complexity

The model must be sophisticated enough to perform scenario

analysis, but not overly complicated to the point that:

– It is impossible to integrate with related model components

– It takes too long to re-run the model if there are problems with previous

processes feeding the model

– When a scenario output does not make sense, no one can figure out why

– The business finds it difficult to explain model outputs

Sample Modeling Techniques Used Include:

Balancing Model Complexity with Model Elegance

Multifactor Linear Regression

Logistic and Probit regression

Panel Data Analysis

ARCH/GARCH

Generalized Linear

Models

ARIMA and ARIMAX

VAR/VECM

Impulse Response Analysis

Artificial Neural Networks

8

Final model results can be problematic due to issues such as:

– Over fitting

– Increased model complexity as a result of obsessing over marginal

improvements to statistical tests

– Data mining/examining too many variables that may be completely unrelated to

the modeled component

“There is no mandate that models be complex. As a matter of fact, we

have actually suggested just the opposite – that in some instances,

models can be simple.”

– Federal Reserve Regulator Speaking at the December, 2013 CRISIL GR&A

Roundtable

Balancing Model Complexity with Model Elegance

9

Adverse scenarios first introduced in CCAR 2013

The adverse scenarios are quite different from a typical stress scenario

– CCAR 2013: Economic contraction coupled with high inflation. (i.e. “Stagflation” not seen

since the 1970s)

– CCAR 2014: Economic contraction coupled with a sharp rise in interest rates due to

global sell off in long-term fixed income assets

Most time and effort spent on the baseline and severely adverse scenarios in

previous CCAR submissions

– In 2014, the Adverse scenario is also going to be used by the regulators to assess capital

Interest Rates & Inflation in Recent CCAR Exercises

Not All Recessions are Created Equal Building Flexible Models for Atypical Stress Scenarios

Regulatory Trends – Emerging Importance of Non-standard Stress Scenarios

Nine Quarter Average CPI 3mo UST 10yr UST

2014 Severely Adverse 1.08% 0.10% 1.26%

2014 Adverse 1.58% 0.10% 5.09%

2013 Severely Adverse 0.98% 0.10% 1.39%

2013 Adverse 3.67% 2.14% 3.66%

10

Not All Recessions are Created Equal Building Flexible Models for Atypical Stress Scenarios

Loss rates between scenarios displayed a counterintuitive rank ordering that

could not be explained

Matter Requiring Immediate Attention (MRIA) issued in CCAR 2013 for lack of

logic surrounding variable selection

Problems Encountered in Modeling Atypical Scenarios

Using factors directly related to model performance instead of proxy variables

reduces the chance of counterintuitive modeling results

Macro-to-micro models that deal with bank specific risk parameters will need to

be created to deal with idiosyncratic scenario risk

How to Build Flexible Models?

Example 2

Model Output = f (Variable A)

Variable B = f (Variable A)

Y = f (Variable B)

Example 1

Model Output = f (Variable A)

Y = f (Variable A)

X

11

www.crisil.com/gra

Not All Recessions are Created Equal Building Flexible Models for Atypical Stress Scenarios

2013 Adverse Scenario

More severe than all but two

recessions in terms of duration

High interest rates and inflation

2014 Adverse Scenario

Smaller peak-to-trough decline

in GDP but still very severe

relative to historical recessions

Atypical rise in interest rates on

the long end of the yield curve

Economic Output (Real GDP)

Interest Rates & Inflation

Nine Quarter Average CPI 3mo UST 10yr UST

2014 Severely Adverse 1.08% 0.10% 1.26%

2014 Adverse 1.58% 0.10% 5.09%

2013 Severely Adverse 0.98% 0.10% 1.39%

2013 Adverse 3.67% 2.14% 3.66%

CCAR Scenarios at A Glance

13

Both Stress Testing Frameworks Require Two Sets of Scenarios

Annexure: UK Versus US Stress Tests

The final shape of UK stress tests yet to be finalized, but the proposed

framework has more similarities than differences with the US CCAR & DFAST

requirements – particularly with regard to scenario design

Risk Factors Included in Common and Internal Scenarios

Internally

Generated Bespoke

Scenarios Common Baseline

& Stress Scenarios

Bank-specific/

Idiosyncratic Risk

Systemic

Risk

Total Risk

Exposure

=

Systemic

Risk

+

Idiosyncratic

Risk

Common baseline and stress

scenarios ‒ Applied across all bank participants

‒ Additional variables often required

Internally generated scenarios ‒ Scenarios designed individually by

each bank

‒ Internal scenarios would typically result

in higher losses than common stress

scenarios

1 2

14