1

Benchmarking the Performance of US Banks

R. Barr, R. Barr, SMUSMU

T. Siems, T. Siems, Federal Reserve Bank of DallasFederal Reserve Bank of Dallas

S. Zimmel, S. Zimmel, SMUSMU

Financial Industry StudiesFinancial Industry Studies, Dec. 1998: www.dallasfed.org, Dec. 1998: www.dallasfed.org

2

Motivations and Goals

MotivationsMotivations Safety and soundness of banking systemSafety and soundness of banking system Protection of FDIC insurance fundProtection of FDIC insurance fund Best allocation of examiner resourcesBest allocation of examiner resources

GoalsGoals Prioritization of on-site examinationsPrioritization of on-site examinations Early-warning indicators of troubled banksEarly-warning indicators of troubled banks

3

Objectives of the Research

Benchmark the U.S. banking system over Benchmark the U.S. banking system over the last decadethe last decade

Assess performance with DEA-based model Assess performance with DEA-based model Isolate best- and worst-practice banksIsolate best- and worst-practice banks Support bank auditors by predicting trouble Support bank auditors by predicting trouble Evaluate DEA in large-scale benchmarking Evaluate DEA in large-scale benchmarking

rolerole

4

Previous Work

Measuring bank management quality with Measuring bank management quality with DEADEA Barr, Seiford, Siems, 1993Barr, Seiford, Siems, 1993

Bank Failure Prediction ModelBank Failure Prediction Model DEA score as input to logit forecasting modelDEA score as input to logit forecasting model Barr and Siems, 1996Barr and Siems, 1996

Technical report versions available at:Technical report versions available at: www.smu.edu/~barrwww.smu.edu/~barr

5

Data Envelopment AnalysisData Envelopment Analysis

A methodology for integrating and A methodology for integrating and analyzing benchmarking data that:analyzing benchmarking data that: Performs a multi-dimensional “gap analysis”Performs a multi-dimensional “gap analysis” Considers interactions, tradeoffs, substitutionsConsiders interactions, tradeoffs, substitutions Integrates all performance measuresIntegrates all performance measures Gives an overall performance ratingGives an overall performance rating Suggests credible organizational goals, Suggests credible organizational goals,

benchmarking partners, ….benchmarking partners, ….

6

Bank Performance Model

Inputs

(Resources, Xs)

Outputs

(Desired outcomes, Ys)

• Salary expense

• Premises & fixed assets

• Other noninterest expense

• Interest expense

• Purchased funds

•Earning assets

• Interest income

• Noninterest income

7

Defining EfficiencyDefining Efficiency

Efficiency = ratio of weighted sums of the Efficiency = ratio of weighted sums of the inputs and outputs (>0)inputs and outputs (>0)

Defines best practice in a DEA modelDefines best practice in a DEA model

E

w Y

v X

j jj

i ii

8

How DEA WorksHow DEA Works

Instead of using fixed weights for all units Instead of using fixed weights for all units under evaluation,under evaluation, DEA computes a separate set of weights for DEA computes a separate set of weights for

each bankeach bank Weights optimized to make that bank’s score Weights optimized to make that bank’s score

the best possiblethe best possible Constraints: no bank’s efficiency exceeds 1 Constraints: no bank’s efficiency exceeds 1

when using the same weightswhen using the same weights

9

Formulating a DEA ModelFormulating a DEA Model

There are many DEA modelsThere are many DEA models The basic idea in each is to choose a set of The basic idea in each is to choose a set of

weights for DMU weights for DMU kk that: that:

positive weightsall

bank each for ,1 :such that

Maximize

pE

E

p

k

10

Measuring DistanceMeasuring Distance

ffff11zz

Ef z

f z

| |

| |1

0

10

20

30

40

50

60

0 5 10 15

Bank Input ($Expenses)

Ban

k O

utp

ut

($ I

nco

me) Efficient frontier of best practice

Inefficient bank

11

Introducing Expert Judgment

Classic models may result in unreasonable Classic models may result in unreasonable weight assignments for inputs & outputsweight assignments for inputs & outputs 0 weights on unflattering dimensions0 weights on unflattering dimensions Can overemphasize secondary factorsCan overemphasize secondary factors

We added weight multipliers to the DEAWe added weight multipliers to the DEA Based on survey of 12 FRB bank examinersBased on survey of 12 FRB bank examiners Used response ranges to set UB/LBs on weightsUsed response ranges to set UB/LBs on weights

12

Survey-Derived Constraints

Analytic Hierarchy Survey range Survey average process weights

Inputs Salary Expense 15.8% - 35.9% 23.10% 25.20% Premises/Fixed Assets 3.1% - 15.7% 9.60% 11.40% Other Noninterest Expense 15.8% - 35.9% 22.70% 19.80% Interest Expense 17.2% - 42.8% 25.90% 23.50% Purchased Funds 12.1% - 34.0% 18.80% 20.20%

Outputs Earning Assets 40.9% - 69.5% 51.30% 52.50% Interest Income 25.7% - 46.9% 34.30% 33.80% Noninterest Income 10.2% - 20.2% 14.40% 13.70%

13

Banking Industry Test Data

End of year data for:End of year data for: 19911991 11,397 banks11,397 banks 19941994 10,224 banks10,224 banks 19971997 8,628 banks8,628 banks

Used constrained CCR-I modelUsed constrained CCR-I model Run with large-scale specialized DEA Run with large-scale specialized DEA

softwaresoftware

14

1991 Profiles by DEA E-Quartile

(Values expressed as a percent of total bank assets)

1991 data DEA Efficiency Quartile most to1 2 3 4 least efficient

most efficient least efficient differenceINPUTS

Salary Expense / Total Assets 1.43% 1.54% 1.65% 1.83% -0.40% *Premises and Fixed Assets / Total Assets 1.00% 1.48% 1.76% 2.22% -1.22% *Other Noninterest Expense / Total Assets 1.53% 1.62% 1.84% 2.41% -0.87% *

Interest Expense / Total Assets 4.71% 4.70% 4.66% 4.62% 0.08% *Purchased Funds / Total Assets 6.29% 8.17% 11.12% 16.07% -9.78% *

OUTPUTSEarning Assets / Total Assets 92.68% 91.67% 90.59% 88.24% 4.44% *Interest Income / Total Assets 8.68% 8.71% 8.67% 8.55% 0.13% *

Noninterest Income / Total Assets 0.95% 0.79% 0.89% 1.00% -0.05%

N 2,850 2,848 2,849 2,850average efficiency score 0.7340 0.5982 0.5387 0.4611 0.2728 *

lower boundary 0.6334 0.5665 0.5092 0.0000upper boundary 1.0000 0.6334 0.5664 0.5091 * Significant

at 0.01

15

1997 Profiles by DEA E-Quartile1997 data DEA Efficiency Quartile most to

1 2 3 4 least efficientmost efficient least efficient difference

INPUTSSalary Expense / Total Assets 1.67% 1.60% 1.64% 1.75% -0.08%

Premises and Fixed Assets / Total Assets 0.98% 1.55% 1.94% 2.44% -1.45% *Other Noninterest Expense / Total Assets 1.85% 1.31% 1.50% 1.92% -0.07%

Interest Expense / Total Assets 3.29% 3.30% 3.27% 3.15% 0.14% *Purchased Funds / Total Assets 10.46% 12.33% 13.63% 15.32% -4.85% *

OUTPUTSEarning Assets / Total Assets 92.99% 92.60% 91.83% 90.65% 2.33% *Interest Income / Total Assets 7.45% 7.41% 7.37% 7.33% 0.13% ~

Noninterest Income / Total Assets 1.80% 0.77% 0.84% 0.90% 0.90% *

N 2,157 2,157 2,157 2,157average efficiency score 0.6685 0.4313 0.3717 0.3067 0.3617 *

lower boundary 0.4722 0.3982 0.3451 0.0000upper boundary 1.0000 0.4721 0.3981 0.3450

16

Analysis of Results

1991 significant 1991 significant differences, Q1-Q4:differences, Q1-Q4: All inputs, and most All inputs, and most

outputsoutputs DEA scoresDEA scores

Changed by 1997:Changed by 1997: Inputs: Salary, other Inputs: Salary, other

non-interest (not sig.)non-interest (not sig.) Outputs: non-interest Outputs: non-interest

income now signif.income now signif.

Noninterest income a Noninterest income a new focus for banksnew focus for banks Fee incomeFee income Off-balance sheet Off-balance sheet

activitiesactivities

17

Other Bank Performance Metrics

DEA Efficiency Quartile m ost to

1 2 3 4 least efficient

1991 data m ost efficient least efficient difference

Return on Average Assets 1.23% 1.00% 0.82% 0.01% 1.22% *Equity / Total Assets 10.35% 8.81% 8.25% 7.76% 2.59% *

Total Loans / Total Assets 48.95% 53.34% 54.74% 56.56% -7.61% *Non-performing Loans / Gross Loans 1.55% 1.65% 1.96% 2.93% -1.38% *

N 2,850 2,848 2,849 2,850average efficiency score 0.7340 0.5982 0.5387 0.4611 0.2728 *

lower boundary 0.6334 0.5665 0.5092 0.0000upper boundary 1.0000 0.6334 0.5664 0.5091 * Significant

at 0.01

18

Relationship with Other Metrics

Efficient banks:Efficient banks: Greater return on Greater return on

assetsassets Higher equity capitalHigher equity capital Fewer risky assetsFewer risky assets

1991 vs. 19971991 vs. 1997 Not comparable scoresNot comparable scores But underlying trends But underlying trends

of variables’ of variables’ importance help importance help explain banking explain banking industry changes industry changes

19

FRB Bank Examination Criteria

CCapital adequacyapital adequacy AAsset qualitysset quality MManagement qualityanagement quality EEarningsarnings LLiquidityiquidity

20

Bank Examiner Ratings

Confidential scores Confidential scores from on-site visitsfrom on-site visits

On each CAMEL On each CAMEL factor and overallfactor and overall

Values from 1 to 5Values from 1 to 5

1 = sound in every1 = sound in every respect respect

2 = sound, modest2 = sound, modest weaknesses weaknesses

3 = weaknesses that give3 = weaknesses that give cause for concern cause for concern

4 = serious weaknesses 4 = serious weaknesses

5 = critical weaknesses,5 = critical weaknesses, failure probable failure probable

21

CAMEL Ratings & DEA Scores

Compared CAMEL Compared CAMEL ratings and DEA ratings and DEA efficiency scoresefficiency scores

Included banks Included banks examined recently:examined recently:1991: 7,487 banks1991: 7,487 banks

1994: 7,679 banks1994: 7,679 banks

1997: 4,494 banks1997: 4,494 banks

CAMEL rating groupsCAMEL rating groups Strong: 1 or 2 ratingStrong: 1 or 2 rating Weak: 3-5 ratingWeak: 3-5 rating

DEA-score groupsDEA-score groups Quintile, by efficiencyQuintile, by efficiency

If no relationship, each If no relationship, each group should contain group should contain 20% of each of the 20% of each of the other metric’s groupsother metric’s groups

22

Efficiency vs. CAMEL Ratings

Efficiency Score Quintiles by CAMEL RatingCombined Data 1991, 1994, 1997

0%

20%

40%

60%

80%

100%

1 2 3 4 5

CAMEL Rating

Per

cent

age

of B

anks

in E

ffici

ency

Qui

ntile

1st quintile (highest DEA scores)2nd quintile3rd quintile4th quintile5th quintile (lowest DEA scores)

23

“Strong” vs. “Weak” CAMELs1991 1994 1997

Strong Weak Strong Weak Strong WeakBanks Banks Banks Banks Banks Banks

INPUTSSalary Expense / Total Assets 1.54% 1.83% 1.65% 2.23% 1.63% 2.04%

Premises and Fixed Assets / Total Assets 1.53% 1.97% 1.65% 1.98% 1.72% 1.94%Other Noninterest Expense / Total Assets 1.64% 2.41% 1.71% 2.92% 1.43% 2.26%

Interest Expense / Total Assets 4.64% 4.82% 2.61% 2.73% 3.25% 3.48%Purchased Funds / Total Assets 10.24% 11.39% 10.95% 10.79% 12.65% 14.83%

OUTPUTSEarning Assets / Total Assets 91.65% 87.85% 91.92% 88.11% 92.18% 89.72%Interest Income / Total Assets 8.55% 8.88% 6.83% 7.32% 7.33% 7.88%

Noninterest Income / Total Assets 0.83% 1.05% 0.93% 1.38% 0.85% 1.25%

DEA EFFICIENCY SCORE 0.5942 0.5235 0.6137 0.5532 0.4272 0.3751

Number of Institutions 5,641 1,846 7,188 491 4,273 221

24

In Summary

DEA useful in benchmarking in service DEA useful in benchmarking in service industryindustry

Can provide information for examiners, but Can provide information for examiners, but not perfect predictornot perfect predictor

Large-scale efficiency analyses can give Large-scale efficiency analyses can give insight into industry dynamics and structure insight into industry dynamics and structure changeschanges