CFPB Proxy Methods: Implications on Fair Lending … Use of Proxies: Implications on Fair Lending...
Transcript of CFPB Proxy Methods: Implications on Fair Lending … Use of Proxies: Implications on Fair Lending...
CFPB Proxy Methods:
Implications on Fair Lending Testing
October 21, 2014
Arthur Baines
2
Use of Proxies: Implications on Fair Lending Testing
Agenda
• When to Use Proxies?
• Which Proxy methods do Regulators use?
• How to calculate BISG Proxies?
• Are they accurate?
• How do Regulators apply proxies in fair lending analysis?
• Key challenges in analyzing disparities in dealer reserve or ‘mark-
up’1
1As defined in CFPB Bulletin 2013-02
3
Use of Proxies: Implications on Fair Lending Testing
When to Use Proxies?
• Relevant Products: Auto, Credit Card, non-HMDA mortgage
• Proxies may be effective in the context of monitoring for fair
lending compliance
• Underwriting outcomes
• Pricing outcomes
• Proxies have some significant limitations
1As defined in CFPB Bulletin 2013-02
4
Use of Proxies: Implications on Fair Lending Testing
Which Proxy methods do Regulators use?
• All agencies estimate race/ethnicity using proxies or neighborhood characteristics
based on publically available Census Bureau data
(http://www.visualwebcaster.com/FederalReserveBankSF/94628/event.html)
• CFPB uses Bayesian Improved Surname Geocoding (BISG) to estimate
race/ethnicity
• Federal Reserve Board uses Majority Minority approach (neighborhood) and
surname (Hispanic) or first name (gender)
• OCC and FDIC use traditional geographic, surname proxies
• BISG -- tested by researchers at Rand on health care claims data with known
race/ethnicity1
1Elliott, Marc N. et al, “Using the Census Bureau’s Surname List to Improve Estimates of Race Ethnicity and Associated
Disparities,” Health Serv Outcomes Res Method (2009) 9:69–83.
5
Use of Proxies: Implications on Fair Lending Testing
http://www.census.gov/genealogy/www/data/2000surnames/index.html
How to calculate BISG Proxies?
Step 1: Surname
Race/Ethnicity Share
Hispanic 1.5%
African American 33.8%
Asian/PI 0.4%
American Indian 0.9%
White 61.6%
2+ Races 1.8%
Total 100.0%Source: Census Bureau
Probabilities for Surname "Johnson"
6
Use of Proxies: Implications on Fair Lending Testing
*Important – BISG does not use the Intra-tract shares commonly used in other geography-based proxies.
How to calculate BISG Proxies?
Step 2: Geography
Race/Ethnicity
Tract
Counts
Intra-Tract
Shares*
U.S. 18+
Population
Count
Share of
U.S.
Hispanic 1,340 24.5% 36,138,485 0.0037%
African American 1,008 18.4% 27,327,470 0.0037%
Asian/PI 307 5.6% 11,637,514 0.0026%
American Indian 15 0.3% 1,600,043 0.0009%
White 2,693 49.2% 157,123,289 0.0017%
2+ Races 109 2.0% 3,177,961 0.0034%
Total 5,472 100.0% 237,004,762 0.0023%Source: Census Bureau
18+ Population of Tract 0050.02 - Washington, DC
7
Use of Proxies: Implications on Fair Lending Testing
1Elliott, Marc N. et al, “Using the Census Bureau’s Surname List to Improve Estimates of Race Ethnicity and Associated
Disparities,” Health Serv Outcomes Res Method (2009) 9:69–83.
How to calculate BISG Proxies?
Step 3: BISG Probabilities
Race/Ethnicity
Surname
"Johnson"
Tract
0050.02
Wash, DC
BISG
Probability
Hispanic 1.5% 0.0037% 2.3%
African American 33.8% 0.0037% 51.1%
Asian/PI 0.4% 0.0026% 0.5%
American Indian 0.9% 0.0009% 0.4%
White 61.6% 0.0017% 43.3%
2+ Races 1.8% 0.0034% 2.6%
Total 100.0% 0.0023% 100.0%Source: Census Bureau & CRA computations
8
Use of Proxies: Implications on Fair Lending Testing
Are race and ethnicity proxies accurate?
• Proxies built upon Census Bureau geography and surname share some
challenges.
• Overestimate the share of African Americans and Hispanics in the
portfolio,
• Fail to identify significant numbers of minority contracts,
• Errors are correlated with geography, FICO, income and relative
income,
• BISG subject to high error rates, but relatively lower than geography or
name alone.
9
Use of Proxies: Implications on Fair Lending Testing
Classifier or
Proxy Hispanic White Black
Asian/Pacific
Islander
American
Indian/Alask
a Native Multiracial
Reported 5.8% 82.9% 6.2% 4.5% 0.1% 0.4%
BISG 6.1% 79.7% 7.5% 5.0% 0.2% 1.4%
Surname Only 7.4% 75.4% 10.0% 4.9% 0.6% 1.7%
Geography Only 7.2% 78.6% 8.1% 4.8% 0.3% 1.0%
Souce: CFPB "Using Publically Available Information to Proxy for Unidentified Race and Ethnicity," September 2014
Table 2: Distribution of loans by race and ethnicity
• The overstatement of minorities populations is high under all three
scenarios, but relatively lower with BISG.
Are race/ethnicity proxies accurate?
Table 10: Classification Over Ranges of BISG Proxy For Non-Hispanic Black
Black BISG
Proxy Probability
Range
Total
Applications
(1)
Estimated
Black (BISG)
(2)
Reported
Black
(3)
Reported
White
(4)
Reported
Other
Minority
(5)
0-10 160,733 1,859 1,466 139,684 19,583
10-20 9,742 1,387 941 8,403 398
20-30 4,916 1,207 906 3,814 196
30-40 3,101 1,072 726 2,242 133
40-50 2,229 997 738 1,408 83
50-60 1,680 922 736 877 67
60-70 1,417 920 765 596 56
70-80 1,407 1,057 963 391 53
80-90 1,517 1,293 1,222 241 54
90-100 3,693 3,548 3,408 200 85
Total 190,435 14,262 11,871 157,856 20,708
Souce: CFPB "Using Publically Available Information to Proxy for Unidentif ied Race and Ethnicity," September 2014
10
Use of Proxies: Implications on Fair Lending Testing
Are race/ethnicity proxies accurate?
Reproduced from CFPB White Paper, Summer 2014, “Using publically available information to proxy for unidentified race and ethnicity”
21% overestimation
BISG probabilities
highly inaccurate
11
Use of Proxies: Implications on Fair Lending Testing
Identified by
Proxy
Correctly
Identified by
Proxy
Not Identified
by Proxy
(false
negatives)
Percent
Wrongly
Included
(false
positives)
African American 12,874 3,379 2,547 832 10,327 19.8% 80.2% 24.6%
Hispanic 13,623 9,561 8,261 1,300 5,362 60.6% 39.4% 13.6%
Asian 7,341 4,072 3,524 548 3,817 48.0% 52.0% 13.5%
non-Hispanic White 157,834 129,793 123,447 6,346 34,387 78.2% 21.8% 4.9%
Proxy
Method Race/Ethnicity
Count of
Borrowers
in Group
Proxy = Yes
Actual = Yes
Proxy = Yes
Actual = No
Proxy = No
Actual = Yes
Percent of Actual Group
Comparison of Proxy Approaches at Identifying Race/Ethnicity
BISG-80%
African American 12,874 1,206 912 294 11,962 7.1% 92.9% 24.4%
Hispanic 13,623 1,349 1,005 344 12,618 7.4% 92.6% 25.5%
Asian 7,341 12 8 4 7,333 0.1% 99.9% 33.3%
non-Hispanic White 157,834 98,410 90,917 7,493 66,917 57.6% 42.4% 7.6%
Tract-80%
African American 12,874 569 462 107 12,412 3.6% 96.4% 18.8%
Hispanic 13,623 11,397 9,423 1,974 4,200 69.2% 30.8% 17.3%
Asian 7,341 3,946 3,405 541 3,936 46.4% 53.6% 13.7%
non-Hispanic White 157,834 93,612 88,850 4,762 68,984 56.3% 43.7% 5.1%
Name-80%
African American 12,874 5,001 2,544 2,457 10,330 19.8% 80.2% 49.1%
Hispanic 13,623 5,555 2,902 2,653 10,721 21.3% 78.7% 47.8%
Asian 7,341 458 251 207 7,090 3.4% 96.6% 45.2%
non-Hispanic White 157,834 167,468 145,636 21,832 12,198 92.3% 7.7% 13.0%
Tract-50%
Are race/ethnicity proxies accurate?
12
Use of Proxies: Implications on Fair Lending Testing
Hispanic Black Asian White
2011 National 11.5% 19.2% 10.1% 6.9%
Year State
Percent of Household with No Vehicle
Comparison of Vehicle Ownership by Race/ethnicity and geography
2011 AL 4.0% 13.6% 2.8% 4.0%
2011 AK 9.8% 15.8% 3.2% 10.7%
2011 AZ 8.3% 13.8% 6.7% 6.3%
2011 AR 5.2% 14.3% 5.2% 4.8%
2011 CA 8.0% 14.9% 7.5% 7.0%
2011 CO 6.4% 13.7% 6.1% 4.9%
2011 CT 18.7% 22.6% 6.8% 6.1%
2011 DE 2.3% 10.7% 6.6% 4.1%
2011 DC 44.1% 40.7% 43.5% 33.1%
Source: Census Bureau, American Community Survey
Why are race and ethnicity proxies are not accurate?
13
Use of Proxies: Implications on Fair Lending Testing
BISG Probabilities for a Hypothetical Portfolio
0% 100%
There are two methods for using the BISG probabilities
1) Continuous or Proportional estimation: measures the change as we
move from lower probabilities to higher probabilities
• Requires the use of regression analysis
2) Threshold-based estimation:
- identify contracts with > 80% probability of being African American
- identify contracts with > 80% probability of being Non-Hispanic White
- compare the outcomes between groups
• Can be calculated with or without use of regression analysis
If no controls are used, the observed differences are referred to as “raw”
14
Use of Proxies: Implications on Fair Lending Testing
Key challenges in analyzing disparities in dealer reserve
• Portfolio level results
• Are they meaningful, or do they simply reflect the different pricing strategies
across dealerships?
• What controls will the CFPB allow?
• Is the information/data available?
• Dealership level results
• Low contact volume from most dealerships
• No identifiable minority contracts from most dealerships
• What do you do with the monitoring results?
• When and what do you communicate to the dealership?
• Remuneration?
• Do you ‘participate’ in the dealer reserve?
15
Use of Proxies: Implications on Fair Lending Testing
Where to start?
• Establish a base line
• Analyzing underwriting and buy rate may be helpful
• Understand what the Regulator/CFPB is going to see.
• Measure dealer reserve in bps (test sensitivity to other measures).
• Use BISG and test threshold vs continuous probability specification.
• Portfolio level:
• Calculate raw differences in likelihood and level of dealer reserve.
• Recalculate differences controlling for geography, basic deal structure, and available
competitive factors.
• Dealer level:
• Establish a contract volume screen
• Calculate raw differences in likelihood and level of dealer reserve
• Identify statistically significant results
• Develop a dealership escalation plan
16
Use of Proxies: Implications on Fair Lending Testing
Helpful Sources:
• Census Bureau Geography level data: • Data: http://www2.census.gov/census_2010/04-Summary_File_1
• Documentation: http://www.census.gov/prod/cen2010/doc/sf1.pdf
• Census Surname List: • Data: http://www.census.gov/genealogy/www/data/2000surnames/names.zip
• Documentation: http://www.census.gov/genealogy/www/data/2000surnames/surnames.pdf
• Additional research and articles • AUTOMOTIVE FINANCE - WILL DEALERSHIP FINANCE RESERVE GO THE WAY OF MORTGAGE
YIELD SPREAD PREMIUMS?
HTTP://WWW.CRAI.COM/UPLOADEDFILES/PUBLICATIONS/AUTOMOTIVE-FINANCE-FE-
WHITEPAPER-0313.PDF.
• HOW THE CFPB’S AUTO FINANCING RULE AFFECTS CONSUMERS; AMERICAN BANKER,
http://www.americanbanker.com/bankthink/how-the-cfpbs-auto-financing-rule-affects-consumers-1058204-
1.html; APRIL 10, 2013.
• COMPLIANCE IN THE INDIRECT AUTOMOTIVE MARKET, KEY ISSUES IN FAIR LENDING ANALYSIS;
ABA BANK COMPLIANCE;
HTTP://WWW.ABA.COM/PRODUCTS/BANKCOMPLIANCE/DOCUMENTS/BANKCOMPL_2013_09_EFEA
TURE.PDF; SEPTEMBER/OCTOBER 2013.
Arthur Baines
Vice President
Financial Economics
Charles River Associates
202-662-7838