EVALUATION OF CREDIT GUARANTEE SCHEMES (CGS): SOME CONSIDERATIONS Allan Riding March 2014.
-
Upload
audra-reynolds -
Category
Documents
-
view
212 -
download
0
Transcript of EVALUATION OF CREDIT GUARANTEE SCHEMES (CGS): SOME CONSIDERATIONS Allan Riding March 2014.
EVALUATION OF CREDIT GUARANTEE SCHEMES (CGS): SOME CONSIDERATIONSAllan RidingMarch 2014
1. Review some of the ways in which credit guarantee schemes (CGS) have been evaluated.
2. Provoke discussion about ways in which credit guarantee schemes (CGS) should be evaluated.
3. Solicit constructive feedback on most recent attempts to evaluate CGS.
Goals for today
• Typical rationales for CGS– According to economic theory intervention is not
warranted without an imperfection, and, when there is an imperfection, the intervention should address the imperfection.
– Information asymmetries between borrower and lender are often cited as an imperfection that provides a rationale for CGS.
– Likewise, scale may also be viewed as an imperfection that supports CGS-type interventions if “good” firms are denied financing.
• Lenders are unable to distinguish good borrowers from bad borrowers because small scale loan balances do not warrant costly due diligence.
1.0 Rationales for Credit Guarantee Schemes (CGS)
• Information asymmetries are more pronounced for…– Smaller firms? (Many)– Innovative firms? (Freel 2007)
• More informationally opaque due to unknown technologies; collateral issues
– Growth-oriented firms? More informationally opaque due to chaos, uncertainty of growth (Ennew and Binks 1996)
– Women-owned firms? (CFIB, 1994; Fay and Williams, 1992)
– Etc.
1.0 On Interventions in the Credit Market
• Development banks– Often with mandate to act as complementary
lenders to those in private sector• Mixed outcomes
– Yeyati et al IIDB, 2004– Rudolph, World Bank 2009
• Credit Guarantee Schemes– The perception that SMEs require interventions
such as CGSs is pervasive:• Green (2003) reports more than 2,250 CGSs across
almost 100 countries. • OECD (2013) the instrument of choice in insulating
SMEs from the financial crisis
2.0 Interventions
• Given the prevalence of CGS, evaluations of cost/benefit balance are necessary, especially when governments are guarantors– Many governments face high debt and are operating
at deficits so value of initiatives must be understood– Financial crisis has prompted commercial lenders to
be more conservative. • Provides incentive for increased demand for CGS as
business borrowers become increasingly likely to resort to CGS
– Basel III allows lenders to use risk rating of guarantor as replacement for risk rating of borrower
• Provides incentive for increased demand for CGS from commercial lenders
3.0 Evaluating Interventions
• Starting point– Need to define precisely what are the goals
of the intervention– Facilitate additional access to financial capital among
firms likely to generate-save for their lack of access to credit-economic welfare?
– UK: to educate lenders with respect to SME lending;– Canada: to facilitate SME growth; – Japan: to rescue firms in disaster situations.– Other?
– Essential that evaluation measures progress towards the stated goals
3.0 Evaluating Interventions
• How to conduct evaluation?– Canada and UK experience (mostly on
Canada)
• “Evaluation” has many definitions– Useful summary (source of next slides):
David Storey (1998) Evaluating the Impact of Public Policies to Support Small Businesses in Developed Economies, University of Warwick Working Paper
3.0 Evaluating Interventions
• UK: – Collected questionnaire data from
borrowers who had received loans under the terms of the (then) DTI Loan Guarantee Scheme (LGS)
– Conducted interviews with primary owners of CGS borrower firms AND with matching lender loan account managers
– Analysis of internal data on frequency of defaults among LGS borrower firms (Cowling et al.)
3.1 Evaluating CGS, UK example
• Survey data issues:– Key respondent problem
• Was the responder to the survey the individual within the firm with best knowledge of issues
– Selection bias• Sampling frame may affect findings
– Non-response bias• Low response rates can compromise results,
especially if non-response was systematic
– Recollected data• Event-related data (e.g., pertaining to date of loan
disbursement) get lost in mists of time.
3.1 Evaluating CGS: On Survey Data
3.1 Six Steps
• Six categories of evaluations frequently observed with respect to government programs
Main concern is to document expenditure, make clear that expenditure is compatible with program goals.
An accounting and legal function, but no economic role.
Example: BDC
(Canada) study
3.1 Six Steps
• Six categories of evaluations frequently observed with respect to government programs
Compares the outcomes of firms assisted by the policy with those of a control group of firms that had not been assisted
Matching based on factors thought to influence sought-for outcomes
Can be difficult to
identify basis for matching
• Canada: – Collected questionnaire data from borrowers
who had received loans under the terms of the (then) Industry Canada CSBF (Canada Small Business Financing program)
– Conducted interviews with primary owners of CGS borrower firms AND with matching lender loan account managers
– Used external data on frequency of defaults among CSBF borrower firms and “comparable” non-CSBF borrowers
3.2 Key resource: Data
• Canada Key Resource– Surveys on Financing of Small and Medium
Enterprises • Initial Survey conducted in 2000 (>10,000
responses)– Follow-up survey in 2001 (~4,000 responses)
• 2004 (>12,000 responses)• 2007 (>12,000 responses)• 2010 (>12,000 responses)• 2013 (only for firms with >10 employees)
– All are stratified and weighted as to size, region, sector and age
3.2 Key resource: Data
• Canada Key Resource– Surveys on Financing of Small and Medium
Enterprises• Initial Survey conducted in 2000 (>10,000
responses)– Follow-up survey in 2001 (~4,000 responses)
• 2004 (>12,000 responses)• 2007 (>12,000 responses)• 2010 (>12,000 responses)• 2013 (only for firms with >10 employees)
– All are stratified and weighted as to size, region, sector and age
3.2 Key resource: Data
Linked to tax data for t=-1 to t=4 using unique business
identifier
• Key task of evaluation is assessing the counter-factual– What would have happened to
businesses in the absence of policy (e.g., CGS)?
• the so-called ‘counter-factual’.
– The effect of policy is defined as the difference between what actually happened and what would have happened in the absence of policy.
3.3 Dealing with the counter-factual
Not directly
observable
• Split data into two groups: (1) firms that had applied for non-guaranteed loans; (2) firms that had applied for and received guaranteed loans.
• Used sample (1) to ‘reverse engineer’ credit scoring algorithm.
• Applied credit scoring model to sample (2) finding that approximately 75% would have been rejected by credit scoring model.
• Compares the outcomes of firms assisted by the policy with outcomes for firms which had not been assisted– inference is that differences in outcomes can
be attributed to the impact of the policy– Problem is that control group firms may not
be directly comparable• For example, if control group simply comprises
other firms in the economy, these may differ systemically from firms that sought loans (arguably less growth oriented) (Recall BDC study)
3.4 Evaluation of Benefits
• Used 2004 Survey of Financing of SMEs.
• Used statistical techniques that seek to explicitly take account of sample selection bias (based on Heckman, 1977).
• Developed statistical model of the selection process and then, taking account of the factors that drive the selection process, estimated a second equation to explain the outcomes.
• Compared job creation by CSBF-assisted SMEs with that of other benchmark samples
• Found that CSBF added significantly to job and value creation
• Essentially, that the CSBF more than pays for itself.
• Two components– Costs of program administration
• Depends on level of government involvement– e.g., in Canada government is passive: keeps
records; compensates commercial lenders to honor defaults; receives fees on loans from lenders
– e.g., in some jurisdictions, government approves loan guarantees and second tier re-insurance is the norm
• Estimated from internal data.
– Costs of honoring defaults• Timing matters!!
3.5 Evaluation of costs.
• Costs of honoring defaults• Timing matters!!
4.0 Evaluation of costs.
Figure 1: Chronology of Claim Receipts
Early claims (a) are costly and (b) may reflect lenders’ poor loan
adjudication practices.Later claims
may not matter much; also
borrowers had paid fees over life of loans
Example: Costs of honoring defaults
• Costs of honoring defaults– Difficult to attribute costs to particular cohorts
• Tendency to use logistic/probit regression models to attribute costs to firm attributes/program parameters/cohort.
• BUT…– Right-censoring of data (need to wait years before
knowing whether or not a given loan will default).– Impacts of attributes etc. may vary over time
» example: age of borrower firm at time of disbursement may be a key factor immediately following disbursement (if young firms are riskier than old firms) but arguably less of a factor ten years later
3.5 Evaluation of costs.
Logistic Regression Example
• Costs of honoring defaults• Event history methods (survival analysis)
– For example, proportional hazards model (aka Cox Regression)
– Probability of default =a) A baseline pattern that depends on time since
disbursementMODIFIED BYb) A function that models how attributes, program
parameters etc. shift the baseline pattern – Used widely in medicine (how does dosage of medicine
influence survival time?), quality control (how does attribute of production item influence useful life?).
3.5 Evaluation of costs.
Example: Proportion Hazards Model Results
• In evaluating CGS, need (at least?) three dimensions– Estimation of
incrementality/additionality– Estimation of benefits (and factors that
influence benefits)– Estimation of costs
• Administrative data• Default data
4.0 Summary
THANK YOUComments, Questions, Discussion