Quality and Accountability in Health: Audit Evidence from Primary Care Providers

SITEJune 2013

Jishnu Das (World Bank and Centre for Policy Research)WithAlaka Holla (World Bank)Karthik Muralidharan (UCSD)

The ProblemStrong theoretical reasons why health care should be public

U(government) ≠ U(Consumer): Pendergast (2003) Patient satisfaction among narcotic addicted patients not a good

measure of how good the doctor is Private sector aggregator of customer feedback

Medical care arguably a credence good You don’t know what you need, but observe utility from what you

get Widely believed to produce inefficiencies in the market

Darby and Karni (1973): Over-treatment Wolinsky (1993): You can’t observe what you bought; treat “low”, charge

“high” Gruber and Owens (1996): Caesarian sections Balfoutas and others (forthcoming): Greek taxi drivers (over provision and

over charging)

And yet…

80% of first-contacts (primary care) in India in private sector New nationwide study: 77 percent of private providers in rural areas do not have

medical training Contrast: All public providers are (supposedly) trained, the majority with an MBBS

77% of providers have no degree, 18% have some other degree (BAMS, BIMS, BUMS, BHMS), and only 4% have an MBBS degree (roughly equivalent to MD in the U.S.). Average village has 3.36 providers with no degree, 0.80 providers with some degree, and 0.18 providers with an MBBS degree

And yet…

Not (just) because there aren’t enough public sector providers

Public share increases from 20% to 35% in villages where there is a public doctor but households still visit private providers in 65% of primary care cases.

Why? What people demand from health care providers very

different from what the public sector provides Hypothesis 1: Decreases quality, increases costs

Example: Demand for injections/steroids leads to lower quality for higher cost

Peer and Administrative accountability from experts in regulated (and in low-income settings) health care beats customer accountability through the market

Hypothesis 2: Increases cost, but at increased quality Example: Poor governance in public sector (Chaudhury and

Hammer 2004, Chaudhury et. al. 2006, Das and Hammer 2007) Low effort arising from poor administrative accountability is

hard to quantify But is potential one large source of losses: `Quiet Corruption’

In health care: Arrow (1963)

This paper Use audit studies with patient and provider fixed-

effects to assess quality in public and private sector 22 people recruited from the local community and

extensively trained visits multiple providers presenting the same set of symptoms. Providers do not know that this is not a real patient

Show effect of practicing in private sector on Adherence to medically required checklists Under-treatment Over-treatment

Assess whether there is a price-quality relationship in private sector

Three literatures it relates to Customer Accountability in private sector versus

administrative accountability in public sector What’s out there and what we don’t know

New empirical and theoretical literature on credence goods

Dulleck and Kerschbamer (2006, 2011, 2012) Schneider (2012), Balfoutas(forthcoming) Bonroy and others (2012)

Audit studies in labor markets and services Primary around issues of discrimination

What are we adding

This paper: What Deploy “standardized patients” (audit study)

People recruited from local communities and extensively trained to present with the same symptoms to multiple providers

Largest such study to date (1105 interactions) 2 Related studies

Compare market care to provider in public clinic (64% not doctors) Compare same doctor in public and private practice

Note: unlike audit studies of car buying or home rentals, we always observe a completed sale Problems arising from potentially off-equilibrium behavior may still

remain We try out various strategies to interpret these results in the light

of known issues with audits

Overview of Results Significant evidence of over-treatment and under-treatment

relative to medical protocols Conditions are not diagnosed or treated appropriately Many medicines are not required

BUT Public clinics provide similar care to private clinics

Effects vary depending on measure of quality used Joint effect of public sector with provider characteristics

72% private sector providers have no medical training The same provider in his/her private clinic provides better care

than in his/her public clinic across all quality measures Customer accountability rewards better quality with higher

prices No link between provider wages in public sector and measures

of quality

Remainder of talk Where we worked (and what does it look

like) What we did What we found Ruling out (some) interpretations of the

data Worry in particular about off-equilibrium

behavior

This paper: Where?

All districts divided into 5 Socio-Cultural Regions (SCRs); one district from each SCR

20 randomly chosen villages from each district Representative sample of all types of providers in 3 districts of Madhya Pradesh

(and public providers in 2 more); majority has no medical training Additional sample from (urban) Delhi

MP Study: The sample1 • In each sampled village, surveyors complete Participatory Resource Assessments (PRAs) in at least 3 different geographical locations and ask for a list of all providers they visit for primary illnesses

2 • A unique list is compiled and a Master Code File (MCF) is filled out. A short survey is administered with each provider listed in the MCF

3 • Then a household census is completed in which members are asked about all illness in the last one month and names and locations of providers they went to

4 • If more than 5% of households report visiting a provider in a location (village/town) outside the village, that village/town is now considered a part of the health-market for the village. These are referred as “clusters”, generally on the main highway near the village

5 • Once all clusters are identified, surveyors visit each cluster and conduct PRAs in the same manner. All providers practicing in the clusters are added to the MCF and a survey is implemented

Rural India: MP 100 villages in MP, randomly selected in 5 districts—

located >1000 health care providers Snapshots of the two remotest districts

Standardized patients sample

Sample restrictions Ruled out 2 remote districts entirely for

private market Ruled out remote locations in other 3

districts Sampled

All MBBS private providers All public clinics in all districts

But no more than 2 doctors per clinic All private clinics of public doctors in all

districts Add in untrained till we have 6 providers per

sampled village

Basic Sample DescriptionTotal Inside village Outside village

(1) (2) (3)Panel A: Number of providers

Total 11.05 3.06 7.99(1.25) (0.37) (1.29)

Public MBBS 0.50 0.05 0.45(0.11) (0.02) (0.10)

Public alternative qualification 0.22 0.07 0.15(0.05) (0.03) (0.04)

Public paramedical 1.58 1.13 0.45(0.19) (0.15) (0.13)

Public unqualified 1.70 0.67 1.03(0.17) (0.10) (0.15)

Private MBBS 0.42 0.00 0.42(0.16) 0.00 (0.16)

Private alternative qualification 1.92 0.23 1.69(0.36) (0.07) (0.37)

Private unqualified 5.40 1.81 3.59(0.60) (0.22) (0.61)

(contd)

Table 1: Health Market Attributes

Total Inside village Outside village(1) (2) (3)

Panel B: Composition of demandPopulation (2001 Census of India) 3885.00 1353.74 2531.26

(385.46) (103.56) (378.58)

0.46(0.00)

0.65 0.35(0.00) (0.00)

1.66 0.40 3.92(0.02) (0.01) (0.03)0.91(0.00)

0.79(0.01)0.03(0.00)0.76(0.00)

Number of villages 100Number of households 23306Number of reported household-visits 18632

Table 1: Health Market Attributes

Probability household visited provider in last 30 days

Probability visited provider was inside/ outside village

Distance traveled to visited provider (km)

Probability visit was to private sector

Probability visit was to private sector in villages with at least 1 public doctor

Probability visit was to MBBS doctor

Probability visit was to unqualified doctor

Standardized patients SPs

22 SPs recruited from the local community Important so that their appearance and manner

conform closely to providers’ expectations Thoroughly trained to make plausible

excuses to avoid invasive exams “palm” medicines if required

150+ hours of training First tried in Delhi pilot

No adverse events; <1% detection rate

Standardized patients Three standardized cases

Unstable Agina: “Doctor, this morning I had a pain in my chest” – Ramlal, Male, 45 years old

Proxy Dysentery: “Doctor, my 2 year old child has been suffering from diarrhea for 2 days” – Shankarlal, Male, 25 years old

Asthma: “Doctor, last night I had a lot of difficulty in breathing” – Rajesh (Male) or Radha (Female), 25 years old

Cases chosen such that Relevance to the Indian context

Increasing incidence of cardiovascular and respiratory illness in India Diarrheal diseases kill approximately 200,000 children per year

(Black et al. 2008) No invasive treatment required

Important to minimize any potential harm to SPs

Standardized patients What is measured

Quality of care through adherence to required and essential checklist of questions and examinations that the provider should complete for each patient Why this may be preferable

Treatment: correctness, incorrectness, use of antibiotics and steroids for cases where they are not required

Diagnosis: whether given, whether correct Time spent, total questions asked, total

examinations completed

Relation between quality measures

Figure 1a Figure 1b

0.1

.2.3

Den

sity

of T

ime

Spe

nt

010

2030

4050

Adh

eren

ce to

Che

cklis

t (P

erce

nt)

0 10 20 30Total Time Spent (Mins)

Consultation Time and Adherence Density of Time Spent

Consultation Time and Adherence to Checklist

0.0

1.0

2.0

3D

ensi

ty o

f Adh

eren

ce

.1.2

.3.4

.5C

orre

ct T

reat

men

t Giv

en

0 20 40 60 80Adherence to Checklist (Percent)

Checklist Adherence and Treatment Adherence Density

Adherence to Checklist and Treatment

1. Doctors under-treat because they figured out that these were not “real patients”. But then, we should see that “correct treatment” is less likely for doctors who spend more time and complete more of the checklist, since they would be more likely to figure out that the patient is not “real”. We find exactly the opposite

2. Little evidence of signaling through medically irrelevant costly effort: more effort leads to better treatment through 90 percent of the distribution

Basic Sample Description

Public Private p-value of (1)-(2) Public Private p-value of

(4)-(5) Public Private p-value of (7)-(8)

(1) (2) (3) (4) (5) (6) (7) (8) (9)Panel A: Provider characteristics

Age of Provider 47.03 43.33 0.059 44.53 45.43 0.523More than 12 years of basic education 0.59 0.53 0.507 0.62 0.69 0.277Has MBBS degree 0.26 0.08 0.001 1.00 1.00No medical training 0.62 0.67 0.473 0.00 0.00Has multiple practices 1.18 1.07 0.022 1.83 2.16 0.000

Panel B: Practice characteristicsTenure in years at current location 15.49 13.45 0.260 7.09 8.08 0.318Dispense medicine 1.00 0.82 0.003 0.57 0.37 0.004Consultation fee (Rs.) 0.95 12.92 0.000 28.78 41.34 0.002Average number of patients per day 28.33 16.25 0.000 24.40 17.05 0.027Electricity 0.95 0.95 0.920 0.98 1.00 0.166 0.96 1.00 0.087Stethoscope 0.97 0.95 0.477 0.99 1.00 0.427 0.99 1.00 0.328Blood pressure cuff 0.84 0.76 0.274 0.99 1.00 0.427 0.99 1.00 0.328Thermometer 0.95 0.92 0.584 0.94 0.98 0.212 0.95 0.98 0.435Weighing Scale 0.87 0.51 0.000 0.94 0.84 0.011 0.93 0.84 0.074Handwash facility 0.89 0.83 0.315 0.86 0.83 0.537 0.83 0.83 0.915

Number of providers 39 206 143 94 94 94Notes:

41.3417.05

0.691.000.00

8.080.37

Table 1. Characteristics of Providers and Practices

Audit 1 Audit 2 Dual

45.43

Results Checklist adherence IRT Scores Treatment Diagnosis Prices

Checklist adherence

Checklist adherence

Checklist adherence

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Mean 22.38 22.32 21.74 21.81 23.29 23.19 23.54 23.03 23.10 23.54SD 16.82 16.39 16.49 16.26 17.26 16.61 17.04 17.61 17.13 17.49Is a public provider -7.932*** -8.302*** -8.046*** -7.749*** -7.226*** -7.808*** -7.483*** -7.390*** -7.726*** -7.280***

(1.637) (1.688) (2.655) (2.664) (1.973) (2.432) (2.434) (1.949) (2.419) (2.384)Has MBBS 2.368 2.173

(2.706) (2.700)Has some qualification 2.205 2.144

(1.562) (1.567)Age of provider -0.023 -0.004 -0.390*** -0.301***

(0.057) (0.056) (0.138) (0.092)Gender of provider (1=Male) -1.097 -0.708 4.314

(3.846) (3.911) (4.160)PCA of clinic infrastructure 1.101* 1.238** 0.046 -0.447 -0.620 -0.452

(0.583) (0.578) (1.968) (1.786) (1.788) (1.795)Number of patients waiting -0.196 -0.112 -0.261 -0.333 -0.290 -0.320

(0.321) (0.440) (0.516) (0.481) (0.776) (0.713)Case II (Dysentery) 19.881*** 1.167 -1.155 -1.370 58.333*** -0.596 0.445 61.149*** 0.488 2.751

(2.528) (4.725) (4.655) (4.657) (4.519) (3.897) (3.687) (5.270) (4.531) (4.412)Case III (Asthma) 24.476*** 6.193 3.250 3.666 25.138*** -31.835*** -31.739*** 26.613*** -32.176*** -31.452***

(2.280) (4.829) (4.869) (4.789) (2.638) (2.415) (2.360) (3.272) (3.106) (2.997)Constant 26.782*** 27.064*** 24.724*** 25.076*** 32.769*** 48.043*** 34.973*** 30.319*** 48.658*** 33.866***

(4.226) (4.841) (3.951) (4.898) (2.725) (4.823) (3.893) (3.602) (4.439) (4.574)Market fixed effects Yes Yes Yes Yes Yes Yes Yes YesProvider fixed effects Yes YesF-stat (All SP FE = 0) 65.3 11.6 12.1 11.8 81.8 38.4 40.7 80.0 33.6 38.0R2 0.334 0.321 0.245 0.253 0.479 0.465 0.505 0.476 0.477 0.480Number of observations 1,129 1,013 668 640 461 373 414 331 272 292Note: All regressions include SP fixed effects

Dependent Variable: Percentage of Checklist Items

Full Sample Audit 1 Audit 2 Dual

(1) (2) (3) (4) (5) (6)

Mean 24.690 24.687 19.033 18.762 23.796 23.911SD (19.881) (19.229) (15.531) (15.017) (14.724) (14.528)Is a public provider -10.344* -12.717* -5.406** -4.486 -8.222*** -9.335***

(6.021) (7.545) (2.484) (2.877) (2.433) (2.896)Has MBBS 3.687 -1.771 4.481

(7.363) (4.382) (3.438)Has some qualification 1.283 3.749 -0.066

(2.912) (3.115) (2.154)Age of provider 0.077 -0.166 0.026

(0.194) (0.133) (0.133)Gender of provider (1=Male) -0.000 -3.168 2.755

(6.419) (6.788) (7.590)PCA of clinic infrastructure 2.997** -0.785 1.945*

(1.416) (0.885) (1.058)Number of patients waiting 0.021 -0.178 -0.787

(0.666) (0.756) (0.632)Constant 28.010*** 22.539*** 28.807*** 38.857*** 60.972*** 23.841**

(6.491) (8.652) (1.999) (5.853) (2.558) (9.488)F-stat (All SP FE = 0) 3.59 3.34 12.51 11.25 32.65 3.89R2 0.526 0.532 0.513 0.523 0.535 0.530Number of observations 327 294 398 358 404 361Notes: 1) All regressions include market and SP fixed effects2) *** Significant at 1%, ** Significant at 5%, * Significant at 10%

Table 3. Public-private differences in the treatment of Standardized Patients

Unstable Angina Dysentery Asthma

Dependent Variable: Percentage of Checklist Items

IRT Score

Full Sample Audit 1 Audit 2 Dual(1) (2) (3) (4)

Is a public provider -0.666*** -0.490 -0.768*** -0.791***(0.176) (0.310) (0.261) (0.234)

Has MBBS 0.289 0.246(0.282) (0.265)

Has some qualification 0.172 0.187(0.152) (0.139)

Age of provider 0.003 0.004 -0.044 -0.051**(0.009) (0.008) (0.030) (0.031)

Gender of provider (1=Male) -0.002 0.037 0.732 -1.181***(0.333) (0.320) (0.619) (0.373)

Constant -0.722 -1.792 0.359 2.531***(0.534) (0.560) (0.770) (1.020)

Market fixed effects Yes Yes Yes YesNumber of providers 390 223 167 126Notes:1) Robust standard errors clustered at the market level are in parenthesis2) *** Significant at 1%, ** Significant at 5%, * Significant at 10%

Dependent Variable: IRT Score (Plausible Values)

Table 2. Public-private differences in the treatment of Standardized Patients

Unstable Angina (N=327)Correct treatment 0.39 0.44 0.35 0.37 0.56Aspirin 0.06 0.03 0.04 0.03 0.20Anti-platelet agents 0.01 0.03 0.01 0.00 0.02Referred 0.25 0.28 0.24 0.24 0.27ECG 0.26 0.23 0.23 0.30 0.34ECG & Referred 0.13 0.10 0.12 0.13 0.15Antibiotic 0.21 0.13 0.17 0.33 0.27Unnecessary or harmful treatment 0.61 0.51 0.53 0.75 0.80

Dysentery (N=398)Correct treatment 0.18 0.10 0.13 0.32 0.20ORS 0.18 0.10 0.13 0.32 0.20Asked to see child 0.24 0.32 0.14 0.25 0.43Antibiotic 0.63 0.46 0.61 0.76 0.59Unnecessary or harmful treatment 0.54 0.56 0.45 0.63 0.60

Asthma (N=404)Correct treatment 0.54 0.37 0.50 0.57 0.68Bronchodilators 0.44 0.32 0.36 0.51 0.59Theophylline 0.24 0.12 0.22 0.27 0.29Oral Corticosteroids 0.25 0.15 0.31 0.17 0.27Antibiotic 0.48 0.41 0.40 0.63 0.52Unnecessary or harmful treatment 0.79 0.76 0.70 0.93 0.86

Table 1. Treatment of Standardized Patients by Case

Public Private Public Private

Audit 1 Audit 2Full Sample

(1) (2) (3) (4) (5) (6) (7) (8)

Mean 0.39 0.39 0.39 0.34 0.45 0.45 0.46 0.46SD (0.49) (0.49) (0.49) (0.47) (0.50) (0.50) (0.50) (0.50)Is a public provider -0.029 -0.026 -0.027 0.034 -0.052 -0.071 -0.051 -0.069

(0.030) (0.042) (0.045) (0.064) (0.042) (0.057) (0.048) (0.053)Has MBBS 0.178** 0.178**

(0.073) (0.070)Has some qualification 0.075 0.075

(0.052) (0.050)Number of patients waiting -0.015** -0.027*** 0.000 -0.013

(0.007) (0.008) (0.011) (0.014)Age of provider -0.002 -0.002 -0.002

(0.002) (0.002) (0.011)Gender of provider (1=Male) 0.041 0.071 -0.002

(0.102) (0.100) (0.477)Case II (Dysentery) 0.151** -0.027 -0.347*** -0.092 -0.189*** 0.585* -0.203*** -0.197**

(0.070) (0.189) (0.039) (0.115) (0.062) (0.311) (0.077) (0.077)Case III (Asthma) 0.585*** 0.399** 0.096* 0.379*** 0.180*** 0.344 0.152* 0.154*

(0.056) (0.182) (0.053) (0.128) (0.069) (0.222) (0.087) (0.079)Constant 0.364*** 0.306* 0.487*** 0.289* 0.481*** 0.666 0.505*** 0.528***

(0.082) (0.158) (0.034) (0.153) (0.059) (0.428) (0.070) (0.076)District fixed effects Yes Yes Yes YesMarket fixed effects Yes Yes YesProvider fixed Effects YesR2 0.166 0.297 0.209 0.274 0.126 0.362 0.115 0.312Number of observations 1,087 1,011 608 605 406 406 330 330Notes: 1) All regressions include SP fixed effects1) Standard errors clustered at the market level in parenthesis

Dependent variable: Correct Treatment

Table2A. Public-Private Differences in Treatment

Full Sample Audit 1 Audit 2 Dual

Coefficient on Public Reported(1) (2) (3) (4) (5) (6) (7) (8)

Panel A: Unstable AnginaCorrect Treatment -0.021 0.195 0.155 0.232* -0.195* -0.258**

(0.072) (0.158) (0.101) (0.133) (0.107) (0.111)Gave/ prescribed Aspirin -0.101*** -0.033 -0.017 -0.034 -0.167** -0.202**

(0.038) (0.070) (0.032) (0.060) (0.076) (0.085)Referred to another provider 0.022 0.071 0.043 0.104 -0.006 -0.109

(0.064) (0.138) (0.085) (0.115) (0.097) (0.105)Asked/ recommended EKG -0.015 -0.024 0.048 0.008 -0.075 -0.079

(0.068) (0.146) (0.093) (0.122) (0.108) (0.120)Both refer and EKG -0.011 -0.078 -0.022 -0.050 -0.009 -0.078

(0.047) (0.108) (0.059) (0.090) (0.078) (0.084)Panel B: Dysentery

Correct Treatment 0.053 0.079 -0.054 -0.014 0.135** 0.125 0.125 0.102(0.047) (0.054) (0.061) (0.075) (0.066) (0.086) (0.078) (0.068)

ORS 0.053 0.079 -0.054 -0.014 0.135** 0.125 0.125 0.102(0.047) (0.054) (0.061) (0.075) (0.066) (0.086) (0.078) (0.068)

Asked to see child -0.007 0.011 0.197** 0.297*** -0.153** -0.203* -0.205** -0.182**(0.062) (0.070) (0.092) (0.096) (0.076) (0.108) (0.097) (0.089)

Panel C: AsthmaCorrect Treatment -0.146*** -0.116 -0.144* -0.127 -0.140* -0.139 -0.139 -0.147*

(0.056) (0.072) (0.086) (0.130) (0.075) (0.098) (0.088) (0.077)Gave/ prescribed Bronchodilators -0.082 -0.070 -0.036 -0.008 -0.118 -0.135 -0.135 -0.111

(0.054) (0.070) (0.086) (0.116) (0.076) (0.099) (0.087) (0.081)Gave/ prescribed Steroids -0.106** -0.073 -0.117 -0.100 -0.088 -0.062 -0.064 -0.095

(0.052) (0.054) (0.101) (0.097) (0.061) (0.073) (0.065) (0.061)District fixed effects Yes Yes Yes YesMarket fixed effects Yes YesProvider Fixed Effects Yes YesOther controlsNotes:1) Standard errors clustered at the market level in parenthesis

SP fixed effects, age, gender, qualification, number of patients waiting

Table2B. Public-Private Differences in Correct Treatment

Full Sample Audit 1 Audit 2 Dual

What exactly is happening with treatment? In audit 1, the two groups behave similarly, but there are some differences

across cases MI identical in both Dysentery public sector providers 20-30% more likely to ask to see

child, no difference in ORS Asthma public sector providers 12-14% less likely to give correct

treatment (not statistically significant) Across all cases, public 13% less likely to give antibiotics

Dual sample, with and without provider fixed effects MI: Equal likelihood of EKG/Referral but private more likely to give

Aspirin Dysentery: Public 10-12% (not significant) more likely to give ORS,

private 18-20% more likely to ask to see child Asthma: Public 13-15% less likely to get it correct

(1) (2) (3) (4) (5) (6) (7) (8)

Mean 0.65 0.64 0.57 0.57 0.76 0.76 0.75 0.75SD (0.48) (0.48) (0.50) (0.50) (0.43) (0.43) (0.43) (0.43)Is a public provider 0.052 0.010 0.011 -0.017 0.020 0.025 0.027 0.042

(0.035) (0.040) (0.068) (0.066) (0.037) (0.051) (0.042) (0.046)Has MBBS 0.229*** 0.236***

(0.074) (0.073)Has some qualification -0.010 -0.011

(0.055) (0.053)Number of patients waiting 0.005 0.005 0.004 0.018

(0.006) (0.009) (0.009) (0.014)Age of provider -0.001 -0.000 -0.011

(0.002) (0.002) (0.011)Gender of provider (1=Male) 0.076 0.086 0.218

(0.095) (0.096) (0.468)Case II (Dysentery) -0.083** 0.055 -0.051 0.004 -0.152*** -0.577 -0.131* -0.124*

(0.041) (0.261) (0.055) (0.107) (0.059) (0.443) (0.077) (0.067)Case III (Asthma) 0.171*** 0.258 0.181*** 0.198* 0.128*** 0.305 0.143** 0.153**

(0.027) (0.255) (0.035) (0.113) (0.044) (0.279) (0.058) (0.061)Constant 0.597*** 0.322** 0.523*** 0.309** 0.756*** 0.994** 0.732*** 0.695***

(0.028) (0.149) (0.034) (0.147) (0.046) (0.416) (0.060) (0.064)District fixed effects Yes Yes Yes YesMarket fixed effects Yes Yes YesProvider fixed Effects YesR2 0.095 0.252 0.059 0.170 0.091 0.358 0.315 0.300Number of observations 1,129 1,052 668 644 461 408 331 331Notes: 1) All regressions include SP fixed effects1) Standard errors clustered at the market level in parenthesis

Dependent variable: Unnecessary or harmful treatment

Table3A. Public-Private Differences in Incorrect Treatment

Full Sample Audit 1 Audit 2 Dual

Coefficient on Public Reported(1) (2) (3) (4) (5) (6) (7) (8)

Panel A: Unstable AnginaUnnecessary or harmful treatment -0.083 -0.168 -0.134 -0.176 -0.070 -0.047

(0.069) (0.148) (0.119) (0.128) (0.079) (0.094)Gave/ prescribed antibiotics 0.012 -0.071 -0.040 -0.075 0.032 0.065

(0.054) (0.104) (0.055) (0.089) (0.094) (0.108)Gave/ prescribed steroids -0.021 -0.028 -0.029 -0.031 -0.014 -0.015

(0.022) (0.044) (0.039) (0.038) (0.037) (0.042)Panel B: Dysentery

Unnecessary or harmful treatment 0.025 0.076 0.028 0.193* 0.005 0.000 0.027 0.016(0.061) (0.070) (0.110) (0.110) (0.075) (0.086) (0.090) (0.078)

Gave/ prescribed antibiotics 0.029 -0.005 -0.181* -0.238** 0.164** 0.149 0.140 0.123(0.062) (0.073) (0.099) (0.119) (0.073) (0.097) (0.087) (0.075)

Panel C: AsthmaUnnecessary or harmful treatment 0.065 0.026 0.021 -0.079 0.078* 0.081 0.080 0.095*

(0.045) (0.053) (0.098) (0.092) (0.047) (0.068) (0.061) (0.056)Gave/ prescribed antibiotics 0.068 0.008 -0.011 -0.132 0.115 0.110 0.109 0.117

(0.062) (0.074) (0.110) (0.124) (0.080) (0.105) (0.094) (0.081)District fixed effects Yes Yes Yes YesMarket fixed effects Yes YesProvider Fixed Effects Yes YesOther controlsNotes:1) Standard errors clustered at the market level in parenthesis

SP fixed effects, age, gender, qualifications, number of patients waiting

Table3B. Public-Private Differences in Incorrect Treatment

Full Sample Audit 1 Audit 2 Dual

Diagnosis Problem: 67% interactions there is no diagnosis Noted in pilot Final survey: randomized SSPs into 2 groups

1 group turns around as they are leaving and ask the provider “Doctor, what is wrong with me?”

Increases rate of diagnosis provision by 20-25 p.p. in all groups

First look at likelihood of providing diagnosis Second, use randomization as instrument in

selection model with binary dependent variables to deduce correct diagnosis rates

Yes No Yes No Yes No Yes No

Gave Diagnosis 0.260 0.353 0.190 0.356 0.294 0.344 0.270 0.381373 756 121 547 252 209 163 168

Diagnosis Correct 0.542 0.539 0.652 0.513 0.507 0.611 0.270 0.38196 267 23 195 73 72 163 168

Gave Diagnosis 0.461 0.200 0.470 0.208 0.449 0.188 0.446 0.202529 600 302 366 227 234 168 163

Diagnosis Correct 0.523 0.575 0.528 0.526 0.515 0.659 0.413 0.579243 120 142 76 101 44 92 57

Notes:2) For each sample, group means and number of observations

Panel A: By Public (Yes = Public)

Panel B: By SP Empowerment (Yes = Empowered)

Summary of Diagnosis by Public and by SP Empowered, by Sample

Full Sample Audit 1 Audit 2 Dual

(1) (2) (3) (4) (5) (6) (7) (8) (9)

Mean 0.32 0.33 0.33 0.33 0.32 0.32 0.33 0.33 0.33SD (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.47)Is a public provider -0.093*** -0.120*** -0.142*** -0.150*** -0.064 -0.095* -0.112** -0.107** -0.095*

(0.029) (0.036) (0.031) (0.044) (0.047) (0.055) (0.053) (0.051) (0.053)SP was empowered 0.243*** 0.251*** 0.226*** 0.230*** 0.228***

(0.033) (0.039) (0.055) (0.057) (0.061)Case II (Dysentery) -0.283*** -0.273*** -0.305*** -0.319*** -0.340***

(0.037) (0.047) (0.057) (0.066) (0.073)Case III (Asthma) -0.077** -0.107** -0.054 -0.045 -0.072

(0.038) (0.043) (0.069) (0.074) (0.084)Number of patients waiting 0.004 0.004 0.003 -0.004 0.001

(0.006) (0.007) (0.012) (0.014) (0.019)Has MBBS -0.081 -0.075

(0.087) (0.083)Has some qualification -0.002 -0.002

(0.055) (0.053)Age of provider -0.001 -0.001 -0.002

(0.002) (0.002) (0.003)Gender of provider (1=Male) -0.073 -0.052 -0.042

(0.093) (0.092) (0.076)Constant 0.353*** 0.536*** 0.352*** 0.458*** 0.351*** 0.390*** 0.381*** 0.532*** 0.419***

(0.015) (0.106) (0.017) (0.099) (0.030) (0.063) (0.034) (0.148) (0.082)District fixed effects Yes Yes Yes YesMarket fixed effects Yes Yes Yes YesProvider fixed effects YesR2 0.018 0.268 0.025 0.211 0.014 0.381 0.027 0.188 0.332Number of observations 1,129 1,052 668 644 461 461 331 302 331Notes:

Public-private differences in diagnosis

Dependent Variable: Diagnosis was given

Full Sample Audit 1 Audit 2 Dual

(1) (2) (3) (4) (5) (6)

Mean 0.46 0.46 0.17 0.17 0.37 0.38SD (0.50) (0.50) (0.37) (0.38) (0.48) (0.48)Is a public provider -0.064 -0.100 -0.086** -0.097* -0.124** -0.126

(0.058) (0.120) (0.037) (0.053) (0.051) (0.086)SP was empowered 0.104 0.228*** 0.311***

(0.096) (0.049) (0.072)Number of patients waiting -0.002 0.005 0.016

(0.010) (0.014) (0.029)Has MBBS -0.092 -0.007 -0.123

(0.177) (0.101) (0.165)Has some qualification 0.053 0.032 -0.078

(0.127) (0.081) (0.080)Age of provider -0.001 -0.003 -0.001

(0.005) (0.003) (0.004)Gender of provider (1=Male) 0.108 -0.071 -0.227

(0.202) (0.120) (0.227)Constant 0.476*** 0.415 0.194*** 0.280* 0.410*** 0.570**

(0.034) (0.314) (0.024) (0.156) (0.027) (0.234)District fixed effects Yes Yes YesMarket fixed effects Yes Yes YesR2 0.016 0.503 0.046 0.471 0.026 0.457Number of observations 327 304 398 372 404 376Notes:1) Standard errors clustered at the market level in parenthesis

Public-private differences in diagnosis, by case

Dependent Variable: Diagnosis was given

AsthmaDysenteryUnstable Angina

(1) (2) (3) (4) (5) (6) (7) (8)

Is a public provider 0.014 0.009 0.154 0.184 -0.075 -0.072 -0.151* -0.119(0.060) (0.067) (0.111) (0.116) (0.076) (0.073) (0.090) (0.089)

Case II (Dysentery) 0.058 0.080 0.139 0.138 -0.064 -0.012 -0.056 -0.038(0.071) (0.073) (0.092) (0.092) (0.108) (0.101) (0.116) (0.109)

Case III (Asthma) 0.137** 0.137** 0.050 0.053 0.210** 0.210* 0.170 0.136(0.056) (0.059) (0.075) (0.074) (0.097) (0.112) (0.112) (0.113)

Number of patients waiting -0.000 -0.001 -0.004 -0.022(0.012) (0.014) (0.017) (0.021)

Has MBBS 0.138* 0.303**(0.076) (0.145)

Has some qualification 0.049 0.043(0.079) (0.080)

Age of provider -0.000 0.000 -0.004 -0.002(0.003) (0.003) (0.004) (0.004)

Gender of provider (1=Male) -0.064 -0.103 0.019 -0.002(0.100) (0.158) (0.109) (0.120)

District fixed effects Yes Yes Yes Yes Yes Yes Yes YesNumber of observations 1,127 1,050 668 644 459 406 330 301Notes:1) Standard errors calculated using the Delta method2) All regressions include district fixed effects

Difference in Diagnosis by Public, by Sample

Y-variable: Diagnosis Given was Correct(marginal effects (dy/ dx) reported)

DualAudit 2Audit 1Full

Prices in the Private Sector Huge variation

within providers

True for all MBBS, other qualified, and unqualified providers

Each vertical line represents a box-plot of prices charged by a provider to real patients. Providers are sorted on the x-axis by quality (measured by number of questions asked and examinations conducted)

Prices, Checklists and Treatment Greater compliance with the checklist is

always rewarded in higher prices Correct treatment leads to higher

prices , but vanishes once we control for checklist adherence Stronger premium among MBBS providers

Works across all cases, weaker for asthma

Prices and Checklist Adherence

0.0

1.0

2.0

3D

ensi

ty o

f Adh

eren

ce

020

4060

80Fe

es C

harg

ed

0 20 40 60 80Checklist Adherence (%)

Fees and Checklist Adherence: All Private

Fees and Checklist Adherence: Only DualDensity of Adherence

Prices and Checklist Adherence

(1) (2) (3) (4) (5) (6) (7)

Mean 28.51 28.46 28.59 28.44 28.59 28.44 28.51SD 26.44 26.45 26.59 26.22 26.59 26.22 26.44Fixed effects District District District District Market Market ProviderPercentage of checklist items 0.434*** 0.419*** 0.455*** 0.340*** 0.450*** 0.346*** 0.297***

(0.072) (0.056) (0.069) (0.082) (0.075) (0.090) (0.078)Dispensed medicine 20.093*** 20.946*** 20.364*** 21.154*** 20.703*** 24.301***

(1.691) (1.707) (1.740) (1.925) (2.005) (2.985)Has MBBS 22.127*** 22.144*** 22.368*** 25.688** 25.888**

(4.061) (3.943) (3.814) (10.453) (10.272)Has some qualification 7.614*** 7.488*** 7.362*** 3.117 3.428

(2.334) (2.465) (2.567) (2.849) (2.865)Referred in Unstable Angina -8.257*** -8.179*** -8.716*** -9.972*** -10.000***

(2.060) (2.003) (1.958) (2.732) (2.797)Asked for child in Dysentery -2.446 -0.296 -0.534 1.132 1.506

(2.085) (2.082) (1.925) (2.044) (2.200)Time spent with SP (minutes) 1.108*** 0.984*** 1.322***

(0.284) (0.338) (0.383)R2 0.122 0.292 0.309 0.339 0.440 0.462 0.626Number of observations 755 742 724 705 724 705 755

Dependent Variable: Prices in Rs.

Table 4. Prices Charged and Checklist Items (Full, Private Only)

(1) (2) (3) (4) (5) (6) (7) (8)

Mean 28.07 28.07 28.21 28.07 28.21 28.21 28.21 28.07SD (26.35) (26.40) (26.55) (26.35) (26.55) (26.55) (26.55) (26.35)Fixed Effects District District Market Provider Market Market Market ProviderCorrect treatment 6.903*** 8.119*** 7.210*** 8.135*** 2.801 4.538** 1.930 2.890

(1.913) (1.988) (2.072) (2.440) (2.458) (2.246) (2.510) (2.689)Dispensed medicine 20.299*** 21.357*** 24.189*** 21.826*** 21.063*** 21.508*** 24.866***

(1.759) (1.999) (3.062) (1.940) (2.048) (2.020) (3.159)Has MBBS 23.310*** 25.059** 23.703** 25.311** 24.196**

(4.427) (10.403) (10.257) (11.500) (10.971)Has some qualification 7.818*** 1.983 1.876 2.748 2.443

(2.321) (2.538) (2.639) (2.513) (2.584)Referred in Unstable Angina -8.609*** -9.015*** -9.525*** -7.774** -8.526***

(2.270) (3.054) (2.877) (3.120) (3.035)Asked for child in Dysentery -3.048 -1.701 -0.176 -1.090 -0.097

(2.770) (2.584) (2.071) (2.124) (2.152)Percentage of checklist items 0.385*** 0.296*** 0.243***

(0.081) (0.092) (0.081)Time spent with SP (minutes) 1.633*** 1.159*** 1.314***

(0.333) (0.357) (0.432)R2 0.062 0.248 0.409 0.594 0.447 0.442 0.461 0.631Number of observations 721 709 691 721 691 691 691 721

continued on next slide

Prices Charged and Correct Treatment (Full Sample, Private Only)

Dependent Variable: Prices in Rs.

continued from previous slide(1) (2) (3) (4) (5) (6) (7) (8)

Average steroid (0-1) -2.726 3.850 -0.345 4.012(7.595) (7.695) (7.576) (7.617)

Average antibiotic (0-1) 9.486*** 9.167*** 9.012*** 8.905***(3.465) (3.476) (3.287) (3.402)

Number of patients waiting 0.338 -0.029 0.314 0.450 0.399 -0.034(0.419) (0.469) (0.444) (0.433) (0.439) (0.416)

Age of provider -0.130 -0.108 -0.115 -0.103(0.096) (0.097) (0.099) (0.099)

Gender of provider (1=Male) -3.220 -3.628 -5.548 -5.185(4.955) (4.988) (5.048) (5.162)

Constant 25.425*** 9.711*** 14.777** 14.753*** 5.927 10.765 5.135 5.870**(1.852) (1.702) (6.965) (1.788) (7.985) (7.605) (8.324) (2.605)

District fixed effects Yes YesMarket fixed effects Yes Yes Yes YesProvider Fixed Effects Yes YesR2 0.062 0.248 0.409 0.594 0.447 0.442 0.461 0.631Number of observations 721 709 691 721 691 691 691 721Note:1) Robust standard errors clustered at the village level in parenthesis2) *** Significant at 1%, ** Significant at 5%, * Significant at 10%

Does the public sector reward quality? Public sector pay in India follows a

matrix Composed of: rank, tenure, qualifications Zero effect of checklist adherence,

treatment, likelihood of discussing diagnosis on wages

(1) (2) (3) (4) (5)

Percentage of checklist items -0.001 0.001(0.002) (0.002)

Time spent with SP (minutes) -0.033** -0.035**(0.013) (0.016)

Correct treatment -0.056 -0.040(0.044) (0.047)

Provider discussed diagnosis 0.026 0.053(0.080) (0.088)

Is a doctor 0.908*** 0.903*** 0.935*** 0.899*** 0.925***(0.168) (0.165) (0.177) (0.171) (0.172)

Is a nurse -0.104 -0.097 -0.098 -0.114 -0.085(0.170) (0.168) (0.172) (0.173) (0.167)

Age of provider 0.017*** 0.017*** 0.017*** 0.017*** 0.017***(0.006) (0.006) (0.006) (0.006) (0.006)

Gender of provider (1=Male) 0.032 0.034 0.039 0.032 0.042(0.157) (0.156) (0.154) (0.156) (0.152)

Born in same district -0.015 -0.009 -0.018 -0.016 -0.010(0.115) (0.113) (0.115) (0.114) (0.109)

Is a dual provider 0.290*** 0.266** 0.280** 0.294*** 0.256**(0.110) (0.108) (0.109) (0.112) (0.107)

Constant 8.351*** 8.407*** 8.346*** 8.331*** 8.395***(0.253) (0.251) (0.262) (0.265) (0.266)

District fixed effects Yes Yes Yes YesR2 0.453 0.459 0.442 0.453 0.449Number of observations 308 308 301 308 301Notes:1) Standard errors clustered at the market level in parenthesis2) *** Significant at 1%, ** Significant at 5%, * Significant at 10%

Dependent Variable: Log of Monthly Wages

Variation in Wages in the Public Sector By Job Position

(1) (2) (3) (4) (5)

Percentage of checklist items -0.002 -0.000(0.002) (0.002)

Time spent with SP (minutes) -0.016 -0.017(0.012) (0.014)

Correct treatment -0.047 -0.038(0.042) (0.044)

Provider discussed diagnosis 0.020 0.052(0.065) (0.073)

Has MBBS 1.283*** 1.271*** 1.308*** 1.275*** 1.303***(0.169) (0.168) (0.174) (0.167) (0.179)

Has some qualification 0.871*** 0.857*** 0.892*** 0.865*** 0.878***(0.297) (0.290) (0.299) (0.293) (0.299)

Age of provider 0.019*** 0.019*** 0.019*** 0.019*** 0.019***(0.006) (0.006) (0.006) (0.006) (0.006)

Gender of provider (1=Male) 0.130 0.130 0.122 0.131 0.121(0.106) (0.105) (0.107) (0.104) (0.105)

Born in same district 0.015 0.017 0.015 0.014 0.022(0.080) (0.080) (0.081) (0.081) (0.081)

Is a dual provider 0.145* 0.137 0.152* 0.149* 0.140(0.085) (0.086) (0.086) (0.087) (0.086)

Constant 8.022*** 8.040*** 7.993*** 7.995*** 8.020***(0.308) (0.315) (0.337) (0.329) (0.344)

District fixed effects Yes Yes Yes Yes YesR2 0.628 0.629 0.620 0.627 0.622Number of observations 308 308 301 308 301Notes:1) Standard errors clustered at the market level in parenthesis2) *** Significant at 1%, ** Significant at 5%, * Significant at 10%

Dependent Variable: Log of Monthly Wages

Variation in Wages in the Public Sector By Qualification

Quick back of the envelope We can provide a back of the envelope

measure of costs and quality in the public and private sectors This is rough—public and private sector

providers provide other services beyond primary care

Public Checklist UsageAverage doctors per facility 2.03 25th percentile 8.33%Monthly salary cost of doctors per month per facility Rs.64,199 50th percentile 15.79%Average number patients per facility per month 984 75th percentile 27.27%Average cost per patient Rs.101 99th percentile 63.18%

Private Facility (parameters from regression)Base price (constant) Rs.2.05Cost per percentage checklist item Rs.0.46MBBS premium Rs.22.14Some qualification premium Rs.7.49

Percentage Checklist Items MBBS Doctor

0% Rs.2425% Rs.3650% Rs.4775% Rs.59100% Rs.70

Cost and Checklist Completion in Public Sector

Cost and Checklist Completion in Private Sector

Rs.33Rs.44Rs.56

No Qualification

Rs.2Rs.14Rs.25Rs.37Rs.48

Per Patient Cost in the Public Sector Facility

Some Qualification

Rs.10Rs.21

Some further interpretation results Audit patients present the same symptoms and same script to multiple

doctors in different conditions. This may be off-equilibrium behavior. 3 sets of issues “Serious” cases never go to the public sector. Therefore, if they do, it is

an indication to the doctor (who is on the equilibrium path) that the patient is not serious

If the same case goes, the patient presents in a different way in the public to the private sector, accounting for lower incentives to put effort

The public-private difference for the same doctors may reflect incentive effects due to the presence of the private sector clinic

Difficult problems: in past led to differing results between audit studies and observational data Famously, Ayres and Spiegelman (1995) versus Goldberg (1996) More recently, discrimination against African American (names):

Bertrand-Mullianathan versus Fryer and Levitt What we do in addition to the audit

First, observe equilibrium path behavior with real patients and check to see if the results are the same

Second, try to assess patient sorting Third, try to rule out deliberately lower effort due to dual practice

Some further interpretation results Is it the case that they treat “real” patients this way?

Yes, we sat in their clinics for 1 day each and find identical results on things that we can measure in both (time spent, questions asked, examinations done) (link to table)

Is it the case that the “regular” patient body is very different for public/private We did exit surveys with patients from all practices. Patients

were not very different in illness and severity, but in private had more access to transport and had more mobile phones (72 vs. 64%)

When we include (means) of the regular patient populations in the audit regressions, nothing changes

It seems like people use the public clinics precisely like they use unqualified providers (link to table)

Some further interpretation results Is it the case that patients “expect”

something very different from public and private? If the patients know what they have, then it

is likely that there will be complete separation by quality and price

Cases deliberately chosen so that same symptom can reflect a minor or major condition

Some further interpretation results Is it the case that public providers were “directing” patients to

their private clinics? Or, would we expect very different care among public sector

providers if they did not have a private clinic? None of our SPs were directed to the private clinic of the public provider. Referrals lower among dual practice People already know where the private clinic is (and sometimes this is not

in the same place) Fully segmented markets Some effect of location on estimated impact in checklist and time-spent, but not

on treatment and diagnosis Further, the guys with clinics in the same location are also worse in their

private practice, suggesting that these guys are just selected worse We cannot tell what would happen where there are is no dual-practice

We note that it is not allowed, but 80 percent of providers have them The providers who have dual practice versus the 20 percent who do not, behave

identically in their public practice in treatment and have lower referral rates, but also have lower checklist completion and diagnosis rates

Some further interpretation results Is it the case more educated patients would get different

results from the public sector? What about urban areas? On all process measures, public providers in Delhi are worse But on treatment, they are better; could reflect higher competence

since we did not observe the same doctor in both practices

Time spent Percentage of checklist items IRT score Uttered

diagnosisCorrect

treatmentIncorrect treatment Antibiotics

Public -5.42*** -16.1*** -1.25*** -0.16 0.29** -0.21 -0.21(0.47) (2.23) (0.068) (0.086) (0.090) (0.13) (0.15)

MBBS 0.80 6.14*** 0.34*** -0.14 0.037 0.17 0.17(0.71) (1.17) (0.042) (0.11) (0.11) (0.14) (0.16)

Some qualification 2.24*** 4.48 0.32*** -0.014 -0.019 -0.13*** -0.068*(0.60) (2.53) (0.054) (0.11) (0.15) (0.023) (0.034)

Dysentery 0.12 6.75*** 0.31*** -0.34*** 0.13*** -0.28*** 0.025**(0.093) (0.22) (0.017) (0.0078) (0.016) (0.014) (0.0084)

Patient load during visit 0.00015 -0.0069 -0.0054 0.00058 -0.0063** 0.0020 0.0011(0.018) (0.054) (0.0047) (0.0015) (0.0025) (0.0039) (0.0023)

Conclusion Significant over and under treatment in both sectors Pure public versus private comparison for the same

provider suggests that public sector has Lower compliance with checklist Lower correct treatment and diagnosis rates Similar rates of over-treatment

Public versus private clinics is more complex Lower compliance with checklist in public sector BUT, no difference in treatment and higher correct

diagnosis rates Several potential explanations (accountability, local,

rules of thumb) Prices reward quality as measured through adherence to

checklist and treatment in private, but not in public clinics

Implications What to do with the public sector

Location subsidies? Massive investments? Better administrative accountability? Reorganizing geographical locations?

What to do with the private sector Better knowledge? What about equity—how to introduce

subsidies