Slide 1

Management field experiments Nick Bloom (Stanford and NBER) www.stanford.edu/~nbloom AOM, August 3 rd 2012

Slide 2

Management field experiments Review of current experimental literature Our projects in India and China Thoughts on running field experiments

Slide 3

The reason is running management experiments is expensive Surge in management experiments, but mainly in: (A) micro-firms (1 or 2 person) in developing countries (B) individual larger firms

Slide 4

Developing countries, micro-firm experiments Karlan & Valdivia in Peru; Bruhn, Karlan & Schoar in Mexico; Karlan & Udry in Ghana; McKenzie & Woodruff in Sri Lanka etc. Provide limited (50 hours) of basic trainings to small firms: accounting, marketing, pricing, strategy etc. Training is provided at random and data collected before & after So far finding not-much impact. I see two potential explanations -management does not matter in tiny firms, or -intervention is very poor quality

Slide 5

Single firms in developed countries, (1/2) Growing literature (surveyed in Bloom & Van Reenen, 2010, Handbook of Labour Economics) Classic examples include: Lazaer (2000, AER) on incentive pay at Safelite Glass, Shearer (2004, REStud) on tree planters and Hamilton et al (2003, JPE) on group incentives in factories

Slide 6

Single firms in developed countries (2/2) Recently Bandiera, Barankay and Rasul have an impressive set of papers. Run experiments on incentives for workers and managers, team selection, and task division on a fruit farm Introduce changes way through season (using last season as the control), finding for example Worker incentive pay increases their performance, especially absolute (rather than relative) incentives Manager incentive pay improves team selection (less favoritism) and the effort they put into monitoring workers

Slide 7

Management field experiments Review of current experimental literature Our projects in India and China India China Thoughts on running field experiments

Slide 8

Does management matter? Evidence from India Nick Bloom (Stanford) Benn Eifert (Berkeley) Aprajit Mahajan (Stanford) David McKenzie (World Bank) John Roberts (Stanford GSB) http://www.stanford.edu/~nbloom/DMM.pdf

Slide 9

Management score Source: www.worldmanagementsurvey.comwww.worldmanagementsurvey.com 2.62.833.23.4 US Japan Germany Sweden Canada Australia UK Italy France New Zealand Mexico Poland Republic of Ireland Portugal Chile Argentina Greece Brazil China India One motivation for looking at management is that country management scores are correlated with GDP

Slide 10

Source: www.worldmanagementsurvey.comwww.worldmanagementsurvey.com Management score US (N=695 firms) India (N=620 firms) Density And firm management spreads look like TFP spreads

Slide 11

But does management cause any of these TFP differences between firms and countries? Massive literature of case-studies and surveys but no consensus Syverson (2011, JEL) no potential driving factor of productivity has seen a higher ratio of speculation to empirical study.

Slide 12

We ran an experiment on large firms to investigate the impact of modern management practices on TFP Experiment on 20 plants in large multi-plant firms (average 300 employees and $7m sales) near Mumbai making cotton fabric Randomized treatment plants got 5 months of management consulting intervention, controls got 1 month Consulting was on 38 specific practices tied to factory operations, quality and inventory control Collect weekly performance data from 2008 to August 2010, and long-run size and management data from 2008 to 2011

Slide 13

Exhibit 1: Plants are large compounds, often containing several buildings.

Slide 14

Fabric weaving Exhibit 2: Plants operate continuously making cotton fabric from yarn

Slide 15

Exhibit 3: Many parts of these Indian plants were dirty and unsafe Garbage outside the plantGarbage inside a plant Chemicals without any coveringFlammable garbage in a plant

Slide 16

Exhibit 4: The plant floors were often disorganized and aisles blocked Instrument not removed after use, blocking hallway. Tools left on the floor after use Dirty and poorly maintained machines Old warp beam, chairs and a desk obstructing the plant floor

Slide 17

Yarn piled up so high and deep that access to back sacks is almost impossible Exhibit 5: The inventory rooms had months of excess yarn, often without any formal storage system or protection from damp or crushing Different types and colors of yarn lying mixed Yarn without labeling, order or damp protection A crushed yarn cone, which is unusable as it leads to irregular yarn tension

Slide 18

Intervention aimed to improve 38 core textile management practices in 6 areas for example Targeted practices in 6 areas: operations, quality, inventory, HR and sales & orders

Slide 19

Months after the diagnostic phase.2.3.4.5.6 -10-8-6-4-2024681012 Adoption of the 38 management practices rose Treatment plants Control plants Share of 38 practices adopted Non-experimental plants in treatment firms Months after the start of the diagnostic phase

Slide 20

In terms of performance looked at four outcomes we have weekly data for Quality Inventory Output Productivity (defined as: Log(VA) 0.42*log(K) 0.58*log(L)) Use weekly data from March 2008 until August 2010 (after which some firms started upgrading to Jacquard looms)

Slide 21

Poor quality meant 19% of labor went on repairs Workers spread cloth over lighted plates to spot defectsLarge room full of repair workers (the day shift) Defects lead to about 5% of cloth being scrappedDefects are repaired by hand or cut out from cloth

Slide 22

22 Previously mending was recorded only to cross- check against customers claims for rebates

Slide 23

Now mending is recorded daily in a standard format, so it can analyzed by loom, shift, design & weaver

Slide 24

The quality data is now collated and analyzed as part of the new daily production meetings Plant managers meet with heads of departments for quality, inventory, weaving, maintenance, warping etc.

Slide 25

Quality improved significantly in treatment plants Control plants Treatment plants Weeks after the start of the experiment Quality defects index (higher score=lower quality) Note: solid lines are point estimates, dashed lines are 95% confidence intervals

Slide 26

Stock is organized, labeled, and entered into the computer with details of the type, age and location. Organizing and racking inventory enables firms to substantially reduce capital stock

Slide 27

Many treated firms have also introduced basic initiatives to organize the plant floor Marking out the area around the model machine Snag tagging to identify the abnormalities

Slide 28

28 Spare parts were also organized, reducing downtime (parts can be found quickly) Nuts & bolts Tools Spare parts

Slide 29

Production data is now collected in a standardized format, for discussion in the daily meetings Before (not standardized, on loose pieces of paper) After (standardized, so easy to enter daily into a computer)

Slide 30

TFP rose in treatment plants vs controls Control plants Treatment plants Weeks after the start of the experiment Total factor productivity Note: solid lines are point estimates, dashed lines are 95% confidence intervals

Slide 31

Why do badly managed firms exist? Competition heavily restricted by trade restrictions, the difficulty of new firms entering (finance is hard to raise), and the difficulty of good current firms expanding (limited by family size) Information is limited: firms either not aware of modern practices or simply do not believe they matter (not worth the it)

Slide 32

Management field experiments Review of current experimental literature Our projects in India and China India China Thoughts on running field experiments

Slide 33

Does Working from Home Work? Evidence from a Chinese Experiment Nick Bloom (Stanford) James Liang (Ctrip & Stanford) John Roberts (Stanford) Jenny Ying (Stanford) http://www.stanford.edu/~nbloom/WFH.pdf

Slide 34

34 CTrip, a large NASDAQ listed Chinese multinational, wondered about introducing working from home CTrip, Chinas largest travel-agent (13,000 employees, and $5bn value on NASDAQ). James Liang is the co-founder, first CEO and Chairman

Slide 35

The randomization into working from home was done publicly and also shown on the firm intranet Open lottery over even/odd treatment Working at Home Working at home Working at Home

Slide 36

Experiment yielded four learnings for the firm: (1) Working-from-home works (on average) Normalized calls per week Before the experimentDuring the experiment Control Treatment

Slide 37

Experiment yielded four learnings for the firm: (2) Better & worse workers both improve when WFH Normalized calls per week: difference between home and work Before experimentDuring experiment

Slide 38

Experiment yielded four learnings for the firm: (3) Selection: Worker choice increases WFH impact Normalized calls per week: difference between home and work During the experiment After the experiment (roll-out) Before the experiment

Slide 39

Experiment yielded four learnings for the firm: 4) Employees value WFH as attrition down 50% Note: average daily commute is 1.41 hours and cost $0.96

Slide 40

Experiment so successful that CTrip is rapidly rolling out WFH across the firm Profit increase per employee WFH about $2,000 per year: Rent: $1,200 per year Retention: $400 per year Labor costs: $300 per year

Slide 41

Management field experiments Review of current experimental literature Our projects in India and China India China Thoughts on running field experiments

Slide 42

Thoughts on experiments 1.Expensive and hard, but worthwhile for the right question 2.Risky for junior faculty as can take many years 3.Think about both measure and intervention both can be tough (in India measuring the control firms was tough) 4.Works best as a team effort design, funding and execution all best as joint production 5.Running pilots and spending time on the ground invaluable for effective operation, analysis and presentation

Slide 43

Management field experiments Nick Bloom (Stanford and NBER) www.stanford.edu/~nbloom AOM, August 2012