7. Unraveling

8/10/2019 7. Unraveling

1/118

Unraveling


2/118

Northwestern Law School: July 2010 News Release

"CHICAGO --- Northwestern University School of Law and

the global law firm Jones Day announced today July 26that the firm will conduct its on-campus interviews for2011 summer associates in September instead of duringthe law school's official on-campus interviewing (OCI)program, which begins Aug. 11. In a move benefiting bothstudents and law firms, Jones Day will conduct interviewson behalf of its 14 U.S. offices on Monday, Sept. 13.

"Jones Day joins Northwestern Law in the belief that the

current recruitment system has created a competitive raceamong law schools and law firms to conduct on-campusinterviews earlier. The result is an inefficient system thatdoes not serve employers or student applicants well,

according to the law school and law firm. 2


3/118

Whats going on?

The announcement refers to the 2010interviewing and hiring of summerassociatesfor the summer of 2011, after

they will have finished only their secondyear of law school. That is, the competitionto find employment and employees for the

summer after the second year of lawschool has now moved a year in advance,to the summer after the first year.

3


4/118

Why are summer hiresimportant?

Because at the biggest law firms, much ofthe hiring of full time associates, who jointhe firm after the completion of their third

year of law school, has unraveled into thesummer associate market; i.e. thecompetition for summer associates is

fierce because summer associates largelybecome full time employees.

4


5/118

Why might this possibly beinefficient?

One reason, made clear by the recent economiccrisis (which reduced corporate demand foroutside legal services) is that hiring more than a

year before the start of employment makes itdifficult for law firms to forecast their demand.Thousands of summer associates at large firmswho accepted permanent offers shortly after

their second year summer associateships inAugust 2008 (for permanent jobs in 2009), hadthem rescinded or deferred.

5


6/118

6

Stages and transitions observed in various other marketswith timing problems

Stage 1: UNRAVELING

Offers are early dispersed in

time exploding

Stage 2: UNIFORM DATES

ENFORCED

Deadlines, congestion

Stage 3:

CENTRALIZED MARKET

CLEARING PROCEDURES


7/118

Roth, A.E. and X. Xing, "Jumping the Gun: Imperfectionsand Institutions Related to the Timing of MarketTransactions,"American Economic Review, 84,

September, 1994, 992-1044. American and British Doctors Federal Court Clerkships

American Law Firms

Postseason college football bowls

Business School Markets: New MBA's and Marketing Professors

Japanese University Graduates

Clinical Psychology Internships

Fraternities and Sororities

Canadian Lawyers Athletes

Marriage

Academic markets for mathematicians, biologists, and chemists

Medieval and Modern Commodity Markets7
http://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/RothAndXing1994.pdf


8/118

8

Unraveling (Roth and Xing 94)

Stage 1begins when the market comes into being (e.g. when a few

hospitals begin offering internships, or when federal court clerkships arecreated by legislation) and the relatively few transactions are made withoutovert timing problems. By the middle of stage 1 the market has grown, andsome appointments are being made rather early, with some participantsfinding that they don't have as wide a range of choices as they would like--students have to decide whether to accept early job offers or take a chanceand wait for better jobs, and some employers find that not all of the students

they are interested in are available by the time they get around to makingoffers. The trade journals start to be full of exhortations urging employers towait until the traditional time to make offers, or at least not to make themany earlier next year than this year. Towards the end of stage 1, the rate ofunraveling accelerates, until sometimes quite suddenly offers are beingmade so early that there are serious difficulties distinguishing among thecandidates. There is no uniform time for offers to be made nor is there acustomary duration for them to be left open, so participants find themselvesfacing unnaturally thin markets, and on both sides of the market a variety ofstrategic behaviors emerge, many of which are regarded as unethicalpractices. Various organizations concerned with the market may haveproposed guidelines intended to regulate it, without notable success. Asstage 1 ends, influential market participants are engaged in a vigorousdebate about what can and should be done. From beginning to end, stage 1may have covered a period of more than fifty years, or fewer than ten.


9/118

9

Causes and cures of unraveling

Unraveling is one of the kinds of market failurethat has often been corrected with appropriateclearinghouse design. But theres still a lot that

we dont understand about its causes.

Unraveling results from massively multiplecauses: you can only use time strategically intwo directions (earlier and later), so there isgoing to be more than one reason to go early.

Avoiding congestion is one reason; early hiringmay even be efficiency enhancing if there isnt

enough time to hire at an otherwise desirable latetime. (This is one attraction of clearinghouses.)


10/118

10

Forestalling: unraveling in

Medieval markets

"Thus at Norwich no one might forestallprovisions by buying, or paying 'earnest money'for them before the Cathedral bell had rung for

the mass of the Blessed Virgin; at Berwick-on-Tweed no one was to buy salmon betweensunset and sunrise, or wool and hides except atthe market-cross between 9 and 12; and at

Salisbury persons bringing victuals into the citywere not to sell them before broad day.


11/118

11

Unraveling in Space "Besides injury through mere proximity, and anticipation in

time, there might be damage due to interception of traffic... "In 1233 Eve de Braose complained that Richard fitz- Stephen

had raised a market at Dartmouth to the injury of hers atTotnes, as ships which ought to come to Totnes were stoppedat Dartmouth and paid customs there. No decision wasreached, and eight years later Eve's husband, William de

Cantelupe, brought a similar suit against Richard's sonGilbert. The latter pleaded that his market was on Wednesdayand that at Totnes on Saturday; but the jury said that themarket at Dartmouth was to the injury of Totnes, becauseDartmouth lies between it and the sea, so that ships touchedthere and paid toll instead of going to Totnes; and also that

cattle and sheep which used to be taken to Totnes marketwere now sold at Dartmouth; the market at Dartmouth wastherefore disallowed.

Both quotes: Salzman, as quoted in Roth and Xing (1994)


12/118

12

Market Failure by Unraveling, in Medicine,Football, Gastroenterology, Law, and in the Lab

Kagel, John H. and A.E. Roth, "The dynamics of reorganization in matchingmarkets: A laboratory experiment motivated by a natural experiment,"Quarterly Journal of Economics, February, 2000, 201-235.

Frchette, Guillaume, Alvin E. Roth, and M. Utku nver,"Unraveling YieldsInefficient Matchings: Evidence from Post-Season College Football Bowls,Rand Journal of Economics, Winter 2007, 967-982.

Niederle, Muriel and Alvin E. Roth, "Unraveling reduces mobility in a labormarket: Gastroenterology with and without a centralized match," Journal ofPolitical Economy, 111, 6, December 2003, 1342-1352.

Avery, Christopher, Christine Jolls, RichardA. Posner, and Alvin E. Roth, "TheMarket for Federal Judicial Law Clerks" , University of Chicago Law

Review, 68, 3, Summer, 2001, 793-902.Haruvy, Ernan, Alvin E. Roth, and M. Utku nver, The Dynamics of Law ClerkMatching: An Experimental and Computational Investigation of Proposalsfor Reform of the Market, Journal of Economic Dynamics and Control, 30,3 , March 2006, Pages 457-486.

Avery, Christopher, Christine Jolls, Richard A. Posner, and Alvin E. Roth, "TheNew Market for Federal Judicial Law Clerks, University of Chicago LawReview, 74, Spring 2007, 447-486.

Niederle, Muriel, and Alvin E. Roth, "Market Culture: How Rules GoverningExploding Offers Affect Market Performance,"American Economic Journal:Microeconomics, 1, 2, 2009, 199-219.

Niederle, Muriel, Alvin E. Roth, and M. Utku Unver, Unraveling Results fromComparableDemand and Supply: An Experimental Investigation, May, 2009.

Roth, Alvin E, "Marketplace institutions related to the timing of transactions, andreply to Priest (2010)" October, 2010. (Here is the shorter version that isforthcoming in Journal of Labor Economics, maybe in April 2012)
http://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20JOLE%20format.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Timing%20of%20transactions%20and%20unraveling%20of%20markets.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1407246http://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/MarketCultureForthcoming.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://lawreview.uchicago.edu/issues/archive/v74/74_2/11.AJPR.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/HaruvyRothUnver.JEDC2006.pdfhttp://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://papers.ssrn.com/sol3/papers.cfm?abstract_id=263988&CFID=647316&CFTOKEN=89632207http://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/unravelingcausesinefficiency.footballbowls.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/KagelRoth.QJE2000.pdf


13/118

13

Organizations concerned with the timing of when offers are made,accepted, rejected

Council of Graduate Schools (CGS): graduate admissions, National Association for College Admission Counseling (NCAC):

undergraduate admissions, (early action, early decision) National Resident Matching Program (NRMP): entry level medical

residencies, (also Canadian Resident Matching ServiceCaRMSandvarious regional matches in Britain)

Specialty Matching Services (SMS): advanced medical residencies andfellowships,

Association of Psychology Postdoctoral and Internship Centers (APPIC):clinical psychology positions, National Association for Law Placement (NALP) for positions in law firms, Judicial Conference of the United States and various ad hoc committees

of judges for federal judicial clerkships, Provincial Law Societies in Canada.

National Association of Colleges and Employers (NACE) for US collegeundergraduates, NCAA: formerly for postseason college football bowls, now regulated by

the Bowl Championship series (BCS), NCAA for recruitment of college athletes, and various drafts National Panhellenic Conference for sorority matching

The Japan Federation of Employers Associations (Nikkeiren) for Japaneseuniversity graduates


14/118

14

1990 College Football Bowl Games: Ratings


15/118

15

Probability of top 2 teams endingthe season as top 2 in AP Poll

1 Week Prior 2 Weeks Prior 3 Weeks Prior 4 Weeks Prior

Probability .690 .586 .310 .345Standard Error .086 .091 .086 .088

Number of Obs. 29 29 29 29


16/118

16

Summary of College Bowl arrangements after the NCAAabandoned its attempt to control the market after 1991

Rose Bowl Fiesta Bowl Orange Bowl Sugar Bowl Cotton Bowl

Starting Year 1902 1971 1935 1934 1937

First

Team

Since 1947 -Champion of

Big TenConference

Until 1978Champion of

WesternConferenceStarting 1978

At Large

Team

Champion ofBig Eight

(Twelve)Conference

Champion ofSoutheastern

Conference

Champion ofSouthwest

Conference

Matchups prior to

Bowl Coalition

Era

( - 1992)

Second

Team

Since 1947 -Champion of

Pacific Ten

(Coast orEight

previously)

Conference

At LargeTeam

At LargeTeam

At Large Team At Large Team

BC

Bowl?

No Yes Yes Yes Yes

First

Team

Champion ofBig Ten

Conference

At LargeTeam possibly

to create 12matchup

Champion ofBig Eight

(Twelve)Conference

Champion ofSoutheastern

Conference

Champion ofSouthwest

Conference

Matchups in Bowl

Coalition Era(1992-1994)

Second

Team

Champion of

Pacific TenConference

At Large

Team possiblyto create 12

At Large

Team possiblyto create 12

At Large Team

possibly tocreate 12

At Large Team

possibly tocreate 12


17/118

17

BA

Bowl?

No Yes Yes Yes No

First

Team

Champion of

Big TenConference

First Team from

BigTwelveConference notgoing to BABowls

Matchups in Bowl

Alliance Era

(1995-1997)

Second

Team

Champion ofPacific TenConference

2 At Large Teams, ACC, Big East, BigTwelve,

Southeastern conference championspossibly to create 1-2 matchup in one of thesebowl games

A team fromPacific Ten orWesternConferences out

of BA BowlsBCS

Bowl?

Yes Yes Yes Yes No

First

Team

First team fromBigTwelveConference outof BCS Bowls

Matchups in Bowl

Championship

Series Era

(1998-)

Second

Team

ACC, Big East, Big Twelve, Big Ten, Pacifi c Ten,

Southeastern conf erence champions, up to 2 highly ranked

other conference or at large teams (with Notre Dame having

priori ty) always to create always 1-2 matchup in BCS rank ings

in one of these bowl games A comparable

team fromSoutheasternConference

During this time the membership of the variousconferences also changed, and a number of

independents joined conferences.

d i d h ffi i f h k


18/118

18

and increased the efficiency of the market:

Average Normalized Nielsen Ratings in BCS Bowls

BC

BC

BC

BABA

BA

BCSBCS

BCS

BCS

BCSBCS

BCS

BCS

2

6

8

10

12

AverageRegularS

easonRatings

2

6

8

10

12

1985 1992 1995 1998 2005Year

Average minus Regular Coalition Regular


19/118

How do clearinghouses helpstop unraveling?

They can establish a thick market at anefficient time.

Weve conjectured that stability plays a

role in helping them do this successfully.

19


20/118

Priority matching (an unstable system)

Edinburgh, 1967 No longer in use

Birmingham 1966, 1971, 1978 " " " "

Newcastle 1970's " " " "

Sheffield 196x " " " "

In a priority matching algorithm, a 'priority' is defined for eachfirm-worker pair as a function of their mutual rankings. Thealgorithm matches all priority 1 couples and removes themfrom the market, then repeats for priority 2 matches, priority 3, etc.

E.g. in Newcastle, priorities for firm-worker rankings wereorganized by theproductof the rankings, (initially) as follows:

1-1, 2-1, 1-2, 1-3, 3-1, 4-1, 2-2, 1-4, 5-1...


21/118

Priority matching (an unstable system)

This can produce unstable matchings -- e.g. if a desirable firm and

worker rank each other 4th, they will have such a low priority(4x4=16) that if they fail to match to one of their first three choices,it is unlikely that they will match to each other. (e.g. the firm mightmatch to its 15th choice worker, if that worker has ranked it first...)

After 3 years, 80% of the submitted rankings were pre-arranged 1-1 rankings without any other choices ranked. This worked to thegreat disadvantage of those who didn't pre-arrange their matches.


22/118

22

What makes a clearinghousesuccessful or unsuccessful?

A matching is stable if there arent a doctor and

residency program, not matched to each other,who would both prefer to be.

Hypothesis: successful clearinghouses producestable matchings.

How to test this?


23/118

23

Market Stable Still in use (halted unraveling)

NRMP yes yes (new design in 98) Edinburgh ('69) yes yes

Cardiff yes yes Birmingham no no Edinburgh ('67) no no Newcastle no no Sheffield no no Cambridge no yes

London Hospital no yes Medical Specialties yes yes (~30 markets, 1 failure) Canadian Lawyers yes yes (Alberta, no BC, Ontario) Dental Residencies yes yes (5 ) (no 2) Osteopaths (< '94) no no Osteopaths (> '94) yes yes

Pharmacists yes yes Reform rabbis yes (first used in 97-98) yes Clinical psych yes (first used in 99) yes

So stability looks like an important feature of a centralized labormarket clearinghouse.


24/118

24

The need for experiments

How to know if the difference betweenstable and unstable matching mechanismsis the key to success?

There are other differences between e.g.Edinburgh and Newcastle

The policy question is whether the newclearinghouse needs to produce stablematchings (along with all the other things itneeds to do like handle couples, etc. ) E.g. rural hospital question


25/118

25

A matching experiment(Kagel and Roth, QJE 2000)

6 firms, 6 workers (half "High productivity" half "low productivity") It is worth $15 plus or minus at most 1 to match to a high It is worth $5 plus or minus at most 1 to match to a low There are three periods in which matches can be made:-2, -1, 0. Your payoff is the value of your match, minus $2 if made in

period -2, minus $1 if made in period -1 Decentralized match technology : firms may make one offer at any

period if they are not already matched. Workers may accept atmost one offer. Each participant learns only of his own offers andresponses until the end of period 0.

After experiencing ten decentralized games, a centralized matching

technology was introduced for period 0 (periods -2 and -1 wereorganized as before). Centralized matching technology: participants who are still

unmatched at period 0 submit rank order preference lists, and arematched by a centralized matching algorithm.

Experimental variable: Newcastle (unstable) or Edinburgh (stable)

algorithm.


26/118

26

Average Cost of Early Markets

0

2

4

6

8

10

1-5 6-10 11-15 16-20 21-25

Markets #

Cost(Dollars)

Decentralized Markets Markets with a Clearinghouse

Stable

(Deferred

Acceptance)

algorithm

New castle

Priority

algorithm


27/118

27

Offers and acceptances


28/118


29/118

29

The Roth-Peranson algorithm is also used inother market clearinghouses since 1998

Postdoctoral Dental Residencies in the United States

Psychology Internships in the United States andCanada

Neuropsychology Residencies in the United Statesand Canada

Osteopathic Internships in the United States

Pharmacy Practice Residencies in the United States

Articling Positions with Law Firms in Alberta,Canada


30/118

30

NRMP also runs Fellowship matches through the Specialties MatchingService (SMS) (using the Roth-Peranson algorithm since 1998)

Abdominal Transplant Surgery (2005)

Child & Adolescent Psychiatry (1995) Colon & Rectal Surgery (1984) Combined Musculoskeletal Matching

Program (CMMP) Hand Surgery (1990)

Medical Specialties Matching Program(MSMP) Cardiovascular Disease (1986)

Gastroenterology (1986-1999;rejoined in 2006)

Hematology (2006) Hematology/Oncology (2006) Infectious Disease (1986-1990;

rejoined in 1994) Oncology (2006)

Pulmonary and Critical Medicine (1986) Rheumatology (2005)

Minimally Invasive and GastrointestinalSurgery (2003)

Obstetrics/Gynecology

Reproductive Endocrinology (1991) Gynecologic Oncology (1993) Maternal-Fetal Medicine (1994) Female Pelvic Medicine &

Reconstructive Surgery (2001)

Ophthalmic Plastic & ReconstructiveSurgery (1991)

Pediatric Cardiology (1999)

Pediatric Critical Care Medicine (2000) Pediatric Emergency Medicine (1994) Pediatric Hematology/Oncology (2001) Pediatric Rheumatology (2004) Pediatric Surgery (1992) Primary Care Sports Medicine (1994) Radiology

Interventional Radiology (2002) Neuroradiology (2001) Pediatric Radiology (2003)

Surgical Critical Care (2004) Thoracic Surgery (1988) Vascular Surgery (1988)


31/118

31

Gastroenterologists Niederle, Murieland Alvin E. Roth, "Unraveling reduces mobility in a

labor market: Gastroenterology with and without a centralized

match," Journal of Political Economy, 1. 111, no. 6, December 2003,1342-1352. .

Niederle, Murieland Alvin E. Roth, "The Gastroenterology FellowshipMatch: How it failed, and why it could succeed onceagain,"Gastroenterology, 127, 2, August 2004, 658-666.

McKinney, C. Nicholas, Niederle, Murieland Alvin E. Roth, "Thecollapse of a medical labor clearinghouse (and why such failures arerare),"American Economic Review, 95, 3, June, 2005, 878-889.

Niederle, Murieland Alvin E. Roth, "The Gastroenterology FellowshipMarket: Should there be a Match?,"American Economic Review,Papers and Proceedings, 95,2, May, 2005, 372-375.

Niederle, Muriel, Deborah D. Proctor and Alvin E. Roth, What will beneeded for the new GI fellowship match to succeed?,Gastroenterology,130, January 2006, 218-224.

Niederle, Muriel, Deborah D. Proctor and Alvin E. Roth, ''TheGastroenterology Fellowship Match: The First Two

Years,''Gastroenterology , 135, 2 (August), 344-346, 2008.
http://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/http://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.gastrojournal.org/article/PIIS0016508508011979/fulltexthttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.2006.AGA.pdfhttp://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/GastroAEAPP.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/gastro.experiment.pdfhttp://www.stanford.edu/~niederle/http://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/Gastroenterology.pdfhttp://www.stanford.edu/~niederle/http://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/gastro.pdfhttp://www.stanford.edu/~niederle/


32/118

32

Timing of Interviews: Unraveling

Interviews for positions starting in 05 for specialties in Match.

Proportion of hospitals that started interviewing

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

aug sept oct nov dec jan feb mar apr

Match Start 05


33/118

33


Interviews are happening earlier and earlier


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.91


Start 05

Start 03

Match Start 05


34/118

34


Interviews are happening earlier and earlier


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Start 07 Survey

Start 06 Survey

Start 05

Start 03

Match Start 05


35/118

35

Timing of Offers

For each of 44 programs:

Use date of first and last offer, andassume that last offer is the longest openoffer: Overestimate length of time they areon the market.

Each program is represented by one line.

Dates during which fellowship programs were making offers. Each


36/118

36

a es du g c e o s p p og a s e e a g o e s acprogram is represented by one of the horizontal lines, indicating the(maximal) dates during which it could have had outstanding offers(2005 survey data, n=44).(As of November 15, 11 (27%) programs had already finished making

offers, 12 (25%) had not yet started, and 21 (48%) were in the midst.)


37/118

37

Market is very dispersed in time

0

0.1

0.2

0.3

0.4

0.5

0.6

1-Au

g

16-Aug

1-Se

p

16-Sep

1-Oc

t

16-Oct

1-No

v

16-Nov

1-De

c

16-Dec

1-Jan

16-Jan1-Fe

b

15-Feb

1-Ma

r

havingoutstandingoffers


38/118


39/118

Eff t f M t h M bilit


40/118

40

Effects of a Match: Mobility

With a Match, fellows are more mobile.

This effect is more pronounced for large hospitals.

(Data: 9180 of 15,187 gastroenterologists who completed residency andfellowship in US after 1977.)

Share of mobility of GI fellows for each year

before match match after match

0.2

0.3

0.4

0.5

0.6

0.7

0.8

1977 1980 1983 1986 1989 1992 1995 1998 2001

Hosp

City

State


41/118

41

Transition to a later market In May 2005, the American Gastroenterological

Association (AGA), the American College of

Gastroenterology (ACG), the American Society forGastrointestinal Endoscopy (ASGE) and the

American Association for the Study of LiverDiseases (AASLD) decided to reintroduce a GIfellowship match, starting in 2006, for positionsbeginning in July 2007.

How to manage the transition? Rates of participation

Concern among programs about whether their chief competitorswill participate.

Change of dates to June 2006 (from as early as July2005)

How to reassure programs that other programs will wait

for Match? (Without a congested transition like 1945-50)


42/118

42

In June 2005, Debbie Proctor, thegastroenterologist who took the lead inreorganizing the match, sent us an email saying,in part

Im answering 3-4 emails per day especially onthis issue. I want to make sure MY competitionis in the match and that they dont cheat. Well,this is another way of saying that if they cheat,then I will too!...Have you ever seen this before?

The distrust amongst program directors? I find ithard to believe that we are unique. Maybe this is[a] social science phenomenon?


43/118

43

Which markets are unraveled? Marketdesign

It appears that markets in which transactions aremade at early, uncoordinated times are marketsin which there are both

Exploding offers Binding commitments

Example: Graduate admissions has a nicelydesigned set of market rules, sent to all admitted

students, that suppresses exploding offers (inpart by facilitating reneges of premature

acceptances).


44/118

44

Students are under no obligation to respondto offers of financial support prior to April 15;earlier deadlines for acceptance of such

offers violate the intent of this Resolution. Inthose instances in which a student acceptsan offer before April 15, and subsequentlydesires to withdraw that acceptance, the

student may submit in writing a resignationof the appointment at any time through April15.


45/118

45

The need for experiments

Note that a simple experimental environment will bequite different from the markets in the table, and from thegastroenterology market.

The laboratory environment, because it is so simple, is

different from each of these markets in more transparentways than they are different from one another. while it is always somewhat risky to draw inferences about the

effect of a rule change in one market from the effects in adifferent market, the inferences may be clearer when one of themarkets is simple.

And in the experiment, the rules are an exogenousexperimental variable, so that their influence can bereadily observed.


46/118

46

An experiment allows us to view differentoffer regimes in a controlled environment

5 firms, 6 applicants, 9 periods. In each period, a firm may make an offer to at most one applicant. Firmsmake offers, applicants decide upon the offers they receive.

Firms and applicants are assigned qualities.

If firm of quality x hires an applicant of quality y, both firm and applicantwill receive a payoff of xy points each.

Firms qualities are simply their assigned participant number, 1,2,3,4,5. Uncertainty about applicants quality is only resolved over time:

In periods 1, 4 and 7, each applicant receives a signal, an integerbetween 1 and 10, each equally likely.

In period 7, the relative ranking of the sum of the 3 signals determinesthe applicants quality. The applicant with the highest sum of 3 signals

has a quality of 6 Ties are broken randomly. Firms see all signals, applicants see only their own signals (as they

become available over time) and their ranking in period 7.

(Note that in this experiment the cost of early matching is bad

matches due to uncertainty about quality.)

E i t l t t t 3 k t lt


47/118

47

Experimental treatments: 3 market cultures

Exploding offers: Firms can make exploding offers andacceptances are binding.

Renege: Firms can make exploding offers, but applicantscan renege on their acceptance, for a small fee (1 point).

Open offers: Firms can only make open offers.

(Many equilibria:One in which all matches are agreed upon inefficiently early.

All environments have a perfect equilibrium with efficient latematching.But the late matching equilibrium is more fragile when offers areexploding and acceptances are binding.)

Ti i f fi l ff ( ff th t t d d t


48/118

48

Timing of final offers (offers that were accepted and notreneged upon for the renege treatment) in terms of the

number of signals that were observed.

Timing of final offers

1

1.5

2

2.5

3

1-5 6-10 11-15 16-20Markets

N

umberofsign

als

available

Exploding

Open

Renege


49/118


50/118

50

Resolution Regarding Gastroenterology Fellowship Applicants,

Including all Research and Clinical Fellowship Applicants and Positions

This resolution concerns the conditions surrounding gastroenterology fellowship offers toapplicants, acceptance by applicants of such offers, and participation by applicants andprograms in the gastroenterology fellowship Match. The general spirit of this resolution isthat each applicant should have an opportunity to consider all programs before making adecision and be able to participate in the Match. The intention of this resolution is to ensureuniformity so that everyone participates fairly and to establish the principle that all positions should be

filled through the Match or after Match Day. It therefore seeks to create rules that give bothprograms and applicants the confidence that applicants and positions will remain availableto be filled through the Match and not withdrawn in advance of it.

This resolution addresses the issue that some applicants may be persuaded or coerced tomake commitments prior to, or outside of, the Match.Early offers and acceptances, and offersoutside of the Match, are violations of the rules and of this resolution and are not condoned.Anyapplicant may participate in the matching processby registering for the Match to interview and

consider match-participating programs; however, an applicant who accepts a position prior to, or outsideof, the Match must comply with the National Resident Matching Program/Specialty Matching Services(NRMP/SMS) Match Participation Agreement by either resigning the accepted position if he/shewishes to submit a rank order list of programs or by withdrawing from the Match prior to therank order list certification deadline, which is the first week in June. In addition, no program maywithdraw a position from the Match after the quota change deadline to offer that position outside thematching process. The spirit of this resolution is to make it unprofitable for programdirectors to press applicants to accept early offers, and to give applicants an opportunity toconsider all offersas well as to provide uniform and widely acceptable rules that protect bothapplicants and fellowship programs.


51/118


52/118

52

Another proposal: (What do you think of it?)


53/118

Another proposal: (What do you think of it?)Uniform notification date forpostdoctoralpositions in

professional psychology

53

O th di h d th bl


54/118

54

Orthopedic surgery had the same problem

But aspiring surgeons may not be able to turn downearly offers after accepting them, even if the orthopedic

organizations were to adopt a policy like thegastroenterologists'.

However, while the (15) orthopedic professionalorganizations also cannot directly prevent employers

from making early offers, unlike the gastroenterologists,they feel they can effectively punish employers whomake early offers, by not allowing them to presentpapers at professional meetings.

Harner, Christopher D., Anil S. Ranawat, Muriel

Niederle, Alvin E. Roth, Peter J. Stern, Shepard R.Hurwitz, William Levine, G. Paul DeRosa, Serena S. Hu,"Current State of Fellowship Hiring: Is a universal matchnecessary? Is it possible?," Journal of Bone and JointSurgery, 90, 2008,1375-1384.
http://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdfhttp://kuznets.fas.harvard.edu/~aroth/papers/Orthomatch.JBJS.pdf


55/118

55

Federal judges hiring law clerks have a similar


56/118

56

Federal judges hiring law clerks have a similarproblem

This market has also unraveled. Like orthopedic surgeons, law students can't change their

minds no matter how early the offer (law students are not ina position to break promises to federal judges).

Like the gastroenterology organizations, the judicialconferences have no way to prevent judges from hiring

early, or from making exploding offers, or punishing thosewho do.

There are now discussions underway among judges andlaw schools about the possibility of instituting aclearinghouse like the ones used by doctors.

But until some way is found to address the issue of earlyand exploding offers before a clearinghouse, the chances ofsuccess are small.

It appears that judges may have to become willing to dosome "community enforcement" of norms against early

exploding offers before a clearinghouse will work for them.


57/118

57

Law Clerks

Avery Christopher Christine Jolls Richard A Posner and


58/118

58

Avery, Christopher, Christine Jolls, Richard A. Posner, andAlvin E. Roth, The Market for Federal Judicial Law Clerks,University of Chicago Law Review, 68, 3, Summer, 2001, 793-902.

The market for clerkships starting in 2003 cleared in the September2001, i.e. at the very beginning of the first semester of the secondyear of law school; nearly two years before the start of employment.

This introduces several potential causes of inefficiency.

because the market clears so early, it clears before informationbecomes available (e.g. students second and third year grades, lawreview articles, etc.) that can help produce efficient matches ofparticular clerks and judges.

because competition among judges to hire earlier than theircompetitors makes the market fast, chaotic, and thin, many studentsand judges have little opportunity to consider a wide range ofoptions, but rather have to transact quickly, before options can bedeveloped.


59/118

59

The proximate cause of that study:

Memo from the Administrative Office of the UnitedStates Courts, to All United States Judges,October 7, 1998.

At its September 15, 1998 session, the JudicialConference of the United States rescinded itsSeptember 1993 policy recommending to all

judicial officers that March 1 of the year before aclerkship begins be the benchmark starting date

for law clerk interviews. This action was takenbecause the policy on law clerk interviews hasnot been universally followed...


60/118

60

Over the next few years, we observed

interviews led very quickly to offers

offers produced very quick responses responses were generally acceptances; and many scheduled interviews were canceled as a result.

Thus, students and judges tended to pair off quickly withthose with whom they have early interviews. As a result,

many students limited the judges to whom they apply toavoid being paired off early with a less preferred judge

We also witnessed complex but binding verbal contracts.

Offers became earlier from year to year (moving backfrom February to September of the second year).

An initial modest reform:


61/118

61

An initial, modest reform:Ninth Circuit Court Announced Positions for

Sept, 2003, as of August, 2001

Haruvy Ernan Alvin E Roth and M Utku nver The


62/118

62

Haruvy, Ernan, Alvin E. Roth, and M. Utku nver, TheDynamics of Law Clerk Matching: An Experimental andComputational Investigation of Proposals for Reform of

the Market, Journal of Economic Dynamics andContro l, 30, 3 , March 2006, Pages 457-486..

Decentralized

Centralized

- idealized

Centralized

- coerced

Without

Announcements

8 cohorts 10 cohorts8 cohorts

With Announcement 10 cohorts11 cohorts 10 cohorts

Experimental Design: 3 period market, in which information aboutmatch quality becomes fully available in the form of accumulatedgrades only in year 3.


63/118

63

Judges have qualities 1 to 4, known from the beginning of the market. In each of three years, students grades are independently drawn

from {0,1,2} with a discrete uniform density. Ties between studentsare broken arbitrarily only after year 3 grades become available (butbefore the application process starts in year 3). After all ties arebroken, the applicant with the highest cumulative grade in year 3 hasquality 4, the applicant with the second highest cumulative grade hasquality 3, the applicant with the second to last grade has quality 2,and the applicant with the lowest grade has quality 1.

The payoff to each subject is the product of his own quality and that of

the subject to whom he is matched Subjects experienced 20 markets, with the roles (judge or applicant)

remaining the same, judges qualities remaining fixed, and applicantsgrades determined randomly in each year of each market. So, offersmade in year 1 can be contingent only on year 1 grades, while offersmade in year 3 are made after all grades, and hence students final

qualities are known. Applicants decide, at the beginning of each year, to which judges, ifany, to submit applications. Applicants can apply to as manyavailable judges as they wish, and no judge may make an offer to anapplicant who has not applied to him. However, when a judgemakes an offer, the applicant must accept, unless a better judgemade an offer simultaneously.


64/118

64

In all six treatments, in each year (1-2 in the centralized treatments and1-3 in the decentralized):

Applicants send applications to available firms.

Firms may choose any one applicant from the pool of applicantswho have applied in a given year, and they are matched to thisapplicant unless a higher quality firm also chooses that applicant.(i.e. applicants cant refuse an offer unless a better one arrives atthe same time)

In the centralized-idealized treatment, year 3:

Firms and applicants not matched by the end of year 2 are allmatched in year 3, at the unique stable matching among thoseremaining in the market. (Notice that even in the centralized-idealized treatment when an applicant applies to a firm and receivesan offer in years 1 and 2, she cannot decline this offer).

In the centralized-coerced treatment, year 3:

Firms and applicants who were not matched by the end of thesecond year were placed via the central match in year 3. But to beeligible for matching to a particular firm following year 2, an applicantneeds to have sent an application to that firm in either year 1 or year2.


65/118

65

Average Group Welfaredashed lines represent treatments with announcements.

23

24

25

26

27

28

29

30

mkts 1-5 mkts 6-10 mkts 11-15 mkts 16-20

Market Block

Welfare

decentralized(NA) central-coerced(NA) central-idealized (NA)


66/118

66

Success of late match

0

0.5

1

1.5

2

2.5

Mkts 1-5 Mkts 6-10 Mkts 11-15 Mkts 16-20

Market

Num

bergoingthro

ughmatch

central-idealized

central-coerced

Proximate cause of our 2007 study


67/118

67

Avery, Christopher, Christine Jolls, Richard A. Posner, and Alvin E. Roth,

The New Market for Federal Judicial Law Clerks, University of Chicago

Law Review, 74, Spring 2007, 447-486

UNITED STATES COURTS OF APPEALS

March 11, 2002Dear Dean:

We are pleased to report that the federal appellate judges havevoted overwhelmingly in favor of a new Plan for Law Clerk Hiring.The Plan includes: (1) a moratorium on law clerk hiring duringthe Fall of 2002, (2) an arrangement ensuring that the hiring oflaw clerks will not be done earlier than the Fall of the third yearof law school, and (3) an agreement that the focus of law clerkhiring will be on third year law students and law graduates.The precise terms of the new Plan are set forth in the attached"Summary," and the history leading to the Plan's adoption appearsin the appended "Background" statement. More than two-hundredfederal appellate judges considered the Plan. Ninety-two percent(92%) of the judges either supported the Plan or indicated that theywould not oppose it. Thus, the "substantial consensus" requirement

that was needed to put the Plan into operation was easily satisfied.

Judges memo cont


68/118

68

Judges memo, cont.There are numerous advantages to a law clerk hiring system that

focuses on third year students and law graduates. Law clerkcandidates will be able to present more information with theirapplications and be judged more fairly. Thus, for example,applicants who are in their third year of law school will be able tooffer

a transcript showing performance during four semesters of school ina good variety of courses;

information on law journal selection, journal publications, andelection to a journal editorial board;

writing samples from seminar courses; information regarding experience gained in clinical courses and

public interest endeavors;

references from law professors for whom the student has worked asa research assistant or a teaching assistant; recommendations associated with judicial internships; significant recommendations from Summer employers; information on moot court competitions; and information on selection or election to positions in student

government.


69/118

69

Initial FAQ (accompanying judges memo)

Q Does the Plan endorse Summer interviewing?A No. Many judges would have opposed the Plan had it endorsed

Summer interviewing. There was a concern that Summer interviewswould be very inconvenient for many people. The reasons aremanifold: many judges are away on vacation during the Summer;law clerk applicants are otherwise occupied with Summer jobs,

vacations, foreign travel, and bar examinations (for recentgraduates); law professors often are away on vacation and thusunavailable to furnish references; and law schools do not releasegrades on any uniform schedule, so official student transcripts fromsome law schools are not available until near September. However,the Plan does not forbid a law student who, say, is fromVirginia and working in Tulsa during the Summer from talking

with a judge who is otherwise available to chat. This hashappened under existing hiring arrangements and the judges sawno reason to prohibit it under the new Plan. The main point,however, is that the formal hiring process will take place in the Fallwhen applications will be submitted and materials and referencesfrom the law schools will be sent to the judges.


70/118

70

Initial FAQ, cont.

QHow is "Fall" determined under the Plan?AThere is no fixed definition of Fall, nor is there any

fixed starting date for the hiring season. Underexisting arrangements, some judges do their hiring inSeptember, some in October, and others do it even later.

The Plan does not change this.QAre judges forbidden from making "exploding offers," i.e.,

offers that require an applicant to respond promptly to anoffer?

AThe Plan does not purport to address how an offer is

given by a judge. This is for each judge to determine.However, no applicant is obliged to act on an offer if theterms are unacceptable, nor is an applicant obliged toaccept the first offer that he or she receives.


71/118

71

Subsequent refinements of rules

From year to year, changes have beenmade to

Put precise start dates for applications (after

Labor Dayalready specified by start ofmarket in 2003)

Specify later precise dates for

Scheduling interviews

Conducting interviews and making offers

No rules regarding exploding offers


72/118

72

2004 modifications

The critical dates under the Revised Law Clerk Hiring Planare as follows:

After Labor Day: Third year law students and law schoolgraduates may submit law clerk applications and lettersof reference may be submitted on their behalf.

From the Day After Labor Day Through the SecondSunday After Labor Day (September 7- 19 in 2004):Reading period.

Beginning on the first Monday after Labor Day(September 13in 2004): Judges may schedule

interviews to be held after the reading period. Beginning on the Second Monday After Labor Day

(September 20in 2004): Judges may conduct interviewsand extend offers.

Subsequent modification of dates in 2005.

Critical Dates under the Law Clerk Hiring Plan for 2011


73/118

Critical Dates under the Law Clerk Hiring Plan for 2011http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/CriticalDates?OpenDocument

73

Offers may be made as soon as interviews are permitted under
http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/CriticalDates?OpenDocumenthttp://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/CriticalDates?OpenDocument


74/118

74

Offers may be made as soon as interviews are permitted underthe Plan. Generally, it is for the judge to determine the termsupon which an offer is extended. However, judges areencouraged not to require an applicant to accept an offerimmediately without reasonable time to weigh it against otherviable options that remain open to the applicant. This wouldnot prohibit an applicant from accepting an offer on the spot.

When setting up an interview with a clerkship applicant, ajudge should make clear to the applicant his or her interviewand offer policies or practices. For example, a judge may havea policy or practice of making offers and entirely filling his orher clerkship slots, even if more interviews are scheduled for

that day. The applicant should be told this in a timely fashion,so that the applicant's decision to accept or decline theinterview is appropriately informed. Applicants should also beinformed if the judge will ask them to make a decision on thespot.

http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/SUMMARY?OpenDocument
http://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/SUMMARY?OpenDocumenthttp://www.cadc.uscourts.gov/internet/lawclerk.nsf/Content/SUMMARY?OpenDocument


75/118

75

Data

We have repeated the surveying of both federal appellatejudges and applicants that we did in our prior study.

Federal appellate judges surveyed in fall of 2004 and fallof 2005.

Third year law students at the four law schools thatprovide the greatest number of clerks surveyed in fall of2004, fall of 2005, and fall of 2006.

About a 50% response rate from both federal appellatejudges and students. On the key measure of judge

reports of adherence versus nonadherence to the startdates, if anything selection bias should lead us tounderestimate the level of nonadherence.

Adherence to start dates (2004 judge


76/118

76

Adherence to start dates (2004 judgesurvey; gray areas = nonadherence)

Number and (in parentheses) cumulative percentage of

responding judges

Before

September

7

September

712

September

1319

September

2026

After

September

26/Not yetDate of first

interview

11

(9%)

6

(15%)

22

(33%)

66

(91%)

11

(100%)

Date of first

offer

5

(4%)

9

(12%)

11

(21%)

67

(78%)

26

(100%)

Judges perceptions: 2004


77/118

77

Judges perceptions: 2004

Number and (in parentheses) cumulative percentage of responding judges

Overall Within

Circuit

Relativelyfew judges

adhered

Toresponding

judges

knowledge,

many judges

adhered buta substantial

number didnot

Toresponding

judges

knowledge,

almost all

judgesadhered

Toresponding

judges

knowledge,

all judges

adhered

At leastone judge

in Circuit

did not

adhere

Start date

for

scheduling

interviews

3(3%)

34

(36%)

41(75%)

26(100%)

36(69%)

Start date

for

conducting

interviews

and making

offers

5(5%)

43

(46%)40

(85%)16

(100%)36

(73%)


78/118

78

Judges perceptions: 2005Number and (in parentheses) cumulative percentage of responding judges

Overall Within

Circuit

Relatively

few judges

adhered

To

responding

judges

knowledge,

many judges

adhered but

a substantialnumber did

not

To

responding

judges

knowledge,

almost all

judges

adhered

To

responding

judges

knowledge,

all judges

adhered

At least

one judge

in Circuit

did not

adhere

Start date

for

scheduling

interviews

5

(6%)

40

(52%)27

(84%)

14

(100%)

35

(87%)

Start date

for

conducting

interviews

and making

offers

4

(5%)

44

(58%)23

(86%)

12

(100%)

34

(81%)

Student reports: 2004


79/118

79

Student reports: 2004(gray areas = nonadherence)

Number and (in parentheses) cumulative percentage ofresponding students

Before

September

7

September

712

September

1319

September

2026

After

September

26/Not yet

Date of first

scheduling ofinterview

8(5%)

39

(31%)94

(92%)8

(97%)5

(100%)

Date of first

interview

9

(6%)

7

(11%)

18

(23%)

101

(91%)

13

(100%)

Date of first

offer

4

(3%)

3

(5%)

8

(12%)84

(77%)

29

(100%)


80/118

Student reports: 2006


81/118

81

Student reports: 2006(gray areas = nonadherence)

Number and (in parentheses) cumulative percentage of

responding students

Before

Sept. 5

Sept. 5-13 Sept. 14-

20

Sept. 21-27 After Sept.

27/Not Yet

Date of first

scheduling ofinterview

13(11%)

31

(38%)68

(97%)3

(99%)1

(100%)

Date of first

interview

13

(11%)

6

(16%)

18

(32%)

77

(97%)

3

(100%)

Date of first

offer

10

(9%)

3

(12%)

10

(22%)69

(87%)

14

(100%)

Exploding offers remain a problem:


82/118

82

Exploding offers remain a problem:Some representative quotes

I received the offer via voicemail while I was in flight tomy second interview. The judge actually left threemessages.First, to make the offer.Second, to tell me that I should respond soon.

Third, to rescind the offer.It was a 35 minute flight.

I had 10 minutes to accept.

I asked for an hour to consider the offer. The judgeagreed; however thirty minutes later [the judge] calledback and informed me that [the judge] wanted to rescindmy offer.


83/118


84/118

84

Fast market clearing: student responses

Fall of 2004 Fall of 2005 Fall of 2006

First offer received on start date

for interviewing and making

offers

38 51 45

First offer received after startdate for interviewing and making

offers

59 52 38

Of first offers received on start

date for interviewing and making

offers, percentage accepted onstart date

42% 63% 62%

So, for a nontrivial proportion of students, the market ends after the firstinterview (and some offers come even before the start date). Similarly forjudges, the market is not thick.

R


85/118

85

Recap

The market has now successfully held the latedate (fall of third year of law school) for severalyears in a row (2003- ). This yields an advantage due to better information

This certainly helps the many non-complying judgeswho make the earliest offers, and apparently alsohelps many of the complying judges.

But congested interviewing, exploding offers arestill a problem

This means that for many participants, the market stillisnt in fact thick.

What are the prospects for the future?

Stages and transitions observed in various other marketsith ti i bl


86/118

86

with timing problems

Stage 1: UNRAVELING

Offers are early, dispersed in

time, exploding

Stage 2: UNIFORM DATESENFORCED

Deadlines, congestion

Stage 3:

CENTRALIZED MARKET

CLEARING PROCEDURES

U if t t d t i th l l k k t


87/118

87

Uniform start dates in the law clerk market

Stage 1: UNRAVELING

Stage 2: UNIFORM

DATES ENFORCED

1983, 1986, 1989, 1990, 1993, 2002

1983: Sept 15 of 3rdyear, abandoned in 84

1986: April 1 of 2ndyear

1989: March 1, not adopted

1990: May 1, 12:00 Noon

1993: March 1 (not formally abandoned until 1998)

Thoughts on the future evolution of the


88/118

88

Thoughts on the future evolution of theclerk market

It has cycled between stage 1 and stage 2multiple times

Comparison to other markets, Stage 2 markets

Psych25 years in stage 2. Japanese universities

The current arrangement has formally heldtogether longer than the previous longest (1993-97) stage 2 arrangement. But theres lots of

non-compliance. Discussions are underway about market

design

Would unraveling occur if


89/118

gsalaries were competitive?

And what are competitive salaries? Recallthe case of law firms

89


90/118

90http://www.nalp.org/reported_salaries_by_city
http://www.nalp.org/reported_salaries_by_cityhttp://www.nalp.org/reported_salaries_by_cityhttp://www.nalp.org/reported_salaries_by_city


91/118

91

salaries of

$160,000accounted for25% of reportedsalaries.

salaries of$160,000accounted

for about18% ofreportedsalaries

Law Firms


92/118

Law Firms

In an attempt to regulate the market, theNALP, formed in 1971, issued regulationsintended to govern the timing and duration

of offers by law firms. E.g. a regulationadopted in 1988 specified that offersshould remain open for at least two weeks,

and others specified dates until whichoffers should remain open.

92


93/118

But Roth and Xing (1994, p1007) note that

some law firms began to give offers which

met the letter of [these regulations], but

which structured the compensation so thatthe offer was competitive because itincluded a signing bonus which could

only be collected if the offer was acceptedmuch more promptly.

93

2010 rules (NALP 2010c),


94/118

All offers to law student candidates(candidates) should remain open for at least

two weeks after the date of the offer letter Q.May employers offer signing bonuses that

decline or evaporate according to the date ofacceptance of the offer of employment?A.This type of signing bonus violates the letterand spirit of the Principles & Standards.

Signing bonuses or other benefits that require a

decision in advance of the dates in Part V or thatvary according to the date of acceptance areconsidered special inducements and violate thisprovision.

94

Which markets are unraveled?


95/118

95

Supply and demand.

Li and Rosen, Li and Suen, insurance Halaburda

Niederle, Roth and Unver In some markets unraveling is attributed to an

imbalance of supply and demand

But to get unraveling you need two things: Firms have to be willing to make early offers

Workers have to be willing to accept them

This is most likely to occur when both firms andworkers can plausibly think of themselves as being onthe long side of the marketa common state of mindin the markets we study

Simple supply and demand hypothesis


96/118

p pp y yp Excess s upp ly or d emand ( in the relevant part of the market)

increases competi t ion, wh ich causes ineff ic ient ly ear lytransact ions.

Menard (book review, 2003), on college admissions:"There are many reasons that college admission has become so complex,

but the main one is demand..(In 1932 Yale admitted 72% of applicants,now 13%).

Avery, Fairbanks, and Zeckhauser (book, 2003, p32) quote a 1990U.S. News and World Report story.

"Many colleges, experiencing a drop in freshman applications as thepopulation of 18-year-olds declines, are heavily promoting early-acceptance plans in recruiting visits to high schools and in campus toursin hopes of corralling top students sooner."

Roth (JPE 1984about the market for new physicians around 1900,"The number of positions offered for interns was, from the beginning,

greater than the number of graduating medical students applying for

such positions, and there was considerable competition amonghospitals for interns. One form in which this competition manifesteditself was that hospitals attempted to set the date at which they wouldfinalize binding agreements with interns a little earlier than their principalcompetitors."

96

Comparable supply and demand


97/118

p pp yhypothesis

Early transactions require both that firmsshould want to make early offers, andworkers should want to accept them.

So too much imbalance in supply anddemand should notbe associated withunraveling.

97

In many markets, both sides feel they are on the long side


98/118

Many positions available

at unselective colleges

Fewer positions at elite

colleges than students

eager to enroll.

College admissions

Fewer top teams than

topbowls

More teams than bowlsPostseason College

Football Bowls (before

BCS)

Fewer top and research

oriented applicants than

positions

More board-certified

applicants than positions

Medical subspecialties

Many more interested

foreign medical graduatesthan American positions

Many more first year

positions than graduatesof American med schools

American medical

residents

Fewer Law Review

editors than Federal

appellate judges

Many more law grads

than judges

Federal court clerkships

Many positions available

at unselective colleges

Fewer positions at elite


eager to enroll.

College admissions Fewer positions at elite


eager to enroll.

College admissions

Fewer top teams than

topbowls



BCS)



BCS)

Fewer top and research

oriented applicants than

positions

More board-certified


Medical subspecialties More board-certified


Medical subspecialties

Many more interested

foreign medical graduatesthan American positions



American medical

residents



American medical

residents

Fewer Law Review

editors than Federal

appellate judges

Many more law grads

than judges

Federal court clerkships Many more law grads

than judges

Federal court clerkships

98


99/118

Supply and demand model


100/118

nFfirms, nFhof high quality

Firm quality is common knowledge fromtime 1

nAapplicants, of which nAhwill eventually

become high quality

Only the proportion nAh/nAis known until the

first period of the latehiring stage, when thehigh quality individuals are realized

Rules: there are multiple early and lateperiods: each unmatched firm may make anoffer to at most one worker in each period,

acceptances are binding100

Rules


101/118

Rules At the beginning of each period, each available firm

can extend an offer to an applicant of its choice. After the firms make offers, the applicants observe all

the offers made by the firms and each applicant canaccept one of the offers she got in the period or rejectthem all.

Acceptances are binding; once an applicant acceptsan offer, firms cannot make her further offers. Each offer is valid only in the period it is extended (it is

an exploding offer; although workers can considermultiple offers if made in the same period )

After the period is over, all decisions made in theperiod become public information.

101

Payoffs


102/118

y Remaining unmatched earns zero. Let EU(i,j) be the expected

payoff of a quality i agent from being matched with a quality jagent. EU(i,j) is increasing in both arguments andsupermodular so that

EU(h,h)-EU(h,)>EU(,h)-EU(,)0 At the beginning of the late stage, a tie-breaking lottery e is

drawn to rank high quality agents among each other and lowquality applicants among each other from the best to the

worst. The lottery results become public information. The payoff of an agent a with quality i being matched with a

partner b of quality j is given byua(b)=EU(i,j)+eb

where eb is the lottery draw of the agent b. E[e]=0 and support

of e is small enough so that being matched with a high qualitypartner is always better than being matched with a low qualitypartner

102

Efficiency and inefficiency due to unraveling


103/118

Well focus on big (and visible) inefficiencies in

which high quality agents are unnecessarilymatched with low quality, i.e. in which thematching isnt as assortative as it could be.

Qual ityw ise Eff ic iency = Sum of the Expected

Payoffs are maximized (without taking tie-breakerinto account) SupermodularityA matching is qualitywise efficient

if and only if it is qualitywise assortative i.e. high typeapplicants are matched with high firms as long as it is

possible and remaining agents are matched with lowquality partners as long as it is possible.

103

Many Nash equilibria


104/118

Many Nash equilibria

But well be able to say something aboutsubgame perfect equilibria.

104


105/118

Subgame perfect equilibria:


106/118

lemmas

Late stage: Any subgame perfect equilibriumproduces assortative matching among thefirms and applicants still unmatched at the

beginning of the late hiring stage. From the beginning: In the cases of excess

supply or demand, nAhnF(Case 1), and

nFnA>nFh(Case 3) the unique subgame

perfect equilibrium outcome is late,assortative matching.

106

What causes late and assortative matchingwhen supply and demand are unbalanced?


107/118

when supply and demand are unbalanced?

Excess supply (case 1): nAh

nFApplicants would accept early offers but firms

prefer to wait and guarantee a high qualityapplicant.

Excess demand (case 3): nFnA>nFh

Applicants wont accept low quality early

offers, since they can always get a low quality

offer late.Applicants would accept high quality early

offers

At equilibrium high quality firms prefer to wait

107

Unraveling that doesnt harm


108/118

efficiency

Lemma: In the case of nFhnA (Case 4,excess demand), the outcome of anysubgame perfect equilibrium is qualitywiseefficient. That is, high quality firms may make early

offers that are accepted, but this doesnt harm(qualitywise) efficiency, since not all highquality firms can be matched anyway. (i.e. ifall applicants are matched, the outcome isqualitywise efficient.)

108

Necessary conditions for inefficient


109/118

unraveling at SPE

Theorem: A qualitywise inefficient earlymatching is an outcome of a subgame perfectequilibrium only if the market is one ofcomparabledemand and supply (Case 2):

nA> nF>nAh

. Inefficiency results at equilibria in which highquality firms wait for the late stages (whenthey can always hire someone), but lowquality firms make early offers which areaccepted (since some applicants will be leftunmatched). But these necessary conditions arent sufficient.

109

Sufficient conditions for full unraveling


110/118

In the case of comparable demand and supply

nA> nF>nAh(Case 2), if0 > nF

huhh+(nF-nFh-nA

h)uh-(nA-nAh)u

and nFh nA

h,

then all low quali ty f i rms h ire in the ear lyhir ing s tage at any subgame perfectequi l ibr ium, leading to a qualitywiseinefficient matching with positive probability.

So we can choose experimentalparameters in which the SPE predictsunraveling.

110


111/118

Experimental design: 4 firms with 1 or 2 position(s)each and 6 or 12 applicants Half of the positions are high quality and


112/118

112

each and 6 or 12 applicants. Half of the positions are high quality andone third of the applicants are high quality.

Treatments6 Appli cants

2 H igh Quali ty Applicants

12 Appli cants

4 H igh Quality Applicants

4 Firm Positi ons

2 H igh Quality

Positions

Baseline Thin

comparable supply/demand

market treatment(Case 2)

Treatment with

excess supply (Case 1)

8 Firm Positions(with f irm quota 2)

4 H igh Quality

Positions

Treatment with

excess demand(Case 3)

Thick comparable

supply/demand market

treatment(Case 2)

Prediction (Theorem) A qualitywise inefficient early matching isan outcome of a subgame perfect equilibrium only if themarket is one of comparable demand and supply (case 2).(Inefficiency results when low quality firms make early offersthat are accepted.)

(SPE) Predictions


113/118

( )

113

SPEnA 6

nAh 2

nA 12

nAh 4

nF 4

nF

h

2

THIN

COMPARABLE

Low quality

firms hire earlyqualitywise-inefficient

outcome

EXCESS

SUPPLY

Late and

assortativematching

efficient outcome

nF 8

nFh 4

EXCESS

DEMAND

Late and

assortative

matching

efficient outcome

THICK

COMPARABLE

Low quality

firms hire early

qualitywise-inefficient

outcome

87.50%

100.00%

Median % High Type Firms Hiring Early


114/118

114

0.00%

12.50%

25.00%37.50%

50.00%

62.50%

75.00%

87.50%

Markets 1-5 Markets 6-10 Markets 11-15 Markets 16-20

THIN COMPARABLE - 6 App - 4 Firm EXCESS SUPPLY -12 App - 4 Firm

EXCESS DEMAND - 6 App - 8 Firm THICK COMPARABLE - 12 App - 8 Firm

0.00%

12.50%

25.00%

37.50%

50.00%

62.50%

75.00%

87.50%

100.00%

Markets 1-5 Markets 6-10 Markets 11-15 Markets 16-20

Median % LowType Firms Hiring Early

THIN COMPARABLE -6 App - 4 Firm EXCESS SUPPLY -12 App - 4 Firm

EXCESS DEMAND - 6 App - 8 Firm THICK COMPARABLE - 12 App - 8 Firm

What have we learned so far?


115/118

115

What have we learned so far? Unraveled markets (with early, dispersed,

exploding offers) produce different outcomes inpredictable ways. Less thick, unsafe, suffer from congestion, inefficient Loss of mobility in the gastro market

Unraveling involves both Market design: e.g. how exploding offers are treated Supply and demand: both firms and workers have to

be willing

Centralized markets can solve these problems Centralized matches are preceded by (potential)

decentralized markets. Even well-designed (stable) matches can unravel Moving from an early to a late equilibrium can be hard

Open questions


116/118

Open questions

what factors make a market prone tounraveling?

Which markets are adequately organized

without a coordinated marketplace(coordinated in time, or space, or both)?

For which markets is a centralizedmarketplace desirable?

For which markets does a centralizedclearinghouse (possibly computer-assisted)offer additional benefits?

116

Experiments have multiple roles to play


117/118

117

Experiments have multiple roles to play

They allow us to investigate questions that thefield data cannot answer E.g. why did the GI match fail in 96?

They allow us to investigate hypotheses

suggested by the field data E.g. does the ability to renege on early acceptancesreduce exploding offers and unraveling?

What conditions of supply and demand promoteunraveling?

They play a role in the considerable amounts ofdiscovery, demonstration, and persuasion thatare necessary to coordinate market participantsto move from one equilibrium to a better one.

Conclusions:


118/118

Markets dont always spring up likeweeds, some of them are hothouseorchids that need care and attention.

We need to understand how marketswork in enough detail so we can fix themwhen theyre broken.

7. Unraveling

Documents

Transcript of 7. Unraveling