Tournament Selection Efficiency: An Analysis of the PGA TOUR’s

FedExCup1

Robert A. Connolly and Richard J. Rendleman, Jr.

October 10, 2012

1Robert A. Connolly is Associate Professor, Kenan-Flagler Business School, University of North Carolina,Chapel Hill. Richard J. Rendleman, Jr. is Visiting Professor, Tuck School of Business at Dartmouth andProfessor Emeritus, Kenan-Flagler Business School, University of North Carolina, Chapel Hill. The authorsthank the PGA TOUR for providing the data used in connection with this study, Pranab Sen, Nicholas Halland Dmitry Ryvkin for helpful comments on an earlier version of the paper and Ken Lovell for providing com-ments on the present version. Please address comments to Robert Connolly (email: robert connolly@unc.edu;phone: (919) 962-0053) or to Richard J. Rendleman, Jr. (e-mail: richard rendleman@unc.edu; phone: (919)962-3188).

Tournament Selection Efficiency: An Analysis of the PGA TOUR’s

FedExCup

Abstract

Analytical descriptions of tournament selection efficiency properties can be elusive for realistic

tournament structures. Combining a Monte Carlo simulation with a statistical model of player

skill and random variation in scoring, we estimate the selection efficiency of the PGA TOUR’s

FedExCup, a very complex multi-stage golf competition, which distributes $35 million in prize

money, including $10 million to the winner. Our assessments of efficiency are based on traditional

selection efficiency measures. We also introduce three new measures of efficiency which focus on

the ability of a given tournament structure to identify properly the relative skills of all tournament

participants and to distribute efficiently all of the tournament’s prize money. We find that reason-

able deviations from the present FedExCup structure do not yield large differences in the various

measures of efficiency.

1 IntroductionIn this study, we analyze the selection efficiency of the PGA TOUR’s FedExCup, alarge-scale athletic competition involving a regular season followed by a series ofplayoff rounds and a “finals” event, where an overall champion is crowned. FedEx-Cup competition began in 2007. Each year, at the completion of the competition,a total of $35 million in prize money is distributed to 150 players, with those inthe top three finishing positions earning $10 million, $3 million and $2 million,respectively.1

Research into selection efficiency highlights the importance of the criterionfor assessing tournament properties.2 Most who study tournament competition em-phasize the probability that the best player will be declared the winner (“predictivepower”) as the critical measure of tournament selection efficiency. Largely main-taining the focus of the selection efficiency literature on a single player, Ryvkin andOrtmann (2008) and Ryvkin (2010) introduce two additional selection efficiencymeasures, the expected skill level of the tournament winner and the expected skillranking of the winner. They develop the properties of these selection efficiencymeasures in simulated tournament competition.

While we use these efficiency measures in our work, we also develop threenew measures of selection efficiency that evaluate the overall efficiency of a tourna-ment structure, not just the the mean skill and mean skill rank of the first-place fin-isher and the expected finishing position of the most highly-skilled player. Much ofthe existing literature (e.g., Ryvkin (2010), Ryvkin and Ortmann (2008)) assumes aspecific set of distributions (e.g., normal, Pareto, and exponential) to describe com-petitor skill and random variation in performance. In this paper, we integrate anempirical model of skill and random variation in performance with a detailed tour-nament simulation to explore the selection efficiency of FedExCup competition.We do not specify the matrix of winning probabilities as in some studies; instead,it is generated naturally from the underlying estimated distributions of competitorskill and random variation and the tournament structure itself.

In the next section of the paper we describe the characteristics of FedExCupcompetition. We develop tournament selection efficiency measures in Section 3.We present an overview of the statistical foundations of our work in Section 4,describe our simulation methods in Section 5, and present results and a discussionof practical implications of our work in Section 6. We summarize our findings inthe final section. Appendix A describes the details of our simulation.

1See http://www.pgatour.com/r/stats/info/?02396.2See Ryvkin and Ortmann (2008) for an excellent recap of existing work along these lines.

2 Characteristics of FedExCup Competition

2.1 Structure of FedExCup Competition

Under current FedExCup rules, similar in structure to NASCAR’s Sprint Cup pointssystem, PGA TOUR members accumulate FedExCup points during the 35-weekregular PGA TOUR season.3 As shown in the “Regular Season Points” portion ofTable 1, points are awarded in each regular season PGA TOUR-sanctioned eventto those who make cuts using a non-linear points distribution schedule, with thegreatest number of points given to top finishers relative to those finishing near thebottom. At the end of the regular season, PGA TOUR members who rank 1 - 125in FedExCup points are eligible to participate in the FedExCup Playoffs, a series offour regular 72-hole stroke play events, beginning in late August.

In the Playoffs, points continue to be accumulated, but at a rate equal to fivetimes that of regular season events. The field of FedExCup participants is reducedto 100 after the first round of the Playoffs (The Barclays), reduced again to 70after the second Playoffs round (the Deutsche Bank Championship), and reducedagain to 30 after the third round (the BMW Championship). At the conclusion ofthe third round, FedExCup points for the final 30 players are reset according to apredetermined schedule, with the FedExCup Finals being conducted in connectionwith THE TOUR Championship. The player who has accumulated the greatestnumber of FedExCup points after THE TOUR Championship wins the FedExCup.4

2.2 FedExCup Competition Objectives

It is clear that the objectives of FedExCup competition are multidimensional andcomplex. From the November 25, 2008 interview with PGA TOUR CommissionerTim Finchem (PGA TOUR (2008)), it is possible to identify a number of thesedimensions.

1. The points system should identify and reward players who have performedexceptionally well throughout the regular season and Playoffs. As such,among those who qualify for the Playoffs, performance during the regularseason should have a bearing on final FedExCup standings.

3The rules associated with FedExCup competition have been changed twice. Detail about therevisions is presented in Hall and Potts (2010).

4A primer on the structure and point accumulation and reset rules may also be found athttp://www.pgatour.com/fedexcup/playoffs-primer/index.html.

2. The Playoffs should build toward a climactic finish, creating a “playoff-typefeel,” holding fan interest and generating significant TV revenue throughoutthe Playoffs.

3. The points system should be structured so that the FedExCup winner is notdetermined prior to the Finals. (In 2008, Vijay Singh only needed to “showup” at the Finals to win the FedExCup. This led to significant changes in thepoints structure at the end of the 2008 PGA TOUR season.)

4. The points system should give each participant in the Finals a mathematicalchance of winning. We note that Bill Haas, the 2011 FedExCup winner andlowest-seeded player to ever win, was seeded 25th among the 30 players whocompeted in the Finals.5

5. The points system should be easy to understand. Under the current system,any player among the top five going into the Finals who wins the final event(THE TOUR Championship) also wins the FedExCup. Otherwise, under-standing the system, especially during the heat of competition, can be verydifficult.

We do not attempt to quantify the PGA TOUR’s objectives, as summarizedabove. Instead, we evaluate the optimal selection efficiency of FedExCup compe-tition based on two decision variables. The first is the Playoffs points multiple.Presently, Playoffs points are five times regular season points. This has a poten-tial impact on Commissioner Finchem’s objective points 1 and 2 above. Talkingwith PGA TOUR officials, we understand that the TOUR reassesses the FedExCuppoints structure at the end of every season and that this multiple is an importantpart of the discussion. Reflecting these discussions, we vary the multiple between1 and 5 in integer increments. Our second decision variable is whether or not toreset accumulated FedExCup points at the end of the third Playoffs round. Thepresent reset system is structured to satisfy objectives 3 and 4 and guarantee thatany player among the top five going into the Finals who wins the final event willwin the FedExCup (objective 5, at least in part).

Although we are able to identify optimal competition structures evaluated interms of our six efficiency measures, we find that the cost of deviating from optimalstructure appears to be small. This finding suggests that the costs of the implicitconstraints associated with the objectives listed above may not be high.

5Although confusing, we adopt the convention used throughout sports competition that a “low”seeding or finishing position is a higher number than a “high” position. For example, in a 10-playercompetition, the “highest” seed is seeding position 1, while the lowest seed is position 10.

3 Measures of EfficiencyIn order to measure the selection efficiency of various FedExCup competition struc-tures, we simulate entire seasons of regular PGA TOUR competition followed byfour Playoffs rounds. In each simulation trial, we begin with a set of “true” playerskills, or expected 18-hole scores. Throughout the regular season and Playoffs com-petition, each simulated score for a given player equals his expected score, as givenby his true skill level, plus a residual random noise component. As the season pro-gresses, and throughout the Playoffs, each player accumulates FedExCup pointsaccording to a defined set of rules as described in Section 4.1. We then estimate theefficiency of the FedExCup points system using the criteria described below.

3.1 Ryvkin/Ortmann Selection Efficiency Measures

We use the following three measures of tournament selection efficiency, examinedin detail by Ryvkin and Ortmann (2008) and Ryvkin (2010).

1. The winning (%) rate of the most highly-skilled player, also known as “pre-dictive power.”

2. The mean skill level (expected 18-hole score) of the tournament winner.3. The mean skill ranking of the tournament winner.

Note that these three criteria focus on a single player, either the most highly-skilled player (predictive power) or the tournament winner. No weight is placedon the finishing positions of other players other than through their effect on thefinishing position of the most highly-skilled player or the mean skill ranking orskill level of the tournament winner.

We propose three new measures of selection efficiency that capture the abil-ity of a given tournament format to properly classify all tournament participantsaccording to their true skill levels, not just the player who is the most highly skilled,and to properly allocate tournament prize money. Even if the most highly-skilledplayer in FedExCup competition wins most of the time, the FedExCup would surelylose credibility if the worst players in the competition could frequently finish nearthe top and win a significant portion of the prize money. Ideally, the FedExCupdesign would not only identify the single best player in the competition with highprobability but would also place players in finishing positions relatively close totheir true skill rankings. As such, tournament prize money would generally be thehighest for the most highly skilled and lowest for the lowest skilled and, there-fore, players would be rewarded in relation to their true skill levels. Our final three

measures of selection efficiency take the form of loss functions that reflect thesetradeoffs.

3.2 Mean Squared Rank Error (LRE)

Consider a tournament of N players, i= 1,2, ...,N, ordered by true skill (or expectedscore) µi, with µ1 < µ2, ... < µN . Let j(i) denote the tournament finishing positionof player i. For example, if the most highly-skilled player finishes the tournament in5th position, j(1) = 5. Then µ j(i) is the inverse transformation of true skill impliedby player i’s tournament finishing position, j(i), which henceforth, we refer to as“implied skill.” Finally, let M j(i) denote the monetary prize to player i if he finishesthe tournament in position j(i), with M1 > M2, ... > MN . Thus, Mi denotes whatplayer i’s prize would have been if his tournament finishing position had equalledhis true skill ranking and M j(i) denotes player i’s actual prize.

Our first loss function, the mean squared ranking error, LRE , measures theextent to which the tournament errs in identifying the true skill rankings of the Ntournament participants.

LRE = 1N

N∑

i=1(i− j (i))2

= 2σ2R (1−ρ) , (1)

where σ2R =

(N2−1

)/12 is the variance of the ranking positions, i= 1,2, ...,N, and

ρ is the Spearman rank order correlation of the true skill ranks, i, and tournamentfinishing positions, j(i). Thus, a tournament scheme that maximizes the Spearmanrank, ρ , will minimize the mean squared ranking error, LRE .6

We note that LRE weights all ranking errors equally, regardless of the actualskill differences of the players who have been miss-ranked. Our final two efficiencymeasures reflect these differences.

3.3 Mean Squared Skill Error (LSE)

The mean squared skill error is defined as follows:

LSE = 1N

N∑

i=1

(µi−µ j(i)

)2

6We note that Spearman’s footrule, another measure of ranking error, is equivalent to minimizingthe sum of absolute ranking errors rather than squared ranking errors.

= 2σ2µ

(1−βµ

). (2)

Here, σ2µ is the variance of true player skill, and βµ is the OLS slope coefficient

associated with a regression of true player skill µi on implied player skill, µ j(i),or vice versa. When true skill rankings and tournament finishing positions are per-fectly aligned, βµ = 1, and LSE = 0. Note that if µ is linear in skill rank, LSE = LRE .The mean squared skill error takes the form of a quadratic loss function, equivalentto the loss function underlying OLS regression and Taguchi’s (2005) loss functionused in quality control.

3.4 Mean Money-Weighted Squared Skill Error (LWSE)

Here we weight each value of(µi−µ j(i)

)2 in (2) by wi = Mi/N∑

i=1Mi, where Mi

denotes what the tournament prize would have been for the player with skill rankingi if his tournament finishing position had equalled his true skill ranking. Thus, themean money-weighted skill error is computed as follows:

LWSE =N

∑i=1

(µi−µ j(i)

)2wi. (3)

Alternatively, we could weight by w j(i) = M j(i)/N∑

i=1Mi, where M j(i) denotes player

i’s actual prize. If the money payout schedule were linear, a given difference be-tween true and implied skill would be penalized the same, regardless of a player’sskill ranking. However, the actual FedExCup money payout schedule is highlyconvex, with the top three finishers earning $10 million, $3 million and $2 million,respectively, and the players in finishing positions 4-150 splitting the remaining $20million of the prize pool, also in non-linear fashion.7 This non-linearity implies thatthe two possible weighting schemes, wi and w j(i), emphasize different types of im-plied ranking errors. Weighting by wi, rather than w j(i), assumes, implicitly, thattournament organizers are more concerned about under-performance by high-skillplayers than over-performance by low-skill players. Unlike the previous two lossfunctions as expressed in Equations (1) and (2), Equation (3) cannot be simpli-fied further without substantial restrictions on the functional form of the weightingfunction (and the consequent loss of generality).

7Details are provided at http://www.pgatour.com/r/stats/info/?02396.

3.5 Mean Squared Error Deflators

Inasmuch as the value of each mean squared error is difficult to interpret without areference point, we deflate each by the corresponding variance of the variable whoseerror we are attempting to estimate (i.e., true skill rankings, true player skill andmoney-weighted true player skill.) Thus, the deflators for the ranking error, skillerror and weighted skill error are, respectively, DRE =

(N2−1

)/12, DSE = σ2

µ , and

DWSE =N∑

i=1µ2

i wi− (N∑

i=1µiwi)

2.

4 Optimizing FedExCup Competition

4.1 FedExCup Points Distribution and Accumulation

The “Regular Season Points” section of Table 1 shows the distribution of regularseason FedExCup points. WGC and Majors are allocated slightly more points thanregular PGA TOUR events. “Additional events,” which are events held opposite ofsome WGC events and majors, are allocated half the points associated with eachregular event finishing position.

During the regular PGA TOUR season, players accumulate FedExCup pointsbased on the regular season points schedule. At the end of the regular season, thetop 125 players in accumulated FedExCup points qualify to participate in the Play-offs. Each participant in the Playoffs carries his accumulated FedExCup points intothe Playoffs, but once in the Playoffs, FedExCup points are awarded and accumu-lated according to the schedule shown in the “Playoffs Points” section of the table.Note that the points distribution schedule for the first three rounds of the Playoffs isexactly five times the points distribution for regular PGA TOUR events conductedprior to the Playoffs.

At the end of the first Playoffs round (The Barclays), only the top 100 play-ers in accumulated FedExCup points are eligible to continue to the second round.After the second round (The Deutsche Bank Championship), only the top 70 playersare eligible to continue to the third round. After the third round (The BMW Cham-pionship) only the top 30 players qualify for the FedExCup Finals (The TOURChampionship). Immediately prior to the Finals, points are reset for each of theFinals qualifiers according to the schedule shown in the “Finals Reset” columnof the “Playoffs Points” section. Points are awarded during the Finals accordingto the schedule shown in the last column of Table 1. (Note that this is exactlythe same distribution of points awarded to finishing positions 1-30 during the firstthree rounds of the Playoffs.) The points reset was put into place after the second

Table 1: FedExCup Points Distribution and Reset Schedule. Points for regularevents during regular season decreased by 0.02 points per finishing position past 70. * =includes THE PLAYERS Championship.

Regular Season Points Playoffs PointsFinishing Regular WGC Additional FinalsPosition Events Events Majors* Events Rounds 1-3 Reset Finals

1 500 550 600 250.0 2,500 2,500 2,5002 300 315 330 150.0 1,500 2,250 1,5003 190 200 210 95.0 1,000 2,000 1,0004 135 140 150 70.0 750 1,800 7505 110 115 120 55.0 550 1,600 5506 100 105 110 50.0 500 1,400 5007 90 95 100 45.0 450 1,200 4508 85 89 94 43.0 425 1,000 4259 80 83 88 40.0 400 800 400

10 75 78 82 37.5 375 600 37511 70 73 77 35.0 350 480 35012 65 69 72 32.5 325 460 32513 60 65 68 30.0 300 440 30014 57 62 64 28.5 285 420 28515 56 59 61 28.0 280 400 28016 55 57 59 27.5 275 380 27517 54 55 57 27.0 270 360 27018 53 53 55 26.5 265 340 26519 52 52 53 26.0 260 320 26020 51 51 51 25.5 255 310 25521 50 50 50 25.0 250 300 25022 49 49 49 24.5 245 290 24523 48 48 48 24.0 240 280 24024 47 47 47 23.5 235 270 23525 46 46 46 23.0 230 260 23026 45 45 45 22.5 225 250 22527 44 44 44 22.0 220 240 22028 43 43 43 21.5 215 230 21529 42 42 42 21.0 210 220 21030 41 41 41 20.5 205 210 205. .. .

66 5 5 5 2.5 2567 4 4 4 2.0 2068 3 3 3 1.5 1569 2 2 2 1.0 1070 1 1 1 0.5 5

71-75 576-85 4

year of FedExCup competition to ensure that no single player could have won theFedExCup prior to the Finals event and also to give each participant in the Finals amathematical chance of winning the FedExCup.

4.2 What We Evaluate

We limit our analysis of selection efficiency to the 125 players who qualify for theFedExCup Playoffs. Using all six efficiency measures, we evaluate the efficiency ofthe regular season points distribution system.8 For this same group of 125 players,we then evaluate each of the six efficiency measures at the end of each round ofthe Playoffs in an attempt to determine if each successive round of the Playoffsimproves selection efficiency for this group of 125 players.

We also evaluate selection efficiency over all six measures at the end ofevery Playoffs round, but only for those players who qualify to play in each round.Our concern is whether the points system improves efficiency incrementally at theend of each round for remaining participating players.

5 Statistical Foundations

5.1 Data

Our data, provided by the PGA TOUR, covers the 2003-2010 PGA TOUR seasons.It includes 18-hole scores for every player in every stroke play event sanctionedby the PGA TOUR for years 2003-2010 for a total of 151,954 scores distributedamong 1,878 players. We limit the sample to players who recorded 10 or more18-hole scores. The resulting sample consists of 148,145 observations of 18-holegolf scores for 699 PGA TOUR players over 366 stroke-play events. Most of theomitted players are not representative of typical PGA TOUR players. For example,711 of the omitted players recorded just one or two 18-hole scores.9

5.2 Player Skill Estimation Model

We employ a variation of the Connolly and Rendleman (2008) model to estimatetime-varying player skill and random variation in scoring for a group of profes-

8We take the regular season points distribution schedule as shown in Table 1 as given as well asthe number of players who qualify for the Playoffs and each of its stages.

9Generally, these are one-time qualifiers for the U.S. Open, British Open and PGA Championshipwho, otherwise, would have little opportunity to participate in PGA TOUR sanctioned events.

sional golfers representative of PGA TOUR participants during the eight-year pe-riod 2003-2010. As in Connolly and Rendleman (2008), we employ the cubicspline methodology of Wang (1998) to estimate skill functions and autocorrelationin residual errors for players with 91 or more scores. We employ a simpler linearrepresentation without autocorrelation, as in Connolly and Rendleman (2012), forplayers with 10 to 90 scores over the full sample period.10 Simultaneously, we esti-mate fixed course effects and random round-course effects. We note that the modeldoes not take account of specific information about playing conditions (e.g., adverseweather as in Brown (2011), pin placements, morning or afternoon starting times,etc.) or, in general, the particular conditions that could make scoring for all playersmore or less difficult, when estimating random round-course effects. Nevertheless,if such conditions combine to produce abnormally high or low scores in a given18-hole round, the effects of these conditions should be reflected in the estimatedround-related random effects.11

Mathematically, the model takes the following form:

s = P f (•)+Cβ +Rb. (4)

In (4), s = (s1, ...,sm)′ is an N = 148,145 vector of 18-hole scores subdivided into

player groups, i, with ni scores per player i and m = 699. Within each player group,the scores are ordered sequentially, with si = (si 1, ...,si ni)

′ denoting the vector ofscores for player i ordered in the chronological sequence gi = 1, 2, ...,ni. We referto gi as the sequence of player i’s “golf times.” The usual error term is part of f (•).

P f (•) captures time variation in skill for each of the m golfers in the sample.P is a matrix that identifies a specific player associated with each score. f (•) =( f1 (•) , ..., fm (•))′ is a vector of m player-specific skill functions described in moredetail in the next subsection.

In (4), the N×109 matrix, C, identifies the 109 individual courses on whichtournament competition is conducted during our sample period, and β is a vectorof estimated fixed course effects. The N× 1,673 matrix R identifies round-courseinteractions associated with each score, defined as the interaction between a regular18-hole round of play in a specific tournament and the course on which the round

10We established the 91-score minimum in Connolly-Rendleman (2008) as a compromise be-tween having a sample size sufficiently large to employ Wang’s (1998) cubic spline model (whichrequires 50 to 100 observations) to estimate player-specific skill functions, while maintaining asmany established PGA TOUR players in the sample as possible. For further details see Connollyand Rendleman (2012).

11Interacted random round-course effects, with similar justification, are also estimated in Berry,Reese and Larkey (1999) and Berry (2001). We also estimate random round-course effects in ouroriginal 2008 model. However, we believe that the course component of a potential round-courseeffect is better modeled as fixed than random.

is played. The vector of estimated random effects associated with each of the dailyround-course interactions is denoted by b.

5.3 Player Skill Functions

Our skill function, as applied to individual player i, takes two forms depending uponthe number of sample scores recorded by player i, and may be written as follows:

fi(•) = zi(gi)+θi

zi(•) = hi(gi) f or ni ≥ 91= li(gi) f or 10≤ ni ≤ 90. (5)

In (5), hi(gi) is Wang’s (1998) smoothing spline function applied to player i’sgolf scores, reduced by estimated fixed course and random round-course effects,over his specific golf times gi = 1, 2, ...,ni, for ni ≥ 91. (As noted above, gicounts player i’s golf scores in chronological order.) The vector of potentially au-tocorrelated random errors associated with player i’s spline fit is denoted θi withθi = (θi 1,θi 2, ....,θi ni)

′ ∼ N(0,σ2i W−1

i ) and σ2i unknown. In Wang’s model, W−1

iis a covariance matrix whose form depends on specific assumptions about depen-dencies in the errors, for example first-order autocorrelation for time series, com-pound symmetry for repeated measures, etc. (See Wang (1998, p. 343) for furtherdetail.) li(gi), applied to players for whom 10≤ ni ≤ 90, is a simple linear functionof player i’s golf times gi = 1, 2, ...,ni. We note that for the 372 players for whomwe estimate skill using Wang’s smoothing spline model, 158 of the spline fits turnout to be linear.

For any given player, i, f = ( f1, .... fn)′ denotes the vector of the player’s

n sequentially ordered golf scores, reduced by estimated fixed course and randomround-course effects. If n ≥ 91, h = (h(t1) , ....h(tn))

′ denotes a vector of valuesfrom the player’s estimated cubic spline function evaluated at points t1, ...., tn, whichrepresent golf times g = 1, 2, ...,n scaled to the [0, 1] interval. If 10 ≤ n ≤ 90,l = (l (t1) , ....l (tn))

′ denotes a vector of values from the player’s estimated linearskill function evaluated at points t1, ...., tn.

In Wang’s model, as applied here, for each player, one chooses the cu-bic spline function h(t), the smoothing parameter, λ , and the first-order auto-correlation coefficient, φ , embedded in W that minimizes 1

n (f−h)′W(f−h) +

λ

1∫0

(d2h(t)/dt2)2 dt. “The parameter λ controls the trade-off between goodness-

of-fit and the smoothness of the [spline] estimate” (Wang (1998, p. 342)).

In (6) below, we break θi into two parts, ϕi +ηi, where ϕi represents theautocorrelated component of θi and ηi is assumed to be white noise.

θi = ϕi +ηi, withϕi = 0 f or 10≤ ni ≤ 90 (6)

Inasmuch as there are likely to be gaps in calendar time between some adja-cent points in a player’s golf time, it is unlikely that random errors around individualplayer spline fits follow higher-order autoregressive processes (i.e., AR(k), k > 1).Therefore, we assume that for players with at least 91 scores, each θi follows aplayer-specific AR(1) processes with first-order autocorrelation coefficient φi. Oth-erwise, we assume residual errors are independent.

As just described, when estimating player skill functions, we also obtainsets of player-specific residual scoring errors, denoted as θ and η . The θ errorsrepresent potentially autocorrelated differences between a player’s actual 18-holescores, reduced by estimated fixed course and random round-course effects, andhis predicted scores. The η errors represent θ errors adjusted for estimated first-order autocorrelation, and are assumed to be white noise. We refer to a player’sskill estimate at a given point in time as an estimate of his “neutral” score, sinceestimated fixed course effects and random round-course effects have been removed.

6 Simulation of FedExCup Competition

6.1 Simulation Design

We structure each of 10,000 simulation trials so that the composition of the playerpool is similar to what one might observe in a typical PGA TOUR season. As such,we do not include all 699 players from the statistical sample in each trial. Instead,the number of players per trial varies between 415 and 459 and reflects the actualnumber of players in the sample in each year, 2003-2010. We also structure thesimulations so that the simulated distributions of player skill (mean neutral scoreper round), scoring, and player tournament participation rates during the simulatedregular season closely approximate those observed in the actual sample. Simulationdetails are provided in the Appendix.

6.2 Simulation Results

Table 2 summarizes the simulation sample mean value of each of six efficiencymeasures at the end of the regular PGA TOUR season and at the end of each round

Tabl

e2:

Effi

cien

cyM

easu

resC

ompu

ted

fora

ll12

5Fe

dExC

upPl

ayof

fsQ

ualifi

ers.

Stag

e0

=en

dof

regu

lars

easo

n;St

age

1=

end

offir

stPl

ayof

fsro

und

(Bar

clay

s);S

tage

2=

end

ofse

cond

Play

offs

roun

d(D

euts

che

Ban

k);S

tage

3=

end

ofth

ird

Play

offs

roun

d(B

MW

);St

age

4N

R=

end

offin

alPl

ayof

fsR

ound

(TO

UR

Cha

mpi

onsh

ip)

with

nopo

ints

rese

t;St

age

4R

=en

dof

final

Play

offs

Rou

nd(T

OU

RC

ham

pion

ship

)w

ithpo

ints

rese

t.“W

eigh

t”is

the

wei

ghtin

gof

FedE

xCup

poin

tsaw

arde

dpe

rto

urna

men

tfin

ishi

ngpo

sitio

ndu

ring

the

Play

offs

rela

tive

toth

ose

awar

ded

duri

ngth

ere

gula

rsea

son.

Eac

hef

ficie

ncy

mea

sure

refle

cts

the

mea

nva

lue

com

pute

dov

er10

,000

sim

ulat

ion

tria

ls,1

,250

pery

earf

orea

chye

ar20

03-2

010.

Fore

ach

effic

ienc

ym

easu

reex

cept

the

first

,a

low

erva

lue

isbe

tter.

The

best

valu

epe

rsta

geis

show

nin

bold

.V

alue

sin

each

stag

eth

atar

eno

tsig

nific

antly

infe

rior

stat

istic

ally

ina

one-

side

dte

stat

the

0.05

leve

lrel

ativ

eto

the

optim

alva

lue

for

the

sam

est

age

are

show

nin

italic

s.T

heop

timal

stag

e-4

valu

ew

ithno

poin

tsre

seti

sal

way

ssi

gnifi

cant

lybe

ttera

tthe

0.05

leve

ltha

nth

eop

timal

valu

ew

itha

poin

tsre

sete

xcep

twhe

reno

ted

with

anas

teri

sk,i

nw

hich

case

the

optim

alva

lue

with

rese

tis

bette

r.

Pane

lA:F

irst

Plac

eR

ate

ofB

estP

laye

rPa

nelD

:Mea

nSq

uare

dR

ank

Err

or(D

eflat

ed)

Stag

eSt

age

Wei

ght

01

23

4N

R4

R0

12

34

NR

4R

10.

396

0.42

00.

446

0.46

80.

498

0.42

20.

872

0.82

60.

794

0.77

70.

775

0.77

32

0.39

60.

438

0.48

40.

526

0.56

30.

439

0.87

20.

805

0.76

00.

737

0.73

40.

733

30.

396

0.44

70.

501

0.54

70.

584

0.44

40.

872

0.79

40.

744

0.71

80.

715

0.71

44

0.39

60.

441

0.49

60.

547

0.58

40.

444

0.87

20.

791

0.73

70.

711

0.70

70.

707

50.

396

0.41

60.

481

0.53

70.

578

0.44

70.

872

0.79

10.

737

0.71

00.

706

0.70

6*

Pane

lB:M

ean

Skill

ofPl

ayer

inFi

rstP

lace

Pane

lE:M

ean

Squa

red

Skill

Err

or(D

eflat

ed)

Stag

eSt

age

Wei

ght

01

23

4N

R4

R0

12

34

NR

4R

169

.009

68.9

2668

.853

68.7

9068

.725

68.8

250.

774

0.72

50.

693

0.67

20.

661

0.66

92

69.0

0968

.873

68.7

6368

.670

68.6

0068

.767

0.77

40.

704

0.65

90.

630

0.61

80.

632

369

.009

68.8

4768

.728

68.6

3768

.571

68.7

500.

774

0.69

60.

647

0.61

60.

603

0.62

04

69.0

0968

.850

68.7

3468

.640

68.5

7568

.750

0.77

40.

696

0.64

70.

615

0.60

10.

617

569

.009

68.8

8068

.759

68.6

5868

.586

68.7

500.

774

0.70

30.

653

0.61

90.

604

0.61

9

Pane

lC:M

ean

Skill

Ran

kof

Play

erin

Firs

tPla

cePa

nelF

:Wei

ghte

dSq

uare

dSk

illE

rror

(Defl

ated

)St

age

Stag

eW

eigh

t0

12

34

NR

4R

01

23

4N

R4

R1

10.6

258.

242

6.72

45.

864

5.41

65.

426

0.65

60.

570

0.50

90.

465

0.43

30.

456

210

.625

7.19

65.

504

4.63

34.

253

4.47

80.

656

0.52

80.

444

0.38

60.

351

0.40

23

10.6

256.

687

5.03

34.

196

3.84

64.

108

0.65

60.

505

0.41

60.

358

0.32

60.

383

410

.625

6.42

24.

861

4.05

23.

702

3.96

00.

656

0.49

50.

411

0.35

30.

321

0.37

85

10.6

256.

298

4.79

74.

004

3.63

13.

879

0.65

60.

499

0.41

60.

358

0.32

20.

377

of the Playoffs evaluated with respect to the 125 players who qualify for the Play-offs in simulated competition. Each of the six panels of Table 2 represents one ofthe six selection efficiency measures. For the efficiency measure shown in PanelA, “First Place Rate of Best Player,” higher values are better. In the remaining fivepanels, lower values indicate greater efficiency. Each efficiency measure is evalu-ated using a Playoffs points to regular season points weighting ratio that varies from1 to 5. All efficiency measures shown in Panels D-F are the values computed fromEquations 1-3, deflated by the corresponding variance of the variable whose errorwe are attempting to estimate. Efficiency for Playoff round 4 is evaluated without apoints reset (NR) and with a reset (R). The points reset schedule is that given in Ta-ble 1 times the weighting ratio divided by 5. We denote the end of the PGA TOURregular season as “Stage 0” and the end of Playoffs rounds 1-4 as Stages 1 through4, respectively.

In each panel, the best efficiency value is shown in bold for each stage 1-4. Except for the few efficiency measures shown in italics, the measures shown inbold are statistically superior in a one-sided test at the 0.05 level relative to all othervalues shown for the same stage.12

Regardless of the points weighting or efficiency measure, efficiency im-proves during each stage of competition during Playoffs rounds 1-3 and from round3 to round 4 when there is no points reset. In Panel C (mean skill rank of player infirst place) and Panel D (mean squared rank error), efficiency measures improvefrom round 3 to round 4 when points are reset after round 3 for all weightingschemes. In Panels E and F, the same is true for weighting scheme 1 only. Thus,from the standpoint of pure mathematical efficiency (i.e., ignoring other factors thatmight argue for a reset), the evidence is mixed as to whether the 125 players inthe Playoffs are ordered more efficiently with a reset after the third round of thePlayoffs than after the final round.

The optimal Playoffs points weight seems to vary by selection efficiencymeasure, but in general, optimal weighting falls between 3 and 5. While in themajority of cases (19 or 30) the present Playoffs points weight of 5 may not beoptimal, we argue in Section 6.3 that these differences may have little practicalsignificance.

We note that with the exception of Panel D (mean squared rank error), effi-ciency measures as of the end of Stage 4 tend to be more favorable without a resetgoing into the final round. (Values in the Stage 4 no reset (NR) column tend tobe higher than corresponding values in the Stage 4 reset (R) column in Panel Aand lower in Panels B-F). Although not shown in the table, with two exceptions

12We estimate statistical significance using 1,000 bootstrap samples drawn from the simulateddata generated by 10,000 trials.

(Panel C, weight 1 and Panel D, all weights) differences between no-reset and cor-responding reset values are favorable to no reset, and differences are statisticallysignificant.

The results summarized above in Table 2 evaluate efficiency for the entire125-player Playoffs field at the end of each Playoffs stage. It is also informativeto evaluate efficiency at the end of each Playoffs stage for only those players whoactually participate in each stage. Table 3, organized similarly to Table 2, summa-rizes ratios of mean before and after efficiency measures at the end of each Playoffsstage for stage participants only. Stated differently, the focus in Table 3 is on selec-tion efficiency computed incrementally for each round of the Playoffs for just thoseplayers participating in each specific round of the Playoffs.

Each value shown in the table reflects the value over 10,000 simulation trialsof the ratio of the mean efficiency measure computed at the end of the stage to themean of the same efficiency measure computed at the beginning of the stage forstage participants only. In all but Panel A, a table entry less than 1 representsan improvement in efficiency from one stage of Playoffs competition to the next.Again, values shown in bold correspond to the best values per stage. Unless shownin italics, all other values in the same stage are are larger than the value shown inbold in more than 95% of bootstrap samples (except in Panel A, where we employa less-than comparison).

As in Table 2, optimal points weightings tend to vary by efficiency measure.Nevertheless, a weighting of 3 to 5 appears to produce the best efficiency measures,but in some cases, the current weight of 5 appears to be optimal. As in Table 2,values shown in Table 3 assume a constant playoffs weighting starting in stage 1.Therefore, it is inappropriate to infer from the Table 3 entries that efficiency can beimproved by changing the weights from one Playoffs stage to another.

For the first two efficiency measures, the first place rate of the best player(Panel A) and the mean skill level of the player in first place (Panel B), selectionefficiency decreases from Stage 3 to Stage 4 with a reset for all weighting schemes.For the other four efficiency measures (Panels C-F), we generally observe morefavorable efficiency outcomes when there is no reset compared with a reset (in 18of 20 cases).

6.3 Practical Significance

Despite finding optimal values for the Playoffs points weighting and the decisionwhether to reset FedExCup points going into the final Playoffs round, we believethat the practical differences are insignificant among efficiency outcomes based onoptimal tournament design and those based on non-optimal design over the range

Table3:

Ratios

ofM

eanB

eforeand

After

Efficiency

Measures

forStage

ParticipantsO

nly.Stage

1=

endof

firstPlayoffs

round(B

arclays);Stage2

=end

ofsecondPlayoffs

round(D

eutscheB

ank);Stage3

=end

ofthirdPlayoffs

round(B

MW

);Stage

4N

R=

endof

finalPlayoffs

Round

(TOU

RC

hampionship)

with

nopoints

reset;Stage

4R

=end

offinal

PlayoffsR

ound(TO

UR

Cham

pionship)with

pointsreset.“W

eight”is

thew

eightingofFedE

xCup

pointsaw

ardedpertournam

entfinishingposition

duringthe

Playoffsrelative

tothose

awarded

duringthe

regularseason.Each

efficiencym

easurereflects

them

eanvalue

oftheratio

oftheefficiency

measure

computed

attheend

ofthestage

tothe

efficiencym

easurecom

putedatthe

beginningofthe

stageforstage

participantsonly

over10,000sim

ulationtrials,1,250

peryearforeachyear2003-2010.Foreach

efficiencym

easureexceptthe

first,a

lower

valueis

better.T

hebestvalue

perstage

isshow

nin

bold.U

nlessshow

nin

italics,allothervalues

inthe

same

stageare

arelarger

thanthe

valueshow

nin

boldin

more

than95%

ofbootstrap

samples

(exceptinPanelA

,where

we

employ

aless-than

comparison).T

heoptim

alstage-4value

with

nopoints

resetislow

er(higherinPanelA

)inm

orethan

95%ofbootstrap

samples

thanthe

optimalvalue

with

apoints

resetexceptwhere

notedw

ithan

asterisk.

PanelA:FirstPlace

Rate

ofBestPlayer

PanelD:M

eanSquared

Rank

Error(D

eflated)W

eightStage

1Stage

2Stage

3Stage

4N

RStage

4R

Stage1

Stage2

Stage3

Stage4

NR

Stage4

R1

1.0611.061

1.0511.062

0.8980.947

0.9440.940

0.9340.893*

21.106

1.1041.087

1.0700.833

0.9230.920

0.9160.919

0.9013

1.1301.119

1.0931.068

0.8130.911

0.9080.904

0.9150.908

41.113

1.1261.102

1.0660.811

0.9070.903

0.8990.913

0.9115

1.0511.156

1.1161.074

0.8320.907

0.9020.896

0.9120.913

PanelB:M

eanSkillofPlayerin

FirstPlacePanelE

:Mean

SquaredSkillE

rror(Deflated)

Weight

Stage1

Stage2

Stage3

Stage4

NR

Stage4

RStage

1Stage

2Stage

3Stage

4N

RStage

4R

10.999

0.9990.999

0.9991.001

0.9370.933

0.9280.915

0.9272

0.9980.998

0.9990.999

1.0010.909

0.9060.897

0.8980.957

30.998

0.9980.999

0.9991.002

0.8990.897

0.8880.894

0.9664

0.9980.998

0.9990.999

1.0020.900

0.8960.884

0.8930.962

50.998

0.9980.999

0.9991.001

0.9080.896

0.8820.888

0.953

PanelC:M

eanSkillR

ankofPlayerin

FirstPlacePanelF:W

eightedSquared

SkillError(D

eflated)W

eightStage

1Stage

2Stage

3Stage

4N

RStage

4R

Stage1

Stage2

Stage3

Stage4

NR

Stage4

R1

0.7760.809

0.8470.872

0.877*0.870

0.8740.879

0.8640.897

20.677

0.7600.821

0.8770.951

0.8050.821

0.8240.838

0.9673

0.6290.748

0.8180.882

0.9680.769

0.8040.816

0.8370.999

40.604

0.7520.820

0.8850.969

0.7550.809

0.8180.839

0.9955

0.5930.757

0.8230.879

0.9570.761

0.8140.819

0.8340.976

of possible Playoffs design schemes that we consider. The entries in Table 4, whichshow the best and worst efficiency outcomes from the corresponding panels of Ta-ble 2 along with efficiency outcomes where the outcomes in each regular seasonand Playoffs event are determined randomly, provide support for this view.13 (Weshow results for random outcomes using a Playoffs weight of 3 only. With randomtournament outcomes, the Playoffs weight has essentially no impact on any of theefficiency measures.)

The outcomes in Panels A-C, based on the efficiency measures of Ryvkinand Ortmann, are the most straightforward to interpret. Panel A shows the rate atwhich the best player in the competition wins. At the end of the competition, thebest and worst outcomes associated with non-random tournament competition fallbetween 58% and 42%. By contrast, with random tournament outcomes, the bestplayer wins less than 1% of the time. Panel B shows the mean skill level (mean neu-tral score per round) of the first-place finisher. The best and worst outcomes at theend of non-random competition fall between 68.57 and 68.83 compared with 70.67when tournament outcomes are determined randomly. Panel C shows the meanskill rank of the player who finishes the competition in first place. Here the best andworst outcomes at the end of non-random tournament competition fall between 3.63and 5.43 compared with 66.61 when outcomes are determined randomly. Clearly,on the basis of these three measures, (non-random) regular tournament competitiondramatically improves each of the three efficiency measures over what might haveotherwise been obtained with random tournament outcomes. Whether tournamentdesign is technically optimal appears to be of second-order importance relative tothe general structure of the competition itself.

Each of the efficiency measures in Panels D-F are the values computed fromEquations 1-3, respectively, deflated by the corresponding variance of the variablewhose error is being estimated. If we further divide the values in Panel D by 2,we obtain

(1−ρi, j(i)

), where ρi, j(i) is the Spearman rank order correlation of the

true skill ranks and tournament finishing positions. Best and worst values fromnon-random tournament competition fall between 0.706 and 0.775, which corre-spond to Spearman rank correlations of 0.647 and 0.613. By contrast, the 1.98value for the same efficiency measure under random competition corresponds to aSpearman rank correlation of 0.01, essentially zero. Clearly the tournament compe-tition, whether optimally designed in terms of Playoffs point weights and the reset,significantly improves the rank ordering of participating players.

If we divide the values in Panel E by 2, we obtain(1−βµ

), where βµ is

13We maintain exactly the same simulation design as described in the appendix, but instead ofbasing tournament outcomes on scores, throughout the regular season and Playoffs, outcomes arebased on random orderings of tournament participants, both before and after cuts.

Table4:

Efficiency

Measures

with

Random

Tournament

Outcom

es.Stage

0=

endof

regularseason;

Stage1

=end

offirst

Playoffsround

(Barclays);

Stage2

=end

ofsecond

Playoffsround

(Deutsche

Bank);

Stage3

=end

ofthird

Playoffsround

(BM

W);Stage

4N

R=

endof

finalPlayoffsR

ound(TO

UR

Cham

pionship)w

ithno

pointsreset;Stage

4R

=end

offinalPlayoffs

Round

(TOU

RC

hampionship)

with

pointsreset.

“Method”

isthe

method

byw

hichvalues

arecom

puted,with

“Optim

al”denoting

theoptim

alvaluefrom

thecorresponding

panelofTable2,“W

orst”denoting

thew

orstvaluefrom

thecorresponding

panelofTable2

and“R

andom”

denotingthe

valuew

henall

tournament

outcomes

aredeterm

inedrandom

lyusing

aPlayoffs

weight

of3.

Each

efficiencym

easurereflects

them

eanvalue

computed

over10,000sim

ulationtrials,1,250

peryearforeachyear2003-2010.Foreach

efficiencym

easureexceptthe

first,alow

ervalueis

better.

PanelA:FirstPlace

Rate

ofBestPlayer

PanelD:M

eanSquared

Rank

Error(D

eflated)Stage

StageW

eight0

12

34

NR

4R

01

23

4N

R4

RB

est0.396

0.4470.501

0.5470.584

0.4470.872

0.7910.737

0.7100.706

0.706W

orst0.396

0.4160.446

0.4680.498

0.4220.872

0.8260.794

0.7770.775

0.773R

andom0.006

0.0080.007

0.0060.007

0.0081.975

1.9771.979

1.9801.981

1.980

PanelB:M

eanSkillofPlayerin

FirstPlacePanelE

:Mean

SquaredSkillE

rror(Deflated)

StageStage

Weight

01

23

4N

R4

R0

12

34

NR

4R

Best

69.00968.847

68.72868.637

68.57168.750

0.7740.696

0.6470.615

0.6010.617

Worst

69.00968.926

68.85368.790

68.72568.825

0.7740.725

0.6930.672

0.6610.669

Random

70.69370.682

70.67570.677

70.67470.672

1.9711.971

1.9751.976

1.9761.976

PanelC:M

eanSkillR

ankofPlayerin

FirstPlacePanelF:M

oney-Weighted

SquaredSkillE

rror(Deflated)

StageStage

Weight

01

23

4N

R4

R0

12

34

NR

4R

Best

10.6256.298

4.7974.004

3.6313.879

0.6560.495

0.4110.353

0.3210.377

Worst

10.6258.242

6.7245.864

5.4165.426

0.6560.570

0.5090.465

0.4330.456

Random

67.56366.940

66.65366.658

66.60566.617

3.3663.336

3.3313.331

3.3303.329

the OLS slope coefficient associated with a regression of true player skill on skillimplied by tournament finishing position. Best and worst values in Panel E fall be-tween 0.601 and 0.669, which correspond to slope coefficients of 0.700 and 0.666.With random ordering, the 1.976 value for the same efficiency measure correspondsto a slope of 0.012, essentially zero. As in Panel D, the efficiency values from non-random competition, whether or not they reflect optimal tournament design, aresubstantially better than that obtained by a random ordering of players.

The values for the money-weighted squared skill error, shown in Panel F,are not as readily interpreted. Nevertheless, best and worst values associated withregular competition fall between 0.321 and 0.456 compared with 3.330 with randomtournament outcomes, suggesting that achieving exactly optimal tournament designis not critical.

Finally, as summarized in the Panel A sections of Tables 2-4, it is clear thatthe reset substantially reduces efficiency, as measured by the winning rate of thebest player. With a points reset, efficiency, as measured at the end of the competi-tion, is approximately the same as efficiency measured at the end of Stage-1 of thePlayoffs (see Table 2). Although it is not so easy for the average person followingprofessional golf to appreciate all the dimensions of the other efficiency measures,we suspect that most would have an intuitive feel for the skill levels of the bestplayers in golf.14 If the best players are not winning the FedExCup at a reasonablyhigh rate, and in particular Tiger Woods over the 2003-2010 period of our study, itisn’t unreasonable to expect that the competition could lose credibility among thosewho follow professional golf. Otherwise, we see little cost in changing Playoffspoints weights or the reset to satisfy PGA TOUR objectives that might not be easyto quantify.

6.4 Competitiveness and Excitement

It is clear from PGA TOUR Commissioner’s November 25, 2008 interview that thePGA TOUR strives to create a competitive and exciting playoffs system, buildingtoward a climactic finish, that will hold fan interest throughout. While aiming toreward players who have performed exceptionally well throughout the regular sea-son, the TOUR does not want the FedExCup winner to be determined prior to theFinals. Thus, the TOUR is seeking to achieve a fine balance between player per-formance during both the regular season and Playoffs. This balance may not be

14In fact, the TOUR publishes player scoring averages and scoring averages adjusted for fieldstrength throughout the PGA TOUR season. Although neither of these measures corresponds ex-actly to our mean neutral score, these averages, along with Official World Golf Rankings and otherperformance measures, make it relatively straightforward to identify the best golfers.

easily quantified in terms of the tournament selection efficiency measures we haveconsidered thus far.

By construction, the points reset ensures that the ultimate winner of theFedExCup cannot be determined until the completion of the final Playoffs event.In the same simulations that underlie the results summarized in Tables 2 and 3, thewinner of the competition would be determined prior to the Finals with probability0.289 under the present Playoffs points weighting scheme (weight = 5) if therewere no reset. With Playoffs points weights of 1, 2, 3 and 4, the probabilitieswould be 0.091, 0.140, 0.194, and 0.245, respectively. Clearly, if the FedExCupwinner were determined prior to the FedExCup Finals, there would be little faninterest in the final event, THE TOUR Championship, which occurs during themiddle of the professional and college football seasons. As such, we believe thatthe PGA TOUR would view these probabilities as being unacceptably high. (IfTiger Woods, or equivalently, a player with his scoring characteristics, is excludedfrom the simulations, the probabilities for Playoffs weights of 1-5 would be 0.048,0.063, 0.078, 0.095, and 0.119, respectively.15 More detailed results for simulationsthat exclude Woods are provided in the online appendix.)

Table 5 shows FedExCup winning percentages for 20 of the 125 players inthe Playoffs. In Panel A, the players are ordered by their seeding positions, 1-20, atthe beginning of the Playoffs. In Panel B, players are ordered by their skill rankings,1-20, in relation to the field of Playoffs participants. We include an online appendixas a supplement to this paper, which shows results in both panels for players in allpositions, 1-125.

Without a reset and without giving more weight to FedExCup points earnedduring the Playoffs relative to the regular season, Table 5 shows that the player whofinished the regular season in first place would win the FedExCup 79.6% of thetime. (If Woods is excluded from the simulations, this estimate is 78.1%.) More-over, without a reset, and with a Playoffs weight of 1, all players in seeding positions6-125 have less that a 1% probability of winning. (This is also the case if Woodsin not included.) Clearly the winning rate of 79.6% for the top-seeded player andthe very low winning rates associated with players seeded beyond position 5 areinconsistent with the TOUR’s objectives. Even with a Playoffs weight of 5, thetop-seeded player going into the Playoffs wins the FedExCup almost 50% of thetime with no reset, and no player beyond seeding position 10 has over a 1% chanceof winning. (Without Woods, 33.8%, and no player past position 14 has more thana 1% chance of winning.) By contrast, regardless of the weight, the winning per-

15Since Woods was such a dominant player during the 2003-2010 period on which our simulationsare based, projections based on the inclusion of a player of Woods’ skill may be misleading, at leastat the top position, for future periods where there may be no equivalently dominant player.

Tabl

e5:

FedE

xCup

Win

ning

Perc

enta

ges

byPl

ayof

fsSe

edin

gPo

sitio

nan

dR

elat

ive

Skill

Ran

k.B

ased

on10

,000

sim

ulat

ion

tria

ls(1

,250

per

year

for

year

s20

03-2

010)

.“W

eigh

t”is

the

wei

ghtin

gof

FedE

xCup

poin

tsaw

arde

dpe

rto

urna

men

tfin

ishi

ngpo

sitio

ndu

ring

the

Play

offs

rela

tive

toth

ose

awar

ded

duri

ngth

ere

gula

rse

ason

.In

Pane

lB,s

kill

rank

ings

are

rela

tive

toth

e12

5pl

ayer

sin

the

Play

offs

.Ifd

ispl

ayed

toa

prec

isio

nof

0.1%

,all

entr

ies

belo

wpo

sitio

n91

wou

ldeq

ualz

ero.

Pane

lA:P

ositi

onB

ased

onPl

ayof

fsSe

edin

gPo

sitio

nPa

nelB

:Pos

ition

Bas

edon

Rel

ativ

eSk

illR

anki

ngs

With

Res

etW

ithou

tRes

etW

ithR

eset

With

outR

eset

Posi

tion

Wei

ght=

1W

eigh

t=5

Wei

ght=

1W

eigh

t=5

Wei

ght=

1W

eigh

t=5

Wei

ght=

1W

eigh

t=5

137

.031

.779

.646

.542

.244

.849

.857

.82

16.5

13.8

11.4

14.6

14.9

13.6

23.9

15.9

39.

87.

73.

36.

68.

48.

37.

76.

24

6.7

5.2

1.6

4.3

5.9

5.5

4.5

3.7

55.

24.

01.

03.

24.

44.

22.

22.

66

4.0

3.2

0.5

2.1

2.6

2.6

1.9

1.8

73.

12.

60.

51.

72.

22.

21.

41.

48

2.3

2.2

0.3

1.6

2.2

2.2

1.1

1.4

92.

01.

90.

31.

31.

61.

51.

01.

010

1.7

1.6

0.2

1.0

1.5

1.3

0.7

0.8

111.

21.

30.

10.

81.

21.

00.

60.

712

1.2

1.3

0.1

0.7

1.1

1.0

0.4

0.7

131.

11.

30.

10.

80.

80.

90.

40.

514

0.8

1.1

0.1

0.7

0.8

0.9

0.4

0.5

150.

60.

90.

10.

50.

80.

80.

40.

416

0.6

0.9

0.0

0.4

0.8

0.8

0.3

0.4

170.

50.

80.

00.

50.

70.

60.

40.

418

0.6

0.9

0.1

0.6

0.7

0.6

0.3

0.3

190.

50.

80.

10.

50.

60.

60.

30.

320

0.4

0.7

0.1

0.5

0.5

0.5

0.2

0.3

centage rate of the top-seeded player is substantially lower with a reset, 31.7% to37.0%, but not so low that his performance during the regular season goes unre-warded, and many more players have a legitimate chance to win. (Without Woods,21.6% to 30.8%.)

From the “Stage 3” column of Table 2, we see that the most-highly skilledplayer in the competition is the number 1 Playoffs seed 47% to 55% of the time.Therefore, ignoring the remote possibility that this player would not make the Play-offs, the number 1 seed would be the most highly-skilled among the 125 Playoffsparticipants in 47% to 55% of FedExCup competitions.16 When this player is notthe top seed, he is very likely be near the top.

Panel B of of Table 5 shows that with a reset, the winning percentage rateof the most highly-skilled player in the Playoffs is greater than that of the number 1seed for both weighting schemes (weights of 1 and 5). This suggests that even if themost highly-skilled player is not the number 1 seed, he still has a reasonably highchance of winning. By contrast, without a reset and with a Playoffs weight of 1, themost highly-skilled player does not win as often as the number 1 seed, but he doeswin more often with a Playoffs weight of 5. (These same relationships do not holdwhen Tiger Woods is excluded from the simulations. See the online appendix.)

Table 6 shows percentage rates per FedExCup finishing position (throughfinishing position eight) for the top-eight-seeded players going into the Finals. (Theonline appendix shows the same results for all 30 players in the Finals over all 30possible finishing positions.) Panels A and B indicate that the percentage rates perfinishing position are hardly affected by the Playoffs weighting scheme when thereis a points reset going into the Finals. With either a weight of 1 or 5, the top 5seeds all have a reasonable chance to win, ranging from 5.3% to 44.1%. Althoughnot shown, winning rates for players in seeding positions 11-30 are all less than 1%for both weighting schemes. Also, we estimate that a player seeded in position 25or worse, the same as Bill Haas’ position going into the 2011 Finals, would winthe FedExCup only 0.19% of the time under the present system with a reset andPlayoffs weight of 5. Thus, Haas’ win was clearly a very rare event.

Panels C and D show percentage rates per finishing position without a reset.With a Playoffs weight of 1, there is little remaining uncertainty about the ultimatewinner and other top finishers; all are very likely to finish in the positions in whichthey started. This problem is mitigated somewhat with a Playoffs weight of 5.Nevertheless, the number 1 seed wins almost four out of five times. These sameresults tend to hold, but to a slightly lesser extent, when Tiger Woods is not includedin the FedExCup competition.

Although not entirely evident, since not all players and finishing positions

16Tiger Woods misses the Playoffs in 36 of 10,000 simulation trials.

Tabl

e6:

Fini

shin

gPo

sitio

nPe

rcen

tage

Rat

esby

Fina

lsSe

edin

gPo

sitio

n.B

ased

on10

,000

sim

ulat

ion

tria

ls(1

,250

per

year

fory

ears

2003

-201

0).“

Play

offs

Wei

ght”

isth

ew

eigh

ting

ofFe

dExC

uppo

ints

awar

ded

pert

ourn

amen

tfini

shin

gpo

sitio

ndu

ring

the

Play

offs

rela

tive

toth

ose

awar

ded

duri

ngth

ere

gula

rsea

son.

Pane

lA:W

ithR

eset

,Pla

yoff

sW

eigh

t=1

Pane

lC:W

ithou

tRes

et,P

layo

ffs

Wei

ght=

1

Fini

shin

gPo

sitio

nFi

nish

ing

Posi

tion

Seed

12

34

56

78

12

34

56

78

141

.637

.717

.92.

60.

20.

00.

00.

091

.48.

00.

60.

00.

00.

00.

00.

02

17.7

20.0

38.6

20.6

3.0

0.1

0.0

0.0

6.2

77.8

14.3

1.7

0.2

0.0

0.0

0.0

310

.37.

918

.539

.720

.43.

10.

20.

01.

58.

768

.618

.02.

90.

30.

00.

04

7.2

5.1

5.1

21.3

40.5

18.3

2.4

0.1

0.6

2.8

8.3

60.5

22.6

4.6

0.6

0.0

55.

53.

13.

15.

620

.538

.120

.43.

60.

21.

33.

68.

754

.225

.55.

60.

96

3.6

2.5

2.1

2.9

4.5

15.2

36.0

27.0

0.1

0.6

1.8

3.9

7.9

49.3

27.7

7.1

72.

71.

11.

91.

62.

83.

614

.937

.30.

10.

30.

82.

23.

87.

245

.030

.18

2.4

1.1

1.0

1.8

1.6

2.2

3.1

13.0

0.0

0.2

0.8

1.5

2.4

3.6

7.9

40.8

Pane

lB:W

ithR

eset

,Pla

yoff

sW

eigh

t=5

Pane

lD:W

ithou

tRes

et,P

layo

ffs

Wei

ght=

5

Fini

shin

gPo

sitio

nFi

nish

ing

Posi

tion

Seed

12

34

56

78

12

34

56

78

144

.136

.517

.02.

40.

10.

00.

00.

077

.318

.33.

80.

60.

10.

00.

00.

02

15.1

20.3

40.1

21.4

3.0

0.1

0.0

0.0

10.3

47.9

30.1

9.5

2.0

0.2

0.0

0.0

39.

57.

218

.040

.221

.73.

30.

10.

04.

49.

832

.934

.114

.83.

40.

60.

14

7.2

4.6

4.9

20.9

39.9

19.5

2.9

0.1

2.5

5.8

7.2

25.0

34.2

18.8

5.4

0.9

55.

33.

23.

05.

119

.337

.322

.44.

11.

33.

54.

26.

221

.132

.521

.67.

46

3.9

3.1

2.4

2.9

4.7

14.8

34.4

27.7

1.1

2.8

3.0

3.9

5.7

18.3

29.7

22.4

72.

91.

32.

41.

92.

83.

714

.236

.10.

82.

12.

32.

63.

35.

815

.827

.78

2.7

1.1

1.4

1.9

1.5

2.3

3.2

12.6

0.6

1.7

1.9

2.1

2.4

3.4

5.6

14.1

are shown, in panels A and B, except for the first and last seeds, each player’smost likely finishing position is worse than his initial seed. In Panel C, where thereis no points reset and points per Playoffs event are the same as those of regularseason events, the most likely finishing position for a player in seeding positions1-11 and 22-30 is his Finals seeding position, and for those in seeding positions12-19 the most likely finishing position is just one position worse. This suggeststhat a competition scheme with no reset and no differential weighting of Playoffsand regular season events leaves very little drama and potential for position changesin the Finals. By contrast, even without a reset, but with a Playoffs points weight of5, players in Finals seeding positions 3-29 are most likely to finish worse than theystarted. (Note, all these results are provided in the online appendix.)

Taken as a whole, we believe that the reset and the weighting of Playoffspoints more heavily than those for regular season events plays a critical role inmaintaining drama and potential fan interest throughout the Playoffs. From a pureefficiency standpoint, the reset tends to be suboptimal. Nevertheless, it is clearthat without a reset, the PGA TOUR could not satisfy its objectives of conducting ameaningful regular season leading to playoffs with a climactic finish that both holdsfan interest and has the potential to generate significant TV revenue.

7 Summary and ConclusionsIn this paper we introduce several new tournament selection efficiency measuresand apply these measures and several existing measures in a systematic evaluationof the selection efficiency of the FedExCup competition run by the PGA TOUR.Our new measures are defined on the full range of tournament outcomes, not justthe characteristics of the top finisher or most highly-skilled player. Using simu-lation, we evaluate the efficiency characteristics of specific alternative tournamentstructures.

Our simulations show that relative to random selection, every variation onthe FedExCup tournament selection method that we consider produces significantimprovements in selection efficiency. Beyond this result, perhaps the most impor-tant regularity is that the points reset impairs tournament efficiency for all efficiencymeasures except the mean squared ranking error. Despite the tendency for the resetto impair efficiency, an important aim of the reset is to ensure that the competitionis in doubt until the last moment. We argue that the reset and weighting of Play-offs points more heavily than those of regular season events are critical elementsin creating an exciting and dramatic set of Playoffs events. We acknowledge thatour analysis of excitement and drama is much less scientific than our more directmathematical assessment of tournament selection efficiency and believe that a more

formal development of this aspect of competition could be an interesting area forfuture research.

AppendixSimulation Methodology

A FedExCup Regular Season and Playoffs Competi-tion

In simulating the accumulation of FedExCup points during the regular PGA TOURseason and Playoffs, we make the following assumptions.

1. Between 415 and 459 players from our statistical sample participate for a full“regular season” prior to the FedExCup Playoffs in 35 4-round stroke playevents.17 The average number of players per event varies by simulation year,reflecting the actual average number of sample players per TOUR event peryear, ranging from 125 to 131. At the same time, the average number of ac-tual players per event ranges from 129 to 140 over the same period, with thedifference reflecting players who actually participated in PGA TOUR eventswho did not meet the 10-round minimum to be included in our statisticalsample. By excluding these players from individual tournament competition,we assume, implicitly, that they would have had little, if any, impact on in-dividual tournament outcomes and overall FedExCup standings. There is no“picking and choosing” of tournaments nor any qualifying requirements.18

The probability that any single player participates in a regular season eventreflects his actual participation frequency on the TOUR in the year of simu-lated tournament competition being. Further details on player sampling areprovided in Appendix Section B.

2. After the first two rounds of each regular season event, the field is cut to thelowest-scoring 70 players who then continue for two more rounds of tourna-ment play.19

1735 regular season events reflects the number of weeks of regular season PGA TOUR compe-tition prior to the FedExCup Playoffs during 2010. In three of the 35 weeks, two PGA TOURsanctioned events were played simultaneously, but no single player could have participated in thetwo events at the same time. Therefore, to simplify the simulations, we treat these weeks as if asingle event were held.

18A standard PGA TOUR event consists of 144 players. In the early and late parts of the PGATOUR season, regular events tend to be reduced in size to 144 players due to limited daylighthours. The TOUR also conducts a few “invitationals” with smaller fields, along with a few smallerfield select events, including tournaments in the World Golf Championship series. In addition, theMasters, one of the four “majors,” is a small field event, with 97 players participating in 2010.

19Generally, the lowest-scoring 70 players and ties make the cut in regular PGA TOUR events.As such, the number of players who make cuts will tend to exceed 70. However, by using fields of

3. FedExCup points are awarded for each tournament using the “PGA TOURRegular Season events points distribution” schedule shown in Table 1, as-suming each of the 35 tournaments is a regular PGA TOUR event rather thana “major,” a World Golf Championship event or an “alternate” event heldopposite tournaments in the World Golf Championship series.

4. At the end of the 35-event regular season, the Playoffs begin with the top 125players in FedExCup points participating in The Barclays, the first of fourPlayoffs events. The Barclays employs a cut after the first two rounds, withthe lowest-scoring 70 players advancing to the final two rounds. At the com-pletion of play, FedExCup points are added to those previously accumulatedfor each of the 125 Playoffs participants according to the schedule of Playoffspoints shown in Table 1.

5. After The Barclays, the top 100 players in FedExCup points advance to theDeutsche Bank Championship. The Deutsche Bank employs a cut after thefirst two rounds, with the lowest-scoring 70 players advancing to the finaltwo rounds. FedExCup points are added to those previously accumulated foreach of the remaining 100 Playoffs participants according to the schedule ofPlayoffs points shown in Table 1.

6. After the Deutsche Bank Championship, the top 70 players in FedExCuppoints advance to the BMW Championship, where there is no cut. FedExCuppoints are added to those previously accumulated for each of the remaining70 Playoffs participants according to the schedule of Playoffs points shownin Table 1.

7. After the BMW Championship, the top 30 players in FedExCup points ad-vance to THE TOUR Championship.

8. When simulating the present TOUR Championship structure, the numberof FedExCup points for the 30 participating players is reset according tothe reset schedule shown in Table 1. Players are then awarded additionalFedExCup points according to their finishing position in THE TOUR Cham-pionship, a four-round stroke play event with no cut, using the points distri-bution schedule for the Finals as shown in Table 1. The FedExCup winner isthe player who has earned the most FedExCup points, not necessarily THETOUR Championship winner.

players from our statistical sample, rather than all competing players, we feel that making a cut at 70players in simulated competition is reasonable. It is almost certain that no ties will occur with oursimulation methodology, but in the unlikely event that a tie does occur, the tie is broken randomly.

B Player SelectionPlayers are selected for regular season tournament participation using the proceduredescribed below. The procedure ensures that in a given year, each player partici-pates in simulated competition at a rate that mimics his participation rate in actualPGA TOUR competition. It also ensures that no player is assigned to participate inthe same event more than once in a given simulation trial. We note that this proce-dure will tend to create tournament fields that are less competitive than majors andother high-prestige selective events and more competitive than weaker field events.

1. A single year from our statistical sample, 2003-2010, is selected, with eachyear being selected exactly 10,000/8 = 1,250 times.

2. All sample players who actually participated in the selected year become theregular season player pool.

3. We illustrate our procedure for assigning players to tournaments in simulatedcompetition using the following hypothetical small-scale example. Assumethere are 8 tournaments in a given year, with an average of 6 players pertournament. Thus, there are 8× 6 = 48 tournament “slots” total. 12 play-ers participate in the 8 tournaments. Appendix Table 1 summarizes how theplayers are assigned to each tournament in a hypothetical PGA TOUR season.Assume that we want to simulate 1,250 trials of tournament play in a givenyear, with 5 events per trial, each of which averages 6 players per event,the same number of players per event as in the hypothetical example. First,we sample the 48 slots in 5× 1,250 = 6,250 groups of 6 players each, withreplacement. Next, we group the sampled slots by player and number thesimulated events 1 through 6,250, with the first five events in the sequencecorresponding to the first trial, the next five to the second trial, etc.Assume that player “A” appears in the simulation sample 2,000 times. Toensure that he is never assigned to the same tournament more than once,we select the tournament numbers, 1-6,250 at random without replacementa total of 2,000 times and assign the randomly-selected tournament numbersto player “A,” doing the same for each of the remaining players, B-L. Theseassignments then define the tournament fields for each simulated event.With this procedure, each player will be represented in the simulation samplein approximately the same proportion of total slots as he is represented in theactual data. The average number of players per event will be exactly 6, butthere is no guarantee that there will be exactly 6 players in each simulatedevent.Our actual sampling procedure differs from the example above only by scale.For example, in 2003, 427 sample players participated in 46 sample events

and took up 5,991 tournament slots, an average of 5,991/46 ≈ 130 playersper event. In simulated regular season play for 2003, we sample the 5,991slots with replacement in 35 tournament groups, 130 players each. We thengroup the sampled slots by player and randomly assign simulation tournamentnumbers 1-35,000, sampled without replacement, for each slot within eachplayer group. The mean number of players per event in our simulation of2003 competition is 130, with a standard deviation of 8 and range of 96 to166. In only 19 of 1,250×35 = 43,750 simulated events for year 2003 doesthe tournament size exceed 156, the standard maximum PGA TOUR fieldsize during periods of maximum daylight.

C Simulated 18-Hole ScoringThe following procedure is used to generate 18-hole scores for players who couldpotentially compete in a given randomly selected PGA TOUR season.

1. A single mean skill level (mean neutral score) for each player is selected atrandom from the portion of his estimated spline-based skill occurring in theselected PGA TOUR season, 2003-2010. This becomes the player’s meanskill level for the entire season.20

2. For each player k, a single θ residual is selected at random from among theentire distribution of nk θ residuals estimated in connection with his cubicspline-based skill function.

3. For each player k, 166 η residuals are selected randomly with replacementfrom among the entire distribution of nk η residuals estimated in connectionwith his cubic spline-based skill function.

4. Using the initial randomly selected θ residual, the vector of 166 randomly-selected η residuals, and player k’s first-order autocorrelation coefficient asestimated in connection with his cubic spline fit, a sequence of 166 estimatedθ residuals is computed.

5. The 166 θ residuals are applied to player k’s skill estimate to produce 166simulated random 18-holes scores. The first 10 scores are not used in simu-lated competition but, instead, are generated to allow the first-order autocor-relation process to “burn in.” The next 156 are the scores required for a playerwho might be selected to play in every regular season tournament and whomisses no cuts during the regular season (35× 4 = 140) or during the four

20We assume that the level of effort for each player throughout the entire regular season andPlayoffs is the same as that reflected, implicitly, in his estimated skill function.

rounds of the Playoffs (4×4 = 16). We note that it is highly unlikely that all156 scores would be used for any single player.

6. Starting with the 11th score, scores for each player k are applied in sequenceas needed to simulate scoring during the regular season and Playoffs.21

21Suppose player 1 makes the cut in the first regular season event and player 2 missed the cut. Ifboth are selected to play in the second regular season event, then simulated scoring in the secondevent will start with scores 15 and 13 for players 1 and 2, respectively.

ReferencesBerry, S. M. (2001), How Ferocious is Tiger?, Chance, 14:3, 51-56.Berry, S. M., C. S. Reese and P. D. Larkey (1999), “Bridging Different Eras in

Sports,” Journal of the American Statistical Association 94:3, 661-676.Brown, Jennifer (2011),“Quitters Never Win: The (Adverse) Incentive Effects of

Competing with Superstar,” Journal of Political Economy 119:5, 982-1013.Connolly, Robert A. and Richard J. Rendleman, Jr. (2008), “Skill, Luck and Streaky

Play on the PGA Tour,” The Journal of The American Statistical Association103:1, 74-88.

Connolly, Robert A. and Richard J. Rendleman, Jr. (2012), “What it Takes to Winon the PGA TOUR (If Your Name is “Tiger” or If It Isn’t),” Interfaces publishedonline May 31.

Hall, Nicholas G. and Chris N. Potts (2012), “A Proposal for Redesign of the FedExCup Playoff Series on the PGA TOUR,” Interfaces 42(2), 166-179.

PGA TOUR, “FedExCup Q&A with Commissioner Finchem,” November 25, 2008.(http://www.pgatour.com/2008/fedexcup/11/25/tuesday.transcript.finchem/index.html)

Ryvkin, Dmitry (2010), “The Selection Efficiency of Tournaments,” EuropeanJournal of Operational Research 206:3, 667-675.

Ryvkin, Dmitry. and Andreas Ortmann (2008), “The Predictive Power of ThreeProminent Tournament Formats,” Management Science 54:3, 492-504.

Taguchi, G. et. al. (2005), Taguchi’s Quality Engineering Handbook “Part III :Quality Loss Function.” John Wiley & Sons, NJ., 171-98.

Wang, Y. (1998), “Smoothing Spline Models with Correlated Random Errors,” TheJournal of The American Statistical Association 93:1, 341-348.

Appendix Table 1: Small-Scale Tournament Participation Example

By Event By PlayerSlot Player Event Slot Player Event

1 A 1 1 A 12 B 1 13 A 33 C 1 25 A 54 D 1 37 A 75 E 1 2 B 16 F 1 14 B 37 G 2 43 B 88 H 2 34 B 69 I 2 3 C 1

10 J 2 15 C 311 K 2 38 C 712 L 3 4 D 113 A 3 16 D 314 B 3 35 D 615 C 3 44 D 816 D 3 5 E 117 E 3 17 E 318 F 3 26 E 519 G 4 39 E 720 H 4 6 F 121 I 4 18 F 322 J 4 27 F 523 K 4 36 F 624 L 4 45 F 825 A 5 7 G 226 E 5 19 G 427 F 5 28 G 528 G 5 40 G 729 H 5 8 H 230 I 5 20 H 431 J 6 29 H 532 K 6 46 H 833 L 6 9 I 234 B 6 21 I 435 D 6 30 I 536 F 6 41 I 737 A 7 10 J 238 C 7 22 J 439 E 7 31 J 640 G 7 47 J 841 I 7 11 K 242 K 7 23 K 443 B 8 32 K 644 D 8 42 K 745 F 8 12 L 346 H 8 24 L 447 J 8 33 L 648 L 8 48 L 8