Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN...

45
Fall 2016 - OR699/SYST699 Final Presentation ESPN NFL Survivor Pool Optimization TEAM ESPN: Audrey Barry Lauren Moore Adam Patterson Dominique Strothers

Transcript of Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN...

Page 1: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Fall 2016 - OR699/SYST699

Final Presentation

ESPN – NFL Survivor Pool Optimization

TEAM ESPN:

Audrey Barry

Lauren Moore

Adam Patterson

Dominique Strothers

Page 2: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Agenda

Executive Summary

Background

Problem Statement

Requirements

Problem Solution

ENSPO System Graphical User Interface

Projections Database

Optimization Model

Validation

Conclusion

Recommendations for the Sponsor

Questions2

Page 3: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Executive Summary

The presentation documents the work completed by Team ESPN. Senior

ESPN Analyst Brian Burke challenged Team ESPN to explore the concept of NFL Survival Pools and create a tool

facilitating the optimal choices contestants should make when participating in a Survival Pool.

The ESPN NFL Survival Pool Optimizer (ENSPO) System was created by Team ESPN for this

project, which proved to be an overall success.

3

Page 4: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Background

The National Football League (NFL) is a professional

American football league consisting of 32 teams, with a 17-

week regular season running from the week after Labor

Day to the week after Christmas.

There are a number of popular contests associated with the

NFL. Typically, contestants pay to enter contests and are

given monetary rewards based on performance in the

contest.

Fantasy Football

Fanduel Football

Survivor Streaks

NFL Pick’em

NFL Survival Pools

4

Page 5: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Background

An NFL Survivor Pool is a contest centered on NFL game

predictions that takes place during an NFL regular season.

Each week of the regular season, a contestant in a pool

chooses 1 team. If that team loses, that contestant is out of the pool.

If that team wins, the contestant remains in the pool and proceeds to pick a

team during the next week.

Contestants can only choose a team once per season.

5

Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Audrey WAS

Lauren BAL

Adam MIA

Dom WAS

Sample Survival Pool Sheet – This would be filled out as Contestants pick teams each week.

Page 6: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Background

Pools can contain any number of contestants. The goal of

the contest is to survive the longest in the pool, i.e. be the

last contestant standing. Contestants must be diligent in

strategizing to win their pool. Some contestants may simply choose the strongest team each week, which is

commonly known as the “greedy” method.

Alternatively, contestants may have a more conservative approach and save

stronger teams for late in a season.

The sponsor challenged Team ESPN to create a tool to

serve as the foundation for a potential service that ESPN

could provide to NFL Survivor Pool contestants. The tool

would help contestants decide the optimal choice of team a

contestant should choose each week.

6

Page 7: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Problem Statement

In any given week of the season, what team

should an NFL Survival Pool contestant select to

optimize their chances of surviving the longest in

the pool? How would this strategy compare to

common heuristics, such as the “greedy” method

of survival?

7

ESPN currently does not host a service to

specifically address these questions.

Page 8: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Requirements

The Team will create an integer optimization model

with the goal to maximize the serial probability of

winning by surviving longest in an NFL Survival

Pool.

The model shall account for uncertainty regarding

the relative strength of each team, both currently

and in the future.

The model shall provide the optimum choice any

week of the season, given the teams a contestant

has selected so far.

The Team will work to demonstrate the superiority

of the model compared to common heuristics.8

Page 9: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The system created to for this project was named the “ESPN NFL Survival Pool Optimizer”, i.e. the

ENSPO System. The ENSPO System is composed of three primary components:

Graphical User Interface (GUI) – Executable file built

in R/Python.

Projections Database – Excel datasets with custom

VBA functions.

Optimization Model – Excel Model that uses the

“OpenSolver” plugin, custom VBA functions, and R

Code.

Problem Solution

9

Page 10: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The Graphical User Interface (GUI)

provides the ability for a contestant to

input information, and view optimization

results.

User Inputs:

Pool Size – Remaining number of Contestants in

the pool.

Pool Strength – Average probability pool

contestants will progress to the next week.

Number of Previously Selected Teams – Results in

dropdown lists to choose teams.

Graphical User Interface

10

Page 11: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Outputs:

Optimum Path, 2 Alternative Paths, Greedy Path

Graphical Comparisons of Paths

Graphical User Interface

11

Example: Optimum Path Probabilities

Example: Graphical Comparison of Paths. The y-axis corresponds to

cumulative probabilities of survival through the Week on the x-axis.

1 WEEK COMPLETED

2 WEEK COMPLETED

3 WEEK COMPLETED

4 WEEK COMPLETED

5 WEEK COMPLETED

6 WEEK COMPLETED

7 WEEK COMPLETED

8 WEEK COMPLETED

9 WEEK COMPLETED

10 WEEK COMPLETED

11 WEEK COMPLETED

12 WEEK COMPLETED

13 WEEK COMPLETED

14 Kansas City Chiefs 0.8123 0.8123

15 Minnesota Vikings 0.6472 0.5257

16 Buffalo Bills 0.7337 0.3857

17 Carolina Panthers 0.8673 0.3345

Week Team Selected

Team

Probability

Cumulative

Probability

Page 12: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The Projections Database (DB) is base source for weekly game probabilities.

Team ESPN chose to use Point Spreads as the

measure of game outcomes.

Point Spread: A type of “betting line” provided to guide bettors on what games to bet on. Point Spreads include a numerical value identifying the estimated difference in Points scored by the Favorite team, and the opponent known as the Underdog.

Example: Week 10 - Favorite: Redskins, Point Spread: - 21

The Redskins play the Cowboys in week 10. The Redskins are estimated to beat the Cowboys by 21 points.

Projections Database

12

Page 13: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Team ESPN chose to use Points Spread from the Vegas Odds as the measure of game outcomes for the current week.

Las Vegas sports books (Vegas Odds) dominate the NFL betting market.

Since Vegas Odds are published only for the current week, Team ESPN decided to use a combination of ESPN NFL Power Rankings, Elo Ratings, and SurvivorGrid projections to generate Point Spreads for future weeks.

ESPN NFL Power Rankings – A list of the 32 NFL Teams ordered from best to worst published each week.

Elo Ratings - 538 provides “Elo Ratings” for NFL Teams each week, via a simple system that estimates each team’s skill level using only the final scores and locations of each game.

SurvivorGrid – A public site built to help NFL Survivor Pool contestants by providing week by week, current and future game predictions.

Projections Database

13

Page 14: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

An Integer Optimization Model was developed

to evaluate the objective function:

Maximize ∑𝒘𝒊𝒋𝒔𝒊𝒋

𝑤𝑖𝑗 = The weight associated with Team i in Week j.

𝑠𝑖𝑗 = A binary decision variable (∀ 𝑠𝑖𝑗 ∈ [0, 1]).

If 𝑠𝑖𝑗= 1, then Team i is selected for Week j

𝑠𝑖𝑗 = 0 otherwise

Subject to the constraints:

Each team must only be selected at most once

Each week must have only one team selected

Optimization Model

14

Page 15: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The Optimization Model is the foundation of the Problem Solution.

The model ingests user inputs from the GUI, and Point Spreads from the

Projections Database.

For this project, Point Spreads were manually input into the model for

analysis purposes on week by week basis.

The model adjusts Point Spreads based on Uncertainty of Future

Weeks, and converts Points Spreads to win probabilities (reference

backup for information on the conversion formula).

Win probabilities are then adjusted to account for Team Consistency,

Divisional Games, the Dome Effect, and Pool Size/Strength to create 𝑤𝑖𝑗

values.

With 𝑤𝑖𝑗 values and the user inputs of previously selected teams, pool

size, and pool strength, the model determines the optimal solution for the

objective function along with two secondary paths and the “Greedy” path.

Optimization Model

15

Page 16: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

𝑤𝑖𝑗 values are based on the probability a team wins in the current

week and future weeks. The Optimization Model automatically

adjusts 𝑤𝑖𝑗 values to account for a number of factors.

Optimization Model

16

Example: Grouping of Weeks Out from Week 6

Current Week Week 7 Week 8 Week 9

Weeks Out 1 Week Out 2 Weeks Out 3 Weeks Out

Pool Size/Strength - The dynamics of being in a small vs large and strong vs weak pool are significant when strategizing in an NFL Survival Pool.

Uncertainty of Future Weeks - The further away a game is from the current week, the more uncertain the prediction for the outcome.

Example: Histogram of Accuracy of

1 Week Out Projections

Page 17: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Divisional Games - There are

eight divisions in the NFL,

and there are four teams

within each division. Over

time, teams likely become

more familiar with all teams in

their division, versus teams

outside their division.

Optimization Model

17

AFC EASTFW% NE MIA BUF NYJ

NE 0.000 0.833 0.800 0.833MIA 0.833 0.000 0.833 0.400BUF 0.800 0.833 0.000 0.600

NYJ 0.833 0.400 0.600 0.000

Example: Analysis of AFC East Division

Dome Effect - Teams with indoor (dome) stadiums have a difficult time playing in cold climates in December.

Team Consistency - Some teams are historically more predictable/unpredictable than others.

Page 18: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The Optimization Model dynamically updates how win

probabilities are adjusted throughout a season.

At the end of each week, an

administrator inputs the outcomes of

games. Based on these outcomes, wij

values could change based on

Divisional Games outcomes and

Team Consistency.

Optimization Model

18

Ex – Team Consistency: The Jets consistently do worse than spread projections. In the most recent week, the Jets beat the spread by 21 points. The model would then change how future wij values for the Jets is adjusted.

Ex – Divisional Games: Based on historical data, the Patriots beat the Bills 80% of matchups. In the most recent week, the Patriots lost to the Bills. The model would then change how future wij values for next game the Patriots play the Bills.

Example: Analysis of Consistency of the Jets

Page 19: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Team ESPN validated the ENSPO System with data from the 2015 season, and compared these results to common

heuristics.

Results using the “greedy” method were first compiled.

This entailed determining how far a contestant would survive by choosing the strongest team each week based on the Vegas Odds for the current week.

Team ESPN desired to compare to an additional common heuristic,

which was labeled as “conservative”.

Results for the conservative method simulate a contestant choosing the Favorite team with the lowest Point Spread based on the Vegas Odds for the current week.

To demonstrate the effectiveness of the ENSPO System, Team ESPN

used their defined projections and generated results for a Survival Pool

with an initial size of 100 contestants.

Validation

19

Page 20: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Validation - 2015

Week 1 2 3 4 5

Team NE NO SEA GB KC

Outcome ✓ ✗ ✓ ✓ ✗

Week 1 2 3 4 5

Team NE PHI SEA GB KC

Outcome ✓ ✗ ✓ ✓ ✗

Week 1 2 3 4 5

Team NE BAL SEA GB KC

Outcome ✓ ✗ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11

Team NE NO SEA GB ATL NYJ ARI BAL CIN PIT PHI

Outcome ✓ ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11

Team NE MIA SEA GB ATL NYJ ARI BAL CIN PIT PHI

Outcome ✓ ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11

Team NE BAL SEA GB ATL NYJ ARI HOU CIN PIT PHI

Outcome ✓ ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Greedy Path 1 – Contestant eliminated week 2. Ignoring week 2, the

contestant is eliminated week 5.

Greedy Path 2 – Contestant chose second highest point spread in week 2, and is

eliminated week 2. Ignoring week 2, the contestant is eliminated week 5.

Greedy Path 3 – Contestant chose third highest point spread in week 2, and is

eliminated week 2. Ignoring week 2, the contestant is eliminated week 5.

ENSPO Path 1 - Contestant eliminated week 2. Ignoring week 2, the contestant is eliminated week 11.

ENSPO Path 2 - Contestant chose alternative path 1 in week 2 and is eliminated week 2. Ignoring week 2, the contestant is eliminated week 11.

ENSPO Path 3 - Contestant chose alternative path 2 in week 2 and is eliminated week 2. Ignoring week 2, the contestant is eliminated week 11.

Disregarding week 2, elimination for the greedy paths happens week 5, and in week 11 for the ENPSO paths. 20

Page 21: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Validation - 2015

Week 1 2 3 4 5 6 7 8

Team NE NO SEA GB NYJ NYG ARZ ATL

Outcome ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11 12

Team NE NO SEA GB ATL NYJ ARI BAL CIN PIT CAR CLE

Outcome ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Team NE NO SEA GB ATL NYJ ARI BAL CIN PIT CAR HOU DEN KC MIN DET IND

Outcome ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Greedy Path 1 – Contestant eliminated week 8.

Greedy Path 2 – Contestant chose

second highest point spread in week 8,

and is eliminated week 12.

Greedy Path 3 – Contestant chose

second highest point spread in week

12, and is eliminated week 12.

ENSPO Path 1 - Contestant eliminated week 12.

ENSPO Path 2 - Contestant chose alternative path 1 in week 12 and is eliminated week 12.

ENSPO Path 3 - Contestant chose alternative path 2 in week 12 and survives the regular season!!!

Week 1 2 3 4 5 6 7 8 9 10 11 12

Team NE NO SEA GB ATL NYJ ARI BAL CIN PIT CAR JAX

Outcome ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11 12

Team NE NO SEA GB NYJ NYG ARZ LA CIN PIT CAR CLE

Outcome ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

Week 1 2 3 4 5 6 7 8 9 10 11 12

Team NE NO SEA GB NYJ NYG ARZ LA CIN PIT CAR JAX

Outcome ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

These paths demonstrate the outcomes if the Saints (NO) had won in week 2, the greedy paths change

week 5 selections, and ENSPO paths change week 11 selections.

21

Page 22: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Validation - 2015

Week 1 2

Team IND TEN

Outcome ✗ ✗

Week 1 2 3

Team KC TEN MIA

Outcome ✓ ✗ ✗

Week 1 2 3

Team KC BUF MIA

Outcome ✓ ✗ ✗

Conservative Path 1 – Contestant eliminated week 1. Ignoring week 1, the

contestant is eliminated week 2.

Conservative Path 2 – Contestant chose second lowest favorite point spread in week

1, and is eliminated week 2. Ignoring week 2, the contestant is eliminated week 3.

Conservative Path 3 – Contestant chose third lowest favorite point spread in week 2,

and is eliminated week 2. Ignoring week 2, the contestant is eliminated week 3.

Don’t be conservative in an NFL Survival Pool. 22

Recall that results for the conservative method simulate a contestant

choosing the Favorite team with the lowest Point Spread based on the

Vegas Odds for the current week.

Don’t be conservative in an NFL Survival Pool.

Page 23: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Conclusion

23

The ENSPO System significantly outperformed

common heuristics, i.e. the “greedy” and

“conservative” approaches.

The ENSPO System is a fully integrated model that

involved a variety of scholarly efforts:

Operations Research

Systems Engineering

Project Management

Computer Science

Statistics

History (NFL, Gambling)

Page 24: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Recommendations for the Sponsor

Explore fine details of the ENSPO System to determine what

features would best benefit the tool supported by ESPN.

Automate manual features of the ENSPO System, i.e.

Automatic input of recent game outcomes

Automatically retrieve Point Spread projections

Simulate ENSPO System with different sources such as Point

Spreads derived from the FPI, and tweak the model based on

these simulations.

Consider running similar analysis on divisional games, team

consistency, and uncertainty on ESPN derived Point Spreads and

adjust how these factors are incorporated in the solution.

Integrate the GUI code into a web based interface.

Engage with Team ESPN for assistance/feedback!

24

Page 25: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Questions

25

Page 26: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Backup

Page 27: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Accounting for Divisional Games

Team ESPN conducted analysis on how teams in the same division perform

against each other, and how performance within divisions compared to point

spreads from the Vegas Odds. To execute this analysis, divisional game

outcomes from the 2012, 2013, and 2014 seasons were consolidated. With

these outcomes, the proportion each favorite team won against each

underdog team in their division was determined. If a favorite team won ≤

66.7%, the matchup of the teams was defined as a “Division Rival”.

Within the Optimization Model, if a matchup (current or future matchup) is

defined as a “Division Rival”, the associated wij values for each team in the

matchup are adjusted: wij = (Original wij + .5)/2

Example -

Fav. Win % NYG DAL PHIL WAS

NYG 0.000 0.667 0.667 0.500

DAL 0.667 0.000 0.500 0.667

PHIL 0.667 0.500 0.000 0.667

WAS 0.500 0.667 0.667 0.000

NFC EAST

Page 28: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Accounting for Team Consistency

Team ESPN analyzed how accurate Vegas Odds historically predict the

performance of individual teams in order to determine how predictable

teams outcomes are. To execute this analysis, game outcomes from the

2012, 2013, and 2014 seasons were consolidated, and compared against

the associated point spread from the Vegas Odds. Team were categorized

based on the deviation of their game outcomes from the point spread:

1. Inconsistent - The team seems to rarely meet the spread and no trend occurs

2. Consistently Bad - The team performs worse than spread projections more often

than not

3. Consistently Good - The team performs better than spread projections more often

than not

4. Consistent - The point spread accurately predicts outcomes for the team

The Optimization Model adjusts wij values based on the consistency

category of both teams in a matchup.

Page 29: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Accounting for Uncertainty

Uncertainty Used Projection Database (as described earlier in the

presentation) to create projected spreads for each team.

Looked at Weeks Out. i.e. how many weeks is this

projecting into the future and group those together.

Current Week Week 7 Week 8 Week 9

Weeks Out 1 Week Out 2 Weeks Out 3 Weeks Out

Week 6 Projections

Page 30: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Compare projections to actual spreads (historical) to see

if the team was an actual underdog or actual favorite.

If Projection is an Underdog Team:

Then if Actual < 0 then (Projection-Actual)

Gives (+) value

Subtract from projected spread

Else if Actual > 0 then (Projection- Actual)

Gives (- or +) value

Add value to projected spread

Project Underdog to.. 1 Week Out 5 Weeks Out 10 Weeks Out 16 Weeks Out

% Actual Underdog 80.83% 81.01% 74.55% 68.75%

% Actual Favorite 19.17% 18.99% 25.45% 31.25%

Accounting for Uncertainty

Page 31: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

1 Weeks Out 2 Weeks Out 3 Weeks Out 4 Weeks Out 5 Weeks Out 6 Weeks Out 7 Weeks Out 8 Weeks Out 9 Weeks Out 10 Weeks Out 11 Weeks Out 12 Weeks Out 13 Weeks Out 14 Weeks Out 15 Weeks Out 16 Weeks Out

Actual Underdog 1240 1118 1012 968 874 797 705 624 560 474 411 350 285 215 147 75

Actual Favorite 178 212 220 178 183 181 184 181 174 172 150 126 93 69 42 21

% Underdog 87.45% 84.06% 82.14% 84.47% 82.69% 81.49% 79.30% 77.52% 76.29% 73.37% 73.26% 73.53% 75.40% 75.70% 77.78% 78.13%

% Favorite 12.55% 15.94% 17.86% 15.53% 17.31% 18.51% 20.70% 22.48% 23.71% 26.63% 26.74% 26.47% 24.60% 24.30% 22.22% 21.88%

Min Value Underdog -9.03 -10.06 -10.60 -11.47 -11.09 -11.12 -11.40 -11.00 -10.99 -11.18 -9.69 -11.12 -10.77 -11.28 -11.44 -9.86

Max Value Underdog 9.61 11.07 13.00 10.15 9.95 10.39 10.22 11.16 11.00 10.25 13.00 8.77 8.91 11.00 11.50 11.50

Average Underdog 0.04 -0.01 -0.02 -0.17 -0.19 -0.32 -0.35 -0.37 -0.62 -0.77 -0.95 -1.22 -1.11 -0.77 -1.00 -0.79

Min Value Favoirite 1.01 0.51 0.56 0.98 0.58 0.61 1.01 1.06 2.00 1.50 1.58 0.91 1.18 2.90 1.34 2.50

Max Value Faorite 14.24 15.73 16.20 16.66 16.41 24.50 15.39 15.62 15.83 21.50 16.00 13.78 15.14 16.20 15.00 12.39

Average Favorite 5.36 5.58 5.60 6.16 6.31 6.28 6.13 6.40 5.81 7.03 7.01 7.03 7.41 7.71 7.68 7.47

Accounting for Uncertainty

Page 32: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Weeks Out Prob of Underdog Prob Of Favorite

Distribution Distribtuion

1 Week Out 87.45% Student T sd= 1.432 12.55% Beta α1= 4.0131 α2= 19.3223 a= 0.2587 b= 29.9278

2 Week Out 84.06% Logistic µ= -0.0337 s= 1.5254 15.94% Beta α1= 6.3187 α2= 49.328 a= 0 b= 56.9722

3 Week Out 82.14% Logistic µ= -0.0581 s= 1.6557 17.86% Beta α1= 4.9997 α2= 65.0405 a= 0 b= 82.5924

4 Week Out 84.47% Logistic µ= -0.1695 s=1.7411 15.53% Beta α1= 5.42789 α2= 63.1356 a= 0 b= 79.5563

5 Week Out 82.69% Logistic µ= -0.1915 s=1.8337 17.31% Logistic µ= 6.1091 α2= 1.4992

6 Week Out 81.49% Logistic µ= -0.3053 s=1.9189 18.51% Beta α1= 4.4741 α2= 74.2769 a= 0 b= 116.9676

7 Week Out 79.30% Logistic µ= -0.3244 s=1.9269 20.70% Beta α1= 3.1462 α2= 10.0251 a= 0.3393 b= 24.5622

8 Week Out 77.52% Logistic µ= -0.3411 s=1.9594 22.48% Beta α1= 4.9264 α2= 35.6215 a= 0.1418 b= 51.6698

9 Week Out 76.29% Logistic µ= -0.6393 s=2.0654 23.71% Gamma α=15.244 β=0.3808 γ=0

10 Week Out 73.37% Logistic µ= -0.7548 s=2.1074 26.63% Beta α1= 4.0762 α2= 25.7337 a= 0.42589 b= 48.772

11 Week Out 73.26% Logistic µ= -1.0010 s=2.1596 26.74% Beta α1= 4.6044 α2= 15.2439 a= 0.0634 b= 30.0045

12 Week Out 73.53% Logistic µ= -1.2774 s=2.2458 26.47% Logistic µ= 6.9272 s= 1.4959

13 Week Out 75.40% Normal µ= -1.0967 σ=4.0328 24.60% Logistic µ= 7.2787 s= 1.7172

14 Week Out 75.70% Beta α1= 6.7232 α2=12.2107 a=-14.4615 b=24.1701 24.30% Beta α1= 1.5444 α2= 3.1513 a= 2.7657 b= 17.9313

15 Week Out 77.78% Logistic µ= -1.2232 s=2.6796 22.22% Logistic µ= 7.4203 s= 1.9926

16 Week Out 78.13% Logistic µ= -1.2277 s= 2.8478 21.88% Beta α1= 11.2743 α2= 7.7803 a= 0 b= 16.0073

Actual Favorite Projection

Parameters Parameters

Actual Underdog Projection

Accounting for Uncertainty

Page 33: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

1 Week Out Underdog 14 Weeks Out Favorite

Accounting for Uncertainty

Page 34: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The Optimization Model ingests the parameters of Pool Size

and Pool Strength from the GUI in order to determine the

probability at least one contestant in a pool will survive through

the remaining weeks in a season.

These probabilities are created by first finding the probability

that zero contestants survive from the current week through all

remaining weeks of the season, then subtracting this value from

1. The binomial distribution is utilized to calculate the probability

that zero contestants in a pool survive a week: b(x; n, P) = nCx * Px * (1 – P)n – x

x = 0, n= number of weeks, P = Pool Strength^Pool Size

1 - b(x; n, P) = probability at least one player survives a week.

The week at which this value falls below .50 is considered the

“tipping point”. For all weeks leading up to this “tipping point”,

weights are not adjusted. Following the “tipping point”, weights

are decreased in an exponential fashion.

Accounting for Pool Size

Page 35: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Based on recommendation from the Sponsor, the Optimization

Model adjusts wij values for the last 5 weeks of the regular

season for teams whose home stadium is a Dome (i.e. indoors)

with playing in a cold weathered climate.

During the last five weeks of a season, wij values for Dome

Teams playing in a cold weather climate as adjusted:

If wij> .03, then wij = Original wij - .03

Else wij = .0001

The associated wij value for the matchup is updated

accordingly so that the total probability of each team in the

matchup winning the game still equals 1.

Accounting for Dome Effect

Page 36: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

TeamRankings.com aggregates a vast repository of sports stats

and data, publishes this data, and uses the data in sports

ratings systems. On this site, the history of Vegas Odds with

respect to how accurate they are in measures the actual of NFL

games is provided, going back through the 2003 season. (NFL

Odds History)

Using this history, Team ESPN used linear regression to come

up with the following formula to convert Point Spreads to win

probabilities: Probability of winning = -0.0293*(Point Spread) + 0.5.

The r^2 value for this formula was 0.96748, hence Team ESPN

felt confident in using the formula to convert Point Spread to

probabilities. This formula would be used to convert Point

Spreads, regardless of the source, into win probabilities within

the Problem Solution.

Probability Conversion

Page 37: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Probability Conversion

Page 38: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

The actors in the ENSPO System

are: Contestant - An NFL Survival Pool

Contestant

Administrator - An administrator,

such as an ESPN analyst, who

adjusts internal settings of the

ENPSO System.

Use Cases

The use cases for the ENSPO System are: Configure Season - An administrator configures the Optimization

Model at beginning of an NFL season.

Configure Week - An administrator configures the Optimization

Model at beginning of a week of an NFL.

Use Tool - The Contestant uses the ENSPO System to determine

what team to select in order to optimize their chances of surviving

the longest in the pool.

Page 39: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Use Cases

Sequence Diagram - Configure Season

Prerequisites - None

Steps

1. inputSchedule - The administrator

inputs the schedule for an upcoming

NFL season into the Optimization

Model.

2. domeEffect - The administrator

identifies matchups in an upcoming

season where a Dome Team plays in

cold weather.

Page 40: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Use Cases

Sequence Diagram - Configure Week

Prerequisites - An administrator has completed the “Configure Season” Use Case

for the current season

Steps

1. getPointSpreads - The administrator obtains Points Spreads

from the Projections DB for the current and future weeks.

2. returnPointSpreads - Points Spreads are returned to the

Administrator

3. inputPointSpreads - The administrator inputs Point Spreads for

the current and future weeks into the Optimization Model.

1. Note: For this project, steps 1-3 are manually complete

by copying data from an Excel sheet, and pasting them

in the Optimization Model.

4. initiateUncertainty - The administrator runs a provided R

function to adjust future weeks for uncertainty.

5. uncertainity - Point Spreads are adjusted to account for

uncertainty in the Optimization Model

6. conversion - Point Spreads are converted to probabilities in the

Optimization Model

7. inputScores - The administrator inputs scores for the previous

week

8. rivals – Team rivalry data is updated in the Optimization Model

9. consistency - Team consistency data is updated in the

Optimization Model

Page 41: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Use Cases

Sequence Diagram – Use Tool

Prerequisites - An administrator has completed the “Configure Season” and

“Configure Week” Use Cases for the current season/week

Steps

1. inputContestantInfo - A Contestant invokes the GUI by inputting

Pool Size, Pool Strength, and Previously Selected Teams.

2. promptSelections - The GUI presents the contestant drop down

lists to select teams previously used.

3. identifySelections - The contestant identifies all the teams they

have previously selected.

4. getPaths - The GUI invokes the Optimization Model to obtain the

optimal, secondary paths, and “Greedy” paths by providing the

Pool Size, Pool Strength, and Previously Selected Teams.

5. adjustWeights - wij values are finalized within the Optimization

Model based on the inputs from the GUI following built in

adjustments for Uncertainty, Team Consistency, Divisional

Rivals, and the Dome Effect.

6. optimize - The Optimization Model solves the Objective Function.

The optimal path is found, along with two secondary paths and

the “Greedy” path.

7. returnPaths - The Optimization Model returns the optimal path,

two secondary paths, and the “Greedy” path to the GUI.

8. generateResults - The GUI ingest the paths return by the

Optimization Model and generate results to display to the

Contestant.

9. displayResults - The GUI displays results to the Contestant.

Page 42: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

• Preseason Administration: Enter each

team’s schedule

Model Administration

Page 43: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

• Preseason Administration: Identify those

games/ teams that are effected by the

Dome Effect constraint. Mark this in the

model.

Model Administration

Page 44: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

• Weekly Administration:

• Update Optimization Model with the previous

week’s game outcomes

• Update the projections and closing spreads

on the Initial Spreads tab for the current week

• Update Point Spreads for Uncertainty

• Run R function which uses the outlined

uncertainty analysis on inputted point spreads and

projections.

Model Administration

Page 45: Fall 2016 - OR699/SYST699 Final Presentation...Fall 2016 - OR699/SYST699 Final Presentation ESPN –NFL Survivor Pool OptimizationTEAM ESPN: Audrey Barry Lauren Moore Adam Patterson

Sensitivity Analysis

Sensitivity analysis was not initially scoped to be a

significant effort for this Project; however given the

number of attributes unique to the ENSPO System,

Team ESPN felt it was necessary to explore a few of

these attributes with respect to sensitivity analysis.

Team ESPN investigated variations on Pool Size,

Pool Strength and the Divisional Games. Variations on Pool Size and Pool Strength do not have a

significant impact on results. Team ESPN suggests that the

“tipping point” of .50 (as explained in the Project Report) be

increased.

Team ESPN performed sensitivity analysis on was Divisional

Games and results support that the inclusion of adjustments for

Divisional Games impact and strengthen results of the ENSPO

System.