redictive Analytics: Prediction Model By Dr. Ash...

Sports Predictive Analytics:NFL Prediction Model

ByDr. Ash Pahwa

IEEE Computer SocietySan Diego Chapter

January 17, 2017

Copyright 2017 Dr. Ash Pahwa 1

Outline� Case Studies of Sports Analytics� Sports� Sports Analytics� Applications of Sports Analytics� Sports Analytics Literature� Data Sources� Sports Predictive Models� Regression Model� Multi Variable Regression with Lasso� NFL Prediction Model� Prediction for Super Bowl 2016� Prediction for NFL 2017 Playoffs


San Diego L.A. Chargers


Case Studies of Sports Analytics?


What is Sports Analytics?Which Pitcher is Better?


2007 BaseballJake Peavy John Lackey

San Diego Padres : National League

L.A. Angles : American League

ERA: 2.54 ERA: 3.01

ERA: Earned Run Average. Mean number of runs yielded per 9 innings

• American league allows Designated Hitter (DH) for pitcher

• National league does not allow Designated Hitter (DH) for pitcher. Pitcher must bat.

Which Variable is the Best Predictor of the Winning Percentage?


Pythagorean Theorem� Used in Baseball. Proposed by Bill James� Suppose

� ‘F’ represents a team’s run scored� ‘A’ represents team’s runs allowed

� 𝑃𝑦ℎ𝑡𝑎𝑔𝑜𝑟𝑒𝑎𝑛 𝑊𝑖𝑛𝑛𝑖𝑛𝑔 𝑃𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 = 𝐹2

𝐹2+𝐴2

� It is called Pythagorean theorem because it is similar to the elementary geometry theorem


Example:� Year 2012: Detroit Tigers

� Scored Runs = F = 726� Allowed Runs = A = 670

� 𝑃𝑦ℎ𝑡𝑎𝑔𝑜𝑟𝑒𝑎𝑛 𝑊𝑖𝑛𝑛𝑖𝑛𝑔 𝑃𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 = 𝐹2

𝐹2+𝐴2= 7262

7262+6702= 527,076

975,976= 0.54

� Total games won = 162*0.54 = 88 games

How to Convey Information to Decision Makers


Data Scientist

• Understand the Sport• Understand the Players• Understand Performance data

Decision Makers

• Management• Operational Personals

Raw Data Meaningful Results

Selection of • Statistical Models• Predictive Models

Goals of Sports Analytics1. Apply Statistical Models to Sporting

Data2. Ratings and Rankings3. Predictive Models4. Player and Team Assessment


Statistical ModelsPredictive Models


Indices of Central

Tendencies and Variability

Statistics Used to Examine

Relationships

Inferential Statistics

Ratings + Rankings

Predictive Models

Histogram (Frequency Distribution)

Normal Distribution (z-values and

p-values)

Normality Ratings + Rankings

Simple Linear Regression

Mean Covariance Outlier Rank Aggregation Multiple Linear Regression

Median Correlation –Pearson

t-test Polynomial Regression

Mode Rank Correlation –Spearman

ANOVA Logistic Regression

Range, Variance Partial Correlation Chi-Square

Standard Deviation

Ranking of Players and Teams?


Pair-Wise Comparison


Pair-Wise ComparisonThe Social Network


Pair-Wise ComparisonCan be Used for Ranking


Sorting the Rating list produces Ranking

Log 5 Method� Developed by Bill James in 1970s� Computes the probability that Team A

will beat Team B� Log 5 formula has nothing to do with

the mathematical function ‘Log’


Log 5 Formula� Suppose Team A true winning percentage is 10 out of 16 games

� Percentage of true winning = 𝑝𝑎 = 10/16 = 0.625� Suppose Team B true winning percentage is 7 out of 16 games

� Percentage of true winning = 𝑝𝑏 = 7/16 = 0.438� --------------------------------------------------------------------------------------� The probability that Team A will beat Team B� 𝑝𝑎,𝑏 =

𝑝𝑎−𝑝𝑎∗𝑝𝑏𝑝𝑎+𝑝𝑏−2∗𝑝𝑎∗𝑝𝑏

= 0.625−0.625∗0.4380.625+0.438−2∗0.625∗0.438

= 0.625−0.2741.063−2∗0.274

= 0.3510.515

= 0.681

� ----------------------------------------------------------------------------------------� The probability that Team B will beat Team A� 𝑝𝑏,𝑎 =

𝑝𝑏−𝑝𝑎∗𝑝𝑏𝑝𝑎+𝑝𝑏−2∗𝑝𝑎∗𝑝𝑏

= 0.438−0.625∗0.4380.625+0.438−2∗0.625∗0.438

= 0.438−0.2741.063−2∗0.274

= 0.1640.515

= 0.318

� ---------------------------------------------------------------------------------------------------� 𝑝𝑎,𝑏 + 𝑝𝑏,𝑎 = 1


𝑝𝑎,𝑏 =𝑝𝑎 − 𝑝𝑎 ∗ 𝑝𝑏

𝑝𝑎 + 𝑝𝑏 − 2 ∗ 𝑝𝑎 ∗ 𝑝𝑏

Arpad Elo� Physics Professor at Marquette

University� Milwaukee, Wisconsin

� Chess Player� Devised a method to rank chess players� His method was adopted by

� US Chess Federation� World Chess Federation

17Copyright 2017 Dr. Ash Pahwa

Points Gained or LostPoints gained/lost by player A = Points gained/lost by player B

• Before the game• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐴 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐴• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐵 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐵• 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑑𝐴𝐵 = 𝑟𝐴 − 𝑟𝐵

• After the game• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐴 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐴′

• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐵 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐵′

• 𝑟𝐴 + 𝑟𝐵 = 𝑟𝐴′ + 𝑟𝐵′

• 𝑃𝑜𝑖𝑛𝑡𝑠 𝑔𝑎𝑖𝑛𝑒𝑑/𝑙𝑜𝑠𝑡 𝑏𝑦 𝑝𝑙𝑎𝑦𝑒𝑟 𝐴 𝑎𝑔𝑎𝑖𝑛𝑠𝑡 𝑝𝑙𝑎𝑦𝑒𝑟 𝐵• 𝜇𝐴𝐵 = 𝐿 𝑑𝐴𝐵

400= 1

1+10−𝑑𝐴𝐵400


Example

� After the game� 𝑃𝑙𝑎𝑦𝑒𝑟 𝐴 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐴′ = 𝑟𝐴 + 𝐾 𝑆𝐴𝐵 − 𝜇𝐴𝐵 = 2400 + 32 1 − 0.91 = 2403� 𝑃𝑙𝑎𝑦𝑒𝑟 𝐵 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐵′ = 𝑟𝐵 + 𝐾 𝑆𝐵𝐴 − 𝜇𝐵𝐴 = 2000 + 32 0 − 0.09 = 1997

� 𝑟𝐴 + 𝑟𝐵 = 𝑟𝐴′ + 𝑟𝐵′

� 2400 + 2000 = 2403 + 1997

If Player A wins

Draw If Player B wins

S(AB) 1 0.5 0

S(BA) 0 0.5 1

Suppose K = 32 for Chess

• Before the game• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐴 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐴 = 2400• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐵 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐵 = 2000• 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑑𝐴𝐵 = 𝑟𝐴 − 𝑟𝐵 = 400• 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑑𝐵𝐴 = 𝑟𝐵 − 𝑟𝐴 = −400

If Player A wins

𝜇𝐴𝐵 = 0.91𝜇𝐵𝐴 = 0.09

• After the game• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐴 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐴′ = 𝑟𝐴 + 𝐾 𝑆𝐴𝐵 − 𝜇𝐴𝐵 = 2400 + 32 0 − 0.91 = 2371• 𝑃𝑙𝑎𝑦𝑒𝑟 𝐵 𝑟𝑎𝑡𝑖𝑛𝑔 𝑟𝐵′ = 𝑟𝐵 + 𝐾 𝑆𝐵𝐴 − 𝜇𝐵𝐴 = 2000 + 32 1 − 0.09 = 2029

• 𝑟𝐴 + 𝑟𝐵 = 𝑟𝐴′ + 𝑟𝐵′

• 2400 + 2000 = 2371 + 2029

If Player B wins


The Social NetworkElo Formula was Used to pair-wise Comparison of Girls


Sports


Sports� Inherent part of Human Culture� Sports competition dates back to the dawn of

our species� Greek Olympics : 776 BC


Sportsman Spirit� Virtues of sports

� fairness� self-control� courage� persistence

� It has been associated with interpersonal concepts of treating others and being treated fairly

� Maintaining self-control if dealing with others, and respect for both authority and opponents

� GOOD SPORTSMEN HAVE ALWAYS BEEN HELD IN HIGH ESTEEM


Most Popular Sports in America


Sports Analytics


Predictive Models� Estimation

� Regression� Classification (win/loss)

� Logistic Regression� Discriminant Analysis

� Linear� Quadratic

� Support Vector MachineCopyright 2017 Dr. Ash Pahwa 27

Goals of Sports AnalyticsPlayer� Discovering hidden talent in a new player� Player Evaluation

� Assessing Player Performance� Which metrics is most important to

assess a players’ performance� Assessing Player Value� How much value a player adds to the

teams’ value


Goals of Sports AnalyticsTeam

� Ranking top teams� Accessing Team Performance

� How to compute the value of a team� Which Team Members are best suited to play

against the opposing team� Which strategy to use to play against a team?

� Anticipating Opponents Behavior� Accessing the probability of a win in a sporting

eventCopyright 2017 Dr. Ash Pahwa 29

Need for Prediction Results� Betting on an sporting event

� People betting on sports need to see the prediction results

� Probability of a win� Point Spread

� Fantasy Sports� DraftKings� FanDuel


Applications of Sports Analytics


Application of Sports PAMovies


The film is based on Michael Lewis' 2003 nonfiction book of the same name, an account of the Oakland Athletics baseball team's 2002 season and their general manager Billy Beane's attempts to assemble a competitive team.

In the film, Beane (Brad Pitt) and assistant GM Peter Brand (Jonah Hill), faced with the franchise's limited budget for players, build a team of undervalued talent by taking a sophisticated sabermetric approach towards scouting and analyzing players.

They acquire "submarine" pitcher Chad Bradford (Casey Bond) and former catcher Scott Hatteberg (Chris Pratt), and win 20 consecutive games, an American League record.

Team Selection: Moneyball


Moneyball – Billy Beane & Paul DePodesta


William Lamar "Billy" Beane III (born March 29, 1962) is an American former professional baseball player and current front office executive.

He is the Executive Vice President of Baseball Operations and minority owner of the Oakland Athletics of Major League Baseball (MLB).

The character of Brand is an invention by the filmmakers; in the excellent Michael Lewis non-fiction book upon which the movie is based, the real-life “Brand” is identified as Paul DePodesta.

Unlike Brand, DePodesta is slender, fit and handsome. He’s also Harvard-educated (not a Yalie – screenwriter Aaron Sorkin‘s private joke).

Application of Sports PABoston Red Sox

� Using Predictive Analytics Strategies Boston Red Sox won 3 world series in Baseball


In 2006, Time named Bill James in the Time 100 as one of the most influential people in the world. He is a Senior Advisor on Baseball Operations for the Boston Red Sox.

Sports Analytics Literature


Predictive Models for SportsLiterature


Statistical Learning� Gareth James� Daniela Witten� Trevor Hastie� Robert Tibshirani

� Stanford University


Data Sources


Data Sources


ValidAccurateCompleteContains derived variables

Data Sources� www.NFL.com� www.NBA.com� www.footballOutsiders.com� www.pro-football-reference.com� www.soccerstats.com� www.basketball-reference.com


Data SourceFootball� www.ArmChairAnalysis.com


Sports Predictive Models


Goals of Predictive Analytics Application: Estimation or Classification

� Estimation – Regression modeling technique is used� Output is a number

� House price� Product sales for next

quarter� GNP growth for the next

quarter� How many points a team

will score


� Classification � Logistic Regression� Support Vector Machine� Discriminant Analysis (Linear,

Quadratic)� Naïve Bayes, Decision Trees etc.

modeling techniques are used� Output is a categorical variable

� Sports team will win or lose� Email is junk or not� Which grade student will get� Tweet is positive or negative

Common PA Techniques

� Regression� Linear 2 variables� Linear multi variables� Logistic� Polynomial

� Clustering� Decision Trees� Neural Networks� Naïve Bayes� ARIMA� A few more …


Regression Model


History of Linear Regression

� Sir Francis Galton and � Karl Pearson� Developed the concepts on

Regression and Correlation in 1900 - 1930


Definition: Linear Regression� 2 Variable Regression

� How a response variable “y” changes� As the predictor (explanatory)

� variable “x” changes� Multiple Regression

� How a response variable “y” changes� As the predictor (explanatory)

� variables “x1”, “x2”, … “xn” change

48

cxy � 1E

cxxxxy nn �� EEEE ...332211

Copyright 2017 Dr. Ash Pahwa

Single Variable Polynomial RegressionFirst degree to Fifth Degree

� The 5th degree polynomial regression looks good for training set data

� But with test data it may not perform well� PROBLEM: Over fit for this data


Overfitting� If there exists a model with estimated parameters 𝑤′ such that

� Training error (w2) < Training Error (w1)� Generalization (Test) error (w2) > Generalization (Test) Error (w1)


Bias and Variance

� Goal is to find out a model complexity where� Generalization (Validation) errors are least� Bias + variance are least

� 𝑀𝑒𝑎𝑛 𝑆𝑞𝑢𝑎𝑟𝑒 𝐸𝑟𝑟𝑜𝑟 = 𝐵𝑖𝑎𝑠2 + 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒� Just like Generalization Error

� We cannot compute Bias and Variance


1st Degree Polynomial Fit𝑦 = 𝑎1𝑥 + 𝑐


> #result = lm(y1~x)> result = lm(y1 ~ poly(x,1,raw=TRUE))> summary(result)

Call:lm(formula = y1 ~ poly(x, 1, raw = TRUE))

Residuals:Min 1Q Median 3Q Max

-0.9872 -0.7277 -0.1394 0.7842 1.2692

Coefficients:Estimate Std. Error t value Pr(>|t|)

(Intercept) 0.1919 0.4285 0.448 0.662poly(x, 1, raw = TRUE) -0.0492 0.1120 -0.439 0.668

Residual standard error: 0.8449 on 12 degrees of freedomMultiple R-squared: 0.01581, Adjusted R-squared: -0.0662 F-statistic: 0.1928 on 1 and 12 DF, p-value: 0.6684

> p = predict(result,list(x=xPredict))> lines(xPredict,p,col='black',lwd=3)>

16thDegree Polynomial Fit𝑦 = 𝑎1𝑥16 + 𝑎2𝑥15 + …+ 𝑎15 𝑥2 + 𝑎16𝑥 + 𝑐


> result = lm(y1 ~ poly(x,16,raw=TRUE))> summary(result)Call:lm(formula = y1 ~ poly(x, 16, raw = TRUE))Residuals:ALL 14 residuals are 0: no residual degrees of freedom!Coefficients: (3 not defined because of singularities)

Estimate Std. Error t value Pr(>|t|)(Intercept) 1.820e-01 NA NA NApoly(x, 16, raw = TRUE)1 1.280e+02 NA NA NApoly(x, 16, raw = TRUE)2 -7.656e+02 NA NA NApoly(x, 16, raw = TRUE)3 1.973e+03 NA NA NApoly(x, 16, raw = TRUE)4 -2.895e+03 NA NA NApoly(x, 16, raw = TRUE)5 2.675e+03 NA NA NApoly(x, 16, raw = TRUE)6 -1.631e+03 NA NA NApoly(x, 16, raw = TRUE)7 6.712e+02 NA NA NApoly(x, 16, raw = TRUE)8 -1.869e+02 NA NA NApoly(x, 16, raw = TRUE)9 3.447e+01 NA NA NApoly(x, 16, raw = TRUE)10 -3.944e+00 NA NA NApoly(x, 16, raw = TRUE)11 2.282e-01 NA NA NApoly(x, 16, raw = TRUE)12 NA NA NA NApoly(x, 16, raw = TRUE)13 -5.901e-04 NA NA NApoly(x, 16, raw = TRUE)14 NA NA NA NApoly(x, 16, raw = TRUE)15 1.468e-06 NA NA NApoly(x, 16, raw = TRUE)16 NA NA NA NA

Residual standard error: NaN on 0 degrees of freedomMultiple R-squared: 1, Adjusted R-squared: NaNF-statistic: NaN on 13 and 0 DF, p-value: NA

Under fit or Over fit Model


# Degree of

Polynomial

Under fit or Over

fit

1 1 Under fit2 2 Under fit3 4 Under fit4 8 OK5 10 Over fit6 16 Over fit

Lasso Regression

Least Absolute Shrinkage and Selection Operator


Cost Function of OLS + Ridge + Lasso


OLS

Ridge Regression

Lasso Regression

NFL Model


Data Source� Seasons

� 2000 – 2016� Weeks 1 – 21

� Week#1 – #17: 16 games + 1 bye� Week#18: 4 games: Wild card� Week#19: 4 games: Divisional Playoff� Week#20: 2 games: Conference Championship� Week#21: 1 game: Super Bowl


Data Tables

� Arm Chair� Game Data� Play Data� Team Data

� External Source� Stadium Data (Wikipedia)� City Coordinates (Wikipedia)� City GDP (Wikipedia + US Govt.)

Arm Chair Data – 26 Tables


Sonny Moore’s Computer Ratinghttp://sonnymoorepowerratings.com/archive


USA Today: Jeff Sagarinhttp://www.usatoday.com/sports/nfl


NFL Prediction

Model Result


Comparison of ErrorsSonny Moore & Jeff Sagarin


Super Bowl 2016 Prediction by Nate Silver


Super Bowl 2016 Prediction by A+


Super Bowl 2016All Predicted Models were Wrong


Panthers BroncosNate Silver 59% 41%

A+ 56.5% 43.5%

2017 NFL PredictionWild Card : Playoff

� Predictions for all the 4 games were 100% correct


2017 NFL PredictionDivisional Title: Playoff


www.NFLPrediction.co

� Predictions � 2 correct� 2 incorrect� 50% correct

2017 NFL PredictionConference Championship: Playoff

� Atlanta Falcons vs Green Bay Packers� Atlanta Falcons 61%

� New England Patriots vs Pittsburgh Steelers� New England Patriots 70%


2017 NFL PredictionSuper Bowl

� New England Patriots vs � Atlanta Falcons

�New England Patriots 60%


Summary� Sports� Sports Analytics� Applications of Sports Analytics� Sports Analytics Literature� Data Sources� Sports Predictive Models� Regression Model� Multi Variable Regression with Lasso� NFL Prediction Model� Prediction for Super Bowl 2016� Prediction for NFL 2017 Playoffs


UCSD Extension Courses

Winter 2017


UCSD Extension coursesWinter 2017

73

Winter 2017

Online

Dates

Sports Predicted Analytics(7 weeks)

January 30 – March 19, 2017

Copyright 2017 Dr. Ash Pahwa

Course Content


UCSD Sports Predictive Analytics

L# Date Subject1 01/30/17 Introduction to Sports Analytics

1.1 What is Sports Analytics?1.2 Tools for Sports Analytics1.3 Science of Learning from Data

1.4 Basic Stat : Data Types + Histograms + Std Dev2 02/06/17 Statistical Methods Applied on Sports Data

2.1 Normal Distribution2.2 Correlation2.3 Rank and Partial Correlation3 02/13/17 Central Limit Theorem + Hypothesis Testing

3.1 CLT + Parameter Estimation3.2 Hypothesis Testing (*)4 02/20/17 Inference Stat: Chi-Sq+ANOVA+Model Selec

4.1 Chi-Square (*)4.2 ANOVA4.3 Stat Model Selection (*)5 02/27/17 Ratings and Rankings + Rank Aggregation

5.1 Ratings and Rankings (Elo System)5.2 Rank Aggregation (Borda Voting)6 03/06/17 Prediction Using Regression Model

6.1 Introduction to Regression6.2 Regression 2 Variables6.3 Regression Multi Variables

7 03/13/17 Prediction Using Logistic Regression Model 7.1 Logistic Regression Mathematics7.2 Logistic Regression Mechanics7.3 Multivariable Logistic Regression

redictive Analytics: Prediction Model By Dr. Ash...

Documents

Transcript of redictive Analytics: Prediction Model By Dr. Ash...