Fitting probability models to frequency data

Chapter 8

Probability models

•  Proportional model: The frequency of events falling in different categories is proportional to the fraction of opportunities. –  e.g. Are babies born at same frequency on all

days of week? (see text) •  Discrete probability models:

–  Binomial: probability of X successes out of n independent trials.

–  Poisson: probability of getting X successes in a block of time or space, when each happens independently.

χ2 Goodness-of-fit test

Compares counts to a discrete probability distribution

Hypotheses for χ2 test

H0: The data come from a particular discrete probability distribution. HA: The data do not come from that distribution.

Test statistic for χ2 test

€

χ 2 =Observedi − Expectedi( )2

Expectediall classes∑

The month of birth for 1245 NHL players

Month Number of players

January 133February 125March 114April 119May 119June 123July 96August 91September 83October 84November 73December 85

Data from http://www.nhl.com/players/search/all.html in 2006

A Goodness-of-Fit test compares count data to a model of the expected frequencies of a set of categories.

Hypotheses for birth month example

H0: The probability of a NHL birth occurring on any given month is equal to national proportions. HA: The probability of a NHL birth occurring on any given month is not equal to national proportions.

Month Number ofplayers

Expected(%)

January 133 7.94February 125 7.63March 114 8.72April 119 8.63May 119 8.95June 123 8.57July 96 8.76August 91 8.5September 83 8.54October 84 8.19November 73 7.70December 85 7.86Total 1245 100

NHL compared to all Canadians

Computing Expected values Month Number of

playersExpected(%)

Expected(of 1245)

January 133 7.94 99February 125 7.63 95March 114 8.72 109April 119 8.63 107May 119 8.95 111June 123 8.57 107July 96 8.76 109August 91 8.5 106September 83 8.54 106October 84 8.19 102November 73 7.70 96December 85 7.86 98Total 1245 100% 1245

Note: For simplicity, we have rounded the expected column to integers. In any real calculation, we would keep a couple decimal places.

The calculation for January

€

Observed − Expected( )2

Expected=133− 99( )2

99=115699

Calculating χ2

€

χ 2 =Observedi − Expectedi( )2

Expectediall classes∑

=115699

+90095

+25109

+144107

+64111

+256107

+

169109

+225106

+529106

+324102

+52996

+16998

= 44.77

The sampling distribution of χ2 by simulation

Sampling distribution of χ2 by the χ2 distribution

Formula for the χ2 distribution

€

f x;k( ) =xk / 2−1e−x / 2

2k / 2Γ k /2( )

€

Γ k /2( ) = t k / 2−10

∞

∫ e− tdtWhere:

Note: You do NOT need to know this slide!

k = degrees of freedom x = χ2

Degrees of freedom

The number of degrees of freedom of a test specifies which of a family of distributions to use.

Degrees of freedom for χ2 test

df = (Number of categories)

– (Number of parameters estimated from the data)

– 1

Degrees of freedom for NHL month of birth

df = 12 - 0 - 1 = 11

Finding the P-value

Critical value

The value of the test statistic where P = α.

Table A - χ2 distribution

The 5% critical value

P < 0.05, so we can reject the null hypothesis. NHL players are not born in the same proportions per month as the population at large.

Table A - χ2 distribution

Using the table, we can even see that P < 0.001 (good to be precise as possible)

Test statistics

A test statistic is a number calculated from the data and the null hypothesis that can be compared to a standard distribution to find the P-value of the test.

Test Statistics and Hypothesis Testing

Data Null Hypothesis

Test Statistic Distribution of possible test statistics

How unusual is this test statistic?(If the null hypothesis is true)

How unusual is this data?(If the null hypothesis is true)

If weird, ! reject H0

If not so weird, ! do not reject H0

UNIQUE TOSITUATION

GENERAL

χ2 test as approximation of binomial test

•  χ2 goodness-of-fit test works even when there are only two categories, so it can be used as a substitute for the binomial test.

•  Very useful if the number of data points is large. –  Imagine if, in our red/blue wrestler example, rather than 16/20

wins by red, we had 1600/2000 wins by red. Imagine calculating:

•  Rather, simply do χ2 test:

€

Pr[1600] =2000!

1600!400!0.516000.5400

Observed Expected Red wins 1600 1000 Blue wins 400 1000

Assumptions of χ2 test

•  No more than 20% of categories have Expected < 5

•  No category with Expected ≤ 1

Probability models

•  Proportional model: The frequency of events falling in different categories is proportional to the fraction of opportunities. –  e.g. Are babies born at same frequency on all

days of week? (see text) •  Discrete probability models:

–  Binomial: probability of X successes out of n independent trials.

–  Poisson: probability of getting X successes in a block of time or space, when each happens independently.

Discrete probability distribution

A probability distribution describing a discrete numerical random variable

For example, •  Number of heads from 10 flips of a coin •  Number of girls in families with two kids •  Number of flowers in a square meter •  Number of disease outbreaks in a year

The Poisson distribution

The Poisson distribution describes the probability that a certain number of events occur in a block of time or space, when those events happen independently of each other and occur with equal probability at every point in time or space.

Poisson distribution

€

Pr X[ ] =e−µµX

X!X = number of successes µ = mean number of successes per unit time or space e = base of natural log, 2.718

Example: Number of goals per side in World

Cup Soccer

Q: Is the outcome of a soccer game (at this level) random?

In other words, is the number of goals per team distributed as expected by pure chance?

Hypotheses

•  H0: Number of goals per side follows a Poisson distribution.

•  HA: Number of goals per side does not follow a Poisson distribution.

World Cup 2002 scores

Number of goals for a team in one game (World Cup 2002)

What�s the mean goals per game?

€

x =37 0( ) + 47 1( ) + 27 2( ) +13 3( ) + 2 4( ) +1 5( ) +1 8( )

128

=161128

=1.26 we will use this as µ

Poisson with µ = 1.26

Example:

€

Pr 2[ ] =e−µµX

X!=e−1.26 1.26( )2

2!=0.284( )1.59

2= 0.225

Poisson with µ = 1.26 X Pr[X] 0 0.284 1 0.357 2 0.225 3 0.095 4 0.030 5 0.008 6 0.002 7 0 ≥8 0

Finding the Expected

X Pr[X] Expected 0 0.284 36.3 1 0.357 45.7 2 0.225 28.8 3 0.095 12.1 4 0.030 3.8 5 0.008 1.0 6 0.002 0.2 7 0 0.04 ≥8 0 0.007

} Too small!

n = 128

Assumptions of χ2 test

•  No more than 20% of categories have Expected<5

•  No category with Expected ≤ 1

Calculating χ2 X Expected Observed 0 36.3 37 0.013 1 45.7 47 0.037 2 28.8 27 0.113 3 12.1 13 0.067 ≥ 4 5.0 4 0.200

€

Observedi − Expectedi( )2

Expectedi

€

χ 2 =Observedi − Expectedi( )2

Expectediall classes∑ = 0.429

Degrees of freedom

df = (Number of categories)

– (Number of parameters estimated from the data)

– 1

= 5 – 1 – 1 = 3

Critical value

Comparing χ2 to the critical value

€

χ 2 = 0.429χ32 = 7.81

0.429 < 7.81

So we cannot reject the null hypothesis. There is no evidence that the score of a World Cup Soccer game is not Poisson distributed.

World Cup 2002 scores

Poisson distribution

Goodness-of-fit tests with a binomial distribution as the null

•  Binomial distribution describes the probability of X successes out of n independent trials, assuming constant probability of success and independence of trials.

•  See text for example: sex ratio in two-child families. Fr

equ

ency

Number of boys

1,400

1,200

1,000

800

600

400

200

00 1 2

ObservedExpected

Fig. 8.5-1 of textbook

Don’t confuse the “Binomial Test” with the “χ2 Goodness-of-Fit Test using the Binomial Distribution”

•  Binomial Test: •  To test whether a proportion of successes in one set of trials is

consistent with the null hypothesis that probability of success is p0 and trials are independent.

•  Null: p = p0

•  χ2 Goodness-of-Fit Test using the Binomial Distribution •  To test whether a distribution of observed number of successes in

many sets of trials is consistent with the null hypothesis that probability of success is constant and independent in all trials.

•  Null: Distribution of successes is binomially distributed.