Functions of random variables

• Sometimes what we can measure is not what we are interested in!

• Example: mass of binary-star system:

• We want M but can only measure V and P.

• Must conserve probability:

M =V2aG

=V3P2πG

Y =g(X) dY=∂g∂X

dx

f(Y)dy= f(X)dx

Y

X

f(X)

f(Y)

Non-linear transformations

• e.g.Flux distributions vs. wavelength, frequency:

• Fluxes and magnitudes:– Gaussian distribution: X ~ G(X0,2)

– Nonlinear transformation induces a bias:

– PROBLEM: evaluate a, (M) in terms of X0 , .

f (ν)dν = f (λ)dλ

ν =cλ

dν =−cλ2 dλ ⇒ f(ν) =λ2

cf(λ)

M ≠−2.5 logX=−2.5 logX + a(M)

X

M=

-2.5

lo

g X

f(M)

f(X)

Nonlinear transformations bias the mean

• To find <Y>, use Taylor expansion around X=<X>:

• Hence

Y =g(X) =g( X ) +g'( X ) X− X[ ] +12g" ( X ) X− X[ ]2 + ...

g(X ) = g( X ) + g' ( X ) X− X[ ] +12g" ( X ) X− X[ ]2 + ...

=g( X ) + g' ( X ) X − X[ ] +12g" ( X ) 2 (X) + ...

X

Y f(Y)

f(X)

Y=g(X)

0

This is the bias.

Variance of a transformed

variable

• Get variance of Y from first principles:X

Y f(Y)

f(X)

Y=g(X)

2 (Y ) = (Y − Y )2

= g( X ) + g' ( X ) X − X[ ] +12

g"( X ) X − X[ ]2 + ... ⎡ ⎣ ⎢

−g( X ) − 0 −1

2g"( X )σ 2 (X) − ...

⎤ ⎦ ⎥2

= g' ( X ) X − X[ ] + ...[ ]2 = g' ( X )[ ]

2σ 2 (X )

What is a statistic?

• Anything you measure or compute from the data.

• Any function of the data.

• Because the data “jiggle”, every satistic also “jiggles”.

• Example: the mean value of a sample of N data points is a statistic:

• It has a definite value for a particular dataset, but it also “jiggles” with the ensemble of datasets to trace out its own PDF.

• NB:

X =1N

Xii=1

N

∑

X ≠ X

Sample mean and variance - 1

• Sample mean:

• The distribution of sample means has a mean:

• ...and a variance:

X =1N

Xii=1

N

∑

X =1N

Xii∑ =

1N

Xii∑ =

1N

Xii∑

2 X( ) = σ 2 1

NXi

i∑

⎛

⎝ ⎜

⎞

⎠ ⎟ =

1

N2σ 2 Xi

i∑ ⎛

⎝ ⎜

⎞

⎠ ⎟ =

1

N 2σ 2

i∑ Xi( )

if the Xi are independent

Sample mean and variance - 2

• If the Xi are all drawn from a single parent distribution with mean <X> and variance 2, then:

• And:

X =1N

Xi=1

N

∑ = X , . . i eX is an unbiased estimator ofX .

2 X( ) =1

N 2σ 2 Xi( )

i∑ ⎛

⎝ ⎜

⎞

⎠ ⎟ =

Nσ 2 Xi( )

N 2=

σ 2 Xi( )

N

⇒ σ X( ) =σ Xi( )

N, i.e. X " jiggles" much less

than a single data value Xi does.

Other unbiased statistics

• Sample median (half points above, half below)

• (Xmax + Xmin) / 2

• Any single point Xi chosen at random from sequence

• Weighted average: wi Xii∑

wii∑

Inverse variance weighting is best!

• Let’s evaluate the variance of the weighted average for some weighting function wi:

• The variance of the weighted average is minimised when:

• Let’s verify this -- it’s important!

wi =1

Var(Xi )≡

1 i

2 .

2

wi Xii

∑wi

i∑

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟=

σ 2 w i Xii

∑ ⎛

⎝ ⎜

⎞

⎠ ⎟

w ii

∑ ⎛

⎝ ⎜

⎞

⎠ ⎟2 =

wi2σ 2 X i( )

i∑

w ii

∑ ⎛

⎝ ⎜

⎞

⎠ ⎟2 .

Choosing the best weighting function

• To minimise the variance of the weighted average, set:

0 =∂

∂wk

wi2 i

2

i∑

wii∑ ⎛ ⎝ ⎜

⎞ ⎠ ⎟2

⎛

⎝

⎜ ⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟ ⎟ ⎟

=2wk k

2

wii∑ ⎛ ⎝ ⎜

⎞ ⎠ ⎟2 −

2 wi2 i

2

i∑

wii∑ ⎛ ⎝ ⎜

⎞ ⎠ ⎟3

=2

wii∑ ⎛ ⎝ ⎜

⎞ ⎠ ⎟2 wk k

2 −wi

2 i2

i∑

wii∑ ⎛ ⎝ ⎜

⎞ ⎠ ⎟

⎛

⎝

⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟ ⎟⇒ wk =

1 k

2 .

(Note: wi2 i

2∑ = wi∑ forwi =1/ i2 )

Using optimal weights

• Good principles for constructing statistics:– Unbiased -> no systematic error

– Minimum variance -> smallest possible statistical error

• Optimally (inverse-variance) weighted average:

• Is unbiased, since:

• And has minimum variance:

ˆ X =wiXi

i∑

wii∑

=Xi / i

2

i∑

1/ i2

i∑

ˆ X = X

2 ˆ X ( ) =1

1/σ i2

i∑