Section 12.1: Covariance and Correlation
16
Section 12.1 R. J. Harris Motivation Concepts Interpretation Example Useful properties Exercise Summary Section 12.1: Covariance and Correlation Rosemary J. Harris School of Mathematical Sciences Queen Mary University of London (Introduction to Probability, Lecture 41)
Transcript of Section 12.1: Covariance and Correlation
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Section 12.1: Covariance and Correlation
Rosemary J. Harris
School of Mathematical SciencesQueen Mary University of London
(Introduction to Probability, Lecture 41)
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Relationship between random variables
Is there a connection between sales of hot chocolate and number of snowmen?
The corresponding random variables are probably not independent......but can we say more about the strength and type of relationship?
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Covariance and correlation
Definition 12.1
Let X and Y be discrete random variables defined on the same sample space. Thecovariance of X and Y is defined by
Cov(X ,Y ) = E( [X − E(X )][Y − E(Y )] ).
If Var(X ) > 0 and Var(Y ) > 0, then the correlation coefficient of X and Y isdefined by
Corr(X ,Y ) =Cov(X ,Y )√
Var(X )Var(Y ).
There’s also a convenient alternative formula for the covariance...
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Alternative formula for covariance
Proposition 12.2
If X and Y are discrete random variables defined on the same sample space, then
Cov(X ,Y ) = E(XY ) − E(X )E(Y ).
Proof:Using Definition 12.1, and properties of expectation, we have
Cov(X ,Y ) = E( [X − E(X )][Y − E(Y )] )
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Uncorrelated random variables
Definition 12.1 (part)
Let X and Y be discrete random variables defined on the same sample space. Thecovariance of X and Y is defined by
Cov(X ,Y ) = E( [X − E(X )][Y − E(Y )] )
When Cov(X ,Y ) = 0 we say that X and Y are uncorrelated.
If X and Y are independent, then Cov(X ,Y ) = 0.
However, if Cov(X ,Y ) = 0, then X and Y are not necessarily independent.[We can have E(XY ) = E(X )E(Y ) even when X and Y are not independent.]
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Positively correlated random variables
Definition 12.1 (part)
Let X and Y be discrete random variables defined on the same sample space. Thecovariance of X and Y is defined by
Cov(X ,Y ) = E( [X − E(X )][Y − E(Y )] ).
Cov(X ,Y ) > 0 if on average [X −E(X )] and [Y −E(Y )] have the same sign.
In this case X and Y tend to deviate together above or below theirexpectation:
When [X − E(X )] is positive, [Y − E(Y )] tends to be positive too;When [X − E(X )] is negative, [Y − E(Y )] tends to be negative too.
For example, number of snowmen and sales of hot chocolate.
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Negatively correlated random variables
Definition 12.1 (part)
Let X and Y be discrete random variables defined on the same sample space. Thecovariance of X and Y is defined by
Cov(X ,Y ) = E( [X − E(X )][Y − E(Y )] ).
Cov(X ,Y ) < 0 if on average [X −E(X )] and [Y −E(Y )] have opposite signs.
In this case X and Y tend to deviate in opposite directions from theirexpectation:
When [X − E(X )] is positive, [Y − E(Y )] tends to be negative;When [X − E(X )] is negative, [Y − E(Y )] tends to be positive.
For example, temperature (to the nearest degree) and sales of hot chocolate.
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Warning!
Correlation does not imply causation...
For example, number of storks and number of babies (in Germany):
[H. Sies, Nature 332 (1988), 495]
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Correlated balls
Example 12.1
Find the covariance of the random variables R and Y of Example 11.1. Discuss theinterpretation of the sign of the covariance in this case.
Using Proposition 12.2, and the results of Exercises 11.4 and 11.6, we have
Cov(R,Y ) =
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Back to the correlation coefficient
Definition 12.1 (part)
Let X and Y be discrete random variables defined on the same sample space. IfVar(X ) > 0 and Var(Y ) > 0, then the correlation coefficient of X and Y isdefined by
Corr(X ,Y ) =Cov(X ,Y )√
Var(X )Var(Y ).
The correlation coefficient is a kind of scaled version of the covariance whichis insensitive, for example, to the choice of units in a measurement.
It has some nice properties...
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Properties of the correlation coefficient
Proposition 12.3
Let X and Y be discrete random variables defined on the same sample space andwith Var(X ) > 0 and Var(Y ) > 0.
(a) If a, b, c , d are real-valued constants with a > 0 and c > 0, then
Corr(aX + b, cY + d) = Corr(X ,Y ).
(b) Corr(X ,Y ) lies in [−1, 1], i.e., −1 ≤ Corr(X ,Y ) ≤ 1.
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Proving properties
Exercise 12.2
Suppose that X and Y are discrete random variables defined on the same sample space(with Var(X ) > 0 and Var(Y ) > 0) and that a, b, c , d are real-valued constants.
(a) Show thatCov(aX + b, cY + d) = acCov(X ,Y ).
(b) If a > 0 and c > 0, use the result of part (a) to show that
Corr(aX + b, cY + d) = Corr(X ,Y ).
(c) Show that if Y = aX + b with a > 0, then Corr(X ,Y ) = 1 (perfect positivecorrelation). What relationship between X and Y would give Corr(X ,Y ) = −1(perfect negative correlation)?
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Proving properties
Exercise 12.2
Suppose that X and Y are discrete random variables defined on the same sample space(with Var(X ) > 0 and Var(Y ) > 0) and that a, b, c , d are real-valued constants.
(a) Show thatCov(aX + b, cY + d) = acCov(X ,Y ).
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Proving properties
Exercise 12.2
Suppose that X and Y are discrete random variables defined on the same sample space(with Var(X ) > 0 and Var(Y ) > 0) and that a, b, c , d are real-valued constants.
(b) If a > 0 and c > 0, use the result of part (a) to show that
Corr(aX + b, cY + d) = Corr(X ,Y ).
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Proving properties
Exercise 12.2
Suppose that X and Y are discrete random variables defined on the same sample space(with Var(X ) > 0 and Var(Y ) > 0) and that a, b, c , d are real-valued constants.
(c) Show that if Y = aX + b with a > 0, then Corr(X ,Y ) = 1 (perfect positivecorrelation). What relationship between X and Y would give Corr(X ,Y ) = −1(perfect negative correlation)?
Section 12.1
R. J. Harris
Motivation
Concepts
Interpretation
Example
Usefulproperties
Exercise
Summary
Summary
Definition 12.1
Let X and Y be discrete random variables defined on the same sample space. Thecovariance of X and Y is defined by
Cov(X ,Y ) = E( [X − E(X )][Y − E(Y )] )
If Var(X ) > 0 and Var(Y ) > 0, then the correlation coefficient of X and Y is defined by
Corr(X ,Y ) =Cov(X ,Y )√
Var(X )Var(Y ).
Positive correlation, Cov(X ,Y ) > 0 means that X and Y tend to deviatetogether above or below their expectations.
Correlation does not imply causation!
