Convergence Notes
-
Upload
heather-oskedasticity -
Category
Documents
-
view
231 -
download
0
Transcript of Convergence Notes
-
8/6/2019 Convergence Notes
1/6
Convergence of random variablesInprobability theory, there exist several different notions ofconvergence ofrandom
variables. The convergence (in one of the senses presented below) ofsequences of
random variables to some limiting random variable is an important concept in
probability theory, and its applications to statistics and stochastic processes. For
example, if the average ofnuncorrelated random variables Yi, i = 1, ..., n, is given by
then as n goes to infinity,Xn converges in probability (see below) to the common
mean, , of the random variables Yi. This result is known as the weak law of large
numbers. Other forms of convergence are important in other useful theorems,
including the central limit theorem.
Throughout the following, we assume that (Xn) is a sequence of random variables, and
Xis a random variable, and all of them are defined on the sameprobability space (,
F, P).
Contents
1 Convergence in distribution 2 Convergence in probability
o 2.1 Lemmao 2.2 Proof of lemmao 2.3 Proof
3 Almost sure convergence 4 Sure convergence 5 Convergence in mean 6 Implications
7 See also 8 External links 9 References
Convergence in distribution
Suppose thatF1,F2, ... is a sequence ofcumulative distribution functions
corresponding to random variablesX1,X2, ..., and thatFis a distribution function
corresponding to a random variableX. We say that the sequenceXn converges towards
Xin distribution, if
-
8/6/2019 Convergence Notes
2/6
for every real numbera at whichFis continuous. SinceF(a) = Pr(X a), this means
that the probability that the value ofXis in a given range is very similar to the
probability that the value ofXn is in that range, provided n is large enough.
Convergence in distribution is often denoted by adding the letter over an arrow
indicating convergence:
Small dis also possible, although less common.
Convergence in distribution is the weakest form of convergence, and is sometimes
called weak convergence (main article: weak convergence of measures). It does not,
in general, imply any other mode of convergence. However, convergence in
distribution is implied by all other modes of convergence mentioned in this article,
and hence, it is the most common and often the most useful form of convergence of
random variables. It is the notion of convergence used in the central limit theorem and
the (weak) law of large numbers.
A useful result, which may be employed in conjunction with law of large numbers
and the central limit theorem, is that if a function g: RR is continuous, then
if Xn converges in distribution to X, then so too does g(Xn) converge in
distribution to g(X). (This may be proved using Skorokhod's representation theorem.)This fact could be taken as a definition for the convergence in distribution.
Convergence in distribution is also called convergence in law, since the word "law"
is sometimes used as a synonym of "probability distribution."
Convergence in probability
To say that the sequenceXn converges towardsXin probability means
for every > 0. Formally, pick any > 0 and any > 0. LetPn be the probability that
Xn is outside a tolerance ofX. Then, ifXn converges in probability toXthen there
exists a valueNsuch that, for all nN,Pn is itself less than .
Convergence in probability is often denoted by adding the letter 'P' over an arrow
indicating convergence:
-
8/6/2019 Convergence Notes
3/6
Convergence in probability is the notion of convergence used in the weak law of large
numbers. Convergence in probability implies convergence in distribution. To prove it,
it's convenient to prove the following, simple lemma:
Lemma
LetX, Ybe random variables, c a real number and > 0; then
Proof of lemma
since
ProofFor every , due to the preceding lemma, we have:
So, we have
Taking the limit for , we obtain:
But is the cumulative distribution functionFX(a), which is continuous
by hypothesis, that is
and so, taking the limit for , we obtain
Almost sure convergenceTo say that the sequenceXn converges almost surely oralmost everywhere orwith
probability 1 orstrongly towardsXmeans
This means that the values ofXn approach the value ofX, in the sense (see almost
surely) that events for whichXn does not converge toXhave probability 0. Using the
probability space (,F, P) and the concept of the random variable as a function from
to R, this is equivalent to the statement
-
8/6/2019 Convergence Notes
4/6
Almost sure convergence is often denoted by adding the letters a.s. over an arrow
indicating convergence:
Almost sure convergence implies convergence in probability, and hence implies
convergence in distribution. It is the notion of convergence used in the strong law of
large numbers.
Sure convergence
To say that the sequence orrandom variables (Xn) defined over the sameprobability
space (i.e., a random process) converges surely oreverywhere orpointwise towards
Xmeans
where is the sample space of the underlyingprobability space over which the
random variables are defined.
This is the notion ofpointwise convergence of sequence functions extended to
sequence ofrandom variables. (Note that random variables themselves are functions).
Sure convergence of a random variable implies all the other kinds of convergence
stated above, but there is no payoff inprobability theory by using sure convergence
compared to using almost sure convergence. The difference between the two only
exists on sets with probability zero. This is why the concept sure convergence of
random variables is very rarely used.
Convergence in mean
We say that the sequenceXn converges in the r-th mean orin theLrnorm towardsX,
ifr 1, E|Xn|r< for all n, and
where the operator E denotes the expected value. Convergence in rth mean tells us
that the expectation of the r-th power of the difference betweenXn andXconverges to
zero.
This type of convergence is often denoted by adding the letterLrover an arrow
indicating convergence:
-
8/6/2019 Convergence Notes
5/6
The most important cases of convergence in r-th mean are:
WhenXn converges in r-th mean toXforr= 1, we say thatXn converges inmean toX.
WhenXn converges in r-th mean toXforr= 2, we say thatXn converges inmean square toX.
Convergence in the r-th mean, forr> 0, implies convergence in probability (by
Chebyshev's inequality), while ifr>s 1, convergence in r-th mean implies
convergence ins-th mean. Hence, convergence in mean square implies convergence
in mean.
Implications
The chain of implications between the various notions of convergence are noted in
their respective sections. They are, using the arrow notation
No other implications other than these hold in general, but a number of special cases
do permit the converse implications:
IfXn converges in distribution to a constant c, thenXn converges in probabilityto c.
IfXn converges in probability toX, and if Pr(|Xn| b) = 1 for all n and some b,thenXn converges in rth mean toXfor all r 1. In other words, ifXn
converges in probability toXand all random variablesXn are almost surely
bounded above and below, thenXn converges toXalso in any rth mean.
If for all > 0,
then we say thatXnconverges almost completely, orfast in probability
towardsX. WhenXn converges almost completely towardsXthen it also
converges almost surely toX. In other words, ifXn converges in probability to
Xsufficiently quickly (i.e. the above sequence of tail probabilities is
summable for all > 0), thenXn also converges almost surely toX. This is a
direct implication from the Borel-Cantelli lemma.
IfSn is a sum ofn real independent random variables:
then Sn converges almost surely if and only ifSn converges in probability.
Lvy's convergence theorem gives sufficient conditions for almost sureconvergence to implyL1-convergence:
-
8/6/2019 Convergence Notes
6/6