2[1]. Discrete Random Variables
-
Upload
nikhil-aggarwal -
Category
Documents
-
view
230 -
download
0
Transcript of 2[1]. Discrete Random Variables
-
8/8/2019 2[1]. Discrete Random Variables
1/67
1
2. Discrete Random Variables
Broadband Network Lab, NCTU, Taiwan
-
8/8/2019 2[1]. Discrete Random Variables
2/67
Broadband Network Lab, NCTU, Taiwan 2
Outline
2.1 Basic Concepts
2.2 Probability Mass Functions
2.3 Functions of Random Variables
2.4 Expectation, Mean, and Variance
2.5 Joint PMFs of Multiple Random
Variables
2.6 Conditioning
2.7 Independence
-
8/8/2019 2[1]. Discrete Random Variables
3/67
Broadband Network Lab, NCTU, Taiwan 3
2.1 Visualization of a RandomVariable
-
8/8/2019 2[1]. Discrete Random Variables
4/67
Broadband Network Lab, NCTU, Taiwan 4
2.1 Main Concepts Related toRandom Variables
A random variable is a real-valued function of
the outcome of the experiment.
A function of a random variable definesanother random variable.
We can associate with each random variable
certain averages of interest, such as themean and the variance.
A random variable can be conditioned on anevent or on another random variables.
There is a notion ofindependence of a randomvariable from an event or from anotherrandom variable.
-
8/8/2019 2[1]. Discrete Random Variables
5/67
Broadband Network Lab, NCTU, Taiwan 5
2.1 Concepts Related to DiscreteRandom Variables
A discrete random variable is a real-valued
function of the outcome of the experimentthat can take a finite or countably infinitenumber of values.
A discrete random variable has an associated
probability mass function (PMF), which givesthe probability of each numerical value thatthe random variable can take.
A function of a discrete random variable
defines another discrete random variable,whose PMF can be obtained from the PMF ofthe original random variable.
-
8/8/2019 2[1]. Discrete Random Variables
6/67
Broadband Network Lab, NCTU, Taiwan 6
2.2 Probability Mass Functions
X: Discrete random variable.
pX: PMF.
x: Any possible value ofX.
pX(x): Probability mass ofx.
( )
Note that :
For any set S of possible values ofX, we
have :
-
8/8/2019 2[1]. Discrete Random Variables
7/67
Broadband Network Lab, NCTU, Taiwan 7
2.2 Calculation of the PMF of aRandom Variable X
For each possible value x of X:
Collect all the possible outcomes that giverise to the event {x = X}.
Add their probabilities to obtain pX(x).
Example: Let Xbe the number of heads obtained in
two independent tosses of a fair coin.
The probability of at least one head is
-
8/8/2019 2[1]. Discrete Random Variables
8/67
Broadband Network Lab, NCTU, Taiwan 8
2.2 The Bernoulli Random Variable
Toss a coin: p(head) = p, p(tail) = 1 p.
The Bernoulli random variable X is
Its PMF is
-
8/8/2019 2[1]. Discrete Random Variables
9/67
Broadband Network Lab, NCTU, Taiwan 9
2.2 The Binomial Random Variable
Toss a coin n times. Tosses are indep..
p(a head) = p, p(a tail) = 1 p.
X: number of heads in the n-toss sequence.
X is a binomial random variable with
parameters n and p.
PMF:
Normalization property:
-
8/8/2019 2[1]. Discrete Random Variables
10/67
Broadband Network Lab, NCTU, Taiwan 10
2.2 The Binomial Random Variable
-
8/8/2019 2[1]. Discrete Random Variables
11/67
Broadband Network Lab, NCTU, Taiwan 11
2.2 The Geometric Random Variable
Repeatedly and independently toss acoin with p(a head) = p, 0 < p < 1.
Geometric random variable X: Numberof tosses need for a head to come up
for the first time. PMF:
-
8/8/2019 2[1]. Discrete Random Variables
12/67
Broadband Network Lab, NCTU, Taiwan 12
2.2 The Geometric Random Variable
It is a legitimate PMF because
1
1 1
0
( ) (1 )
(1 )
1
1 (1 )
1
k
x
k k
k
k
p k p p
p p
pp
= =
=
=
=
=
=
-
8/8/2019 2[1]. Discrete Random Variables
13/67
Broadband Network Lab, NCTU, Taiwan 13
2.2 The Poisson Random Variable
PMF:
is a positive parameter characterizing
the PMF.
-
8/8/2019 2[1]. Discrete Random Variables
14/67
Broadband Network Lab, NCTU, Taiwan 14
2.2 The Poisson Random Variable
Binomial RV with small p and large n
Poisson RV with = np.
Justification: See Problem 12.
Example of Poisson RV: Number of typos in a book.
Number of cars involved in accidents in acity on a given day.
-
8/8/2019 2[1]. Discrete Random Variables
15/67
Broadband Network Lab, NCTU, Taiwan 15
2.3 Functions of Random Variables
IfY = g(X) is a function of a RV X, then
Y is also a RV.
The PMF ofY is
-
8/8/2019 2[1]. Discrete Random Variables
16/67
-
8/8/2019 2[1]. Discrete Random Variables
17/67
Broadband Network Lab, NCTU, Taiwan 17
2.3 Example 2.1 (2/2)
Possible values ofy = 0, 1, 2, 3, 4.
pY(0) = pX(0) = 1/9.
pY(1) = pX(-1) + pX(1) = 2/9.
Similar for y = 2, 3, 4.
PMF ofY is
PMF ofZ is
-
8/8/2019 2[1]. Discrete Random Variables
18/67
Broadband Network Lab, NCTU, Taiwan 18
2.4 Expectation
We define the expected value (also
called the expectation or the mean) of arandom variable X, with PMF pX, by
Physical meaning: Center of gravity of
the PMF.
-
8/8/2019 2[1]. Discrete Random Variables
19/67
Broadband Network Lab, NCTU, Taiwan 19
2.4 Example 2.2
Example 2.2 : Consider two independent coin
tosses, each with a 3/4 probability of a head,and let X be the number of heads obtained.This is a binomial random variable, its PMF is :
So the mean is :
2 4 Moments Va iance and
-
8/8/2019 2[1]. Discrete Random Variables
20/67
Broadband Network Lab, NCTU, Taiwan 20
2.4 Moments, Variance, andStandard Deviation
nth moment = E[Xn].
The 1st moment is just the mean.
Variance: var(X) = E[(X - E[X])2].
The variance provides a measure of
dispersion ofX around its mean.Another measure of dispersion is the
standard deviation ofX:
X has the same unit as X.
-
8/8/2019 2[1]. Discrete Random Variables
21/67
Broadband Network Lab, NCTU, Taiwan 21
2.4 Example 2.3
Recall Example 2.1.
;
2 4 Expected Value Rule for
-
8/8/2019 2[1]. Discrete Random Variables
22/67
Broadband Network Lab, NCTU, Taiwan 22
2.4 Expected Value Rule forFunctions of Random Variables
Let X be a random variable with PMF pX . Then,
the expected value of the random variable g(X)is given by :
Proof:
-
8/8/2019 2[1]. Discrete Random Variables
23/67
Broadband Network Lab, NCTU, Taiwan 23
2.4 Example 2.3 (continued)
The variance ofX can also be calculated as :
2 4 Mean and Variance of a Linear
-
8/8/2019 2[1]. Discrete Random Variables
24/67
Broadband Network Lab, NCTU, Taiwan 24
2.4 Mean and Variance of a LinearFunction of a Random Variable
Let X be a random variable and let Y = aX + b,
where a and b are given scalars. Then,
Proof : [ ] ( ) ( )
( ) ( )
[ ]
X
x
X X
x x
E Y ax b p x
a xp x b p x
aE x b
= +
= +
= +
2 4 Variance in Terms of Moments
-
8/8/2019 2[1]. Discrete Random Variables
25/67
Broadband Network Lab, NCTU, Taiwan 25
2.4 Variance in Terms of MomentsExpression
Proof:
-
8/8/2019 2[1]. Discrete Random Variables
26/67
Broadband Network Lab, NCTU, Taiwan 26
2.4 Example 2.4 (1/2)
If the weather is good (which happens with
probability 0.6), Alice walks the 2 miles toclass at a speed ofV = 5 miles per hour, andotherwise drives her motorcycle at a speed ofV = 30 miles per hour. What is the mean of
the time T to get to class? Sol:
-
8/8/2019 2[1]. Discrete Random Variables
27/67
Broadband Network Lab, NCTU, Taiwan 27
2.4 Example 2.4 (2/2)
However, it is wrong to calculate the mean of
the speed V .
To summarize, in this example we have
-
8/8/2019 2[1]. Discrete Random Variables
28/67
Broadband Network Lab, NCTU, Taiwan 28
2.4 Example 2.5
Mean and Variance of the Bernoulli.
PMF:
2 4 Discrete Uniformly Distributed
-
8/8/2019 2[1]. Discrete Random Variables
29/67
Broadband Network Lab, NCTU, Taiwan 29
2.4 Discrete Uniformly DistributedRandom Variable.
Discrete Uniformly Distributed Random
Variable. PMF:
2 4 Discrete Uniformly Distributed
-
8/8/2019 2[1]. Discrete Random Variables
30/67
Broadband Network Lab, NCTU, Taiwan 30
2.4 Discrete Uniformly DistributedRandom Variable.
Consider the case where a = 1 and b = n.
General case :
-
8/8/2019 2[1]. Discrete Random Variables
31/67
Broadband Network Lab, NCTU, Taiwan 31
2.4 Example 2.7
The Mean of the Poisson.
PMF:
var(X) = . (See Problem 24.)
-
8/8/2019 2[1]. Discrete Random Variables
32/67
Broadband Network Lab, NCTU, Taiwan 32
2.4 Example 2.8 (1/2)
Consider a quiz game where a person is given
two questions and must decide which questionto answer first. Question 1 will be answeredcorrectly with probability 0.8, and the personwill then receive as prize $100, while question2 will be answered correctly with probability0.5, and the person will then receive as prize
$200. If the first question attempted isanswered incorrectly, the quiz terminates, i.e.,the person is not allowed to attempt thesecond question. If the first question isanswered correctly, the person is allowed to
attempt the second question. Which questionshould be answered first to maximize theexpected value of the total prize moneyreceived?
-
8/8/2019 2[1]. Discrete Random Variables
33/67
Broadband Network Lab, NCTU, Taiwan 33
2.4 Example 2.8 (2/2)
Sol:
2 5 Joint PMF of Two Random
-
8/8/2019 2[1]. Discrete Random Variables
34/67
Broadband Network Lab, NCTU, Taiwan 34
2.5 Joint PMF of Two RandomVariables
X, Y: Two discrete random variables.
pX,Y : Joint PMF ofX and Y.
(x, y): A pair of possible values ofX and Y.
pX,Y (x, y) = P(X = x, Y = y).
Marginal PMFs:
Verify:
-
8/8/2019 2[1]. Discrete Random Variables
35/67
Broadband Network Lab, NCTU, Taiwan 35
2.5 Example 2.9
The marginal PMF ofX or Y at a given value is
obtained by adding the table entries along acorresponding column or row.
2 5 Functions of Multiple Random
-
8/8/2019 2[1]. Discrete Random Variables
36/67
Broadband Network Lab, NCTU, Taiwan 36
2.5 Functions of Multiple RandomVariables
Z = g(X, Y) defines another random
variables.
PMF:
Expected value:
When g(X, Y) = aX + bY + c,
-
8/8/2019 2[1]. Discrete Random Variables
37/67
Broadband Network Lab, NCTU, Taiwan 37
2.5 Example 2.9 (continued)
Now, a new random variable Z is defined by:
Please find E[z]=?
Alternatively, we can find E[z] from the PFM ofvariable Z. (see textbook p.95)
2 Z X Y = +
[ ] [ ] 2 [ ]E Z E X E Y = +
3 6 8 3 51[ ] 1 2 3 4
20 20 20 20 20E X = + + + =
3 7 7 3 50[ ] 1 2 3 4
20 20 20 20 20E Y = + + + =
51 50[ ] 2 7.55
20 20E Z = + =
2 5 More than Two Random
-
8/8/2019 2[1]. Discrete Random Variables
38/67
Broadband Network Lab, NCTU, Taiwan 38
2.5 More than Two RandomVariables
PMF of three random variables:
Marginal PMFs:
Expected value:
g(X1, X2,, Xn) = a1X1 + a2X2 + +anXn
2 l 2 0
-
8/8/2019 2[1]. Discrete Random Variables
39/67
Broadband Network Lab, NCTU, Taiwan 39
2.5 Example 2.10
Mean of the Binomial : Your probability class
has 300 students and each student hasprobability 1/3 of getting an A, independentlyof any other student. What is the mean ofX,the number of students that get an A?
Sol:
1 2 1[ ] 1 0
3 3 3iE X = + =
2 5 E l 2 11
-
8/8/2019 2[1]. Discrete Random Variables
40/67
Broadband Network Lab, NCTU, Taiwan 40
2.5 Example 2.11
The Hat Problem : Suppose that n people
throw their hats in a box and then each picksone hat at random. What is the expectedvalue ofX, the number of people that get backtheir own hat?
Sol: We introduce a random variable Xi that takesthe value 1 if the ith person selects his/her ownhat, and takes the value 0 otherwise.
;
2 6 Conditioning a Random Variable
-
8/8/2019 2[1]. Discrete Random Variables
41/67
Broadband Network Lab, NCTU, Taiwan 41
2.6 Conditioning a Random Variableon an Event
The conditional PMF of a random variable X,
conditioned on a particular event A withP(A) > 0, is defined by
Note that are disjoint for different x,therefore
Combining above, we can see that
so pX|A is a legitimate PMF.
{X = x} A
2 6 Vi li ti f ( )
-
8/8/2019 2[1]. Discrete Random Variables
42/67
Broadband Network Lab, NCTU, Taiwan 42
2.6 Visualization ofpX|A(x)
2 6 E l 2 12
-
8/8/2019 2[1]. Discrete Random Variables
43/67
Broadband Network Lab, NCTU, Taiwan 43
2.6 Example 2.12
Roll a Die. X: The roll of a fair six-sided die.
A: Event that the roll is an even number.Find pX|A(x).
Sol:
2 6 E l 2 13
-
8/8/2019 2[1]. Discrete Random Variables
44/67
Broadband Network Lab, NCTU, Taiwan 44
2.6 Example 2.13
A student will take a certain test repeatedly,
up to a maximum of n times, each time with aprobability p of passing, independently of thenumber of pervious attempts. What is the PMFof the number of attempts, given that the
student passes the test? Sol : Let A be the event that the student
passes the test.1
1
( ) (1 )n
m
m
P A p p
=
= 1
1
|
1
(1 ), 1,..., .
(1 )( )
0, otherwise
k
nm
X Am
p pif k n
p pP k
=
=
=
2.6 Conditioning one Random
-
8/8/2019 2[1]. Discrete Random Variables
45/67
Broadband Network Lab, NCTU, Taiwan 45
2.6 Conditioning one RandomVariable on Another
The conditional PMF pX|Y ofX given Y is
Normalization property:
The conditional PMF is often convenient for the
calculation of the joint PMF,
2 6 Vi li ti f ( | )
-
8/8/2019 2[1]. Discrete Random Variables
46/67
Broadband Network Lab, NCTU, Taiwan 46
2.6 Visualization ofpX|Y(x|y)
2 6 E l 2 14 (1/2)
-
8/8/2019 2[1]. Discrete Random Variables
47/67
Broadband Network Lab, NCTU, Taiwan 47
2.6 Example 2.14 (1/2)
Professor May B. Right often has her facts
wrong, and answers each of her studentsquestions incorrectly with probability 1/4,independently of other questions. In eachlecture May is asked 0, 1, or 2 questions with
equal probability 1/3. Let X and Y be thenumber of questions May is asked and thenumber of questions she answers wrong in agiven lecture, respectively. Please constructthe joint PMF pX,Y (x, y).
2 6 E ample 2 14 (2/2)
-
8/8/2019 2[1]. Discrete Random Variables
48/67
Broadband Network Lab, NCTU, Taiwan 48
2.6 Example 2.14 (2/2)
Sol:
2 6 Example 2 15 (1/2)
-
8/8/2019 2[1]. Discrete Random Variables
49/67
Broadband Network Lab, NCTU, Taiwan 49
2.6 Example 2.15 (1/2)
Consider a transmitter that is sending
messages over a computer network. Definethe following two random variables:
X : the travel time of a given message,
Y : the length of the given message.
Assume that the travel time X of the messagedepends on its length Y. The travel time is
104Y secs with probability 1/2, 103Y secswith probability 1/3, and 102Y secs withprobability 1/6. Please find the PMF of X.
2 6 Example 2 15 (2/2)
-
8/8/2019 2[1]. Discrete Random Variables
50/67
Broadband Network Lab, NCTU, Taiwan 50
2.6 Example 2.15 (2/2)
Sol: We can know:
and
So:
2 6 Conditional Expectations
-
8/8/2019 2[1]. Discrete Random Variables
51/67
Broadband Network Lab, NCTU, Taiwan 51
2.6 Conditional Expectations
The conditional expectation ofX given
an event A with P(A) > 0, is defined by
For a function g(X), we have
The conditional expectation ofX given avalue y ofY is defined by
2 6 Total Expectation Theorem
-
8/8/2019 2[1]. Discrete Random Variables
52/67
Broadband Network Lab, NCTU, Taiwan 52
2.6 Total Expectation Theorem
Total expectation theorem:
Let A1, , An form a partition of thesample space and P(Ai) > 0 for all i, then
Let A1, , An form a partition of anevent B and for all i, theniP(A B) > 0
2.6 Verify the Total Expectation
-
8/8/2019 2[1]. Discrete Random Variables
53/67
Broadband Network Lab, NCTU, Taiwan 53
y pTheorem
Verify
2 6 Example 2 16
-
8/8/2019 2[1]. Discrete Random Variables
54/67
Broadband Network Lab, NCTU, Taiwan 54
2.6 Example 2.16
Messages Transmission :
P(Boston to New York) = 0.5.P(Boston to Chicago) = 0.3.
P(Boston to San Francisco) = 0.2.
X: transit time of a message (random).
E[X | New York] = 0.05 s.
E[X | Chicago] = 0.1 s.
E[X | San Francisco] = 0.3 s.
Please find E[X]. Sol:
2 6 Example 2 17 (1/2)
-
8/8/2019 2[1]. Discrete Random Variables
55/67
Broadband Network Lab, NCTU, Taiwan 55
2.6 Example 2.17 (1/2)
Mean and Variance of the Geometric Random
Variable.
Alternative way to fine E[X] and var(X).
Define A1 = {X = 1} = {first try is a success},
A2 = {X > 1} = {first try is a failure}.
(1)
2 6 Example 2 17 (2/2)
-
8/8/2019 2[1]. Discrete Random Variables
56/67
Broadband Network Lab, NCTU, Taiwan 56
2.6 Example 2.17 (2/2)
So
(2)
2.7 Independence of a Random
-
8/8/2019 2[1]. Discrete Random Variables
57/67
Broadband Network Lab, NCTU, Taiwan 57
pVariable from an Event
We say the random variable X is
independent of the event A if
From the definition of the conditional
PMF, we have
If P(A) > 0, independence is the sameas the condition
-
8/8/2019 2[1]. Discrete Random Variables
58/67
2.7 Independence of Random
-
8/8/2019 2[1]. Discrete Random Variables
59/67
Broadband Network Lab, NCTU, Taiwan 59
pVariables
We say that two random variables X
and Y are independent if
or, equivalently,
X and Y are said to be conditionallyindependent, given P(A) > 0, if
Once more, this is equivalent to
2.7 Facts About Independent
-
8/8/2019 2[1]. Discrete Random Variables
60/67
Broadband Network Lab, NCTU, Taiwan 60
pRandom Variables
IfX and Y are independent random
variables, then E[XY] = E[X] E[Y].
For any functions g and h, g(X) and h(Y)are independent.
E[g(X) h(Y)] = E[g(X)] E[h(Y)].
var(X + Y) = var(X) + var(Y).
-
8/8/2019 2[1]. Discrete Random Variables
61/67
2.7 Independence of Several
-
8/8/2019 2[1]. Discrete Random Variables
62/67
Broadband Network Lab, NCTU, Taiwan 62
pRandom Variables
Three random variables X, Y, and Z are
said to be independent if
IfX, Y, and Z are independent random
variables, then f(X), g(Y), and h(Z) are also independent.
g(X, Y) and h(Z) are independent.
g(X, Y) and h(Y, Z) are usually not
independent.
2.7 Variance of the Sum of
-
8/8/2019 2[1]. Discrete Random Variables
63/67
Broadband Network Lab, NCTU, Taiwan 63
Independent Random Variables
IfX1, X2, , Xn are independent random
variables, then
Example 2.20 :
Xi ~ Bernoulli(p).X = X1 + + Xn ~ Binomial(n, p).
{Xi} are independent.
Sol:
2 7 Example 2 21 (1/2)
-
8/8/2019 2[1]. Discrete Random Variables
64/67
Broadband Network Lab, NCTU, Taiwan 64
2.7 Example 2.21 (1/2)
We wish to estimate the approval rating of a president,to be called C. To this end, we ask n persons drawn atrandom from the voter population, and we let Xi be arandom variable that encodes the response of the ithperson:
We model X1,X2, . . . , Xn as independent Bernoulli
random variables with common mean p and variancep(1 p). Naturally, we view p as the true approvalrating of C. We average the responses and computethe sample mean Sn, defined as
Please find E[Sn] and var[Sn].
2 7 Example 2 21 (2/2)
-
8/8/2019 2[1]. Discrete Random Variables
65/67
Broadband Network Lab, NCTU, Taiwan 65
2.7 Example 2.21 (2/2)
Sol:
-
8/8/2019 2[1]. Discrete Random Variables
66/67
2 7 Example 2 22 (2/2)
-
8/8/2019 2[1]. Discrete Random Variables
67/67
2.7 Example 2.22 (2/2)
To see how accurate this process is, consider nindependent Bernoulli random variables X1, . . . , Xn,
each with PMF
In a simulation context, Xi corresponds to the ithoutcome, and takes the value 1 if the ith outcomebelongs to the event A. The value of the randomvariable
is the estimate ofP(A) provided by the simulation.According to Example 2.21, X has mean P(A) andvariance P(A)(1P(A))/n, so that for large n, it providesan accurate estimate ofP(A).