Review of Probability Theory
-
Upload
clarke-soto -
Category
Documents
-
view
44 -
download
1
description
Transcript of Review of Probability Theory
1
Review of Probability Theory
[Source: Stanford University]
2
A random experiment with set of outcomes
Random variable is a function from set of outcomes to real numbers
Random Variable
3
Example
Indicator random variable:
A : A subset of is called an event
4
CDF and PDF Discrete random variable:
The possible values are discrete (countable)
Continuous random variable: The rv can take a range of values in R
Cumulative Distribution Function (CDF):
PDF and PMF:
5
Expectation and higher moments
Expectation (mean):
if X>0 :
Variance:
6
Two or more random variables
Joint CDF:
Covariance:
7
Independence
For two events A and B:
Two random variables
IID : Independent and Identically Distributed
8
Useful Distributions
9
Bernoulli Distribution
The same as indicator rv:
IID Bernoulli rvs (e.g. sequence of coin flips)
10
Binomial Distribution Repeated Trials:
Repeat the same random experiment n times. (Experiments are independent of each other)
Number of times an event A happens among n trials has Binomial distribution (e.g., number of heads in n coin tosses, number of arrivals in n time
slots,…)
Binomial is sum of n IID Bernoulli rvs
11
Mean of Binomial
Note that:
12
1 2 3 4 5 6 7 8 9 10 11 120
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Binomial - Example
n=4
n=40
n=10
n=20
p=0.2
13
Binomial – Example (ball-bin)
There are B bins, n balls are randomly dropped into bins.
: Probability that a ball goes to bin i : Number of balls in bin i after n drops
14
Multinomial Distribution Generalization of Binomial Repeated Trails (we are interested in
more than just one event A) A partition of into A1,A2,…,Al
Xi shows the number of times Ai occurs
among n trials.
15
Poisson Distribution
Used to model number of arrivals
16
0 5 10 150
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Poisson Graphs
=10
=.5
=1
=4
17
Poisson as limit of Binomial
Poisson is the limit of Binomial(n,p) as
Let
18
0 1 2 3 4 5 6 7 8 9 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Poisson and Binomial
Poisson(4)
n=5,p=4/5
n=20, p=.2
n=10,p=.4
n=50,p=.08
19
Geometric Distribution
Repeated Trials: Number of trials till some event occurs
20
Exponential Distribution
Continuous random variable Models lifetime, inter-arrivals,…
21
Minimum of Independent Exponential rvs
: Independent Exponentials
22
Memoryless property
True for Geometric and Exponential Dist.:
The coin does not remember that it came up tails l times
Root cause of Markov Property.
23
Proof for Geometric
24
Moment Generating Function (MGF) For continuous rvs (similar to Laplace
transform)
For Discrete rvs (similar to Z-transform):
Characteristic Function
25
Characteristic Function
Can be used to compute mean or higher moments:
If X and Y are independent and T=X+Y
26
Useful CFs
Bernoulli(p) :
Binomial(n,p) :
Multinomial:
Poisson: