RPRA 2. Elements of Probability Theory 1
Engineering Risk Benefit Analysis1.155, 2.943, 3.577, 6.938, 10.816, 13.621, 16.862, 22.82,
ESD.72, ESD.721
RPRA 2. Elements of Probability Theory
George E. ApostolakisMassachusetts Institute of Technology
Spring 2007
RPRA 2. Elements of Probability Theory 2
Probability: Axiomatic Formulation
The probability of an event A is a number that satisfies thefollowing axioms (Kolmogorov):
0 ≤ P(A) ≤ 1
P(certain event) = 1
For two mutually exclusive events A and B:
P(A or B) = P(A) + P(B)
RPRA 2. Elements of Probability Theory 3
Relative-frequency interpretation
• Imagine a large number n of repetitions of the “experiment” of which A is a possible outcome.
• If A occurs k times, then its relative frequency is:
• It is postulated that:
nk
)A(Pnklim
n≡
∞→
RPRA 2. Elements of Probability Theory 4
Degree-of-belief (Bayesian) interpretation
• No need for “identical” trials.
• The concept of “likelihood” is primitive, i.e., it is meaningful to compare the likelihood of two events.
• P(A) < P(B) simply means that the assessor judges B to be more likely than A.
• Subjective probabilities must be coherent, i.e., must satisfy the mathematical theory of probability and must be consistent with the assessor’s knowledge and beliefs.
RPRA 2. Elements of Probability Theory 5
Basic rules of probability: Negation
S
EVenn Diagram___E SEE =∪
⇒==+ 1)S(P)E(P)E(P
)E(P1)E(P −=
RPRA 2. Elements of Probability Theory 6
Basic rules of probability: Union
( ) ( )
( ) ⎟⎠⎞
⎜⎝⎛−+
+−=⎟⎠⎞
⎜⎝⎛
+
−
= +==∑ ∑∑
I
U
KN
1i
1N
1N
1i
N
1ijji
N
1ii
N
1i
AP1
AAPAPAP
( )∑=
≅⎟⎟⎠
⎞⎜⎜⎝
⎛ N
1ii
N
1i APAP U
Rare-Event Approximation:
RPRA 2. Elements of Probability Theory 7
Union (cont’d)
• For two events: P(A∪B) = P(A) + P(B) – P(AB)
• For mutually exclusive events:
P(A∪B) = P(A) + P(B)
RPRA 2. Elements of Probability Theory 8
Example: Fair Die
Sample Space: {1, 2, 3, 4, 5, 6} (discrete)
“Fair”: The outcomes are equally likely (1/6).
P(even) = P(2 ∪ 4 ∪ 6) = ½ (mutually exclusive)
RPRA 2. Elements of Probability Theory 9
Union of minimal cut sets
∏−∑ ∑∑=
+−
= +==++−=
N
1ii
1N1N
1i
N
1ijji
N
1iiT MMMMX 1)(...
Rare-event approximation: ( ) ( )∑≅N
1iT MPXP
From RPRA 1, slide 15
⎟⎟⎠
⎞⎜⎜⎝
⎛++−= ∏−∑ ∑∑
=
+−
= +==
N
ii
NN
i
N
ijji
N
iiT MP...)MM(P)M(P)X(P )(
1
11
1 111
RPRA 2. Elements of Probability Theory 10
Upper and lower bounds
( ) ( )∑≤N
1iT MPXP
∑ ∑∑−
= +==−≥
1N
1i
N
1ijji
N
1iiT )MM(P)M(P)X(P
∏−∑ ∑∑=
+−
= +==++−=
N
1ii
1N1N
1i
N
1ijji
N
1iiT )M(P...)MM(P)M(P)X(P )1(
The first “term,” i.e., sum, gives an upper bound.
The first two “terms,” give a lower bound.
RPRA 2. Elements of Probability Theory 11
Conditional probability
( ) ( ) ( ) ( ) ( )APA/BPBPBAPABP ==
( ) ( )( )BPABPBAP ≡
For independent events:
( ) ( ) ( )( ) ( )BPA/BP
BPAPABP=
=
Learning that A is true has no impact on our probability of B.
RPRA 2. Elements of Probability Theory 12
Example: 2-out-of-4 System
1
2
3
4
M1 = X1 X2 X3 M2 = X2 X3 X4
M3 = X3 X4 X1 M4 = X1 X2 X4
XT = 1 – (1 – M1) (1 – M2) (1 – M3) (1 – M4)
XT = (X1 X2 X3 + X2 X3 X4 + X3 X4 X1 + X1 X2 X4) - 3X1 X2 X3X4
RPRA 2. Elements of Probability Theory 13
2-out-of-4 System (cont’d)
P(XT = 1) = P(X1 X2 X3 + X2 X3 X4 + X3 X4 X1 + X1 X2 X4) –3P(X1 X2 X3X4)
Assume that the components are independent and nominallyidentical with failure probability q. Then,
P(XT = 1) = 4q3 – 3q4
Rare-event approximation: P(XT = 1) ≅ 4q3
RPRA 2. Elements of Probability Theory 14
Updating probabilities (1)
• The events, Hi, i = 1...N, are mutually exclusive and exhaustive, i.e., Hi∩Hj= Ø, for i≠j, ∪Hi = S, the sample space.
• Their probabilities are P(Hi).
• Given an event E, we can always write
)H(P)H/E(P)E(P iN
1i∑=
H1
H2
H3
E
RPRA 2. Elements of Probability Theory 15
Updating probabilities (2)
• Evidence E becomes available.
• What are the new (updated) probabilities P(Hi/E)?Start with the definition of conditional probabilities, slide 11.
• Using the expression on slide 14 for P(E), we get
⇒== )E(P)E/H(P)H(P)H/E(P)EH(P iiii
)E(P)H(P)H/E(P)E/H(P ii
i =
RPRA 2. Elements of Probability Theory 16
Bayes’ Theorem
PosteriorProbability
PriorProbability
Likelihood of theEvidence
( ) ( ) ( )( ) ( )∑
= N
1ii
iii
HPHEP
HPHEPEHP
RPRA 2. Elements of Probability Theory 17
Example: Let’s Make A Deal
• Suppose that you are on a TV game show and the host has offered you what's behind any one of three doors. You are told that behind one of the doors is a Ferrari, but behind each of the other two doors is a Yugo. You select door A.
At this time, the host opens up door B and reveals a Yugo. He offers you a deal. You can keep door A or you can trade it for door C.
• What do you do?
RPRA 2. Elements of Probability Theory 18
Let’s Make A Deal: Solution (1)• Setting up the problem in mathematical terms:
A = {The Ferrari is behind Door A}B = {The Ferrari is behind Door B}C = {The Ferrari is behind Door C}
• The events A, B, C are mutually exclusive and exhaustive.• P(A) = P(B) = P(C) = 1/3
E = {The host opens door B and a Yugo is behind it}
What is P(A/E)? ⇒ Bayes’ Theorem
RPRA 2. Elements of Probability Theory 19
Let’s Make A Deal: Solution (2)
( )( ) ( )
( ) ( ) ( ) ( ) ( ) ( )CPCEPBPBEPAPAEPAPAEP
EAP
++=
=
But
P(E/B) = 0 (A Yugo is behind door B).
P(E/C) = 1 (The host must open door B, if the Ferrari is behind door C; he cannot open door A under any circumstances).
RPRA 2. Elements of Probability Theory 20
Let’s Make A Deal: Solution (3)
•Let P(A/E) = x and P(E/A) = p
•Bayes' theorem gives:p1
px+
=
Therefore
• For P(E/A) = p = 1/2 (the host opens door B randomly, if the Ferrari is behind door A)
⇒ P(A/E) = x = 1/3 = P(A) (the evidence has had no impact)
RPRA 2. Elements of Probability Theory 21
Let’s Make A Deal: Solution (4)
• Since P(A/E) + P(C/E) = 1 ⇒
• P(C/E) = 1 - P(A/E) = 2/3 ⇒
⇒ The player should switch to door C
• For P(E/A) = p = 1 (the host always opens door B, if the Ferrari is behind door A)
⇒ P(A/E) = 1/2 ⇒ P(C/E) = 1/2, switching to door C does not offer any advantage.
RPRA 2. Elements of Probability Theory 22
Random Variables
• Sample Space: The set of all possible outcomes of an experiment.
• Random Variable: A function that maps sample points onto the real line.
• Example: For a die ⇒ S = {1,2,3,4,5,6}
• For the coin: S = {H, T} ≡ {0, 1}
RPRA 2. Elements of Probability Theory 23
Events
- ∞ ∞3 542 1 6
3.6
0
We say that {X ≤ x} is an event, where x is any number on the real line.
For example (die experiment):
{X ≤ 3.6} = {1, 2, 3} ≡ {1 or 2 or 3}
{X ≤ 96} = S (the certain event)
{X ≤ -62} = ∅ (the impossible event)
RPRA 2. Elements of Probability Theory 24
Sample Spaces
• The SS for the die is an example of a discrete sample space and X is a discrete random variable (DRV).
• A SS is discrete if it has a finite or countably infinite number of sample points.
• A SS is continuous if it has an infinite (and uncountable) number of sample points. The corresponding RV is a continuous random variable (CRV).
• Example:{T ≤ t} = {failure occurs before t}
RPRA 2. Elements of Probability Theory 25
Cumulative Distribution Function (CDF)
• The cumulative distribution function (CDF) is
F(x) ≡ Pr[X ≤ x]
• This is true for both DRV and CRV.
Properties:1. F(x) is a non-decreasing function of x.2. F(-∞) = 03. F(∞) = 1
RPRA 2. Elements of Probability Theory 27
Probability Mass Function (pmf)
• For DRV: probability mass function
( ) ii pxXP ≡=( ) ∑ ≤= xxallforpxF ii ,
1p)S(Pi
i == ∑ normalization
Example: For the die, pi = 1/6 and
31
61
61p)21(P)3.2(F
2
1i =+==∪= ∑
1p6
1i =∑
RPRA 2. Elements of Probability Theory 28
Probability Density Function (pdf)
f(x)dx = P{x < X < x+dx}
( ) ( )dx
xdFxf =
∫∞
∞−==∞= 1ds)s(f)(F)S(P
∫∞−
=x
ds)s(f)x(F
normalization
RPRA 2. Elements of Probability Theory 29
Example of a pdf (1)•Determine k so that
( ) 1x0forkxxf 2 ≤≤= ,
( ) ,0xf = otherwise
is a pdf.
Answer:
The normalization condition gives:
∫ =⇒=1
0
2 3k1dxkx
RPRA 2. Elements of Probability Theory 30
Example of a pdf (2)
3xxF =)(
=∫=− dxx3)75.0(F)875.0(F875.0
75.0
2
= 0.67 - 0.42 = 0.25 =
= P{0.75 < X < 0.875}
1
1
f(x)
F(x)
3
0.67
0.42
0.75 0.875
1
RPRA 2. Elements of Probability Theory 31
Moments
Expected (or mean, or average) value
[ ]( )
⎪⎪⎩
⎪⎪⎨
⎧
≡≡∑
∫∞
∞−
jjj DRVpx
CRVdxxfxmXE
Variance (standard deviation σ )
( )[ ]( ) ( )
( )⎪⎪⎩
⎪⎪⎨
⎧
−
−=≡−
∑
∫σ
∞
∞−
jj
2j
222
DRVpmx
CRVdxxfmxmXE
RPRA 2. Elements of Probability Theory 32
Percentiles
• Median: The value xm for which
• F(xm) = 0.50
• For CRV we define the 100γ percentile asthat value of x for which
( )∫γ
∞−γ=
x
dxxf
Top Related