Statistics and Probability for Engineers
Joost Slingerlandbased (heavily) on notes by Ollie Mason
Semester 1, 2017-2018
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Introduction/Motivation
Why study statistics and probability?Apart from the intrinsic interest and beauty, there are many, manyapplications. For example,
Probability is fundamental in Queueing Theory and the designof modern communications systems.
Information Theory, which is key to data compression and thedesign of error correcting codes relies heavily on probability.
Modern methods for information retrieval from the Web andmathematical models of the Web and other complex real worldnetworks are built on probability theory. GOOGLE ranks pagesaccording to the probability that a random surfer visits them!
Control systems in the real world are subject to randomdisturbances and noise - modelling and designing such systemsrequires probability.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Introduction/Motivation
Statistical methods are developed using Probability Theory.
Quality improvement: In comparing different products andprocesses, statistics is key to determining if there is asignificant difference between them.
Determining the reliability, lifetime of products based onsampling relies on statistical methods.
Market research methods depend heavily on statistics.
In planning how to allocate resources (schools, transportfacilities, manpower), statistical methods are required.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Course Outline
Basic Probability
Definitions and basic rules of probability
Counting Techniques
Conditional Probability, Independence and Bayes Theorem
Random Variables RVs
Discrete RVs and probability mass functions
Binomial and Poisson distributions
Markov and Chebyshev Inequalities for Discrete RVs
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Course Outline
Random Variables
Continuous RVs - probability density and cumulativedistribution functions
Gaussian (Normal) distribution and applications
Exponential distribution
Gamma and Weibull distributions and applications
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Course Outline
Statistics
Introduction to Sampling
Central Limit Theorem
Confidence intervals and Hypothesis Tests for populationmeans and proportions
Chi-square Tests
Correlation and regression - prediction intervals andhypothesis testing
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Sample Spaces and Events
We start our investigation of probability with some basic conceptsand definitions. In probability we are always concerned withexperiments in which the outcome is not determined in advance.For example:
Rolling a 6 sided die;
Dealing a hand of 5 cards from a deck of 52;
Measuring the time to failure of a mechanical component.
Measuring rainfall on a given day.
Measuring CO2 emissions from a car or a factory.
Measuring how many people use a rail link between 7am and9am on a given day.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Sample Spaces and Events
Definition
The sample space of an experiment is the set of all possibleoutcomes of the experiment.
We often denote the sample space by Ω
If we roll an unbiased (fair) 6-sided die, then the sample spaceis Ω = 1, 2, 3, 4, 5, 6.If we toss a fair coin twice, and denote heads by H and tailsby T then the sample space is Ω = HH,HT ,TH,TT.
In all of these examples, the sample space is finite.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Sample Spaces and Events
In many examples, sample spaces are not finite.
If we toss a fair coin until heads comes up, then the samplespace is H,TH,TTH,TTTH, . . . , . This time the samplespace is infinite. However, the sample space is still countableor discrete.
If we are measuring the emissions from cars in an NCT centreand are interested in how many cars we need to test before wefind one that fails, then our sample space is again countablyinfinite.
If we are measuring the actual emissions in g/km then oursample space would be continuous.
Note that the definition of a sample space depends on what we areinterested in measuring. We may only be interested in whether ornot a car fails an emission test in which case the sample spacewould be Pass,Fail.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Sample Spaces and Events
In probability theory, we are concerned with quantifying thelikelihood of events. For our purposes, we shall think of events assubsets of the sample space. For example:
Example
If we toss an unbiased coin twice, then the following are examplesof events.
Getting heads twice - HH.Getting at least one tail - HT ,TH,TT.Getting heads exactly once - HT ,TH.
All of these are subsets of the sample spaceΩ = HH,HT ,TH,TT
In some examples, not all subsets of the sample space will beevents.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Sample Spaces and Events
Example
If we roll a fair 6-sided die twice, then the following are allexamples of events.
Getting two 5’s - (5, 5).Getting two numbers that sum up to 10 -(4, 6), (5, 5), (6, 4).Getting two numbers that differ by 4 -(1, 5), (2, 6), (5, 1), (6, 2).Getting two numbers that sum to 8 -(2, 6), (3, 5), (4, 4), (5, 3), (6, 2).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
Example
Two different models of electric motor are tested for 1,000 hoursand the time to failure of each model is measured. Here oursample space is continuous
(x1, x2) : 0 ≤ x1 ≤ 1000, 0 ≤ x2 ≤ 1000.
The following are examples of events:
Model 1 fails first: (x1, x2)|x1 < x2.A failure occurs before 500 hours: (x1, x2|min(x1, x2) < 500.Both models fail at exactly the same time : (x1, x2)|x1 = x2.
Before defining some basic laws on how to combine events, weneed to review some elementary notions from set theory.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
Events can be combined in a variety of ways corresponding tostandard operations on sets. The three basic operations are union,intersection and complement.
Union E ∪ F is the event consisting of all outcomes that areeither in E or F (or both).
Intersection E ∩ F is the event consisting of all outcomes that arein both E and F .
Complement E c is the event consisting of all outcomes notbelonging to E .
E c is sometimes written as E ′ (e.g. in Montgomery & Runger)
Standard Notations∅ denotes the empty set or the impossible eventE\F denotes E ∩ F c .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
Another way to think about the combinations of events we justdescribed is as follows. If E and F are events, then
Union E ∪ F is the event that E or F or both happen.(Mathematicians would just say E or F happens, butthey mean that both may happen)
Intersection E ∩ F is the event that both E and F happen.
Complement E c is the event that E does not happen.
We can extend the list with more complicated events, for exampleE\F is the event that E happens, but F does not
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
Venn diagrams are a useful way to think about events andcombinations of events:
E F
E ∪ F is the entire coloured region;
E ∩ F is the purple shaded region;
(E ∪ F )c is the unshaded region.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Mutually Exclusive Events
Events with E ∩ F = ∅ are said to be mutually exclusive.
EF
G
E and F are mutually exclusive
E and G are not
F and G are not
Another word for mutually exclusive is disjoint
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Sets - Some Simple Rules
E ∪ (F ∪ G ) = (E ∪ F ) ∪ G ;
E ∩ (F ∩ G ) = (E ∩ F ) ∩ G ;
E ∩ (F ∪ G ) = (E ∩ F ) ∪ (E ∩ G );
E ∪ (F ∩ G ) = (E ∪ F ) ∩ (E ∪ G ).
de Morgan’s laws Verify using Venn diagrams and with the algebraof sets that
(E ∪ F )c = E c ∩ F c ;
(E ∩ F )c = E c ∪ F c .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
Example
An unbiased coin is tossed twice. Consider the events:
Getting heads twice - E = HH.Getting at least one tail - F = HT ,TH,TT.Getting heads exactly once - G = HT ,TH.
Then E and F are mutually exclusive.E ∪ G = HH,TH,HT. F ∩ G = G .E c = HT ,TH,TT, G c = TT ,HH.E c ∩ G c = TT = (E ∪ G )c .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
E is a subset of F if every element (outcome) in E is also in F .Formally, we write
E ⊆ F ⇐⇒ x ∈ E ⇒ x ∈ F .
E
F
This means that if the event E occurs, then F has occurred.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Events and Sets
Example
A fair 6-sided die is rolled twice. Consider the following events.
Getting two 5’s - E1 = (5, 5).Getting two numbers that sum up to 10 -E2 = (4, 6), (5, 5), (6, 4).Getting two numbers that differ by 4 -E3 = (1, 5), (2, 6), (5, 1), (6, 2).Getting two numbers that sum to 8 -E4 = (2, 6), (3, 5), (4, 4), (5, 3), (6, 2).
Then E1 ⊆ E2. E3 ∩ E4 = (2, 6), (6, 2). E3\E4 = (1, 5), (5, 1).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Axioms of Probability and Interpretations
Axioms of Probability and Interpretations
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Classical Probability
The sample space describes all possible outcomes of anexperiment. Events represent sets of outcomes that are of interest.We are now interested in quantitatively describing the likelihood ofdifferent events.
Classical Probability
An experiment has N outcomes that are equally likely tooccur;
The event E contains exactly k of these outcomes;
Then the probability of E is P(E ) = kN .
Essentially, in this case the probability of an event is the ratio offavourable outcomes to all possible outcomes .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Classical Probability
Example
If the sample space for an experiment is a, b, c , d , e and alloutcomes are equally likely, what is:
P(a, b): Answer: 25
P(c , d , e): Answer: 35
Example
If a card is drawn from a well shuffled pack, what is the probabilitythat:
it is an ace? Answer: 452
it is a spade? Answer: 1352 = 1
4
it is red? Answer: 12 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Classical Probability
Example
If a fair die is rolled twice, what is the probability that the sum ofthe two scores is equal to 10?
Let E be the event that the sum is equal to 10.
The size of the sample space is N = 36.
E = (4, 6), (5, 5), (6, 4) so k = 3.
Therefore P(E ) = 336 = 1
12 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Problems with Classical Probability
The classical concept of probability has limited applicability. Inmany real situations, not all outcomes are equally likely.
Football matches or horse races;
Lifetime of a light bulb - 100 hours, 1020 hours, lifetime of theuniverse.
Height of a person selected at random from the population.
Wind speed measured at a potential site for a wind farm.
Does a windscreen shatter or not when subjected to a specificstress?
Survey on how people travel to work - bus, cycle, rail, drive,walk, hang-glide, helicoptor.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Frequency Interpretation of Probability
An alternative concept of probability is the relative frequencyinterpretation.
Relative Frequency
The experiment is repeated N times (N is very large);
The event E occurs N(E ) times.
Then the probability of E is (approximately) P(E ) = N(E)N .
The probability of an event is the proportion of time the eventoccurs if the experiment is repeated a large number of times.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Frequency Interpretation of Probability
Example
A particular rail service has run 1000 times and has been on timeon exactly 970 of these times. Using the frequency interpretation ofprobability, what is the probability of this service running on time?
SolutionE - event that the service runs on time. Then P(E ) = 970
1000 = 0.97.
Similarly, if 70 of the last 10,000 integrated circuit chips producedby a factory have had a crack, then the probability of a chip fromthis factory being cracked is 70
10,000 = .007.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Frequency Interpretation
Example
A sample of 150 plastic panels was selected and tested for shockresistance and scratch resistance. The results are summarised inthe table below.
shock resistancehigh low
scratch high 125 12resistance low 7 6
If a panel is selected at random, what is the probability that it has:
high scratch and high shock resistance? Answer: 125150 .
high scratch or high shock resistance? Answer: 144150 .
high scratch and low shock resistance? Answer: 12150 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Problems with Frequency Interpretation
How large does N have to be?
Is the concept well-defined? If we carry out two separate runsof N experiments, the numbers do not need to be equal.
One approach is to define P(E ) as
limN→∞N(E )
N.
The question then arises if this limit exists and, if so, how dowe calculate it in practice.
The problems with constructing a meaningful theory using eitherthe frequency or classical concepts of probability led to thedevelopment of an axiomatic approach (due to A. N. Kolmogorov).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Axioms of Probability
Sample space We begin with a sample space Ω. Events are subsetsof Ω but not all subsets are necessarily events.
Set of events We call the set of all events F , so E ∈ F ⇒ E ⊂ Ω.Simple combinations of events are also events, so Fmust be closed under complement and undercountable unions and countable intersections. Such aset of events is called a sigma algebra.
Probability measure A probability measure P : F → R is a mapwhich assigns a probability to each event.
Probability measures must satisfy the following axioms.
1 P(E ) ≥ 0 for all events E .
2 P(Ω) = 1. This means that the probability of the entiresample space must equal 1 - something must occur;
3 If events E1,E2,E3, . . ., are disjoint, meaning Ei ∩ Ej = ∅ forall i 6= j , then P(∪∞i=1Ei ) =
∑∞i=1 P(Ei ).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Axioms of Probability
These axioms make sense in terms of the frequency interpretationof probability.
Repeat an experiment N times. The event E occurs k times.Obviously, 0 ≤ k ≤ N. So 0 ≤ P(E ) ≤ 1.(We will soon show that the axioms indeed give P(E ) ≤ 1)
Ω is the collection of all possible outcomes, so the event Ωmust happen each time we repeat the experiment. SoP(Ω) = 1.
If E and F are mutually exclusive and if E occurs kE times, Foccurs kF times, then E ∪ F must occur exactly kE + kFtimes. So
P(E ∪ F ) =kE + kF
N= P(E ) + P(F ).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Mutually Exclusive Events
Example
A fair 6-sided die is rolled twice. What is the probability that thesum of the scores is either strictly greater than 10 or strictly lessthan 5?
Let E denote the event that the sum is strictly greater than10 and F the event that the sum is strictly less than 5.
E = (5, 6), (6, 5), (6, 6),F = (1, 3), (1, 2), (1, 1), (2, 2), (2, 1), (3, 1).P(E ) = 3
36 = 112 , P(F ) = 6
36 = 16 .
As these are mutually exclusive (why?) we can writeP(E ∪ F ) = 1
12 + 16 = 1
4 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Simple properties of P
We have P(∅) = 0.To see this, take the sequence in axiom 3 to be Ei = ∅ for all i .Clearly this gives Ei ∩ Ej = ∅ ∩ ∅ = ∅. Also, ∪∞i=1Ei = ∅.The axiom then gives P(∅) = P(∅) + P(∅) + P(∅) + . . .Clearly this implies P(∅) = 0.
If E1 ∩ E2 = ∅ then we have P(E1 ∪ E2) = P(E1) + P(E2).This comes from axiom 3 if we take E3 = E4 = . . . = ∅.Similarly for any finite number of mutually exclusive eventsE1, . . .En we have the finite version of axiom 3,P(∪ni=1Ei ) =
∑ni=1 P(Ei ).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Complementary Events
The event E and the event E c = S\E are said to becomplementary. Complementary events are always mutuallyexclusive.
Thus we know that P(E ∪ E c) = P(E ) + P(E c).
But E ∪ E c = S and P(S) = 1.
This means that P(E c) = 1− P(E ) for any event E .
Theorem
For any event E , P(E c) = 1− P(E ).
This also shows that P(E ) ≤ 1 for any event (as expected).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Example
A fair die is rolled twice. What is the probability that the sum ofthe scores is less than or equal to 10?
There are many possible ways that this event can occur.However this event is the complement of the event that thesum is strictly greater than 10.
Let E denote the event that the sum is strictly greater than10. We have previously calculated that P(E ) = 1
12
Hence the answer to our problem is 1− 112 = 11
12 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Complementary Events
Example
A computer is used to randomly generate 3 digit PINs for access toa building. (0 is allowed as a digit in any position). What is theprobability that a PIN generated in this way contains at least 2distinct digits?
First note that the set of all possible PINs can be identifiedwith the set of integers from 0 to 999. Thus the size of thesample space is 1000.
Let E be the event that a PIN contains the same digitrepeated three times.
The number of ways E can occur is 10. So P(E ) = 1100 .
We are interested in the complement of E . Thus theprobability we want is 1− P(E ) = 99
100 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
More Laws of Probability
Theorem
If the event E := e1, e2, . . . , ek, then
P(E ) = P(e1) + P(e2) + · · ·P(ek).
Events consisting of a single outcome such as e1 are elementaryevents.
Theorem
If the event E is a subset of the event F , E ⊆ F , thenP(E ) ≤ P(F ).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Examples
Example
5 tools are used to produce a certain machine part. If a machinepart is equally likely to have been produced by each tool, what isthe probability that a part selected at random:
Was produced by tool 1 or tool 2?Answer: P(1, 2) = 2
5 .
was produced by tool 2, tool 4 or tool 5?Answer: P(2, 4, 5) = 3
5 .
was not produced by tool 5?Answer: P(5c) = 1− P(5) = 4
5 .
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
General Addition Formula
How do we calculate P(E ∪ F ) for two events that are notmutually exclusive?
First, let’s write E ∪ F as a combination of mutually exclusiveevents.
E ∪ F = (E ∩ F c) ∪ (E ∩ F ) ∪ (E c ∩ F ).
Thus we can write using the previous result on mutually exclusiveevents:
P(E ∪ F ) = P(E ∩ F c) + P(E ∩ F ) + P(E c ∩ F ).
Adding and subtracting P(E ∩ F ) and we find that
P(E∪F ) = (P(E∩F c)+P(E∩F ))+(P(E∩F )+P(E c∩F ))−P(E∩F )
.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
General Addition Formula
ButP(E ) = P(E ∩ F c) + P(E ∩ F )
andP(F ) = (P(E ∩ F ) + P(E c ∩ F ))
Hence the calculation on the last slide gives us the following result.
Theorem
For any two events E and F ,P(E ∪ F ) = P(E ) + P(F )− P(E ∩ F ).
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Example
Example
E and F are events and P(E ) = 0.3, P(F ) = 0.4 andP(E ∩ F ) = 0.2. What is
P(E c)?
P(E ∪ F )?
P(E c ∩ F c)?
P(E c ∪ F c)?
P(E ∪ F c)?
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Example
Example
E and F are events and P(E ) = 0.3, P(F ) = 0.4 andP(E ∩ F ) = 0.2. What is
P(E c) = 0.7.
P(E ∪ F ) = 0.3 + 0.4− 0.2 = 0.5.
P(E c ∩ F c) = P((E ∪ F )c) = 1− 0.5 = 0.5
P(E c ∪ F c) = 0.8
P(E ∪ F c): P(E ∩ F c) = 0.1 soP(E ∪ F c) = 0.3 + 0.6− 0.1 = 0.8
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Examples
Example
A, B and C are events and A and C are mutually exclusive. IfP(A) = 0.2, P(B) = 0.6, P(C ) = 0.5 and P(A ∩ Bc) = 0.1:
What is the probability P(A ∩ C )?Answer As A and C are mutually exclusive,P(A ∩ C ) = P(∅) = 0.
What is the probability P(A ∪ C )?Answer As A and C are mutually exclusive,P(A ∪ C ) = P(A) + P(C ) = 0.7.
What is the probability P(A ∪ B)?Answer P(A ∩ B) = 0.1 so P(A ∪ B) = 0.2 + 0.6− 0.1 = 0.7.
What can you say about P(B ∩ C )?P(B ∩ C ) ≥ 0.2.
Joost Slingerland based (heavily) on notes by Ollie Mason Statistics and Probability for Engineers
Top Related