IE 4521 Practice Midterm Problems

John Gunnar Carlsson

October 24, 2009

1 Probability problems

1.1 Dice

Suppose we roll a pair of dice:

1. What is the probability that the second die lands on a higher value than the first die?

Solution By writing out all pairs as in slide 4 of lecture 2, we find that the second die is higher than the firstdie in 15 of the 36 cases, so the answer is 15/36 = 5/12.

2. What is the probability that the sum of the upturned faces is i, for i ∈ {2, . . . , 12}?

Solution See slide 6 of lecture 2.

3. Suppose that we know the sum of the upturned faces is either 5 or 7. What is the probability that the sum is5?

Solution Let A be the event that the sum of the faces is 5 and B the event that the sum of the faces is 7. LetC = A ∪ B be the event that the sum of the faces is either 5 or 7. Then Pr (A) = 4/36, Pr (B) = 6/36, andPr (C) = Pr (A) + Pr (B) = 10/36. Then

Pr ( A|C) =Pr (A ∩ C)

Pr (C)=

Pr (A)Pr (C)

=4/3610/36

= 2/5

1.2 Machine failure

A system is composed of 5 components, each of which is either working or broken. Consider an experiment thatconsists of observing the status of each component, and let the outcome of the experiment be given by the vector(x1, x2, x3, x4, x5), where xi = 1 if component i works and 0 if component i is broken.

1. How many outcomes are in the sample space of this experiment?

Solution There are 25 = 32 states.

2. Suppose that the system will work if components 1 and 2 are both working, or if components 3 and 4 areboth working, or if components 1, 3, and 5 are all working. Let W denote the event that the system works.Specify all the outcomes in W.

Solution The event W consists of all events of the form (1, 1, x3, x4, x5), (x1, x2, 1, 1, x5), and (1, x2, 1, x4, 1).There are 15 total events which can be computed by writing all the possibilities out exhaustively.

3. Let A denote the event that components 4 and 5 are broken. How many outcomes are contained in event A?

1

Solution Event A consists of all outcomes of the form (x1, x2, x3, 0, 0). Since we can freely choose x1, x2,and x3, there are 23 = 8 outcomes.

4. How many outcomes are contained in event A ∩W?

Solution By enumerating possibilities (i.e. just writing them all out), we find that outcomes (1, 1, 1, 0, 0)and (1, 1, 0, 0, 0) are common to A and W.

2 Conditional probability problems

2.1 Witness reliability

You are a member of a jury judging a hit-and-run driving case. A taxi hit a pedestrian one night and fled thescene. The entire case against the taxi company rests on the evidence of one witness who saw the accident from hiswindow some distance away. He says that he saw the pedestrian struck by a blue taxi. The lawyer for the injuredpedestrian establishes the following facts:

1. There are only two taxi colors in town, blue taxis and black taxis. On the night in question, 85% of all taxison the road were black and 15% were blue.

2. The witness has undergone an extensive vision test and has demonstrated that he can successfully identifythe color of a taxi 80% of the time.

Using Bayes’ rule, compute the probability that the car was, in fact, black.

Solution We have

Pr (taxi is black) = 0.85Pr (taxi is blue) = 0.15

Pr (witness says "black"| taxi is black) = 0.8Pr (witness says "blue"| taxi is black) = 0.2Pr (witness says "blue"| taxi is blue) = 0.8

Pr (witness says "blue"| taxi is black) = 0.2

and therefore, by Bayes’ rule,

Pr ( taxi is black|witness says "blue")

=Pr (witness says "blue"| taxi is black) Pr (taxi is black)

Pr (witness says "blue")

=Pr (witness says "blue"| taxi is black) Pr (taxi is black)

Pr (witness says "blue"| taxi is black) Pr (taxi is black) + Pr (witness says "blue"| taxi is blue) Pr (taxi is blue)

=0.2 · 0.85

0.2 · 0.85 + 0.8 · 0.15≈ 0.59

Hence, there is still a 59% probability that the witness is incorrect, which gives us reasonable doubt as a jury.

2.2 Socks

Sock drawer A contains 2 white socks and 1 black sock, whereas sock drawer B contains 1 white sock and 5 blacksocks. A sock is drawn at random from sock drawer A and placed in sock drawer B. A sock is then drawn fromsock drawer B. It happens to be white. What is the probability that the sock transferred was white?

2

Solution Let C denote the event that the transferred sock is white, and let D denote the event that the visiblesock is white. Hence, we can let Cc (the complement of C) denote the event that the transferred sock is black, andDc the event that the visible sock is black. We have

Pr (C) = 2/3Pr (Cc) = 1/3

Pr ( D|C) = 2/7Pr ( D|Cc) = 1/7

And therefore, by Bayes’ rule,

Pr (C|D) =Pr ( D|C) Pr (C)

Pr (D)

=Pr ( D|C) Pr (C)

Pr ( D|C) Pr (C) + Pr ( D|Cc) Pr (Cc)

=2/7 · 2/3

2/7 · 2/3 + 1/7 · 1/3= 0.8

3 Random variables

3.1 Guessing game

One of the numbers 1 through 16 is uniformly chosen at random. You have to guess the number chosen by asking“yes-no” questions. Compute the expected number of questions you will need to ask in each of the two cases:

1. Your ith question is “is it i?” for i = 1, . . . , 16.

Solution Clearly, if the chosen number is x, then we will take exactly x guesses to reach that number, so

E (#of guesses) =1

16+

216

+ · · ·+ 1616

= 8.5

2. Your ith question is “is it Zi?”, where Zi is chosen uniformly at random between 1 and 16. (note: this wouldbe a very stupid way to guess, because we could guess the same wrong guess twice, for example)

Solution At each guess, we have a probability of 1/16 of guessing correctly. Hence, the number of guesses,X, is a geometric random variable with p = 1/16. Therefore,

E (X) = 1/p = 16

3. At each turn, you try to eliminate one-half of the remaining numbers (so that your first question is “is it lessthan or equal to 8?” and your second question is either “is it less than or equal to 4?” or “is it less than orequal to 12?” depending on the response, and so forth)

Solution If we use this strategy, then after the ith guess we have 16/2i remaining numbers to choose from,because we’re halving the remaining numbers. Since 16/24 = 1, this means that after 4 guesses we musthave the correct answer, so we will always take exactly 4 guesses.

3.2 Fitting

The density function of X is given by

f (x) =

{a + bx2 0 ≤ x ≤ 10 otherwise

If E (X) = 3/5, find a and b.

3

Solution We require thatˆ 1

0a + bx2 dx = 1

ˆ 1

0x(

a + bx2)

dx = 3/5

and thereforeˆ 1

0a + bx2 dx = 1[

ax + bx3/3]1

0= 1

a + b/3 = 1

We also haveˆ 1

0x(

a + bx2)

dx = 3/5[ax2/2 + bx4/4

]1

0= 3/5

a/2 + b/4 = 3/5

So, we have

a + b/3 = 1a/2 + b/4 = 3/5

whence a = 3/5, b = 6/5.

3.3 Machine breakdown

There are two possible causes for a breakdown of a machine. To check the first possibility would cost C1 dollars,and, if that were the cause of the breakdown, the trouble could be repaired at a cost of R1 dollars. Similarly, thereare costs C2 and R2 associated with the second possibility. Let p and 1− p denote, respectively, the probabilitiesthat the breakdown is caused by the first and second possibilities. Under what conditions on p, Ci, Ri (for i = 1, 2)should we check the first possible cause of breakdown and then the second, as opposed to reversing the checkingorder, so as to minimize the expected cost involved in returning the machine to working order?

Note If the first check is negative, we must still check the other possibility. Assume that only one error is possibleat a time.

Solution Suppose that we check the first cause of breakdown and then the second. Then, with probability p, ourcost will be C1 + R1, and with probability 1− p our cost will be C1 + C2 + R2. Therefore, our expected cost is

p (C1 + R1) + (1− p) (C1 + C2 + R2)

If we check the second cause of breakdown first, our expected cost is

(1− p) (C2 + R2) + p (C1 + R1 + C2)

Hence, it is desirable to check the first cause of breakdown only if

p (C1 + R1) + (1− p) (C1 + C2 + R2) < (1− p) (C2 + R2) + p (C1 + R1 + C2)

which we can simplify to(1− p) C1 < pC2

In other words, the money wasted due to checking the first type of failure must be less than the money wasteddue to checking the second type of failure.

4

3.4 A queueing problem

Consider a post office that is staffed by two clerks. Suppose that when Mr. Smith enters the system, he discoversthat Ms. Jones is being served by one of the clerks and Mr. Brown by the other. Suppose also that Mr. Smith is toldthat his service will begin as soon as either Jones or Brown leaves. If the amount of time that a clerk spends witha customer is exponentially distributed with parameter λ, what is the probability that, of the three customers, Mr.Smith is the last to leave the post office?

Solution Consider the time at which Mr. Smith first finds a free clerk. At this point either Ms. Jones or Mr.Brown would have just left and the other one would still be in service. However, by the memoryless property ofexponential random variables, it follows that the additional amount of time that this other person (either Jonesor Brown) would still have to spend in the post office is exponentially distributed with parameter λ. That is, it isthe same as if service for this person were just starting at this point. Hence, by symmetry, the probability that theremaining person finishes before Smith must equal 1/2.

We can do this mathematically as follows: let x be the time that the first person leaves the system, let Y be thetotal waiting time that the remaining person has (starting at time 0), and let Z be the total waiting time that Mr.Smith has (starting at time x). Then the probability that Mr. Smith is not the last person to leave the post office is

Pr (Y ≥ Z + x|Y ≥ x)

The memoryless property of the exponential says that, if X is exponentially distributed, then

Pr (X ≥ x + y|X ≥ x) = Pr (X ≥ y)

plugging this into the above with X = Y, y = Z, we find that

Pr (Y ≥ Z + x|Y ≥ x) = Pr (Y ≥ Z)

Now, Y and Z are both exponentially distributed random variables with parameter λ, so Pr (Y ≥ Z) = 1/2.

3.5 Bus arrivals

Buses arrive at a specified stop at 15-minute intervals starting at 7:00 a.m. That is, they arrive at 7:00, 7:15, 7:30,7:45, and so on. If a passenger arrives at the stop at a time that is uniformly distributed between 7 and 7:30, findthe probability that he waits

1. less than 5 minutes for a bus

Solution The passenger’s arrival time is uniformly distributed between 7:00 and 7:30. We can write this asa uniform random variable with

f (x) =

{1/30 if 0 ≤ x ≤ 300 otherwise

The passenger will wait less than 5 minutes for a bus if he arrives between 7:10 and 7:15, or between 7:25 and7:30. The probability of this happening is

ˆ 15

101/30 dx +

ˆ 30

251/30 dx = 1/3

2. more than 10 minutes for a bus.

Solution The passenger will wait more than 10 minutes for a bus if he arrives between 7:00 and 7:05, orbetween 7:15 and 7:20. The probability of this happening is

ˆ 5

01/30 dx +

ˆ 20

151/30 dx = 1/3

5

3.6 Bus breakdown

A bus travels between two cities A and B, which are 100 miles apart. If the bus has a breakdown, the distance fromthe breakdown to city A has a uniform distribution over (0, 100). There is a bus service station in city A, in B, andin the center of the route between A and B. It is suggested that it would be more efficient to have the three stationslocated 25, 50, and 75 miles, respectively, from A. Do you agree? Why?

Solution Let X ∼ U (0, 100) denote the place where the bus has a breakdown. Let d1 (X) denote the distancefrom X to the nearest service station using the current setup, and let d2 (X) denote the distance from X to thenearest service station using the newly suggested setup. We have

d1 (X) =

X if X ≤ 2550− X if 25 < X ≤ 50X− 50 if 50 < X ≤ 75100− X if X > 75

d2 (X) =

25− X if X ≤ 25X− 25 if 25 < X ≤ 37.550− X if 37.5 < X ≤ 50X− 50 if 50 < X ≤ 62.575− X if 62.5 < X ≤ 75X− 75 if X > 75

The expected waiting time under the current scheme is

E (d1 (X)) =ˆ 100

0d1 (x) f (x) dx

=ˆ 100

0d1 (x) (1/100) dx

=1

100

ˆ 25

0x dx +

1100

ˆ 50

2550− x dx +

1100

ˆ 75

50x− 50 dx +

1100

ˆ 100

75100− x dx

= 12.5

The expected waiting time under the new scheme is

E (d2 (X)) =ˆ 100

0d2 (x) f (x) dx

=ˆ 100

0d2 (x) (1/100) dx

=1

100

ˆ 25

025− x dx +

1100

ˆ 37.5

25x− 25 dx +

1100

ˆ 50

37.550− x dx +

1100

ˆ 62.5

50x− 50 dx

+1

100

ˆ 75

62.575− x dx +

1100

ˆ 100

75x− 75 dx

= 9.375

So, this is indeed a preferable scheme.

3.7 Gamma distribution (this one’s hard)

If X is an exponential random variable with mean 1/λ, show that

E(

Xk)

= k!/λk

for k = 1, 2, . . . .

6

Hint Write out the expectation

E(

Xk)

=ˆ ∞

0xk f (x) dx =

ˆ ∞

0xkλe−λx dx = λ

ˆ ∞

0xke−λx dx

Then, apply a change of variables y = λx and use the identity

Γ (t) = (t− 1)! =ˆ ∞

0e−yyt−1 dy

Solution Following the hint, we write

E(

Xk)

= λ

ˆ ∞

0xke−λx dx

We let y = λx and hence dx = 1/λdy, so

E(

Xk)

= λ

ˆ ∞

0(y/λ)k e−y · (1/λ) dy

= λ

ˆ ∞

0(1/λ)k yke−y · (1/λ) dy

= (1/λ)kˆ ∞

0yke−y dy

= (1/λ)k Γ (k + 1) =k!λk

as desired.

4 The normal distribution

4.1 Normal approximation to binomial

Let X be the number of times that a fair coin, flipped 40 times, lands heads. Find the probability that X = 20. Usethe normal approximation and then compare it to the exact solution.

Solution The normal approximation for Pr (X = 20) is Pr (19.5 ≤ Y ≤ 20.5), where Y ∼ N (20, 10), so

Pr (19.5 ≤ Y ≤ 20.5) = Pr

19.5− 20√10

≤ Y− 20√10︸ ︷︷ ︸

∼N (0,1)

≤ 20.5− 20√10

≈ Φ (0.16)−Φ (−0.16) ≈ 0.1272

The exact value isPr (X = 20) = C (40, 20) (0.5)40 ≈ 0.1254

where C (n, k) = n!k!(n−k)! .

4.2 Signal transmission

Suppose that a binary message – either 0 or 1 – must be transmitted by wire from location A to location B. However,the data sent over the wire are subject to a channel noise disturbance, so to reduce the possibility of error, the value2 is sent over the wire when the message is 1 and the value −2 is sent when the message is 0. If m ∈ {−2, 2} is thevalue sent at location A, then R, the value received at location B, is given by R = m + X, where X is the channelnoise disturbance. When the message is received at location B, the receiver decodes it according to the followingrule:

7

Figure 1: A system of components

• If R ≥ 0.5, then 1 is concluded.

• If R < 0.5, then 0 is concluded.

Assume that X ∼ N (0, 1). Compute the probability that an error occurs in decoding (for the case that m = −2and the case that m = 2).

Solution There are two types of errors that can occur; we either mistake a 1 for a 0 or a 0 for a 1. Suppose thatthe initial message is 1, and so m = 2. The probability of an error is precisely the probability that R = 2 + X ≤ 0.5.This is

Pr (2 + X ≤ 0.5) = Pr (X ≤ −1.5)= Φ (−1.5) ≈ 0.0668

If the initial message is 0, so that m = −2, then the probability of an error is precisely the probability that R =−2 + X ≥ 0.5. This is

Pr (−2 + X ≥ 0.5) = Pr (X ≥ 2.5)= 1−Φ (2.5) ≈ 0.0062

Hence, the probability of an error is approximately 0.073.

5 Reliability theory

5.1 System reliability

Find the reliability of the system shown in figure 1.

8