Theory of Probability

Prof Tasneem Chherawala

NIBM

Uncertain Events

• See a Zebra the classroom today?

• Win in a lottery?

• Toss a coin and get heads / roll a die (or two) and get 5/ pick a card from a deck and get an Ace?

• Make a 10% profit from investment in shares over 1 month?

• Face a default on money lent?

• Earn positive returns on a fixed deposit?

• Count the total number of hours today and get 24?

Probability

• Probability quantifies how uncertain we are about a

future outcome / event

• A probability refers to the percentage chance that

something will happen, from 0 (it is impossible) to 1 (it is

certain to occur), and the scale going from less likely to

more likely

• An interpretation based on data - Probability can be

interpreted as the relative frequency of the outcomes

(values) of uncertain events (variable) after a great many

(infinitely many) repetitions / parallel independent trials of

an experiment

Why Measure Uncertainty

• Is something at stake?

• To make tradeoffs among uncertain events

• Measure combined effect of several uncertain events

• To communicate about uncertainty

• To draw inferences about a population from a sample

Probability Theory in Finance

• Financial decisions – a game of chance!

• What chance?– Probability of making or losing money in an investment!

• Why chance? – Uncertainty and variability of future events (price movements

and value of investments)

• Probability concepts help define financial risk by quantifying the prospects for unintended and negative outcomes (losses)

• Probability also quantifies expected values of future events, which gives us a fair estimate of current value of investment

Probability Theory in Finance

• Probability theory applications:– Make logical and consistent investment decisions

– Manage expectations in an environment of risk

• Hypothesis: No system for making financial choices from those offered to us can both (1) be certain to avoid losses and (2) have a reasonable chance of making us rich

• Expected values in a probability model are the prices of alternative financial decisions

Some Definitions

• Random Experiment: a process that leads to one of several possible outcomes

• Outcome: result of the experiment– Examples: Head in a coin toss, two heads when tossing two

coins, win in a lottery, 10% returns from the stock in one day, Borrower repays Re 1 from Rs. 100 borrowed at 12% rate of interest

• Sample Space: a set of all possible outcomes of the experiment – relates to population data– Examples:

• Toss one coin: S = {H, T}

• Toss two coins / toss a coin twice: S = {HH, HT, TH, TT}

• Lottery: S = {Win, Lose}

• One day stock returns: S = {-100% to +Infinity?}

• Repayment on Rs. 100 loan by borrower at 12% rate of int.: S = {0,1,2,….., 100, 101,…,112}

Sample Space

• Finite sample space: finite number of outcomes in the space S.

• Countable infinite sample space: ex. natural numbers.

• Discrete sample space: if it has finite or countable infinite number of outcomes.

• Continuous sample space: If the outcomes constitute a continuum. Ex. All the points in a line.

Events

• Event: subset of a sample space – a combination of possible outcomes of an uncertain process – Examples

• Head in one toss of a coin: E1 = {H}

• One head and one tail in two coin tosses: E2 = {HT, TH}

• Mean of the dice values greater than or equal to 5 in two dice rolls: E3 = {4.6, 5.5, 5.6, 6.4, 6.5, 6.6}

• One day stock returns greater than equal to 10%: E4 = {10% to +Infinity}

• Loss to bank if Borrower doesn’t repay principal: D = {1%,2%,…..,99%,100%}

Events

• Mutually Exclusive Events: events which have no outcome in common and thus cannot happen together

• If a list of events is mutually exclusive, it means that only one of them can possibly take place– Examples

• Toss one coin: E1 = {H}, E2 = {T}

• Toss two coins: E1: first toss is Head = {HT, HH}, E2: first toss is Tail = {TH, TT}

• Invest in a stock for one day: E1: One day stock returns greater than equal to 10% = {10% to +Infinity}, E2: One day stock returns between 2% to 5% = {2% to 5%}

– How would E1 compare with E3: One day stock returns less than or equal to 10% = {-100% to 10%)

• Lend money: E1: Borrower repays entire principal = {100,101,102,…112}, E2: Borrower repays 50% of principal = {50}

– How would E1 compare with E3: Borrower repays principal with 5% interest = {105} ?

Properties of Probabilities defined

on a Sample Space• Sample space must be exhaustive: List all possible

outcomes

• Outcomes in the sample space must be mutually exclusive

• The exhaustive and mutually exclusive characteristic of a sample space together imply that

– The probability (likelihood of occurrence) of any one outcome or event must lie between 0 and 1

• 0 < P(E) <1

– The sum of the probabilities of all the outcomes in the sample space must be 1

• P(S) = 1

Properties of Probabilities defined

on a Sample Space• By extension of the above rationale

– The probability of any event would lie between 0 (impossible) and 1 (certain), or 0 < P(E) < 1.

– There is no such thing as a negative probability (less than impossible?) or a probability greater than 1 (more certain than certain?).

– The sum of all probabilities of all outcomes would equal 1, provided the outcomes are both mutually exclusive and exhaustive.

– If outcomes are not mutually exclusive, the probabilities would add up to a number greater than 1, and if they were not exhaustive, the sum of probabilities would be less than 1.

Classical / A Priori Approach to

Probabilities• When the range of possible uncertain outcomes in a

sample space is known and equally likely

• Probability for each outcome or event can be determined by logic

• Examples– Tossing a fair coin – outcomes can only be heads or tails and

both are equally likely

– Rolling a fair die – outcomes can only be 1,2…,6 and all six are equally likely

– Drawing a card from a pack – outcomes can only be 52, and all are equally likely

• The probability of each outcome in the above examples has been determined by construction of the coin/die/pack of cards

Defining A Priori Probabilities

• In a sample of N mutually exclusive, exhaustive and equally likely outcomes we assign a chance (or weight) of 1/N to each outcome

• We define the probability of an event for such a sample as follows:

• The probability of an event E occurring is defined as:

• P(E) = n(E)/n(S)

– n(E) is the number of outcomes favourable to E and

– n(S) is the total number of equally likely outcomes in the sample space S of the experiment

• By extension, the probability of event E not occurring

• P(not E) = 1-P(E) = 1- n(E)/n(S)

Example

• What is the probability that a card drawn at random from a deck of cards will be an ace?

• Total no. of outcomes in the sample space?

• Are they equally likely?

• Event of interest (E)?

• No. of Outcomes favourable to the Event?

• P(E)?

• What is the probability that a card drawn at random from the deck will not be an ace?

A Priori Probabilities

• The same principle can be applied to the problem of determining the probability of obtaining different totals from a pair of dice.

• Possible Outcomes?

• Are they equally likely?

• Event 1 = A = sum of the two dice will equal 5

• Event 2 = B = the absolute difference will equal 1

• P (Event 1) =

• P (Event 2) =

A Priori Probabilities

• In certain cases where outcomes are not equally likely,

one can still deduce rationally the a priori probability of

an event

• For example

– If we forecast that a company is 70% likely to win a

bid on a contract (irrespective of how this probability

is derived), and we know this firm has just one

business competitor, then we can also make an a

priori forecast that there is a 30% probability that the

bid will go to the competitor

Empirical Probabilities

• In finance, we cannot depend upon the exactness of a process to determine a priori probabilities

• The financial analyst would have to depend upon historical occurrence of the event(s) or repeat an experiment multiple times to determine the probability of the event empirically

• For example, the range of outcomes of returns on a financial asset are virtually infinite and that too, not all outcomes are a priori, equally likely

• Thus, the financial analyst would have to observe many movements in asset prices to determine the probability of future price changes of a given magnitude

• Of course, we know that past performance does not guarantee future results, so a purely empirical approach has its drawbacks

Defining Empirical Probabilities• The probability of an event (or outcome) is the proportion

of times the event occurs in a long run of repeated

experiment.

• The empirical probability of a given outcome / event Z is

defined as

– P(Z) = no. of Z occurrences/no. of trials of the experiment

• This is the same as analysis of relative frequency of

observations in a sufficiently large sample

• Consider a financial analyst who is interested in knowing

what will be the probable one day returns on a particular

stock.

– He tracks the past 100 days of price movements and returns of

a particular stock.

– Each of the 100 days would constitute trial and each day’s

returns would constitute the outcome

Subjective Probabilities

• Probability under this approach is simply defined as the strength of belief that an event will occur

• It is based upon experience and judgment

• Such probabilities are applied to many business problems where a priori probabilities are not possible, nor are there sufficient empirical observations upon which to base probability estimates

• For example, subjective probability is incorporated in the forecasting of company profits by investment analysts

• Of course, subjective probabilities are unique to the person making them and to the specific assumptions made

Rules of Probability

• There are a number of formal probability rules applied to

probability estimates

• Which of these rules is applicable will depend upon

whether

– We are concerned with a single event, in which case

the outcomes relate only to that event

– We are concerned with combinations of several

events, for example the changes in Sensex and

Exchange Rates together

– The combined events are independent or mutually

exclusive

Rules of Probability

• The rules are

– Complement rule: When we are concerned with

whether an event A will not occur

– Multiplication rule: when we are concerned with event

A and B occurring together. This requires us to know

whether A and B are independent of each other

– Addition rule: when we are concerned with event A or

B happening. This requires us to know whether A and

B are mutually exclusive

Intersection of Events and Joint

Probability

• Joint probability: The probability of both event A and

event B occurring: P(A and B) / P(A ∩ B) / P(AB)

• Intersection: Event defined as both A and B occur

• Example Using a priori probabilities:

– P(A and B) = No. of Outcomes favourable to the Joint Event /

Total no. of outcomes in the sample space

– Event A is you draw a spade from a deck of cards

Event B is you draw a king

– Intersection is event you draw a king of spades

– Joint probability = P(A and B) = 1/52

Joint Probability Table

Job_statusDecision

accept reject

governmen 0.116 0.072

military 0.003 0.003

misc 0.000 0.003

private_s 0.266 0.352

retired 0.014 0.005

self_empl 0.035 0.051

student 0.005 0.008

unemploye 0.013 0.056

• Interpretation of 0.266?

Marginal Probability

• Marginal probability: Probability of a single event

• When outcomes are exhaustive and mutually exclusive, can be calculated by adding all the joint probabilities containing the single event

Marginal Probability

Job_statusDecision

accept reject Total

governmen 0.116 0.072 0.187

military 0.003 0.003 0.005

misc 0.000 0.003 0.003

private_s 0.266 0.352 0.617

retired 0.014 0.005 0.019

self_empl 0.035 0.051 0.086

student 0.005 0.008 0.013

unemploye 0.013 0.056 0.069

Total 0.451 0.549 1.000

• Interpretation of 0.187, 0.451?

Conditional Probability

• A conditional probability is the probability of an event given that another event has occurred

• Conditional probability assumes that one event has taken place or will take place, and then asks for the probability of the other (A, given B)

• P(B | A): probability of B given A

• P(A | B): probability of A given B

• For example, – Probability of drawing a king given that a spade is drawn: P(king

| spade) = 1/13

– Probability of drawing a spade given that a king is drawn: P(spade | king) = 1/4

– What is the probability that the total of two dice will be greater than 8 given that the first die is a 6?

Conditional Probability in Terms of

Joint Probability

Job_statusDecision

accept reject Total

governmen 0.116 0.072 0.187

military 0.003 0.003 0.005

misc 0.000 0.003 0.003

private_s 0.266 0.352 0.617

retired 0.014 0.005 0.019

self_empl 0.035 0.051 0.086

student 0.005 0.008 0.013

unemploye 0.013 0.056 0.069

Total 0.451 0.549 1.000

•Given that an application

is accepted, what is the

probability that the

customer is a Pvt. Sector

employee?

•What kind of probability

asked for?

•Imagine 1000 Customer

Applications. You would

expect 451 to be accepted

and 266 of those to be

Pvt. Sector employees

•P(Pvt. S Empl | Accept) =

116/451 = 0.257

Conditional Probability in Terms of

Joint Probability• Joint Probability can be rescaled to find Conditional

Probability

P(A|B) = P(A and B) / P(B)

• Can think about this as rescaling the Accept Column so it sums to one

• Interpretation– P(Pvt. S Empl | Accept) = 266/451 = 0.59– P(Non Pvt. S Empl | Accept) = 185/451 = 0.41

– Means Pvt. S Employees are more likely to be accepted than Non Pvt. Sector Employees?

– P(Accept | Pvt. S Empl) = 266/617 = 0.43

– P(Accept | Non Pvt. S Empl) = 185/383 = 0.483

– Means more of the accepted applicants are Non Pvt. Sector Employees than Pvt. Sector Employees?

– If the bank’s objective was to specifically target a group for marketing of the product, which probabilities would it rely upon to give it an accurate picture?

Joint Probability in Terms of

Conditional Probability• Another way to calculate Joint Probability of Events A

and B isP(A and B) = P(A) x P(B|A) = P(B) x P(A|B)

• where P(B|A) is the conditional probability of B given A and P(A|B) is the conditional probability of A given B

• Example: – If we believe that a stock is 70% likely to return 15% in the next

year, as long as GDP growth is at least 8%, then we have made our prediction conditional on a second event (GDP growth). In other words, event A is the stock will rise 15% in the next year; event B is GDP growth is at least 8%; and our conditional probability is P(A | B) = 0.7

– Now if we know that there is a 20% unconditional probability that GDP will grow at 8% or above P(B) = 0.2

– The probability that GDP will grow at least at 8% and stock will return 15% is P(A and B) = 0.7 *0.2 = 0.14

Joint Probability Vs Conditional

Probability• Joint probability is not the same as conditional

probability, though the two concepts are often confused.

• Joint probability sets no conditions on the occurrence of events but simply provides the chance that both events will happen together

• In a problem, to help distinguish between the two, look for qualifiers that one event is conditional on the other (conditional) or whether they will happen concurrently (joint).

Joint Probability and Independence

of Events• Independence of Event A and B: When Probability of

Event A occurring does not depend upon occurrence of Event B and vice versa

• Two events are independent if and only if

• P(A | B) = P(A) [implies P(B | A) = P(B)]

• If A and B are independent, then the probability that events A and B both occur is:P(A and B) = P(A) x P(B | A) = P(A) x P(B)

• Examples:– What is the probability that a fair coin will come up with heads

twice in a row?

– Now consider a similar problem: Someone draws a card at random out of a deck, replaces it, and then draws another card at random. What is the probability that the first card is the ace of clubs and the second card is a club (any club)?

Joint Probability and Independence

of Events• Is the customer application being accepted independent

of being a Pvt. sector employee?

• P(Accept | Pvt. Sector employee) = 0.43

• P(Accept) = 0.451

• The rule generalizes for more than two events provided they are all independent of one another, so the joint probability of three events P(ABC) = P(A) * (P(B) * P(C), again assuming independence

Joint Probability and Mutually

Exclusive Events• The joint probability of two mutually exclusive events

occurring is 0

• This is because by definition of mutually exclusive, events A and B cannot occur together

• For example:– Roll a die once: Event A = 6, Event B <5

– Are A and B mutually exclusive?

– P(A and B) = ?

– Roll 2 dice: Event A = {1,4}, Event B = {4,1}

– Are A and B mutually exclusive?

– P(A and B) = ?

– Roll 2 dice: Event A = {1,4}, Event B = at least one die shows 1

– Are A and B mutually exclusive?

Union of Events and Probability

• Union: Union of events A and B is event that either A or B or both occur: (A or B) or (AUB)

• P(A or B) is interpreted as the probability that at one of the two events A and B will occur

• If events A and B are mutually exclusive, P(A and B) = 0

• P(A or B) = P(A) + P(B)

• Example:– What is the probability of rolling a die and getting either a 1 or a

6?

– Since it is impossible to get both a 1 and a 6, these two events are mutually exclusive

– P(1 or 6) = P(1) + P(6) = 1/6 + 1/6 = 1/3

Union of Events and Probability

• If events A and B are not mutually exclusive, P(A and B) ≠ 0

• P(A or B) = P(A) + P(B) – P(A and B)

• The logic behind this formula is that when P(A) and P(B) are added, the occasions on which A and B both occur are counted twice. To adjust for this, P(A and B) is subtracted.

• Example:– What is the probability that a card selected from a deck will be

either an ace or a spade?

– P(Ace) = 4/52

– P(Spade) = 13/52

– P(Ace and Space) = ?

– P(Ace or Space) = ?

Union of Events and Probability

• Consider the probability of rolling a die twice and getting a 6 on at least one of the rolls. The events are defined in the following way:

• Event A: 6 on the first roll: P(A) = 1/6

• Event B: 6 on the second roll: P(B) = 1/6

• P(A and B) = 1/6 x 1/6 (why?)

• P(A or B) = 1/6 + 1/6 - 1/6 x 1/6 = 11/36

• The same answer can be computed using the following admittedly convoluted approach:

• Getting a 6 on either roll is the same thing as not getting a number from 1 to 5 on both rolls.

• This is equal to: 1 - P(1 to 5 on both rolls) = 1 – 5/6 x 5/6 = 1 – 25/36 = 11/36

Union of Events and Probability

• Despite the convoluted nature of this method, it has the advantage of being easy to generalize to three or more events.

• For example, the probability of rolling a die three times and getting a six on at least one of the three rolls is?

1 - 5/6 x 5/6 x 5/6 = 0.421

• In general, the probability that at least one of k independent events will occur is:

1 - (1 - α)k

where each of the events has probability α of occurring

Union of Events and Probability

• Example:

• Assume that the bank had lent to 5 different borrowers, each of whom was assigned a probability of default = 0.1

• The bank wished to hedge the risk of its portfolio such that it bought insurance such that the loss against at least one borrower defaulting was made good

• How would the insurance company estimate the probability of at least one borrower defaulting, assuming that the default event of each borrower was independent?

• P(At least one default) = 1 – (1-0.1)5 = 0.41

Union of Events and Probability

Job_statu

s

Decision

accept reject Total

governme

n 0.116 0.072 0.187

military 0.003 0.003 0.005

misc 0.000 0.003 0.003

private_s 0.266 0.352 0.617

retired 0.014 0.005 0.019

self_empl 0.035 0.051 0.086

student 0.005 0.008 0.013

unemploy

e 0.013 0.056 0.069

Total 0.451 0.549 1.000

•What is the probability that the

customer application is accepted

or the customer is a government

employee?

•What kind of probability asked

for?

•P(Accept) = 0.451

•P(Govt. Employee) = 0.187

•P(Accept and Govt. Employee =

0.116

•P(Accept or Govt. Employee) =

0.451 + 0.187 – 0.116 = 0.522

Union of Events and Probabilities

• Example:

• A Fund manager has invested in the stocks of two companies 1 and 2 and is interested in knowing what is the probability that the equity price of either company will rise

• Are the two events mutually exclusive?

• P(Co. 1 Equity ↑) = 0.55

• P(Co. 2 Equity ↑) = 0.35

• P(Co. 1 Equity ↑ and Co. 2 Equity ↑) = 0.3

• P(Co. 1 Equity ↑ or Co. 2 Equity ↑) = 0.55+0.35-0.30 = 0.60

Union of Events and Probability

• What if we want to know the probability of at least one

of 3 events A, B and C happening?

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )P A B C P A P B P C P AB P AC P BC P ABC

A B

C

Probability Rules Summary

• Multiplication Rule: Joint probability of any two events A and B is:– P(A and B) = P(A | B) * P(B)

– Follows from definition of conditional probability

• Multiplication Rule Independent Events: If A and B are independent, joint probability is:– P(A and B) = P(A) * P(B)

• Addition Rule mutually exclusive events:– P(A or B) = P(A) + P(B)

– Mutually exclusive if both cannot occur

• Addition Rule: Probability that event A or event B or both occur is– P(A or B) = P(A) + P(B) – P(A and B)

• Are mutually exclusive events independent?

• Does pr(A|B) = pr(A)?

• NO!

• If B has happened, then A can not happen

• pr(A|B) = 0

• So

• Mutually exclusive events are DEPENDENT.

Probability Tree and Total

Probability Rule

Job_statusDecision

accept reject

Private

Sector 0.29 0.35

Non Pvt

Sector 0.16 0.20

Probability Tree

Accept: 0.45

Reject: 0.55

Pvt. Sector | Accept: 0.59

Not Pvt. Sector | Accept: 0.41

Pvt. Sector | Reject: 0.64

Not Pvt. Sector | Reject: 0.36

0.29

0.16

0.35

0.20

Pvt. S & Accept

Not Pvt. S & Accept

Pvt. S & Reject

Not Pvt. S & Reject

Total Probability Rule

• The Total Probability Rule: The total probability rule explains an unconditional probability of an event A, in terms of that event's conditional probabilities in a series of mutually exclusive, exhaustive scenarios of event B

• P(A) = P(A | B) x P(B) + P(A | not B) x P(not B)

• With the total probability rule, event A has a conditional probability based on each scenario of event B (i.e. the likelihood of event A, given that scenario), with each conditional probability weighted by the probability of that scenario for event B occurring

• Example: – What is the probability of a customer employed in the Private

Sector:

– P(Pvt. S. Empl) = P(Pvt. S Empl | Accept) x P(Accept) + P(Pvt. S Empl | Reject) x P(Reject) = 0.59*0.45+0.64*0.55 = 0.61

Total Probability Rule

• A model that predicts whether a borrower has defaulted or not is 95 percent effective in predicting that a borrower has defaulted when it actually has. However, the model also yields a “false positive” result for 1 percent of the non-defaulted borrowers. That is, there is 1 percent chance that a borrower who has not defaulted will be identified as a defaulted borrower by the model.

• Q: If 0.5 percent of the bank’s portfolio has actually defaulted, what is the probability that a borrower has defaulted given that the model predicts default?

Probability Tree

Default: 0.005

No Default: 0.995

Def Pred | Default: 0.95

No Def Pred | Default: 0.05

Def Pred | No Default: 0.01

No Def Pred | No Default: 0.99

0.00475

0.00025

0.00995

0.98505

Total Probability Rule And Baye’s

Theorem

• Let D be the event that a borrower has defaulted

• Let E be the event that the model predicts default

• We need to estimate P(D|E)

• We know: P(E|D)=0.95, P(E|notD)=0.01, P(D)=0.005,

P(notD)=1-P(D)=0.995

0147.0995.001.0005.095.0)()()()()(

00475.0005.095.0)()()(

3231.00147.0

00475.0

)(

)()(

notDPnotDEPDPDEPEP

DPDEPEDP

EP

EDPEDP