Intro to MarkocChain Chutes-And-Ladders

8/2/2019 Intro to MarkocChain Chutes-And-Ladders

1/38

Mehran University of Engineering and Technology, Jamshoro

-This class will meet @: 8:30 a.m -10:30 a.m (Mondays)

10:30 a.m - 12:30 p.m (Thursdays)10:30 a.m - 12:30 p.m (Wednesdays)10:00 a.m -11:30 a.m (Fridays)

Today's Lecture: Lecture # 35 Lec # 37

15-03-2012

Introduction to Markov ChainsPS: Some slides are taken from Dr. Deckelman site


2/38

Mr. Markov Plays Chutes and

Ladders

Introduction

The Concept of a Markov Chain

The Chutes and Ladders Transition Matrix

Simulation Techniques

Repeated Play

Conclusion

4/24/2012 Principles of Teletraffic Engineering 2


3/38



4/38

Introduction



5/38

How the Game is Played [1]Chutes and Ladders is a board game where players

spin a pointer to determine how they will advance

The board consists of 100 numbered squarese o ec ve s o o sq re 100

The spin of the pointer determines how manysquares the player will advance at his turn with

equal probability of advancing from 1 to 6 squaresHowever, the board is filled with chutes, which

move a player backward if landed on



6/38

How the Game is Played [2]There are also ladders, which advance a player

Chutes have pictures of bad behavior which leadsto disasters

Ladders have pictures of ood behavior leadin


to rewards

Most of the chutes and ladders produce relativelysmall changes in position, but several produce

large gains or losses


7/38

The Concept of aov



8/38

Topics CoveredTransition matrix

Probability vectors

Absorbing vs. non-absorbing Markov chainsSteady-state matrices



9/38

Markov ChainsA Markov Chain is a weighted digraph

representing a discrete-time system that can be

in any number of discrete statesThe transition matrix for a Markov chain is the

transpose of a matrix of probabilities of movingfrom one state to another

Pij = probability of moving from state i to j



10/38

Transition Matrix

Below, is the layout of the transition matrix for

Chutes and Ladders (101x101)

p0,0 p0,1 p0,100

p1,0 p1,1 p1,100

. . .

. . .

p100,0 p100, 1 . p100,100



11/38

Three properties which identify a state

model as being a Markov model:

1) The probability of moving from state i to j isindependent of what happened before moving to

state j and how one got to state i (Markovss o

2) Sum of probabilities for each state must be one

3) X(t) = probability distribution vector of the

probability of the system being in each of thestates at time n



12/38

Probability VectorThe probability vector is a column vector in which the

entries are nonnegative and add up to one

The entries can represent the probabilities of finding a



13/38

Two types of Markov Chains1) Absorbing Markov Chains

Can not get out of certain states

Once a system enters an absorbing state, the systeme s s e o e o

2) Non-absorbing Markov Chains

Can always get out of every state



14/38

Absorbing Markov Chains

(Chutes and Ladders)

Two conditions must be met for a MarkovChain to be absorbing:

Must have at least one state which cannot be leftonce as een en ere

It must be possible, through a series of one or moremoves, to reach at least one absorbing state fromevery non-absorbing state (given enough time, everysubject will eventually be trapped in an absorbingstate)



15/38

Common QuestionA common question arising in Markov-chain models

is, what is the long-term probability that the system

will be in each state?-

called the steady-state vector of the Markov chain



16/38

Steady-state matrixThe steady-state probabilities are average probabilities

that the system will be in a certain state after a large

number of transition periods-

independent of the initial distribution

Long-term probabilities of being on certain squares

mPn

n=

lim



17/38

The Chutes and

Matrix



18/38

Analytical ComputationProgramming Language: Java

Objectives:

Compute the transition matrixCompute t e pro a i ity vectors



19/38

Techniques of Transition MatrixHow we did the analytic computations2D array of integers of size 101 by 101

Each array entry represents a move from one square too e

i.e. Pij represents the players move from position i to position j

A square with a ladder beginning in that square is considered apseudo-state and has a 0 probability of landing on it

A square with a chute beginning in that square is considered apseudo-state and also has a 0 probability of landing on it



20/38

Results of Transition MatrixThe matrix is way too large to show on this slide

Some example probabilities computed:

If a player is on square 97, the probability of staying ona square s 3 an e pro a y o mov ng o

square 78 is 1/6



21/38

Techniques of Probability VectorsHow we did the analytic computations

1D array of integers of size 101

Probability vectors Vn represent the probability of being

Vo = {1,0,0} and means that the probability of being onsquare 0 is 1 or 100%

Vo * P = V1;

V1 * P = V2, etc



22/38

Results of Transition MatrixAfter 1000 moves, the probability vector reached a

limit of {0,0,1}

This means that after 1000 moves, the game is



23/38

Simulation

Techniques



24/38

SimulationProgramming Language: C++

Objectives:

Find frequencies for being at each positionFind mean num er o moves to win

Find standard deviation

Simulate a large number of games



25/38

Technique OneHow we simulated the gameArray of 101 integers representing the board

Each array entry represented a state Psuedo-states held the value at the end of the ladder or the

chutei.e. Index 1 represented square 1 and held 38. If index became1 it would move to square 38.

Normal states held the index of that state. Index would be

compared to value and be the same so index would stay at itsvalue.



26/38

Technique OneWhen index became 100 game would be over.

If index was greater than 100 it would reset to itsprevious. This repeats until index 100 is hit.

Mean and standard deviation were calculatedthroughout the game.

Printed most moves to win and least moves.

Printed number of times each square was landed on for

250,000 games and frequency of each.



27/38

Technique One ResultsWe ran 250,000 games

Mean is approximately 39.65 moves to reach square

100.e stan ar ev at on s approx mate y 24.00



28/38

Technique TwoRun the game as a non-absorbing.

i.e. If the index is 100 or above subtract 100 to run the

game as a non-ending board..

This was done to compare to the non-absorbinganalytical model.



29/38

Repeated Play



30/38

Analytic ComputationProgramming Language: Java

Objectives:

Compute the non-absorbing matrixCompute t e corresponding pro a i ity vectors



31/38

Techniques of Repeated PlayHow we did the analytic computations:

Similar to transition matrix, except square 100 cannot be

landed on,

board (Similar to Monopoly)

The game cannot be won using this method

Comparison will be done to simulation results



32/38

Theoretical Experimental

=.0 =.0

%581.05 =P%552.05 =P

%89.226 =P%94.226 =P

%09.242 =P %13.242 =P

%573.065 =P%599.065 =P

%452.099 =P %441.099 =P


33/38

Conclusion


34/38

The Theoretical Result and TheExperimental Result matched

The Markov Chain works in Chutes andLadders


35/38

Chutes and Ladders is a game for 3-6 years.

We can make it more interesting by

changing some rules


36/38

Special Thanks to Dr. Deckelman


37/38

Sources Mooney and Swift. A Course In Mathematical

Modeling. MAA Publications, 1999.



38/38

End


Intro to MarkocChain Chutes-And-Ladders

Documents

Transcript of Intro to MarkocChain Chutes-And-Ladders