Introduction to Monte Carlo Methods - sites.tufts.edu · Intro to MCMC Introduction Introduction to...

Intro to MCMC

Introduction

Introduction to Monte Carlo Methods

Daryl DeFord

VRDIMGGG

June 6, 2018

Intro to MCMC

Introduction

Outline

1 Introduction

2 Motivating Problems: Why Randomness?

3 Monte Carlo Methods

4 Historical Overview

5 Markov Chain Methods

6 MCMC on Graphs

Intro to MCMC

Introduction

Code for this talk:

Remember you can open the page source and copy things tomath.dartmouth.edu/~ddeford/sage_cell to make modifications andrun variations. (links are clickable)

1 math.dartmouth.edu/~ddeford/war

2 math.dartmouth.edu/~ddeford/cube_dist

3 math.dartmouth.edu/~ddeford/pisimple

4 math.dartmouth.edu/~ddeford/mcmc1

Intro to MCMC

Motivating Problems: Why Randomness?

Warmup Card Problems

1 In a shuffled deck, what is the probability that the top card is red anda queen?

2 In a shuffled deck, what is the probability that the top card is red or aqueen?

3 In a shuffled deck, what is the probability that at least one of the toptwo cards is an ace?

4 In a shuffled deck, what is the probability that exactly one of the toptwo cards is an ace?

5 In a shuffled deck, what is the probability that at most one of thetop two cards is an ace?

6 In a shuffled deck, what is the probability that a 5 card hand containsat least one card of each suit?

7 In a shuffled deck, what is the probability that a 5 card hand containsno cards of exactly one suit?

Intro to MCMC

Game: Cooperative War

Rules:1 Nominate a dealer in your group of 3 players

2 Deal 2 cards to each player

3 Each round begins with the dealer playing the highest value card fromtheir hand.

4 The round continues counterclockwise with each player playing thehighest card in their hand only if it is higher than the previouslyplayed card. If not, skip that player and move on to the next.

5 The round ends once each player has had an opportunity to play acard.

6 You (collectively) win if all players have played all of their cards at theend of the second round.

Intro to MCMC

Cooperative War Results?

Question

What is the probability that you win, given a randomly shuffled deck?

Answer

Try it out!

Intro to MCMC

Cooperative War Results?

Question

What is the probability that you win, given a randomly shuffled deck?

Answer

Try it out!

Intro to MCMC

Geometric Probability

Question

What is the expected distance between two random points on [0, 1]?

Answer ∫ 1

|x− y|dxdy =1

Question

What is the expected distance between two random points on [0, 1]n?

Answer ∫ 1

· · ·∫ 1

√√√√ n∑j=1

(xj − yj)2dx1 · · · dxndy1 · · · dyn = : (

Intro to MCMC

Numerical Integration

Question

What is the area “under” the curve?∫ 1

∫ √1−x2

Intro to MCMC

Different Perspectives on “Friendship”

Intro to MCMC

Krackhardt D. (1987). Cognitive social structures. Social Networks, 9, 104-134.

Intro to MCMC

“Friendship” over Time

Newcomb T. (1961). The acquaintance process. New York: Holt, Reinhard and Winston.

Intro to MCMC

Monte Carlo Methods

Properties of Monte Carlo Methods

• Draw (independent) samples from a random distribution

• Compute some measure for each draw

• Repeat lots and lots of times

• Average/aggregate the derived data

Intro to MCMC

Historical Overview

Moderately Ancient

• Buffon’s Needle Experiment

• Lord Kelvin (out of a hat)

• Everyone...

Intro to MCMC

Historical Overview

Intro to MCMC

Markov Chain Methods

What is a Markov chain?

Definition (Markov Chain)

A sequence of random variables X1, X2, . . . , is called a Markov Chain if

P(Xn = xn : X1 = x1, . . . , Xn−1 = xn−1) = P(Xn = xn : Xn−1 = xn−1).

Examples

• Snakes & Ladders

• Text generation

• Walks on graphs (PageRank)

• Walks on (families of) graphs (markovchain)

Intro to MCMC

Markov Formalism

Given a finite state space X = x1, x2, . . . , xn we can specify a Markovchain over X with transition probabilities pi,j = P(Xm = i : Xm−1 = j)and associated transition matrix P = [pi,j ].Desirable adjectives:

• Irreducible: A chain is irreducible if each state is (eventually)reachable from every other state.

• Aperiodic: A chain is aperiodic if for each state, the GCD of thelengths of the loops, starting and ending at that state is equal to 1.

• Steady State Distribution: A distribution π is said to be a steadystate of the chain if π = πP .

Intro to MCMC

Key Theorem

If the chain is irreducible and periodic then limm→∞ Pm = 1π for aunique π. Even better, if y1, y2, . . . , ym are samples from π then,

limm→∞

m∑i=1

f(yi) = E[f ]

The key step of MCMC is to create an irreducible, aperiodic Markov chainwhose steady state distribution π is the distribution we are trying tosample from.

Intro to MCMC

What is MCMC?

In our Monte Carlo methods we just required that we sample from ourspace uniformly but this isn’t always easy to do. MCMC gives us a way tosample from a desired pre–defined distribution by forming a related Markovchain (or walk) over our state space, with transition probabilitiesdetermined by a multiple of the distribution that we are trying to samplefrom.

Intro to MCMC

Proportional to a distribution!?!

A common way this arises is when we have a score function or a rankingon our state space and want to draw proportionally to these scores. Givena score s : X → R we want to sample from X with probabilities

P(Xi) =s(Xi)∑

When |X| is enormous, we don’t want to/can’t compute the denominatordirectly. Also, uniform sampling over–prioritizes low score spaces. This isalso an advantage to local methods.

Intro to MCMC

Simplifications

• Discrete probability distribution/state space

• Score function distributions

• Symmetric proposal distributions

• ...

Intro to MCMC

MCMC on Graphs

Terminology

Score: A function s : X → R≥0 that determines our target distribution.

Proposal Distribution: A Markov chain Q over X with the propertythat Q(xj : xi) = Q(xi : xj).

Metric: Another function f : X → R that is our quantity of interest forthe distribution.

Intro to MCMC

MCMC on Graphs

Terminology

Score: A function s : X → R≥0 that determines our target distribution.

Proposal Distribution: A Markov chain Q over X with the propertythat Q(xj : xi) = Q(xi : xj).

Metric: Another function f : X → R that is our quantity of interest forthe distribution.

Intro to MCMC

MCMC on Graphs

Metropolis Procedure

Given that we have a given score function, proposal distribution, metric,and initial graph g0 we generate new graphs gn by:

1 Generating g according to the proposal distribution Q(g : gi).

2 Compute the acceptance probability: α = min

(1,s(g)

)3 Pick a number β uniformly on [0, 1]

gi+1 =

{g if β < α

gi otherwise/

Intro to MCMC

MCMC on Graphs

Example:

Score: s(G) is the number of edges within each color

Proposal Distribution: Uniform on partitions with two colors, whereeach color forms a connected subgraph and the maximum imbalancebetween the colors is 2 nodes.

Metric: f(G) is the number of edges between two nodes of differentcolors

Intro to MCMC

MCMC on Graphs

Example: Initial graph

Intro to MCMC

MCMC on Graphs

Example: Proposed g

Intro to MCMC

MCMC on Graphs

Example: Proposed g

Old: 7 edges between yellow nodes and 3 edges between green nodes.New: 4 edges between yellow nodes and 5 edges between green nodes.

Intro to MCMC

MCMC on Graphs

Example: α and β

α = min

(1,s(g)

)= min

β = very random = .31415

Success!!

Intro to MCMC

MCMC on Graphs

Example: Metric of interest

Old: There are 8 edges between the colors. New: There are 6 edgesbetween the colors.

Intro to MCMC

MCMC on Graphs

Scores

• Compactness

• Legal Constraints

• Network measures

• ....

Intro to MCMC

MCMC on Graphs

Proposal Distributions

• Boundary edge flips

• Uniform

• Node flips

• ...

Intro to MCMC

MCMC on Graphs

Metrics

• Compactness

• Legal Constraints

• Network measures

• ....

Introduction to Monte Carlo Methods - sites.tufts.edu · Intro to MCMC Introduction Introduction to...

Documents

Transcript of Introduction to Monte Carlo Methods - sites.tufts.edu · Intro to MCMC Introduction Introduction to...

Markov chain Monte Carlo (MCMC) methods

Approximate Inference using MCMC - mit.edu9.520/spring10/Classes/class21_mcmc_2010.pdf · •In this class, we will concentrate on Markov Chain Monte Carlo (MCMC) methods for performing

Particle Markov chain Monte Carlo methods...Introduction Introduction Markov chain Monte Carlo (MCMC) methods have become a standard tool for statisticians. Methods like the Metropolis-Hastings

Computer Vision Lab. SNU Young Ki Baik An Introduction to MCMC for Machine Learning (Markov Chain Monte Carlo)

An Introduction to MCMC for Machine Learning - Vision Labsvision.psych.umn.edu/users/schrater/schrater_lab/courses/AI2/MCMC... · An Introduction to MCMC for Machine Learning CHRISTOPHE

MCMC: Does it work? How can we tell? - School of Statisticsusers.stat.umn.edu/~geyer/jsm09.pdf · MCMC Markov chain Monte Carlo (MCMC) is great stu . MCMC revitalized Bayesian inference

Adaptive Increasingly Rarely Markov Chain Monte Carlo (AirMCMC) · 2017-07-28 · Adaptive MCMC Air MCMC Adaptive Increasingly Rarely Markov Chain Monte Carlo (AirMCMC) Krys Latuszynski

Markov Chain Monte Carlo (MCMC)

Lecture I A Gentle Introduction to Markov Chain Monte ... · PDF fileLecture I A Gentle Introduction to Markov Chain Monte Carlo (MCMC) Ed George University of Pennsylvania Seminaire

Markov Chain Monte Carlo. - HEC Montréalneumann.hec.ca/~p240/c660176/MCMC.pdf · 2010. 4. 6. · MCMC G. Gauthier Introduction Chaîne de Markov MCMC Ex.: estimation Algorithme de

Introduction to Discrete MCMC for Redistricting · 2019-06-16 · 1 Introduction This is intended as a friendly introduction to the ideas and background of Markov Chain Monte Carlo

Markov Chain Monte Carlo (MCMC) Inferencenlp.chonbuk.ac.kr/PGM2019/slides_jbnu/MCMC.pdf · 2019-05-29 · Markov Chain Monte Carlo (MCMC) Inference Seung-Hoon Na Chonbuk National

An introduction to adaptive MCMC - University of Warwick€¦ · An introduction to adaptive MCMC Gareth Roberts MIRAW Day on Monte Carlo methods March 2011 Mainly joint work with

An Introduction to MCMC for Machine Learning - cs.ubc.caarnaud/andrieu_defreitas_doucet_jordan... · algorithms, known as Markov chain Monte Carlo (MCMC). These algorithms have played

Statistics for Data Scientists: Monte Carlo and MCMC ...faculty.ucr.edu/~jflegal/StatisticsForDataScientists.pdfStatistics for Data Scientists: Monte Carlo and MCMC Simulations James

Comparison of two Markov chain Monte Carlo (MCMC) methods · Markov chain Monte Carlo (MCMC) draws samples by running a cleverly constructed Markov chain for a long time. Some MCMC

Introduction to MCMC methods

eprints.uad.ac.ideprints.uad.ac.id/2426/1/080902_B1_4_Semnas_UTY_5.pdfKeywords : Bayesian model, Hybrid MCMC methods, Signal Segmentation PENDAHULUAN ... Chain Monte Carlo (MCMC),

An introduction to STAN and SAS PROC MCMC · 2013-06-21 · 244 C. Markov-chains Monte-Carlo methods than with a i.i.d. sampling. The advantage of MCMC methods is to allow the sampling

§❹ The Bayesian Revolution: Markov Chain Monte Carlo (MCMC)