Applied Bayesian Nonparametrics 3. Infinite Hidden Markov...

40
Applied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR 2012 Erik Sudderth Brown University Work by E. Fox, E. Sudderth, M. Jordan, & A. Willsky AOAS 2011: A Sticky HDP-HMM with Application to Speaker Diarization IEEE TSP 2011 & NIPS 2008: Bayesian Nonparametric Inference of Switching Dynamic Linear Models NIPS 2009: Sharing Features among Dynamical Systems with Beta Processes

Transcript of Applied Bayesian Nonparametrics 3. Infinite Hidden Markov...

Page 1: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Applied Bayesian Nonparametrics

3. Infinite Hidden Markov Models

Tutorial at CVPR 2012Erik Sudderth Brown University

Work by E. Fox, E. Sudderth, M. Jordan, & A. Willsky AOAS 2011: A Sticky HDP-HMM with Application to Speaker Diarization IEEE TSP 2011 & NIPS 2008: Bayesian Nonparametric Inference of Switching Dynamic Linear Models NIPS 2009: Sharing Features among Dynamical Systems with Beta Processes

Page 2: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Observations True mode sequence •! Markov switching models for time series data

•! Cluster based on underlying mode dynamics

Temporal Segmentation

Hidden Markov Model modes

observations

Page 3: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Outline Temporal Segmentation !!How many dynamical modes?

!!Mode persistence

!!Complex local dynamics

!!Multiple time series

Spatial Segmentation

!! Ising and Potts MRFs

!!Gaussian processes

Page 4: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Hidden Markov Models

Time

Mod

e

modes

observations

Page 5: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Hidden Markov Models

Time modes

observations

Page 6: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Hidden Markov Models

Time modes

observations

Page 7: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Hidden Markov Models

Time modes

observations

Page 8: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Issue 1: How many modes?

•! Dirichlet process (DP): !! Mode space of unbounded size !! Model complexity adapts to

observations

•! Hierarchical: !! Ties mode transition

distributions !! Shared sparsity

Time

Mod

e

Infinite HMM: Beal, et.al., NIPS 2002 HDP-HMM: Teh, et. al., JASA 2006

Hierarchical Dirichlet Process HMM

Page 9: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

•! Global transition distribution:!

HDP-HMM

sparsity of ! is shared

•! Mode-specific transition distributions:!

Hierarchical Dirichlet Process HMM

Page 10: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Issue 2: Temporal Persistence

Hidden Markov Model

True mode sequence HDP-HMM inferred mode sequence

Page 11: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Sticky HDP-HMM

Time

Mod

e

Page 12: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Sticky HDP-HMM

mode-specific base measure

Increased probability of self-transition

sticky original

Infinite HMM: Beal, et.al., NIPS 2002

Page 13: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Direct Assignment Sampler •! Marginalize:

!! Transition densities !! Emission parameters

•! Sequentially sample:

Conjugate base measure "" closed form

Chinese restaurant

prior likelihood

Collapsed Gibbs Sampler

Splits true mode, hard to

merge

Page 14: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

•! Approximate HDP: "! Average transition density "! (" transition densities)

•! Sample:

Blocked Resampling HDP-HMM weak limit approximation HDP-HMM weak limit approximation

•! Compute backwards messages:

•! Block sample as:

" Average transition density " (" transition densities)

Page 15: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Results: Gaussian Emissions

Blocked sampler

HDP-HMM Sticky HDP-HMM

Sequential sampler

Page 16: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Sticky HDP-HMM

HDP-HMM

Results: Fast Switching

Observations

True mode sequence

Page 17: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Hyperparameters •! Place priors on hyperparameters and infer them from data •! Weakly informative priors •! All results use the same settings

hyperparameters

can be set using the data

Related self-transition parameter: Beal, et.al., NIPS 2002

Page 18: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

HDP-HMM: Multimodal Emissions

•! Approximate multimodal emissions with DP mixture

•! Temporal mode persistence disambiguates model

modes

mixture components

observations

Page 19: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

John Jane Bob John Bob

Ji l l

0 1 2 3 4 5 6 7 8 9 10

x 104

-30

-20

-10

0

10

20

30

40

Speaker Diarization

Page 20: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Results: 21 meetings

Overall DER

Best DER

Worst DER

Sticky HDP-HMM 17.84% 1.26% 34.29% Non-Sticky HDP-HMM

23.91% 6.26% 46.95%

ICSI 18.37% 4.39% 32.23%

0 10 20 30 40 500

10

20

30

40

50

Sticky DERsIC

SI D

ERs

! "! #! $! %! &!!

"!

#!

$!

%!

&!

'()*+,-./01

234

'()*+,-./0

1

Page 21: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Results: Meeting 1

Sticky DER = 1.26%

ICSI DER = 7.56%

Page 22: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Results: Meeting 18

Sticky DER = 20.48%

ICSI DER = 22.00%

4.81%

Page 23: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

= set of dynamic parameters

Issue 3: Complex Local Dynamics

•! Discrete clusters may not accurately capture high-dimensional data

•! Autoregressive HMM: Discrete-mode switching of smooth observation dynamics

Switching Dynamical Processes

modes

observations

Page 24: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Linear Dynamical Systems •! State space LTI model:

•! Vector autoregressive (VAR) process:

Page 25: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Linear Dynamical Systems •! State space LTI model:

State space models

VAR processes

•! Vector autoregressive (VAR) process:

Page 26: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Switching Dynamical Systems Switching linear dynamical system (SLDS):

Switching VAR process:

Page 27: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

HDP-AR-HMM and HDP-SLDS HDP-AR-HMM HDP-SLDS

Page 28: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Dancing Honey Bees

Page 29: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Honey Bee Results: HDP-AR(1)-HMM

Sequence 1 Sequence 2 Sequence 3

HDP-AR-HMM: 88.1% SLDS [Oh]: 93.4%

HDP-AR-HMM: 92.5% SLDS [Oh]: 90.2%

HDP-AR-HMM: 88.2% SLDS [Oh]: 90.4%

Page 30: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Issue 4: Multiple Time Series

•! Goal: !!Transfer knowledge between related time series !!Allow each system to switch between an

arbitrarily large set of dynamical modes •! Method:

!!Beta process prior !!Predictive distribution: Indian buffet process

Page 31: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

IBP-AR-HMM •! Latent features determine

which dynamical modes are used

•! Beta process prior: !! Encourages sharing !! Unbounded features

Features/Modes

Sequ

ence

s

!

Page 32: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Motion Capture

CMU MoCap: http://mocap.cs.cmu.edu/ 6 videos of exercise routines:

Page 33: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

Library of MoCap Behaviors

Page 34: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()*+$,-'-.*'$ /0122*$*'$-34$05$678$

!"#$%&'()#*(*+,-#."#/0(1#'+23#0'2%4'#2+*'5(067#8+*'5(06#,+9',)#:(*#40($%&'&#*(#;<=>??7#@')A,*)#/0(1#>A53')#B#CA&&'0*3-#<D8E#F(0G)3(4-#8E<@#H".H7#

Page 35: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()'!)%&*+)+,#-.+/)

,&.(19*2*:$%&'()*+$;*)-9&12.$

I(2+*%(:)#(/#)','2*#9'3+$%(0)#+20())#+,,#$%&'()#

Page 36: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()'!)%&*+)+,#-.+/)

<&=)'$>?&'()$<0(*(2(,#0'JA%0')#)K%*23%:5#,%53*#(:L(//#+*#)*+0*#+:&#M%:%)3#

,&.(19*2*:$%&'()*+$;*)-9&12.$

Page 37: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()'!)%&*+)+,#-.+/)

@A*+$B2&:=*$<%NN+#:''&)#23'')'-#C+:&K%23#:''&)#O',,6#*(#9'5%:#40'4+0+*%(:#

,&.(19*2*:$%&'()*+$;*)-9&12.$

Page 38: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()'!)%&*+)+,#-.+/)

>*'$@9*+$;(*3#<%NN+)#+:&#;0(K:%')#:''&#*(#9'#9+G'&#*(#2(:2,A&'#40'4+0+*%(:#

,&.(19*2*:$%&'()*+$;*)-9&12.$

Page 39: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()'!)%&*+)+,#-.+/)

C2-'*$!)**.*$D:,6#%:#<%NN+#E%&'()#

,&.(19*2*:$%&'()*+$;*)-9&12.$

Page 40: Applied Bayesian Nonparametrics 3. Infinite Hidden Markov ...sudderth/bnpCVPR12/slides/BNP3hdphmm.pdfApplied Bayesian Nonparametrics 3. Infinite Hidden Markov Models Tutorial at CVPR

!"#$%&'()'!)%&*+)+,#-.+/)

>'&2$;1?3$P:%JA'#*(#;0(K:%')-#9A*#1A,*%4,'#)*6,')#'Q%)*R#C*+99%:5#$)7#)K%0,%:5-#'*27#

,&.(19*2*:$%&'()*+$;*)-9&12.$