Dynamic Time Warping (DTW)
-
Upload
jana-callahan -
Category
Documents
-
view
172 -
download
9
description
Transcript of Dynamic Time Warping (DTW)
Dynamic Time Warping (DTW)
J.-S Roger Jang (張智星 )
http://mirlab.org/jang
MIR Lab, CSIE Dept
National Taiwan University
-2-
Dynamic Time Warping
Goal To align two sequences under certain constraints, such that the distance between these two sequences is as small as possible.
Method Dynamic programming
-3-
Distance between Same-length Sequences
Distance between
Alignment
1 2 1 2, , and , ,n nx x x x y y y y
1 2 3 4 5 6 7 8 9, , , , , , , ,x x x x x x x x x x
1 2 3 4 5 6 7 8 9, , , , , , , ,y y y y y y y y y y
1 1
2
11
, 1
( , ), 2
max ,
n
i ii
n pp np i i
i iii
i ii
x y p
L norm x y x yx y p
x y p
-4-
Distance between Different-length Sequences
1 2 1 2
1 1 2 2
, , 1
, , , , ,
We want to find an alignment path , , , , ,
(subject to the alignment constraints), such that the distance along the
path is minimized:
( , ) mini i
i i
m n
k k
p qp q i k
x x x x y y y y
p q p q p q
dist x y x y
2
1
1 1
,
with the bounary conditions:
Anchored beginning: , 1,1
Anchored end: , ,
k
i
k k
p q
p q m n
-5-
Alignment Constraints: Type 1
Temporal constraints
Local path constraints One-to-one mapping No consecutive skip-over
1 2 1 2,k kp p p q q q
x1 x2 x3
y1
x4 x5
y2 y3 y4 y5 y6 y7 y8
Alignment path =
1,1 , 2,3 , 3,5 , 4,6 , 5,8
-6-
Type-1 Alignment Path
Alignment path (mapping path)
x1 x2 x3
y1
x4 x5
y2 y3 y4 y5 y6 y7 y8
Alignment path =
1,1 , 2,3 , 3,5 , 4,6 , 5,8
i
1
),( jiD
2
3
4
5
6
7
8
1 2 3 4 5
j
-7-
Alignment Constraints: Type 2
Temporal constraints
Local path constraints 1-to-1, 1-to-many, or many-to-1 mapping
No skip-over
1 2 1 2,k kp p p q q q
x1 x2 x3
y1
x4 x5
y2 y3 y4 y5 y6 y7 y8
Alignment path =
1,1 , 2,2 , 2,3 , 3,4 , 3,5 , 4,6 , 5,7 , 5,8
-8-
Type-2 Alignment Path
Alignment path
x1 x2 x3
y1
x4 x5
y2 y3 y4 y5 y6 y7 y8
Alignment path =
1,1 , 2,2 , 2,3 , 3,4 , 3,5 , 4,6 , 5,7 , 5,8
i
1
2
3
4
5
6
7
8
1 2 3 4 5
j
-9-
Type-1 DTW: Table Fillup
i
j
x(i-1)
y(j) 1. Optimum-value function ( , ):
The DTW distance between (1: ) and (1: )
2. Recurrent equation for ( , ) :
( 1, 2)
( , ) ( ) ( ) min ( 1, 1) ,
( 2, 1)
with (1,1) (1) (1) .
3. Answ
D i j
x i y j
D i j
D i j
D i j x i y j D i j
D i j
D x y
er ( ( ), ( ))D len x len y
),( jiD
x, y: input vector/matrixLocal paths: 27-45-63 degrees
DTW formulation:
y(j-1)
x(i)
-10-
Type-2 DTW: Table Fillup
1. Optimum-value function ( , ):
The DTW distance between (1: ) and (1: )
2. Recurrent equation for ( , ) :
( , 1)
( , ) ( ) ( ) min ( 1, 1) ,
( 1, )
with (1,1) (1) (1) .
3. Answer
D i j
x i y j
D i j
D i j
D i j x i y j D i j
D i j
D x y
D
( ( ), ( ))len x len y
x, y: input vector/matrixLocal paths: 0-45-90 degrees
DTW formulation:
i
j
x(i-1)
y(j) ),( jiD
y(j-1)
x(i)
-11-
Local Path Constraints
Type 1: 27-45-63 local paths
Type 2: 0-45-90 local paths
jiD ,
jiD ,
( , ) ( ) ( )
( , 1)
min ( 1, 1)
( 1, )
D i j x i y j
D i j
D i j
D i j
( , ) ( ) ( )
( 1, 2)
min ( 1, 1)
( 2, 1)
D i j x i y j
D i j
D i j
D i j
2,1 jiD
1, jiD 1,1 jiD
jiD ,1
1,1 jiD 1,2 jiD
-12-
Path Penalty for Type-1 DTW
Alignment path of type-1 DTW 45-degree paths are likely to be avoided since we
are minimizing the total distance. So we can add penalty for paths deviated from 45-
degree.
( 1, 2)
( , ) ( ) ( ) min ( 1, 1)
( 2, 1)
D i j
D i j x i y j D i j
D i j
0
jiD ,
2,1 jiD
1,1 jiD 1,2 jiD
-13-
Path Penalty for Type-2 DTW
Alignment path of type-1 DTW 45-degree paths are likely to be taken since we are
minimizing the total distance. So we can add penalty for paths of 45-degree.
( , 1)
( , ) ( ) ( ) min ( 1, 1)
( 1, )
D i j
D i j x i y j D i j
D i j
jiD ,
1, jiD 1,1 jiD
jiD ,1
0
0
-14-
Other Minutes about DTW
Typical applications Speech recognition: MFCC matrices as inputs
(where x(i) is the MFCC vector of frame i) Query by singing/humming: Pitch vectors as
inputs (where x(i) is the pitch value of frame i)
Abundant variants for various applications Recurrent formulas Local path constraints
-15-
Applications
Applications of DTW DTW for speech recognition DTW for query by singing/humming