A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
description
Transcript of A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
![Page 1: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/1.jpg)
A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
Sajid M. SiddiqiByron Boots
Geoffrey J. Gordon
Carnegie Mellon University
NIPS 2007poster W22
![Page 2: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/2.jpg)
Objective• Learn parameters of a Linear
Dynamical System from data while ensuring a stable dynamics matrix A more efficiently and accurately than previous methods
![Page 3: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/3.jpg)
• Formulate an approximation of the problem as a semi-definite program (SDP)
• Start with a quadratic program (QP) relaxation of this SDP, and incrementally add constraints
• Because the SDP is an inner approximation of the problem, we reach stability early, before reaching the feasible set of the SDP
• We interpolate the solution to return the best stable matrix possible
Our Approach
![Page 4: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/4.jpg)
Outline• Linear Dynamical Systems• Stability• Convexity• Constraint Generation Algorithm• Empirical Results• Conclusion
![Page 5: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/5.jpg)
Application: Dynamic Textures1
• Videos of moving scenes that exhibit stationarity properties
• Can be modeled by a Linear Dynamical System• Learned models can efficiently simulate realistic sequences• Applications: compression, recognition, synthesis of videos• But there is a problem …
steam
river
fountain
1 S. Soatto, D. Doretto and Y. Wu. Dynamic Textures. Proceedings of the ICCV, 2001
![Page 6: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/6.jpg)
Linear Dynamical Systems
• Input data sequence
• Assume a latent variable that explains the data
• Assume a Markov model on the latent variable
x1 x2 x3
y1 y2 y3
… xT
yT…
![Page 7: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/7.jpg)
Linear Dynamical Systems2
• State and observation updates:
• Dynamics matrix:• Observation matrix:
2 Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Trans.ASME-JBE
![Page 8: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/8.jpg)
Learning Linear Dynamical Systems
• Suppose we have an estimated state sequence
• Define state reconstruction error as the objective:
• We would like to learn such thati.e.
![Page 9: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/9.jpg)
Subspace Identification3
• Subspace ID uses matrix decomposition to estimate the state sequence
• Build a Hankel matrix D of stacked observations
3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996
![Page 10: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/10.jpg)
Subspace ID with Hankel matrices
Stacking multiple observations in D forces latent states to model the future
e.g. annual sunspots data with 12-year cycles
=
First 2 columns of U
k = 1
k = 12
t t
![Page 11: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/11.jpg)
Subspace Identification3
• In expectation, the Hankel matrix is inherently low-rank!
• Can use SVD to obtain the low-dimensional state sequence
3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996
=
For D with k observations per column,
![Page 12: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/12.jpg)
Training LDS models• Lets train an LDS for steam textures
using this algorithm, and simulate a video from it!
= [ …]xt 2 R40
![Page 13: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/13.jpg)
Simulating from a learned model
xt
t
The model is unstable
![Page 14: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/14.jpg)
Notation• 1 , … , n : eigenvalues of A (|1| > … > |n| )
• 1,…,n : unit-length eigenvectors of A
• 1,…,n : singular values of A (1 > 2 > … n)
• S matrices with || · 1
• Smatrices with 1 · 1
![Page 15: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/15.jpg)
Stability• a matrix A is stable if |1|· 1, i.e. if
• Why?
matrix of eigenvectors
eigenvalues in decreasing order of magnitude
![Page 16: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/16.jpg)
Stability
• We would like to solve
0 5 10 15 20-20
0
20
0 5 10 15 200
500
1000
|1| = 0.3
|1| = 1.295
xt(1), xt(2)
xt(1), xt(2)
s.t.
![Page 17: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/17.jpg)
• The state space of a stable LDS lies inside some ellipse
• The set of matrices that map a particular ellipse into itself (and hence are stable) is convex
• If we knew in advance which ellipse contains our state space, finding A would be a convex problem. But we don’t
• …and the set of all stable matrices is non-convex
?
??
x̂
x̂
Stability and Convexity
![Page 18: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/18.jpg)
Stability and Convexity• So, S is non-convex!
• Lets look at Sinstead …
– Sis convex
– Sµ S
• Previous work4 exploits these properties to learn a stable A by solving an SDP for the problem:
4 S. L. Lacy and D. S. Bernstein. Subspace identification with guaranteed stability using constrained optimization. In Proc. of the ACC (2002), IEEE Trans. Automatic Control (2003)
A1 A2
s.t.
![Page 19: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/19.jpg)
Simulating from a Lacy-Bernstein stable texture model
xt
t
Model is over-constrained. Can we do better?
![Page 20: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/20.jpg)
• Formulate the S approximation of the problem as a semi-definite program (SDP)
• Start with a quadratic program (QP) relaxation of this SDP, and incrementally add constraints
• Because the SDP is an inner approximation of the problem, we reach stability early, before reaching the feasible set of the SDP
• We interpolate the solution to return the best stable matrix possible
Our Approach
![Page 21: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/21.jpg)
The Algorithm
• A1: unconstrained QP solution (least squares estimate)• A2: QP solution after 1 constraint (happens to be stable)• Afinal: Interpolation of stable solution with the last one• Aprevious method: Lacy Bernstein (2002)
objective function contours
![Page 22: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/22.jpg)
Quadratic Program Formulation
![Page 23: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/23.jpg)
Quadratic Program Formulation
Constraints are of the form
![Page 24: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/24.jpg)
Constraint Generation
![Page 25: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/25.jpg)
Simulating from a Constraint Generation stable texture model
xt
t
Model captures more dynamics and is still stable
![Page 26: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/26.jpg)
![Page 27: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/27.jpg)
Empirical Evaluation• Algorithms:
– Constraint Generation – CG (our method)– Lacy and Bernstein (2002) –LB-1
• finds a 1 · 1 solution
– Lacy and Bernstein (2003)–LB-2 • solves a similar problem in a transformed space
• Data sets– Dynamic textures– Biosurveillance
![Page 28: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/28.jpg)
Reconstruction error• Steam video
% d
ecr
ease
in
ob
ject
ive
(lo
wer
is b
ett
er)
number of latent dimensions
![Page 29: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/29.jpg)
Running time• Steam video
Ru
nn
ing
tim
e (
s)
number of latent dimensions
![Page 30: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/30.jpg)
Reconstruction error• Fountain video
% d
ecr
ease
in
ob
ject
ive
(lo
wer
is b
ett
er)
number of latent dimensions
![Page 31: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/31.jpg)
Running time• Fountain video
Ru
nn
ing
tim
e (
s)
number of latent dimensions
![Page 32: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/32.jpg)
![Page 33: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/33.jpg)
Other Domains
![Page 34: A Constraint Generation Approach to Learning Stable Linear Dynamical Systems](https://reader033.fdocuments.us/reader033/viewer/2022051019/56815989550346895dc6cba0/html5/thumbnails/34.jpg)
Conclusion• A novel constraint generation algorithm for
learning stable linear dynamical systems
• SDP relaxation enables us to optimize over a larger set of matrices while being more efficient
• Future work:– Adding stability constraints to EM– Stable models for more structured dynamic textures