Unsupervised Learning of Compositional Sparse Code for Natural Image Representation
description
Transcript of Unsupervised Learning of Compositional Sparse Code for Natural Image Representation
![Page 1: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/1.jpg)
Unsupervised Learning of Compositional Sparse Code
for Natural Image Representation
Ying Nian WuUCLA Department of Statistics
October 5, 2012, MURI Meeting
Based on joint work with Yi Hong, Zhangzhang Si, Wenze Hu, Song-Chun Zhu
![Page 2: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/2.jpg)
Sparse Representation
Sparsity: most of coefficients are zero Matching pursuit: Mallat, Zhang 1993 Basis pursuit/Lasso/CS: Chen, Donoho, Saunders 1999; Tibshirani 1996 LARS: Efron, Hastie, Johnstone, Tibshirani, 2004 SCAD: Fan, Li 2001
Dictionary learning Sparse component analysis: Olshausen, Field 1996
K-SVD: Aharon, Elad, Bruckstein 2006 Unsupervised learning: SCA, ICA, RBM, NMF FA
![Page 3: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/3.jpg)
Group Sparsity
Group Lasso: Yuan, Lin 2006 The basis functions form groups (multi-level factors/additive model)
Our goal: Learn recurring compositional patterns of groups Compositionality (S. Geman; Zhu, Mumford) Active basis models for deformable templates Atomic decomposition molecular structures
![Page 4: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/4.jpg)
The first 7 iterations
Learning in the 10th iteration
Learned dictionary of composition patterns from training image
Generalize to testing images
![Page 5: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/5.jpg)
Shared matching pursuit
Support union regressionMulti-task learningAvoid early decision
Active basis model
![Page 6: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/6.jpg)
Active basis model: non-Gaussian background
Della Pietra, Della Pietra, Lafferty, 97; Zhu, Wu, Mumford, 97; Jin, S. Geman, 06; Wu, Guo, Zhu, 08
![Page 7: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/7.jpg)
Log-likelihood
![Page 8: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/8.jpg)
After learning template, find object in testing image
![Page 9: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/9.jpg)
Sparse coding model
Rewrite active basis model in packed form
Represent image by a dictionary of active basis models
![Page 10: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/10.jpg)
Olshausen-Field: coding units are wavelets
Our model: coding units are deformable compositions of wavelets
The coding units allow variations, making it generalizable (1) variations in geometric deformations (2) variations in coefficients of wavelets (lighting variations) (3) AND-OR units (Pearl, 1984; Zhu, Mumford 2006) (4) Log-likelihood
![Page 11: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/11.jpg)
Our model: coding units are deformable compositions of wavelets
Learning algorithm: specify number and size of templates
Image encoding: template matching pursuit
Dictionary re-learning: shared matching pursuit collect and align image patches currently encoded by each template re-learn each template from the collected and aligned image patches
Inhibition
The first 7 iterations
Learning in the 10th iteration
![Page 12: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/12.jpg)
1385 1950
1831 1818
![Page 13: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/13.jpg)
1247725
1096 844
![Page 14: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/14.jpg)
1887 2838
2737 2644
![Page 15: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/15.jpg)
![Page 16: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/16.jpg)
![Page 17: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/17.jpg)
![Page 18: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/18.jpg)
![Page 19: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/19.jpg)
![Page 20: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/20.jpg)
![Page 21: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/21.jpg)
![Page 22: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/22.jpg)
![Page 23: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/23.jpg)
![Page 24: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/24.jpg)
![Page 25: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/25.jpg)
![Page 26: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/26.jpg)
![Page 27: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/27.jpg)
![Page 28: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/28.jpg)
![Page 29: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/29.jpg)
![Page 30: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/30.jpg)
![Page 31: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/31.jpg)
![Page 32: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/32.jpg)
![Page 33: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/33.jpg)
![Page 34: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/34.jpg)
15 training images: 61.63 \pm 2.2 %30 training images: 68.49 \pm 0.9%
![Page 35: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/35.jpg)
Information scaling
fine coarse
Wu, Zhu, Guo 2008
GeometryTexture Image patterns of different statistical properties are connected by scale A common framework for modeling different regimes of image patterns
Change of statistical/information-theoretical properties of imagesover the change of viewing distance/camera resolution
![Page 36: Unsupervised Learning of Compositional Sparse Code for Natural Image Representation](https://reader030.fdocuments.us/reader030/viewer/2022033022/56816354550346895dd3fe04/html5/thumbnails/36.jpg)