Epitome Tuition - Free Notes and ...€¦ · . Epitome Tuition - Free Notes and Video Lessons @
University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information...
-
Upload
gavin-chambers -
Category
Documents
-
view
214 -
download
0
Transcript of University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information...
![Page 1: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/1.jpg)
Aug. 11, 2004
Universityof Toronto
Learning the “Epitome”of a Video Sequence
Information Processing Workshop 2004
Vincent Cheung
Probabilistic and Statistical Inference Group
Electrical & Computer Engineering
University of Toronto
Toronto, Ontario, Canada
Advisor: Dr. Brendan J. Frey
![Page 2: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/2.jpg)
Cheung2 / 12
Information Processing Workshop 2004
Outline
● Image epitome► What?► Why?
● Implementation computation issues► Efficiently implementing the learning algorithm
● Video epitome► Extension to videos► Video inpainting
![Page 3: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/3.jpg)
Cheung3 / 12
Information Processing Workshop 2004Im
age
Lea
rnin
gV
ide
o
Image Epitome
● Jojic, N., Frey, B., & Kannan, A. (2003). Epitomic analysis of appearance and shape. In Proc. IEEE ICCV.
● Miniature, condensed version of the image
● Accurately accounts for the interesting properties of the image
● Applications► object detection► texture segmentation► image retrieval► compression
![Page 4: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/4.jpg)
Cheung4 / 12
Information Processing Workshop 2004Im
age
Lea
rnin
gV
ide
o
Image Epitome Examples
![Page 5: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/5.jpg)
Cheung5 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Epitome
Input image
Training Set
SamplePatches
UnsupervisedLearning
Learning the Image Epitome
e
Z1 Z2 ZM
…
TMT2T1Bayesiannetwork
e – epitomeTk – mappingZk – image patch
![Page 6: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/6.jpg)
Cheung6 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Xe(i,j)
KK
N
N
Cumsum
2
N
N
Xe(i,j)
KK
N
N
Cumsum
2
N
N
Shifted Cumulative Sum Algorithm
(2, 2), (i+1, j+1)
(1, 2), (i, j+1)
(2, 1), (i+1, j)
– col 1+ col (P+1)
– row 1+ row (P+1)
–
+
+ + pixel (1,1)+ pixel (P+1, P+1)
(1, 1), (i, j)
(2, 2), (i+1, j+1)(2, 2), (i+1, j+1)
(1, 2), (i, j+1)(1, 2), (i, j+1)
(2, 1), (i+1, j)(2, 1), (i+1, j)
– col 1+ col (P+1)
– row 1+ row (P+1)
–
+
+ + pixel (1,1)+ pixel (P+1, P+1)
(1, 1), (i, j)(1, 1), (i, j)
+-
-+
![Page 7: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/7.jpg)
Cheung7 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
X e
P
P
P
P
Ta
Tß
X e
P
P
P
P
Ta
Tß
Collecting Sufficient Statistics
![Page 8: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/8.jpg)
Cheung8 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Extending Epitomes to Videos
● Desire a miniature, condensed version of a video sequence
● Want it to accurately account for the interesting properties of the video
● Applications► optic flow► segmentation► texture transfer► layer separation► compression► noise reduction► inpainting
![Page 9: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/9.jpg)
Cheung9 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Input Video
Frame 1Frame 2
Frame 3
Training Set
SamplePatches
Video Epitome
UnsupervisedLearning
Video Epitome
![Page 10: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/10.jpg)
Cheung10 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Video Epitome Example
Temporally Compressed
Spatially Compressed
![Page 11: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/11.jpg)
Cheung11 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Video Inpainting (1)
● Fill in missing portions of a video► damaged films► occluding objects
● Reconstruct the missing pixels from the video epitome
![Page 12: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/12.jpg)
Cheung12 / 12
Imag
eL
earn
ing
Vid
eo
Information Processing Workshop 2004
Video Inpainting (2)
![Page 13: University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.](https://reader035.fdocuments.us/reader035/viewer/2022062806/56649ece5503460f94bdaaa1/html5/thumbnails/13.jpg)
Cheung13 / 12
Information Processing Workshop 2004
Conclusion
● Improved the efficiency of learning image epitomes
● Extended the concept of epitomes to video sequences
● Demonstrated the ability of video epitomes to model motion patterns through video inpainting