Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana...
description
Transcript of Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana...
![Page 1: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/1.jpg)
Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories
Svetlana Lazebnik Cordelia Schmid Jean PonceBeckman Institute Gravir Laboratory Beckman InstituteUIUC, USA INRIA, France UIUC, USA
Supported in part by the UIUC Campus Research Board, the UIUC/CNRS Collaborative Research Agreement, and the National Science Foundation under grant IRI-990709.
![Page 2: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/2.jpg)
• 3D objects are never planar in the large,but they are always planar in the small.
• Representation: Local invariants andtheir spatial layout.
• Affine-invariant patches.
LeCun’03
![Page 3: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/3.jpg)
(Lindeberg & Garding’97)(Mikolcajczyk & Schmid’02)
![Page 4: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/4.jpg)
• Spatial selection • Shape selection• Affine adaption
Schaffalitzky & Zisserman (2001); Tuytelaars & Van Gool (2003)
![Page 5: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/5.jpg)
(0,0)
(0,1)
(1,0)
Image 1 Image 2
Affine adaptation/Rectification process
Lindeberg & Garding (1997)Mikolcajczyk & Schmid (2002)Rectified patch
![Page 6: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/6.jpg)
[Range spin images: Johnson & Hebert (1998)]
Intensity-Domain Spin Images
![Page 7: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/7.jpg)
System architecture (Lazebnik, Schmid, & Ponce, CVPR’03)
[Signatures and EMD for image retrieval: Rubner, Tomasi, & Guibas (1998)]
• Signature: SS = { ( m1 , w1 ) , … , ( mk , wk ) }• Earth Mover’s Distance: D( SS , SS’’ ) = [i,j fij d( mi , m’j)] / [i,j fij ]
![Page 8: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/8.jpg)
Texture retrieval/classification experiments
Schmid (2001); Varma & Zisserman (2002)
10 texture classes, with 20 samples per class.
NN classification
Ave
rage
reco
gniti
on ra
te
Ave
rage
reco
gniti
on ra
te
![Page 9: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/9.jpg)
More retrieval/classification experiments: Brodatz database
• Picard et al. (1993, 1996)• Xu et al. (2000)
111 images divided into 9 windows
111 classes with 9 samples per class
Ave
rage
reco
gniti
on ra
te
Ave
rage
reco
gniti
on ra
te
![Page 10: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/10.jpg)
T1 (brick) T2 (carpet) T3 (chair) T4 (floor 1) T5 (floor 2) T6 (marble) T7 (wood)
Multi-texture Samples
Texture Classes [NOTE: we do NOT use color information.]
![Page 11: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/11.jpg)
A Two-Layer Architecture(Lazebnik, Schmid, & Ponce, ICCV’03)
Modeling:1. Use EM to learn a mixture-of-Gaussians model of
each texture class.2. Compute co-occurrence statistics of sub-class labels
over affinely adapted neighborhoods.
Recognition:1. Use the generative model to obtain initial class
membership probabilities.2. Use relaxation (Rosenfeld et al., 1976) to refine these
probabilities.
Malik, Belongie, Leung, & Shi (2001); Schmid (2001); Kumar & Hebert (2003)
![Page 12: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/12.jpg)
Neighborhood Statistics
Estimate:• probability p(c,c’),• correlation r(c,c’).
![Page 13: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/13.jpg)
Relaxation (Rosenfeld et al., 1976)
Iterate, for all regions i:
where
and wij=0 is region j is not in the neighborhood of i, with j wij=1.
![Page 14: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/14.jpg)
Classification rates for single-texture images
10 training images per class, 10 test images per class.
![Page 15: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/15.jpg)
Weakly-Supervised Modeling
Idea: Replace L mixture models with M components by a single mixture model with L x M components.
• Annotate each image with the set C of labels associated with classes occurring in it.
• Run EM:• E step: update class membership probabilities:
p (clm | x, C ) / p ( x | clm ) p ( clm | C ).• M step: update model parameters.
Nigam, McCallum, Thrun & Mitchell (2000)
![Page 16: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/16.jpg)
T1 (brick) T2 (carpet) T3 (chair) T4 (floor 1) T5 (floor 2) T6 (marble) T7 (wood)
T1 (brick) T2 (carpet) T3 (chair) T4 (floor 1) T5 (floor 2) T6 (marble) T7 (wood)
Single-texture training images only
Single- and multi-texture training images
ROC Curves
10 single-texture images per class, 13 two-texture training images, 45 multi-texturetest images.
![Page 17: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/17.jpg)
Effect of relaxation on labelingOriginal image
Top: before relaxation, bottom: after relaxation
![Page 18: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/18.jpg)
Successful Segmentation Examples
![Page 19: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/19.jpg)
Unsuccessful Segmentation Examples
![Page 20: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/20.jpg)
Animal Dataset
• 10 training images for each animal + background, 20 test images per class.
Bradshaw, Scholkopf, & Platt (2001); Schmid (2001); Kumar & Hebert (2003)
• No manual segmentation.
![Page 21: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/21.jpg)
![Page 22: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/22.jpg)
Oh well..
![Page 23: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/23.jpg)
![Page 24: Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories Svetlana LazebnikCordelia SchmidJean Ponce](https://reader035.fdocuments.us/reader035/viewer/2022062521/5681684b550346895dde3f61/html5/thumbnails/24.jpg)
• 3D Objects without distinctive texture
• Category-level recognition of 3D objects
• Please join us in trying to solve the 3D object recognition problem..