Subtasks of Unconstrained Face...
Transcript of Subtasks of Unconstrained Face...
![Page 1: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/1.jpg)
Subtasks of Unconstrained Face Recognition
Leibo*, Liao*, and Poggio * = equal contribution
![Page 2: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/2.jpg)
(Klontz & Jain 2013)
![Page 3: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/3.jpg)
Unconstrained face recognition
![Page 4: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/4.jpg)
Unconstrained face recognition
![Page 5: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/5.jpg)
Unconstrained face recognition
“Factors such as aging, pose, illumination, and expression (A-PIE) can not only decrease performance, they can cause its catastrophic failure” - (IARPA JANUS announcement)
![Page 6: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/6.jpg)
Transformations
(Anselmi et al. 2013)
![Page 7: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/7.jpg)
Transformations
View-specificView-invariant
(Freiwald & Tsao 2010)
![Page 8: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/8.jpg)
Subtasks of unconstrained face recognition
A collection of synthetic datasets constituting a partial decomposition of the unconstrained problem into subtasks
![Page 9: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/9.jpg)
Subtasks of unconstrained face recognition
A collection of synthetic datasets constituting a partial decomposition of the unconstrained problem into subtasks
- 400 faces- All rendered under specific transformation conditions for each “subtask”- Same-different tasks (like LabeledFaces in the Wild).
![Page 10: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/10.jpg)
A collection of synthetic datasets constituting a partial decomposition of the unconstrained problem into subtasks
- 400 faces- All rendered under specific transformation conditions for each “subtask”- Same-different tasks (like LabeledFaces in the Wild).
Subtasks of unconstrained face recognition
Unit tests for unconstrained face recognition
![Page 11: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/11.jpg)
Affine subtasks
- Translation- Scaling- In-plane rotation
![Page 12: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/12.jpg)
Non-affine subtasks
- Yaw rotation- Pitch rotation- Illumination
![Page 13: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/13.jpg)
Clutter subtasks
![Page 14: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/14.jpg)
Interaction subtasks
![Page 15: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/15.jpg)
Alternative subtasks
Detection Alignment Recognition
![Page 16: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/16.jpg)
Alternative subtasks
Detection Alignment Recognition
-Huang et al. 2008-Leibo et al. 2014
Labeled faces in the wild
![Page 17: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/17.jpg)
Alternative subtasks
Detection Alignment Recognition
-Huang et al. 2008-Leibo et al. 2014-Taigman et al. Etc.
Labeled faces in the wild
- Aligned
95+ of 123 papers on LFW indexed by Google Scholar actually used LFW-a, closely cropped
![Page 18: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/18.jpg)
SUFR-in the Wild (SUFR-W)- A new benchmark we proposed.
- Comparable to LFW.
- 13,661 images to LFW's 13,233
![Page 19: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/19.jpg)
SUFR-in the Wild (SUFR-W)- A new benchmark we proposed.
- Comparable to LFW.
- 13,661 images to LFW's 13,233
![Page 20: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/20.jpg)
Conclusion
● We advocate an algorithm development approach combining both synthetic and natural, unconstrained data.● Even if you will ultimately be working within a DAR pipeline, recognition systems that can handle transformation invariance is useful for recovering from errors of alignment. No single point of failure.● More explicit connections with neuroscience and other parts of computer vision.● The next paper we wrote after this one was entitled:
![Page 21: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/21.jpg)
Conclusion
● We advocate an algorithm development approach combining both synthetic and natural, unconstrained data.● Even if you will ultimately be working within a DAR pipeline, recognition systems that can handle transformation invariance is useful for recovering from errors of alignment. No single point of failure.● More explicit connections with neuroscience and other parts of computer vision.● The next paper we wrote after this one was entitled: Can a biologically-plausible hierarchy effectively replace face detection, alignment, recognition pipelines? (available on arxiv)
![Page 22: Subtasks of Unconstrained Face Recognitioncbcl.mit.edu/publications/ps/Subtasks_Presentation_VISAPP2014.pdf · Face Recognition Leibo*, Liao*, and Poggio * = equal contribution (Klontz](https://reader034.fdocuments.us/reader034/viewer/2022050417/5f8d215cf04b555bf7116500/html5/thumbnails/22.jpg)
ConclusionAll datasets available from CBMM.MIT.EDU
● We advocate an algorithm development approach combining both synthetic and natural, unconstrained data.● Even if you will ultimately be working within a DAR pipeline, recognition systems that can handle transformation invariance is useful for recovering from errors of alignment. No single point of failure.● More explicit connections with neuroscience and other parts of computer vision.● The next paper we wrote after this one was entitled: Can a biologically-plausible hierarchy effectively replace face detection, alignment, recognition pipelines? (available on arxiv)