Detecting Faces in Images: A Survey - CS Course...
Transcript of Detecting Faces in Images: A Survey - CS Course...
![Page 1: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/1.jpg)
Detecting Faces in Images: A Survey
By: Ming-Hsuan Yang, David J. Kriegman, and Narendra Ahuja
Presented By: Neal Audenaert
![Page 2: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/2.jpg)
Agenda
IntroductionApproaches
Knowledge-basedFeature invariantTemplate matchingAppearnce-based
Databases and EvaluationDiscussion
![Page 3: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/3.jpg)
Agenda
IntroductionApproaches
Knowledge-basedFeature invariantTemplate matchingAppearnce-based
Databases and EvaluationDiscussion
![Page 4: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/4.jpg)
Introduction
DomainFace detection (not recognition)Still images
ObjectivesComprehensive survey of techniquesDiscussion of performance measures
LimitationsMethods are not directly comparable
![Page 5: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/5.jpg)
Challenges
PoseStructural componentsFacial expressionOcclusionImage orientationImaging conditions
![Page 6: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/6.jpg)
General Tasks
Face localizationFacial feature detectionFace recognitionFace authenticationFace trackingFacial expression recognition
![Page 7: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/7.jpg)
Agenda
IntroductionApproaches
Knowledge-basedFeature invariantTemplate matchingAppearnce-based
Databases and EvaluationDiscussion
![Page 8: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/8.jpg)
Survey of Techniques
Knowledge BasedTop-downBottom-up
Template BasedDefined templatesLearned templates
Knowledge-basedFeature invariant
Template matchingAppearance-based
![Page 9: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/9.jpg)
Survey of Techniques
XAppearance-based
XXTemplate Matching
XFeature Invariant
XKnowledge-basedDet.Loc.Approach
Knowledge-based Feature Invariant Template Matching Appearance-based
![Page 10: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/10.jpg)
Knowledge-Based Top-Down Methods
Main Idea: Use knowledge about what constitutes a face faces to define rules
Strengths: Frontal faces in uncluttered scenes
Weaknesses:Translating knowledge into rulesEnumeration of cases
Knowledge-based Feature Invariant Template Matching Appearance-based
![Page 11: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/11.jpg)
Knowledge-Based Top-Down Methods
![Page 12: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/12.jpg)
Bottom-Up Feature-Based Methods
Main Idea: Describe relationships between invariant features using statistical models
Strengths: Improved invariance for different poses and lighting conditions
Weaknesses: Corruption of individual due to illumination, noise, or occlusion
Knowledge-based Feature Invariant Template Matching Appearance-based
![Page 13: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/13.jpg)
Bottom-Up Feature-Based Methods
Facial FeaturesTextureSkin ColorMultiple Features
Knowledge-based Feature Invariant Template Matching Appearance-based
![Page 14: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/14.jpg)
Template Matching
Main Idea: Find correlation values with a standard face pattern for face contour, eyes, nose, and mouth
Strengths: Simple to implement
Weaknesses: Cannot deal with variation in scale, pose, and shape
Alternatives: Multiresolution, multiscale, subtemplates, and deformable templates
Knowledge-based Feature Invariant Template Matching Appearance-based
![Page 15: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/15.jpg)
Appearance-Based Methods
Main Idea: Use statistical analysis and machine learning techniques to learn “template” characteristics
Strengths: Most successful approach
Weaknesses: Relatively complex to implement, high-dimensionality requires many training examples
Knowledge-based Feature Invariant Template Matching Appearance-based
![Page 16: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/16.jpg)
Overview of Techniques
EigenfacesDistribution-Based MethodsNeural Networks (ANN)Support Vector Machines (SVM)Sparse Network of Winnows (SNoW)Naïve Bayes ClassifierHidden Markov Models (HMM)Information Theoretic ApproachesInductive Learning
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 17: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/17.jpg)
Eigenfaces
Pedro?
Main Idea: Calculate distance between an instance and exemplary data in a reduced dimensional space
Build a face map
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 18: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/18.jpg)
Distribution-Based Methods
Fit a distribution model to examples
Project example into reduced dimensional space
Build classifier to decide face/non-face
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 19: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/19.jpg)
Sung and Poggio
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 20: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/20.jpg)
Sung and Poggio
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 21: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/21.jpg)
Sung and Poggio
MahalanobisPCA for each cluster
Representative sample of non-face images?
Bootstrap approach
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 22: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/22.jpg)
Yang, Ahuja, Kriegman
Method 1:Factor Analysis
Instead of PCADoes not define a mixture model
Estimate mixture model using EMMethod 2:
Fisher’s Linear DiscriminantClass decomposistion using Kohonen’s Self Organizing MapsML decision rule to detect faces
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 23: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/23.jpg)
Neural Networks
Two class pattern recognition Advantage: capture complex class conditional desnsityDrawback: Requires extensively tuning
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 24: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/24.jpg)
Support Vector Machines
Estimating hyperplane is expensiveEvaluation is fast
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 25: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/25.jpg)
Sparse Network of Winnows
Detect images with:Different features and expressionsDifferent posesDifferent lighting conditions
Primitive and multiscale featuresTailored for domains where
Number of features is largeFeatures unknown a priori
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 26: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/26.jpg)
Naïve Bayes Classifier
Estimate joint probability of local appearance
c.f. bottom-up methods
Emphasize local appearanceSome patterns are more unique
Detects some ratated and profile faces
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 27: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/27.jpg)
Hidden Markov Model
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 28: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/28.jpg)
Hidden Markov Model
AlternativesKarhunen Loeve Traformcoefficients as input to HMMUse HMM to learn face to non-face transition
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 29: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/29.jpg)
Information Theoretic Approaches
Markov Random Fields (MRF)Model context-dependent entities
Kullback relative inforamtionMaximize information-based discriminant between the two classes
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. TheoryInductive
![Page 30: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/30.jpg)
Inductive Learning
C4.5 AlgorithmBuilds a decision tree8x8 pixel window
Represented as 30 value vectorEntropy, mean, std dev. of pixel value
Find-SGaussian clusters to aproximatedistribution of face patternsFind-S to learn thresholding
Knowledge-based Feature Invariant Template Matching Appearance-based
EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive
![Page 31: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/31.jpg)
Agenda
IntroductionApproaches
Knowledge-basedFeature invariantTemplate matchingAppearnce-based
Databases and EvaluationDiscussion
![Page 32: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/32.jpg)
AT&T Cambridge Laboratories Face Database
![Page 33: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/33.jpg)
Face Image Databases
![Page 34: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/34.jpg)
Features of Databases
Designed for single studySmallHighly constrainedOriented to face recognition
![Page 35: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/35.jpg)
Benchmark Test Sets
![Page 36: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/36.jpg)
Benchmark Test Sets
![Page 37: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/37.jpg)
Performance Evaluation
![Page 38: Detecting Faces in Images: A Survey - CS Course Webpagescourses.cs.tamu.edu/rgutier/cpsc689_s07/yang2002faceDetection... · Detecting Faces in Images: A Survey By: Ming-Hsuan Yang,](https://reader033.fdocuments.us/reader033/viewer/2022052917/5a9f1f397f8b9a89178c5b3b/html5/thumbnails/38.jpg)
Agenda
IntroductionApproaches
Knowledge-basedFeature invariantTemplate matchingAppearnce-based
Databases and EvaluationDiscussion