Automated Face Detection
description
Transcript of Automated Face Detection
![Page 1: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/1.jpg)
Automated Face Automated Face DetectionDetection
Peter BrendePeter Brende
David Black-SchafferDavid Black-Schaffer
Veni BourakovVeni Bourakov
![Page 2: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/2.jpg)
Primary ChallengesPrimary Challenges
Scale differences Scale differences Overlapping/obstructed facesOverlapping/obstructed faces Lighting variationLighting variation
![Page 3: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/3.jpg)
Implementation OverviewImplementation Overview
1.1. Color-based skin separationColor-based skin separation
2.2. Spatial analysis to generate candidate facesSpatial analysis to generate candidate faces Morphological?Morphological? or Template Matching?or Template Matching?
3.3. Eliminate false hitsEliminate false hits Hands & arms, based on shapeHands & arms, based on shape Roof, based on textureRoof, based on texture Neck, based on relative positionNeck, based on relative position
![Page 4: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/4.jpg)
Color-based Skin SeparationColor-based Skin Separation
What color space to use?What color space to use?
How to separate out the skin color?How to separate out the skin color?
![Page 5: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/5.jpg)
Marginal Color DistributionsMarginal Color Distributions
![Page 6: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/6.jpg)
Parametric SeparationParametric Separation
Simple & Fast (h>0.98 or h<0.01)Simple & Fast (h>0.98 or h<0.01) Problems with non-linearity of HSV space in bright areasProblems with non-linearity of HSV space in bright areas
![Page 7: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/7.jpg)
Full Joint-Probability DistributionFull Joint-Probability Distribution
We have enough data, so why not?We have enough data, so why not?
Provides most accurate per-pixel Provides most accurate per-pixel classification. classification.
Allows use to circumvent choosing a decision Allows use to circumvent choosing a decision boundary. We can simply use Bayes rule.boundary. We can simply use Bayes rule.
![Page 8: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/8.jpg)
Slices from 3D Joint DistributionSlices from 3D Joint Distribution
![Page 9: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/9.jpg)
Skin-probability Image Obtained Skin-probability Image Obtained From Applying Bayes RuleFrom Applying Bayes Rule
![Page 10: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/10.jpg)
Color-based Skin SeparationColor-based Skin Separation
What color space to use?What color space to use? HSV if separability of distributions is necessaryHSV if separability of distributions is necessary
How to separate out the skin color?How to separate out the skin color? Parametric is fast but loose in HSVParametric is fast but loose in HSV
Provides a binary mapping and requires choosing Provides a binary mapping and requires choosing thresholdsthresholds
Full PDF is accurate in any color spaceFull PDF is accurate in any color space Can be fast if done correctly (table lookup)Can be fast if done correctly (table lookup) No thresholds: produces a pure probability mapNo thresholds: produces a pure probability map
![Page 11: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/11.jpg)
Spatial Analysis MethodSpatial Analysis Method
MorphologicalMorphological Obtain binary mask through thresholdingObtain binary mask through thresholding Perform morphological operations to separate and Perform morphological operations to separate and
identify blobs corresponding to facesidentify blobs corresponding to faces Difficult due to overlapping facesDifficult due to overlapping faces
Template MatchTemplate Match Search a scene for prototypical face imageSearch a scene for prototypical face image Need to decide which data to work with Need to decide which data to work with
(luminance vs. skin probability)(luminance vs. skin probability)
![Page 12: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/12.jpg)
Template MatchingTemplate Matching
Using the skin-probability image:Using the skin-probability image: Greatly simplifies information contentGreatly simplifies information content
Simple information Simple information simple algorithm simple algorithm
Allows algorithm to focus on the single best facial Allows algorithm to focus on the single best facial clue: oval-shaped clue: oval-shaped skinskin regions regions
Allows us to avoid creating a binary maskAllows us to avoid creating a binary mask
![Page 13: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/13.jpg)
Correlation of simple template Correlation of simple template with Skin-probability imagewith Skin-probability image
![Page 14: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/14.jpg)
Process of Inclusion/EliminationProcess of Inclusion/Elimination
Iteratively pick ‘strong’ regions of the skin-Iteratively pick ‘strong’ regions of the skin-probability image as faces:probability image as faces: For each template search for matching face shapes For each template search for matching face shapes
(convolution peaks)(convolution peaks) For each detected face, ‘subtract/erase’ the region For each detected face, ‘subtract/erase’ the region
from image to avoid duplicate detectionfrom image to avoid duplicate detection Stop when no significant skin regions remainingStop when no significant skin regions remaining
![Page 15: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/15.jpg)
Positive Detection and Positive Detection and EliminationElimination
![Page 16: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/16.jpg)
![Page 17: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/17.jpg)
![Page 18: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/18.jpg)
![Page 19: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/19.jpg)
![Page 20: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/20.jpg)
![Page 21: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/21.jpg)
![Page 22: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/22.jpg)
![Page 23: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/23.jpg)
![Page 24: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/24.jpg)
Template and Threshold Template and Threshold SelectionSelection
Critical step of our algorithmCritical step of our algorithm Potential problems:Potential problems:
Template matching face of a different sizeTemplate matching face of a different size Small template leads to double hits laterSmall template leads to double hits later Large template leads to missed facesLarge template leads to missed faces
Bad thresholds Bad thresholds low sensitivity or low specificity low sensitivity or low specificity Solution:Solution:
Use templates of many sizes, going from largest to smallestUse templates of many sizes, going from largest to smallest Set threshold as high as possible without sacrificing Set threshold as high as possible without sacrificing
sensitivitysensitivity
![Page 25: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/25.jpg)
Algorithm ImplementationAlgorithm Implementation
1.1. Load probability and template data Load probability and template data2.2. Down-sample the image by factor 2:1 Down-sample the image by factor 2:13.3. Calculate the face-probability image by color Calculate the face-probability image by color4.4. Remove hands/arms Remove hands/arms5.5. Template match with skin-probability image Template match with skin-probability image6.6. Eliminate false positive hits on necks of large Eliminate false positive hits on necks of large
facesfaces7.7. Remove patterned hits Remove patterned hits
![Page 26: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/26.jpg)
Overall ResultsOverall ResultsImage 1 2 3 4 5 6 7 Total hits 21 23 25 24 23 23 22 161 misses 0 1 0 0 1 0 0 2 2nd hits 0 1 0 0 0 0 1 2 faces 21 24 25 24 24 24 22 164 Image 1 2 3 4 5 6 7 Total %hits 100% 96% 100% 100% 96% 96% 100% 98% %misses 0% 4% 0% 0% 4% 0% 0% 1% %2nd hits 0% 4% 0% 0% 0% 0% 5% 1% Image 1 2 3 4 5 6 7 Total hits 23 6 13 5 26 24 5 102 misses 3 17 10 17 1 2 17 67 2nd hits 2 0 0 0 0 2 0 4 faces 25 23 23 22 27 26 23 169 Image 1 2 3 4 5 6 7 Total %hits 92% 26% 57% 23% 96% 92% 22% 60% %misses 12% 74% 43% 77% 4% 8% 74% 40% %2ndhits 8% 0% 0% 0% 0% 8% 0% 2%
![Page 27: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/27.jpg)
ConclusionsConclusions
Recognition of face-shaped blobs from the skin-Recognition of face-shaped blobs from the skin-probability map works excellentlyprobability map works excellently Requires that the skin colors be well knownRequires that the skin colors be well known Requires that the general face sizes be well knownRequires that the general face sizes be well known
Our set of images was relatively consistent in terms Our set of images was relatively consistent in terms of these factorsof these factors
![Page 28: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/28.jpg)
![Page 29: Automated Face Detection](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815735550346895dc4d5a7/html5/thumbnails/29.jpg)
17
16
15
14
5 (0)4 (19)13
5 (0)4 (19)12
5 (0)13 (08)11
2 (1)2 (20)10
2 (1)1 (22)9
1 (2)11 (14)8
5 (0)10 (16)7
5 (0)9 (17)6
5 (0)12 (13)5
5 (0)7 (18)4
5 (0)7 (18)3
5 (0)2 (20)2
2 (1)4 (19)1
Gender RecognitionFace Detection
17
16
15
14
5 (0)4 (19)13
5 (0)4 (19)12
5 (0)13 (08)11
2 (1)2 (20)10
2 (1)1 (22)9
1 (2)11 (14)8
5 (0)10 (16)7
5 (0)9 (17)6
5 (0)12 (13)5
5 (0)7 (18)4
5 (0)7 (18)3
5 (0)2 (20)2
2 (1)4 (19)1
Gender RecognitionFace Detection