Facial Feature Extraction Yuri Vanzine C490/B657 Computer Vision.
Transcript of Facial Feature Extraction Yuri Vanzine C490/B657 Computer Vision.
Potentially Bigger Project
Future Goal: Application Component
• Receive Frontal and Profile Images of User’s Head
• Localize face, locate position of eyes, mouth to determine face geometry
• Use face geometry to locate secondary features, eg. nose, facial hair, ears.
• Extract face geometry and features and transform them into quantative feature vectors.
• Apply features and geometry to 3-D model in a standard 3-D model format, eg. 3DS, OpenNURBS, DXF, DWG, etc.
Need: Provide Ability to import Avatar appearance from web cam
Application needs to perform the following:
Part of the Project for C490/B657
Face Localization, Locating Eyes and Mouth
• Face Localization
• Mouth Location
• Left and Right Eye Location
Process phases:
Face Localization
Existing Techniques:
• RGB, YCbCr, HSV Late Fusion• Several weak, fast classifiers instead of ‘expensive’ ones• Environment is controlled: sufficient lighting, head position, subject is told to place
head a certain way, but our classifier is rotation, scale and translation invariant!
• Skin-tone techniques in various colorspaces
• Template-based
• Eigen-faces
• Other
Technique used for Face Detection:
RGB Colorspace
Matlab Code:
for xc=1:x for yc = 1:y iR = (int)R(xc,yc); iG = (int)G(xc,yc); iB = B(xc,yc); mxRGB = max(max(iR, iG), iB); mnRGB = min(min(iR, iG), iB); if iR>95 && iG>40 && iB>20 && mxRGB-mnRGB>15 && abs(iR-iG) > 15 && iR > iG && iR > iB else F(xc,yc,1)=0; F(xc,yc,2)=0; F(xc,yc,3)=0; end endend
HSV Colorspace
Bounding planes, defining skin-color, determined in [3]:
Matlab Code:for xc=1:x for yc = 1:y iH = double(H(xc,yc))*255; iS = double(S(xc,yc))*255; iV = double(V(xc,yc))*255; if iH>=0 && iS>=15 && iS>=(0.75*iH+0.3*iV-30) || iS<=-iH-0.1*iV+110 && iH<=-0.4*iV+75 && iS<=0.08 * (100-iV) *iH +0.6* iV else F(xc,yc,1)=0; F(xc,yc,2)=0; F(xc,yc,3)=0; end endend
Late Fusion of Colorspaces
RGB
HSV
YCbCr To be implemented…
Bounding Box
RGB and HSV in this implementation is luminance-dependent. Frequent false positives are likely.This implementation should be replaced by YCbCr from [4].
Eye Location (YCbCr)
Complexity Map
5 6 5
7 150 6
7 150 6
Average of Differencesbetween pixels in the 8-neighborhood,followed by dilation using a disk of 5 pixels
Eye Location (YCbCr)
Eye Map and Complexity Map Combined
AND
Notice problems because bounding box needs to be replaced by Connected Component Mask
AND
Current Problems and Future Work
• Problem: Current skin-tone clustering is not robust.• Solution: YCbCr luma-independent detection needs to be implemented.
• Problem: Manual thresholding is implemented. • Solution: Automatic thresholding is desired.
• Problem: Bounding Box instead of Skin-Tone Connected component• Solution: Improve algorithm extracting just face-mask
YCbCr Colorspace (continued)
This model alone is sufficient. Taken from [4] (to be implemented in the future). YCbCr is luminance-dependent. Direct use of YCbCr results in false positives. Therefore YCbCr must be made luma-independent.
References
1. A Survey on Pixel-Based Skin Color Detection Techniques, Vladimir Vezhnevets, Vassili Sazonov, Alla Andreeva, http://www.graphicon.ru/2003/Proceedings/Technical/paper509.pdf
2. Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, A Robust Skin Color Based Face Detection Algorithm, http://www2.tku.edu.tw/~tkjse/6-4/6-4-6.pdf
3. Face Detection in Color Images using Wavelet Packet Analysis, C. Garcia, G. Zikos, G. Tziritas, http://www.csd.uch.gr/~tziritas/papers/ICMCS-99.pdf
4. Face Detection in Color Images, Rein-Lien Hsu, Mohamed Abdel-Mottaleb, Anil K. Jain, http://www.csd.uch.gr/~hy471/papers/FaceDetectionColor.pdf