Facial Feature Extraction Yuri Vanzine C490/B657 Computer Vision.

22
Facial Feature Extraction Yuri Vanzine C490/B657 Computer Vision

Transcript of Facial Feature Extraction Yuri Vanzine C490/B657 Computer Vision.

Facial Feature Extraction

Yuri Vanzine C490/B657

Computer Vision

Potentially Bigger Project

Future Goal: Application Component

• Receive Frontal and Profile Images of User’s Head

• Localize face, locate position of eyes, mouth to determine face geometry

• Use face geometry to locate secondary features, eg. nose, facial hair, ears.

• Extract face geometry and features and transform them into quantative feature vectors.

• Apply features and geometry to 3-D model in a standard 3-D model format, eg. 3DS, OpenNURBS, DXF, DWG, etc.

Need: Provide Ability to import Avatar appearance from web cam

Application needs to perform the following:

Part of the Project for C490/B657

Face Localization, Locating Eyes and Mouth

• Face Localization

• Mouth Location

• Left and Right Eye Location

Process phases:

Face Localization

Existing Techniques:

• RGB, YCbCr, HSV Late Fusion• Several weak, fast classifiers instead of ‘expensive’ ones• Environment is controlled: sufficient lighting, head position, subject is told to place

head a certain way, but our classifier is rotation, scale and translation invariant!

• Skin-tone techniques in various colorspaces

• Template-based

• Eigen-faces

• Other

Technique used for Face Detection:

RGB, YCbCr, HSV Late Fusion

RGB Colorspace

Matlab Code:

for xc=1:x for yc = 1:y iR = (int)R(xc,yc); iG = (int)G(xc,yc); iB = B(xc,yc); mxRGB = max(max(iR, iG), iB); mnRGB = min(min(iR, iG), iB); if iR>95 && iG>40 && iB>20 && mxRGB-mnRGB>15 && abs(iR-iG) > 15 && iR > iG && iR > iB else F(xc,yc,1)=0; F(xc,yc,2)=0; F(xc,yc,3)=0; end endend

HSV Colorspace

Bounding planes, defining skin-color, determined in [3]:

Matlab Code:for xc=1:x for yc = 1:y iH = double(H(xc,yc))*255; iS = double(S(xc,yc))*255; iV = double(V(xc,yc))*255; if iH>=0 && iS>=15 && iS>=(0.75*iH+0.3*iV-30) || iS<=-iH-0.1*iV+110 && iH<=-0.4*iV+75 && iS<=0.08 * (100-iV) *iH +0.6* iV else F(xc,yc,1)=0; F(xc,yc,2)=0; F(xc,yc,3)=0; end endend

YCbCr Colorspace

“YCbCr simple” from [3] (could not get it to work):

Late Fusion of Colorspaces

RGB

HSV

YCbCr To be implemented…

Bounding Box

RGB and HSV in this implementation is luminance-dependent. Frequent false positives are likely.This implementation should be replaced by YCbCr from [4].

More examples of Bounded Images

Mouth Location (YCbCr)

See [4] for further detail.

Mouth Location (continued)

Mouth MaskConvex Face Bounding Box

Eye Location (YCbCr)

Eye Map

Eye Location (YCbCr)

Complexity Map

5 6 5

7 150 6

7 150 6

Average of Differencesbetween pixels in the 8-neighborhood,followed by dilation using a disk of 5 pixels

Eye Location (YCbCr)

Eye Map and Complexity Map Combined

AND

Notice problems because bounding box needs to be replaced by Connected Component Mask

AND

Matlab Demo of Bounding Box, EyeMap and MouthMap

Current Problems and Future Work

• Problem: Current skin-tone clustering is not robust.• Solution: YCbCr luma-independent detection needs to be implemented.

• Problem: Manual thresholding is implemented. • Solution: Automatic thresholding is desired.

• Problem: Bounding Box instead of Skin-Tone Connected component• Solution: Improve algorithm extracting just face-mask

YCbCr Colorspace (continued)

YCbCr Colorspace (continued)

This model alone is sufficient. Taken from [4] (to be implemented in the future). YCbCr is luminance-dependent. Direct use of YCbCr results in false positives. Therefore YCbCr must be made luma-independent.

YCbCr Colorspace (continued)

YCbCr Colorspace (continued)

References

1. A Survey on Pixel-Based Skin Color Detection Techniques, Vladimir Vezhnevets, Vassili Sazonov, Alla Andreeva, http://www.graphicon.ru/2003/Proceedings/Technical/paper509.pdf

2. Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, A Robust Skin Color Based Face Detection Algorithm, http://www2.tku.edu.tw/~tkjse/6-4/6-4-6.pdf

3. Face Detection in Color Images using Wavelet Packet Analysis, C. Garcia, G. Zikos, G. Tziritas, http://www.csd.uch.gr/~tziritas/papers/ICMCS-99.pdf

4. Face Detection in Color Images, Rein-Lien Hsu, Mohamed Abdel-Mottaleb, Anil K. Jain, http://www.csd.uch.gr/~hy471/papers/FaceDetectionColor.pdf