ECE738 Advanced Image Processing

Face Sketch Recognition

[Tang04] IEEE Trans. CSVT, Jan 2004

(C) 2005 by Yu Hen Hu 2ECE738 Advanced Image Processing

Problem Statement

• Goal:– Match face images to

face sketches– Useful when test faces

are in sketch forms (e.g. no photo of suspect).

• Approach– Transform face images

to sketches– Match sketches of

faces using an eigen-sketch approach

(C) 2005 by Yu Hen Hu 3ECE738 Advanced Image Processing

Geometric Methods

• Fiducial grid model– Identify anchor points tip of

noise, corners of mouth– Derive other fiducial points

from the identified anchor point

• Geometric features– 35 fiducial points used– 26 distances between key

fiducial points measured.– Including sizes of nose, eyes,

eyebrows, face contours, etc.• Measure geometric features

from both face image and sketches and perform matching

(C) 2005 by Yu Hen Hu 4ECE738 Advanced Image Processing

Eigen-Sketch Method

• Convert face image into eigenface representation.• Generate eigen-sketch from each eigenface.• Project probe sketches (test sketches) onto the

subspace spanned by eigen-sketches. • Use feature vectors of eigen-face representation to

match the weight vectors • Assumption:

– Face images and sketches are normalized in size, lighting, poses, and expression neutral

– No occlusions on test images (eye glasses, beard, etc.)

(C) 2005 by Yu Hen Hu 5ECE738 Advanced Image Processing

Photo-to-sketch transformation

Photo-to-sketch transformation examples. (a) Original photo. (b) Reconstructed

photo. (c) Reconstructed sketch. (d) Original sketch.

Photo – eigen-face – eigen-sketch - sketch

(C) 2005 by Yu Hen Hu 6ECE738 Advanced Image Processing

Results

• Using Feret test set, the proposed method (sketch transform) out-performs conventional method (geometric and eigen-face) by large margin.

• Possible reason: – Geometric face: features measured on photo may be different from

sketches– Eigenface: gray scale image value differ very much between phton

and sketch.

Rank 1 2 3 4 5 6 7 8 9 10

Geometric 30 37 45 48 53 59 62 66 67 70

Eigenface 31 43 48 55 61 63 65 65 67 67

Sketch transform 71 78 81 84 88 90 94 94 95 96