UNIVERSITI PUTRA MALAYSIA

LAILI HAYATI BINTI ANUAR

FK 2011 81

3D FACE REGISTRATION ACROSS POSE VARIATION AND FACIAL EXPRESSION USING CROSS PROFILE ALIGNMENT

© COPYRIG

HT UPM

3D FACE REGISTRATION ACROSS POSE VARIATION

AND FACIAL EXPRESSION USING CROSS PROFILE ALIGNMENT

By

LAILI HAYATI BINTI ANUAR

Thesis Submitted to the School of Graduate Studies, Universiti Putra Malaysia, in Fulfilment of the Requirements for the Degree of Master of Science

JUNE 2011

© COPYRIG

HT UPM

Thesis abstract presented to the Senate of Universiti Putra Malaysia in fulfilment of the requirement for the degree of Master of Science

3D FACE REGISTRATION ACROSS POSE VARIATION AND FACIAL EXPRESSION USING CROSS PROFILE ALIGNMENT

By

LAILI HAYATI BINTI ANUAR

June 2011

Chair : Syamsiah binti Mashohor, PhD

Faculty: Faculty of Engineering

In a 3D face recognition system, face registration is usually employed to compensate

the pose variation in a 3D face model. Most previous methods in 3D face

registration are based on the well known global-based approach, Iterative Closest

Point (ICP). The experiments are usually conducted using cleaned and frontal-

viewed face models, neglecting the facial variation that often occur in real-time

scenarios, such as pose variation, facial expression, facial outliers and occlusion.

The proposed thesis uses a local-based approach known as Cross Profile Alignment

(CPA) as an alternative to the global-based approach, utilizing the facial feature of a

face surface as an attempt to cater all the above problems.

Among all features on a face surface, nose tip is the most commonly used feature for

facial feature landmarking. It is crucial to accurately detect the nose tip as it will

affect the overall performance of the registration process. Most of the presented nose

tip detection algorithms were developed merely based on the assumption that the

nose tip is the highest point on a face, which is not robust enough for face model

© COPYRIG

HT UPM

under large rotation variation and having large facial outliers. Thus, as the first step

prior face registration, the thesis proposed a novel nose tip region detection algorithm

using localized point signature, developed specially to locate the nose tip region

across various facial variation.

The experiment conducted on challenging 3D face databases yields good results with

94.77% detection rate for the nose tip region detection algorithm. Based on the nose

tip region location, a cross-profile is extracted and face model is compensated for

rotation variation and translation displacement. The registration framework with

CPA which gained accuracy rate of 93.9% when tested within 10 degrees error

margin, outperforms the registration framework with ICP using Average Face Model

(AFM) with accuracy rate of 87.7%, with lower processing time. The findings

during this work indicate the accuracy and the reliability of the proposed registration

framework towards 3D face model with challenging facial variation.

© COPYRIG

HT UPM

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagai memenuhi keperluan untuk ijazah Master Sains

PENDAFTARAN WAJAH 3D MERENTASI VARIASI GAYA DAN EKSPRESI WAJAH MENGGUNAKAN PENJAJARAN PROFIL

BERSILANG

Oleh

LAILI HAYATI BINTI ANUAR

Jun 2011

Pengerusi: Syamsiah binti Mashohor, PhD

Fakulti: Fakulti Kejuruteraan

Di dalam sistem pengecaman wajah 3D, pendaftaran wajah lazimnya dilaksanakan

untuk mengatasi masalah variasi gaya pada model wajah 3D. Kebanyakan kaedah

pendaftaran wajah 3D yang terdahulu adalah berdasarkan kepada kaedah berasaskan

global yang diketahui umum, Iterative Closest Point (ICP). Eksperimen yang telah

dilaksanakan lazimnya menggunakan model wajah yang telah dibersihkan dan

menghadap posisi hadapan, tanpa mengambil kira variasi wajah yang selalu wujud

dalam persekitaran realiti-masa seperti variasi gaya, ekspresi wajah, lebihan dan

penambahan pada wajah. Tesis ini mencadangkan penggunakan kaedah berasaskan

tempatan yang dikenali sebagai Penjajaran Profil Bersilang (CPA) sebagai alternatif

kepada kaedah berasaskan sejagat, dengan menggunakan ciri-ciri pada permukaan

wajah sebagai percubaan untuk mengatasi semua masalah di atas.

Di antara semua ciri-ciri yang terdapat pada permukaan wajah, puncak hidung

didapati paling banyak digunakan untuk tujuan penandaan wajah. Adalah sangat

© COPYRIG

HT UPM

penting untuk mengesan puncak hidung dengan tepat kerana ia akan mempengaruhi

prestasi proses pendaftaran secara keseluruhannya. Kebanyakan algoritma

pengesanan puncak hidung yang telah dikemukakan sebelum ini adalah semata-mata

berdasarkan anggapan bahawa puncak hidung merupakan titik paling tinggi pada

wajah, di mana kurang tepat bagi model wajah yang mempunyai putaran yang besar

dan unsur asing yang banyak. Oleh sebab itu, sebagai langkah pertama sebelum

pendaftaran, tesis ini mencadangkan algoritma baru pengesanan bahagian puncak

hidung dengan menggunakan point signature yang telah diubahsuai, dibangunkan

khas untuk mengesan bahagian puncak hidung dalam pelbagai variasi wajah.

Eksperimen yang telah dijalankan ke atas pangkalan data wajah 3D yang mencabar

telah menghasilkan keputusan yang baik dengan kadar pengesanan 94.77% untuk

algoritma pengesanan bahagian puncak hidung. Berdasarkan lokasi puncak hidung

ini, profil bersilang boleh diekstrak dan model wajah dibetulkan dari segi putaran dan

sesaran peralihan. Rangka kerja pendaftaran menggunakan CPA memperolehi kadar

ketepatan 93.9% apabila diuji dengan jidar ralat 10 darjah, mengatasi rangka kerja

pendaftaran menggunakan ICP dengan Average Face Model (AFM) dengan kadar

ketepatan 87.7%, dalam masa pemprosesan yang lebih rendah. Penemuan-penemuan

sepanjang kerja ini menunjukkan ketepatan dan keboleh percayaan rangka kerja

pendaftaran yang dicadangkan ke atas model wajah 3D yang mengandungi variasi

wajah yang mencabar.

© COPYRIG

HT UPM

ACKNOWLEDGEMENTS

First of all, all praises to the supreme almighty Allah s.w.t., whom without His

blessing, this thesis will not come true.

I would like to take this opportunity to say many thanks to those who directly and

indirectly supported me throughout my study. My utmost gratitude goes to my

supervisor, Dr. Syamsiah Mashohor and my co-supervisor, Dr. Makhfudzah Mokhtar

for their helpful guidance and efforts in making this thesis a reality. Many thanks

also go to Dr. Wan Azizun Wan Adnan for the opportunity that she had given me and

for supporting me with her research funds. I also would like to express my

appreciation to the University Putra Malaysia for the financial support via the

Graduate Student Fellowship (GRF) programme. Special thanks to the Institute of

Advance Technology (ITMA) for letting me using the 3D laser scanner, and for the

helpful technicians during the acquisition of 3D face models for UPMFace face

database.

Not forgotten, to my parents and my family members for believing in me and for

their undeniable support in my easy and tough days. To my lab mates, for their help,

presence, love and friendship. Lastly, my deepest gratitude to my beloved husband,

M. Iqbal Saripan, who is always by my side through my ups and downs, and my

wonderful kids who always inspire me in their special way. And also for those who I

do not mention here, thank you very much for everything.

This work is partly funded by research grant Science Fund Malaysia (5450083). I

also would like to acknowledge the use of GavaDB face database by Moreno and

Sánchez.

© COPYRIG

HT UPM

I certify that a Thesis Examination Committee has met on 6th June 2011 to conduct the final examination of Laili Hayati binti Anuar on her thesis entitled "3D Face Registration Across Pose Variation and Facial Expression using Cross Profile Alignment" in accordance with the Universities and University Colleges Act 1971 and the Constitution of the Universiti Putra Malaysia [P.U.(A) 106] 15 March 1998. The Committee recommends that the student be awarded the Master of Science.

Members of the Thesis Examination Committee were as follows:

Abdul Rahman bin Ramli, PhD Associate Professor Faculty of Engineering Universiti Putra Malaysia (Chairman)

Mohammad Hamiruce bin Marhaban, PhD Associate Professor Faculty of Engineering Universiti Putra Malaysia (Internal Examiner)

Rahmita Wirza binti O.K Rahmat, PhD Associate Professor Faculty of Computer Science and Information Technology Universiti Putra Malaysia (Internal Examiner)

Salina binti Abdul Samad, PhD Professor Faculty of Engineering Universiti Kebangsaan Malaysia Malaysia (External Examiner)

NORITAH OMAR, PhD Associate Professor and Deputy Dean School of Graduate Studies Universiti Putra Malaysia

Date:

© COPYRIG

HT UPM

This thesis was submitted to the Senate of Universiti Putra Malaysia and has been

accepted as fulfilment of the requirement for the degree of Master of Science. The

members of the Supervisory Committee were as follows:

Syamsiah binti Mashohor, PhD Lecturer Department of Computer and Communication Systems Engineering Faculty of Engineering Universiti Putra Malaysia (Chairman)

Makhfudzah binti Mokhtar, PhD Lecturer Department of Computer and Communication Systems Engineering Faculty of Engineering Universiti Putra Malaysia (Member)

________________________________ HASANAH MOHD. GHAZALI, PhD Professor and Dean School of Graduate Studies Universiti Putra Malaysia Date :   

© COPYRIG

HT UPM

DECLARATION

I declare that the thesis is my original work except for equations and citations which

has been duly acknowledged. I also declare that it has not been previously, and is not

concurrently, submitted for any other degree at Universiti Putra Malaysia or at any

other institution.

____________________________

LAILI HAYATI BINTI ANUAR

Date :

© COPYRIG

HT UPM

TABLE OF CONTENTS

Page ABSTRACT ii ABSTRAK iv ACKNOWLEDGEMENTS vi APPROVAL vii DECLARATION ix LIST OF TABLES xii LIST OF FIGURES xiii LIST OF ABBREVIATIONS xvi LIST OF SYMBOLS xvii CHAPTER

1 INTRODUCTION 1 1.1 Face for Biometrics 1 1.2 Introduction to Face Recognition 2 1.3 Introduction to 3D Face Registration 4 1.4 Motivations 5 1.5 Aim and Objective 6 1.6 Scope of Work 7 1.7 Thesis Structure 9

2 LITERATURE REVIEW 11 2.1 3D Face Recognition 11 2.1.1 Introduction 11 2.1.2 2D and 3D Face 12 2.1.3 3D Face Recognition System 15 2.1.4 3D Face Representations 18 2.2 3D Face Registration 21 2.2.1 Introduction 21 2.2.2 Global and Local Approaches 22 2.2.3 Iterative Closest Point (ICP) 24 2.2.4 Average Face Model (AFM) 26 2.3 Local-based Face Registration Approach 28 2.3.1 Nose Tip Detection for Pose Estimation 29 2.4 Point Signature 34 2.4.1 Introduction 34 2.4.2 Point Signature Implementations 34 2.4.3 Definition of Point Signature 35 2.4.4 Matching of Point Signatures 38 2.5 Summary 39

3 METHODOLOGY 40 3.1 Introduction 40 3.2 Rotation Variations in 3D Face Model 41

© COPYRIG

HT UPM

3.3 3D Face Databases 42 3.3.1 GavaDB 42 3.3.2 UPMFace 45 3.3.3 Experimental Data Sets 48 3.4 Nose Tip Region Detection 48 3.4.1 Convex Point Classification 49 3.4.2 Search for Candidate Regions Using

Morphology Operations 52

3.4.3 Training for Tolerance Band 55 3.4.4 Localized Point Signature for Nose Tip Region

Detection 56

3.5 Coarse Alignment 62 3.6 Registration with ICP using AFM 64 3.6.1 Construction of an AFM 65 3.6.2 3D Face Registration with ICP using AFM 68 3.7 Pose Estimation and Registration with CPA 71 3.7.1 Detecting the Centre of the Nose Tip Region 72 3.7.2 Cross Profile Alignment (CPA) for Fine Alignment 73 3.7.3 Translation Displacement Calculation 75 3.8 Ground Truth Annotation 76 3.9 Transformation Function for Registration 77 3.10 Performance Analysis 79 3.11 Summary 80

4 RESULTS AND DISCUSSION 81 4.1 Introduction 81 4.2 Experimental Setup 81 4.3 Training for Tolerance Band 85 4.4 Experimental Works 86 4.4.1 Nose Tip Region Detection 87 4.4.2 Registration with CPA 94 4.4.3 Registration with ICP using AFM 95 4.5 Performance Analysis 96 4.6 Summary 102

5 CONCLUSIONS AND FUTURE WORKS 103 5.1 Conclusions 103 5.2 Future Works 105

REFERENCES 107 APPENDICES 113 BIODATA OF STUDENT 115 LIST OF PUBLICATIONS 116