Face alignment by deep convolutional network with adaptive learning rate
-
Upload
zhiwen-shao -
Category
Presentations & Public Speaking
-
view
27 -
download
0
Transcript of Face alignment by deep convolutional network with adaptive learning rate
![Page 1: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/1.jpg)
Face Alignment by Deep Convolutional Network with Adaptive Learning Rate
Zhiwen Shao, Shouhong Ding, and Lizhuang Ma
Shanghai Jiao Tong University
![Page 2: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/2.jpg)
Applications
Face preprocessing
Face animation 1
Face beautification
1 The two images were downloaded from https://www.mixamo.com/faceplus
![Page 3: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/3.jpg)
ChallengesLarge pose, illumination and expression variationsPartial occlusionLow quality
![Page 4: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/4.jpg)
How to solve these problems
Deep convolutional neural networka small number of training images annotated with landmarks
We propose an effective data augmentation strategy
![Page 5: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/5.jpg)
Data augmentation
Translation
Rotation
JPEGcompression
improve the robustness of landmark detection in the condition of tiny face shift
adapt pose variation of in-plane rotation
be robust to poor-quality images
Training data are enlarged dozens of times
Also
![Page 6: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/6.jpg)
Deep convolutional network
Eight convolutional layers followed by two fully-connected layers
Convolution operation:
*j ij i j
i
y k x b
![Page 7: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/7.jpg)
Deep convolutional network
Every two continuous convolutional layers connect with a max-pooling layer
Max-pooling operation:
, ,0 ,max { }i i
j k j h m k h nm n hy x
![Page 8: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/8.jpg)
Deep convolutional network
VGG net: multiple continuous convolutional layers, jointly extract complex features
Batch normalization ( )( )
x E xyVar x
ReLU nonlinearity max(0, )y xaccelerate training
![Page 9: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/9.jpg)
Deep convolutional network
The network input is 50×50×3 for color face patches
The output is predicted coordinates of five landmarks
guarantee numerical stabilityreduce computational cost
Shrink the coordinates
Euclidean loss: 21 ( )2
L f f
![Page 10: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/10.jpg)
Initially the value of Euclidean loss may be several hundred or even several thousand
Initial learning rate must be very small
converge slowlysearch the solution in a small range
How?
Adaptive learning rate
Deep convolutional network
![Page 11: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/11.jpg)
Adaptive learning rate
We firstly assign learning rate depended on initial testing loss, to avoid the network link weights changed sharply
![Page 12: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/12.jpg)
Adaptive learning rate
When network loss was reduced significantly, changinglearning rate to be a larger value
![Page 13: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/13.jpg)
Adaptive learning rate
The algorithm consists of two iterative procedures for adaptive learning rate decrease
![Page 14: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/14.jpg)
Experiments
![Page 15: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/15.jpg)
Datasets and evaluation metric
LFPW 1432 face Images, we use 1030 images as same as cascaded CNN [9]
AFLW 25993 face Images, we use 2995 images as same as TCDCN [17]
Datasets
Evaluation metric: mean errorthe distance between estimated landmarks and the ground truths,normalized with the inter-ocular distance,averaged over all landmarks and images
![Page 16: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/16.jpg)
Algorithm discussions
Train our network with adaptive learning rate algorithm
Train our network with only one iterative procedure (the learning rate is gradually reduced from a small value)
VS.
![Page 17: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/17.jpg)
Algorithm discussions
The minimal loss are 0.6823 and 0.7938 respectively
The difference of average distance is approximately20.1115 / ( ) 2 / 5 1.0559, 0.2
![Page 18: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/18.jpg)
Algorithm discussions
Method LFPW AFLWOur algorithm (one iterative procedure) 1.50 8.15
Our algorithm 1.17 7.42
Mean error (%)
![Page 19: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/19.jpg)
Comparison with other methods
Method LFPW AFLWESR [7] 5.36 12.4
RCPR [8] 4.58 11.6SDM [11] 2.26 8.5
cascaded CNN [9] 2.10 8.72TCDCN [17] 1.33 8.0
Our approach 1.17 7.42
![Page 20: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/20.jpg)
Comparison with other methods
Method LFPW AFLWESR [7] 5.36 12.4
RCPR [8] 4.58 11.6SDM [11] 2.26 8.5
cascaded CNN [9] 2.10 8.72TCDCN [17] 1.33 8.0
Our approach (quick) 1.28 7.86Our approach 1.17 7.42
Our approach (quick)
Changing convolutional layers:
2, 2, 2, 2 1, 1, 2, 2
![Page 21: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/21.jpg)
Deep model Time (Intel Core i5 CPU)cascaded CNN [9] 120 ms
TCDCN [17] 17 msOur approach (quick) 39 ms
Our approach 67 ms
Comparison with other methods
![Page 22: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/22.jpg)
More examples
![Page 23: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/23.jpg)
Conclusions
We propose an effective deep convolutional network based on data augmentation and adaptive learning rate
We achieve state-of-the-art performance
The data augmentation and adaptive learning rate can also be applied to other problems like face recognition
![Page 24: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/24.jpg)
Demo
![Page 25: Face alignment by deep convolutional network with adaptive learning rate](https://reader035.fdocuments.us/reader035/viewer/2022062822/587d0e901a28abae148b5483/html5/thumbnails/25.jpg)
Thank you!