Igor Markov Face Detection and Classification on Mobile Devices.
-
Upload
whitney-caple -
Category
Documents
-
view
221 -
download
1
Transcript of Igor Markov Face Detection and Classification on Mobile Devices.
![Page 1: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/1.jpg)
Igor Markov
Face Detection and Classification on Mobile
Devices
![Page 2: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/2.jpg)
2
AgendaIntroductionAlgorithmsThe projectFree frameworks
2
![Page 3: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/3.jpg)
3
What is face detection for?Camera focusingTagging faces on photosMarketing studiesSurveillanceSpecial effects (Augmented Reality)Robotics
3
![Page 4: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/4.jpg)
4
On mobile devices?The same thing.
4
![Page 5: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/5.jpg)
5
Face classificationGenderAgeEmotionEthnic group
5
![Page 6: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/6.jpg)
6
Face trackingIs this the same person in the next video frame?
6
![Page 7: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/7.jpg)
7
Traditional algorithmsSearch for eyes, nose, mouth, etcEstimate relative positions of these points... or, comparison with templates
EigenfacesLinear Discriminate AnalysisElastic Bunch Graph MatchingMultilinear Subspace LearningDynamic link matching
7
![Page 8: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/8.jpg)
8
Proposed in 2001 by Paul Viola and Michael JonesReal-time enoughA face can be rotated by angle up to 30°Good for embedded solutionsLearning is rather slow
8
Viola–Jones Object Detection Framework
![Page 9: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/9.jpg)
9
Sub-windowSize is 24×24Moves through all possible positions
9
![Page 10: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/10.jpg)
10
The light part is addedThe dark part is subtracted
10
Haar-like features
![Page 11: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/11.jpg)
11
Haar Feature Example
11
![Page 12: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/12.jpg)
12
Integral Image
12
![Page 13: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/13.jpg)
13
Classifiers Cascade
13
![Page 14: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/14.jpg)
14
Learning: Photo SetAt&T FacedatabaseYale Facedatabase AExtended Yale Facedatabase BFERET
14
![Page 15: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/15.jpg)
15
Machine LearningBoostingAdaBoost
15
![Page 16: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/16.jpg)
16
ClassificationLearning: AdaBoostClassifications: Local Binary Patterns, EigenFaces, etc.
16
?
?
![Page 17: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/17.jpg)
17
Use CaseFace detection and classification for marketing studyVideo stream from a camera, real timeUsing Android phoneHigh performance
17
![Page 18: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/18.jpg)
18
Generic scheme on AndroidScheme - camera, native, overlays
18
![Page 19: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/19.jpg)
19
OptimizationsAvoid large data copyingdouble ➙ intEarly exit from loopsParallelizationSIMD
19
![Page 20: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/20.jpg)
20
Parallel DetectionThread pool (max threads = CPU cores number)For each possible sub-window size:
Put a task to the thread poolWait for results
20
![Page 21: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/21.jpg)
21
NEON code
21
loop: vldmia %0!, {%%d8, %%d9} //q4 <- data[i][j] vldmia %3!, {%%d28, %%d29} //q14 <- integral_fi[i-1][j] vldmia %5!, {%%d30, %%d31} //q15 <- sq_integral_fi[i-1][j] vmul.f32 %%q5, %%q4, %%q4 //q5 <- data^2 vmov %%d1, %%d8 // q0[2-3] <- q4[0-1] vadd.f32 %%q4, %%q0 vext.32 %%d3, %%d8, %%d9, #1 // q1[2-3] <- q4[1-2] vmov %%s5, %%s16 // q1[1] <- q4[0] vadd.f32 %%q4, %%q1 //data is summed in q4 vmov %%d5, %%d10 // q2[2-3] <- q5[0-1] vadd.f32 %%q5, %%q2 vext.32 %%d7, %%d10, %%d11, #1 // q3[2-3] <- q5[1-2] vmov %%s13, %%s20 // q3[1] <- q5[0]
![Page 22: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/22.jpg)
22
Public FrameworksOpenCV (FaceRecognizer)Android SDK (Camera Face Listener)iOS SDK (Core Image)Lots of them (facedetection.com)
22
![Page 23: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/23.jpg)
23
OpenCVOpen sourceC++Many useful algorithms and primitives
23
FaceRecognizer model = createEigenFaceRecognizer();
....
int predictedLabel = model->predict(testSample);
![Page 24: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/24.jpg)
24
Android SDK Face Detection
24
class MyFaceDetectionListener implements Camera.FaceDetectionListener {
public void onFaceDetection(Face[] faces, Camera camera) { int i = 0;
for (Face face : faces) {
Log.i("FD", "face detected: " + (++i) + " of " + faces.length + "X: " + faces.rect.centerX() + "Y: " + faces.rect.centerY());
} }}
![Page 25: Igor Markov Face Detection and Classification on Mobile Devices.](https://reader034.fdocuments.us/reader034/viewer/2022052701/56649c755503460f949297c4/html5/thumbnails/25.jpg)
25
iOS Core Image
25
CIContext *context = [CIContext contextWithOptions:nil];
NSDictionary *opts = @{ CIDetectorAccuracy : CIDetectorAccuracyHigh };
CIDetector *detector = [CIDetector detectorOfType:CIDetectorTypeFace context:context options:opts]; opts = @{ CIDetectorImageOrientation : [[myImage properties] valueForKey:kCGImagePropertyOrientation] };
NSArray *features = [detector featuresInImage:myImage options:opts];