HBSI Automation Using the Kinect
-
Upload
international-center-for-biometric-research -
Category
Technology
-
view
490 -
download
0
Transcript of HBSI Automation Using the Kinect
HUMAN-BIOMETRIC SENSOR INTERACTION AUTOMATION
USING THE KINECTZACH MOORE
•Can the Kinect 2 be used to determine Human-Biometric Sensor Interaction errors automatically in real-time?
RESEARCH QUESTION
METHODOLOGY
•Phase 1: Programming
•Phase 2: Construction
•Phase 3: Pilot Study
•Phase 4: Data Collection
PHASES
KINECT BODY TRACKING
• All face values are a built in feature of the Kinect.• These track the eyes, nose,
and mouth corners.
• 17 upper body points tracked not including the face.
KINECT MEASUREMENT CHECKS
CORRECT PRESENTATION
INCORRECT PRESENTATION
CLASSIFYING ERRORS
•Subject chooses the type of luggage that closely represents what they usually carry in an airport• They can bring their own, or choose from a
selection
•Given mock passport and immigration form
PROTOCOL
• They walk up to the booth and give the forms to the agent (test admin)
• The test admin asks them to provide their 10-print samples
• Once that’s done, they start he iris capture process.• This is where the Kinect is determining any errors
• They provide one sample, gather their belongings, and walk away from the booth
PROTOCOL
Pilot Ground Truth
Scenario
PROCESS MAPResearch Question: Can the Kinect 2 be used to determine Human-Biometric Sensor Interaction errors automatically in real-time?
Booth and usability study. Proved the Kinect was reliable.
My thesis. Will determine if the Kinect can be used to classify errors automatically.
Future work. Provide real-time feedback to users to test if Kinect affects throughput.
•Reviewed the video footage of all 100 subjects• Used to determine if presentation was correct or
incorrect
•Exported the AOptix logs• Used to determine the HBSI metric
•All done after the data collection had concluded
GROUND TRUTH CLASSIFICATION
•Used the body points from the Kinect sensor• This data was used to determine if the presentation
was correct or incorrect
•Monitored the AOptix state change over the network• Used to determine HBSI Metric
•All done in real-time
KINECT CLASSIFICATION
AOPTIX STATES[1] [2] [3] [4] [5]
[6] [8] [11] [13] [15]
[21] [22] [23] [25]
CLASSIFICATION PROCESS
RESULTS
GENDER REPORT
FemaleMale
Category
47.0%
53.0%
Gender
Gender CountMale 47
Female 53
Total 100
AGE BREAKDOWN
49+41-4833-4026-3218-25
40
30
20
10
0
Age Group
Cou
nt
1110
7
31
41
Age Breakdown
ETHNICITY
IndianAra
bMixe
d
Hispani
c
Asian
or Pa
cific I
sland
erOth
er
Africa
n Ameri
canAsian
Cauca
sian
70
60
50
40
30
20
10
0
Ethnicity
Coun
t
1122248
23
57
Subject Ethnicity
CLASSIFICATION RESULTS
GROUND TRUTH CLASSIFICATIONS
SPSFT PFT DFIDICI
120
100
80
60
40
20
0
120
100
80
60
40
20
0
Metric
Cou
nt 67
10
120
21
37
4
Ground Truth HBSI Metric Classifications
EXAMPLE INTERACTION
Subject ID Ground Truth Classification Kinect Classification Correct Classification
066 FTD FTD Y
066 FTD FTD Y
066 FTD FTD Y
066 FTD FTD Y
066 SPS SPS Y
GROUND TRUTH COMPARED TO KINECT
SPSFTPFTDFIDICI
120
100
80
60
40
20
0
120
100
80
60
40
20
0
Metric
Cou
nt 67
10
120
21
37
4
Ground Truth HBSI Metric Classifications
SPSNONEFTPFTDFIDICI
120
100
80
60
40
20
0
120
100
80
60
40
20
0
Metric
Cou
nt
58
70
9
67
2725
3
Kinect HBSI Metric Classifications
•Cause:• The AOptix device switched states so quickly, that
the Kinect did not detect the change
• The Kinect has a fixed frame refresh rate (30fps)
•From the Kinect’s point of view, no error occurred, so it did not classify the presentation
“NONE” CLASSIFICATION
“NONE” CLASSIFCATION
Refresh FrameRefresh Frame
Kinect
AOptix
Subject ID Ground Truth Classification Kinect Classification Correct Classification
028 FTD FTD Y
028 FTD FTD Y
028 FTD NONE N
028 FTD NONE N
028 FTD NONE N
028 SPS SPS Y
“NONE” EXAMPLE
HBSI METRICS CLASSIFIED AS “NONE”
CIDIFTDFTPSPS
Category
31
52
13
1
HBSI Metrics Classified as "NONE" by Kinect• 70 instances of “NONE” classification total
• Of these 70, the ground truth equivalent metric classification is shown
PRESENTATION ACCURACY
Correct PresentationIncorrect Presentation
Category
48.3%51.7%
Kinect Presentation Classifications
Correct PresentationIncorrect Presentation
Category
23.9%
76.1%
Ground Truth Presentation Classifications
ACCURACY OF KINECT CLASSIFICATIONS
Different ClassificationSame Classification
Category
62.9%
37.1%
Kinect Classifications Compared to Ground Truth
ACCURACY BY METRICCI DI FI
FTD FTP SPS
Different ClassificationSame Classification
Category
50.0%
50.0%
51.4%
48.6%
81.0%
19.0%
52.5%47.5%
80.0%
20.0%
80.6%
19.4%
Kinect Classifications Compared to Ground Truth by Metric
•How accurate was the Kinect at determining these errors when it did notice the state change?
•By removing the observations that include “NONE”, does the accuracy improve?
FURTHER QUESTIONS RAISED
REMOVING “NONE’ CLASSIFICATIONS
Subject ID Ground Truth Classification Kinect Classification Correct Classification
028 FTD FTD Y
028 FTD FTD Y
028 FTD NONE N
028 FTD NONE N
028 FTD NONE N
028 SPS SPS Y
Subject ID Ground Truth Classification Kinect Classification Correct Classification
028 FTD FTD Y
028 FTD FTD Y
028 SPS SPS Y
PRESENTATION ACCURACY – WITHOUT “NONE”
Correct PresentationIncorrect Presentation
Category
25.4%
74.6%
Ground Truth Presentation Classifications
Correct PresentationIncorrect Presentation
Category
29.1%
70.9%
Kinect Presentation Classifications
ACCURACY OF KINECT CLASSIFICATIONS – WITHOUT “NONE”
Different ClassificationSame Classification
Category
85.7%
14.3%
Kinect Classifications Compared to Ground Truth
ACCURACY BY METRIC – WITHOUT “NONE”
CI DI FI
FTD FTP SPS
Different ClassificationSame Classification
Category
66.7%
33.3%
79.2%
20.8%
81.0%
19.0%
91.2%
8.8%
88.9%
11.1%
84.4%
15.6%
Kinect Classifications Compared to Ground Truth by Metric
CONCLUSIONS AND FUTURE WORK
•The Kinect can be used to determine HBSI errors in real-time• The accuracy of which depends on the thresholds the
Kinect operates under
•The refresh rate of the Kinect was not high enough to detect all state changes from the AOptix device
•This research provides a foundation for future work
CONCLUSIONS
• Increasing Kinect refresh rate or using different sensor
• Developing real-time feedback to both subject and test administrator• Test change in throughput and performance
• Adjusting Kinect thresholds for correct/incorrect presentation classifications
• Use Kinect gesture recognition to use for different modalities (fingerprint)
• Implement in operational testing
FUTURE WORK
QUESTIONS?