Iris Recognition Device Usability as Observed in Comparative Testing The International Workshop on...

Iris Recognition Device Usability as Iris Recognition Device Usability as Observed in Comparative TestingObserved in Comparative Testing

The International Workshop on Usability and BiometricsThe International Workshop on Usability and Biometrics24 June 200824 June 2008

Michael ThiemeMichael ThiemeDirector of Special Projects Director of Special Projects

International Biometric GroupInternational Biometric [email protected]

Copyright © 2008Copyright © 2008International Biometric GroupInternational Biometric Group

© Copyright 2007 International Biometric Group © Copyright 2008 International Biometric Group Page 2

TopicsTopics Iris recognition capture device landscape Application-specific usability Special considerations in mobile collection Evaluating usability through scenario testing Usability-related performance test results Observations on iris usability Potential research areas


About International Biometric GroupAbout International Biometric Group International Biometric Group, LLC

– Founded 1996; 40 employees– Staff in New York, DC Metro, San Francisco– Technology-neutral, vendor-independent

Biometric systems integration– Design, develop, implement biometric identification systems– From mobile data collection to large-scale matching

Biometric R&D and evaluation – 10+ years of biometric R&D and evaluation expertise– Conducted $5m+ in USG-funded biometric R&D

Biometric consulting– SMEs for USG agencies: DHS, Executive Branch, IC– Insight into biometric vendors, industry landscape– Hands-on experience with all biometric technologies


Iris Recognition Testing ExperienceIris Recognition Testing Experience Comparative Biometric Testing (1998-present)

– Tested Panasonic AuthentiCam, LG2200, IrisGuard, JIRIS– Standardized scenario test methodology applicable to fingerprint,

face, voice, signature, hand, vein– First test effort to measure and report false non-match and

failure-to-enroll rates for iris recognition DHS Independent Testing of Iris Recognition Technology

(2004-2005)– Collected approximately 120,000 irises, conducted 2b matches– First test to evaluate interoperability across multiple iris devices– First test to measure statistically significant false match rates

Currently performing operational testing for tactical field deployments of iris recognition devices (GWOT)– Factors: usability, data quality


Iris Recognition Capture Device LandscapeIris Recognition Capture Device Landscape Considerable variation across iris recognition devices

– Single vs. dual-iris imagers– Simultaneous vs. serial capture– User-adjusted vs. auto-adjusting– Fixed-focal length vs. auto-focus – Volumetric range– Audio and visual prompts– Single-capture enrollment vs. multi-capture enrollment

Arguably greater inter-device variation than in any other modality


Application-Specific UsabilityApplication-Specific Usability Registered traveler

– Educated, tech-savvy, motivated users; low habituation; low throughput demands

– Perception of invasiveness not tolerable High-end access control

– Motivated, habituated users; rapid throughput required– Device must be calibrated for fast capture

Benefits issuance / immigration control (Pak / UAE)– Low habituation, limited familiarity with technology; motivation

may be questionable; can be invasive – Deployer may deploy head brace or other harness

Mobile population control– Similar to preceding, with addition of environmental challenges

(see discussion below); perceived invasiveness may be the point


Special Considerations in Mobile CollectionSpecial Considerations in Mobile Collection Mobile iris collection a dynamic area

– ~1m irises collected through BAT Adds the operator as a variable

– What prompts are most instructive to an operator?– Does the operator have two hands free?– How close is too close?– Tactile vs. distance-based devices


Measuring Usability in Scenario Testing Measuring Usability in Scenario Testing Performance metrics applicable to usability

– Failure to enroll / failure to acquire rate– Transaction time– False non-match rate– These are indirect measures of usability, conflated with other

factors Challenge: how to habituate users in a highly compressed

test timeframe, while still allowing for device-to-device differentiation– More training penalizes easy-to-use devices; less training

penalizes difficult-to-use devices– Can run multiple, sequential recognition transactions to drive

muscle memory in test subjects


Usability and Transaction DurationUsability and Transaction Duration


Usability and FTE / Non-Match ErrorsUsability and FTE / Non-Match Errors


FTA by Age RangeFTA by Age Range

LG IrisAccess 3000

Age Range

Recognition Attempts

Recognition Attempts with No Sample Acquired FTA

Total18-35 15930 760 4.77%36-50 10598 415 3.92%51-65 3666 247 6.74%

OKI IRISPASS-WG

Age Range



Total18-35 15890 407 2.56%36-50 10552 356 3.37%51-65 3707 171 4.61%

Panasonic BM-ET300

Age Range



Total18-35 15930 363 2.28%36-50 10602 371 3.50%51-65 3690 259 7.02%


Observations on Iris UsabilityObservations on Iris Usability Iris recognition trails most other biometrics in terms of

usability, but improvement is continual Dual-iris “mirror” imagers are problematic

– Adjustment for stronger eyes lead to misalignment, looking with wrong eye – leads to failures or off-axis captures

– Difficult to develop an interface intuitive for non-engineers– Primary value is in reducing sequencing errors, not simplifying capture

Older users have more difficulty interacting with devices– Vision? Technology exposure? Other factors…

Substantial progress has been made in “vicinity” capture– User stands in specified area, looks in general direction of imager; no

explicit alignment– Attempt to eliminate problem of initial orientation (distance from

device, where to look)


Potential Research AreasPotential Research Areas Enrollment / recognition through multiple, lower-quality

video frames Addressing dominant and weak eyes Correcting sequence errors Usability as a function of operator skills (mobile)


ContactContact

Michael [email protected]

212-809-9491 (NY Office)917-375-8206 (Mobile)

Iris Recognition Device Usability as Observed in Comparative Testing The International Workshop on...

Documents

Transcript of Iris Recognition Device Usability as Observed in Comparative Testing The International Workshop on...