18/10/2018
1
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Touchscreen Biometrics:Desktop Signature goes Mobile
Prof. Julian Fierrez
http://atvs.ii.uam.es/fierrez
BiDA Lab - Biometrics and Data Pattern Analytics
Universidad Autonoma de Madrid (UAM), SPAIN
(Based on PhD material
from Dr. Ruben Tolosana)
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Signature Biometrics: From Lab to Market
2002…
ResearchResearchat UAM
• J. Fierrez and J. Ortega-Garcia, "On-line signature verification", A. K. Jain, et al. (Eds.), Handbook of Biometrics, 2008.
• M. Martinez-Diaz and J. Fierrez, "Signature Databases and Evaluation", Stan Z. Li et al. (Eds.), Encyclopedia of Biometrics, 2015.
http://atvs.ii.uam.es/fierrez
18/10/2018
2
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
On-line Writer/Signature Verification
Feature-based (Global Features)
Distance-based classifiers• Mahalanobis• Euclidean [Nelson et al., 1994]
Statistical/other classifiers• Gaussian Mixture Models
(GMM)• Parzen Windows
Function-based (Local Features)
Time-Sequence matching techniques
• Hidden Markov Models (HMM) [Dolfing et al., 1998]
• Gaussian Mixture Models (GMM) [Richiardi et al., 2005]
• Dynamic Time Warping (DTW) [Sato and Kogure, 1982]
sample index
0 50 100 150 200 250 300 350 4000
2000
4000
x
0 50 100 150 200 250 300 350 4000
1000
2000
y
0 50 100 150 200 250 300 350 4000
500
1000
z
0 50 100 150 200 250 300 350 4001000
1200
1400
azim
uth
0 50 100 150 200 250 300 350 400400
500
600
altitu
de
Dynamic signature matching
• J. Fierrez and J. Ortega-Garcia, "On-line signature verification",
A.K. Jain et al. (Eds), Handbook of Biometrics, 2008.
• M. Martinez-Diaz and J. Fierrez, "Signature Databases and
Evaluation", Stan Z. Li and Anil K. Jain (Eds.), Encyclopedia
of Biometrics, Springer, pp. 1367-1375, 2015.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Feature Extraction: Global Features Example
-0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8-1
-0.5
0
0.5
1
1.5
2
2.5
Global Feature 1
Glo
bal F
eatu
re 2
Genuine Signatures (All)
Skilled Forgeries (All)
Genuine Signatures (Shown)
Skilled Forgery (Shown)
-0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
Global Feature 3
Glo
bal F
eatu
re 4
• M. Martinez, J. Fierrez, et al., "Mobile Signature Verification: Feature Robustness
and Performance Comparison", IET Biometrics, December 2014.
18/10/2018
3
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Dynamic Time Warping (DTW)Hidden Markov Models (HMM)
Point-to-point correspondenceStatistical modeling of signature regions
• M. Martinez-Diaz, J. Fierrez and S. Hangai, "Signature Matching", Stan Z. Li and
Anil K. Jain (Eds.), Encyclopedia of Biometrics, Springer, pp. 1382-1387, 2015.
Similarity Computation
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Resources: Multimodal Databases w Signature
• MCYT Database (Spanish Project 2000-2003)
➢ Fingerprint (with human-labeled quality) and on-line Signature of 330 donors
• J. Ortega-Garcia, J. Fierrez-Aguilar, et al., "MCYT baseline corpus: A bimodal biometric database",
IEE Proceedings Vision, Image and Signal Processing, December 2003.
• BiosecurID Database (Spanish Project 2003-2006)
➢ 8 Modalities: speech, iris, face, Signature and handwriting (on-line and off-line), fingerprints, hand and keystroking of 400 donors in 4 acquisition sessions
• Biosecure Database (EU Project 2004-2007)
➢ 3 Datasets: Web scenario, Office scenario, Mobile scenario
➢ 667 donors
See: https://atvs.ii.uam.es/atvs/databases.jsp
• J. Ortega, J. Fierrez, et al., “The BioSecure Multimodal Database", IEEE Trans. PAMI, June 2010.
• J. Fierrez, J. Galbally, et al., "BiosecurID: A Multimodal Biometric Database",
Pattern Analysis and Applications, Vol. 13, n. 2, pp. 235-246, May 2010.
https://atvs.ii.uam.es/atvs/databases.jsp
18/10/2018
4
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Our HMM system
SVC-04 skilled forgeries
SVC-04 random (zero-effort, casual) impostors
http://www.cs.ust.hk/svc2004/
Benchmarks: SVC 2004
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
BioSec Signature Evaluation Campaign, BSEC 2009
• DTW, HMM and Global Systems
• Score normalization
• Fusion of systems
False Acceptance Rate (%)
Fals
e R
eje
ctio
nR
ate
(%)
• N. Houmani, et al., "BioSecure signature evaluation campaign
(BSEC2009): Evaluating online signature algorithms depending
on the quality of signatures", Pattern Recognition, March 2012.
Our DTW system
http://www.cs.ust.hk/svc2004/
18/10/2018
5
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Signature Biometrics: From Lab to Market
• Development and Transfer of signature biometric technology:
– Cecabank is a Spanish wholesale bank that provides support services to banking
business processes (~45% of the Spanish Banking Sector)
2002… 2014-2015
1st Tech. Transfer
UAM → CECABANK
Research
UAM→ CECABANK
Researchat UAM
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Development and Transfer of signature biometric technology:
– Cecabank’s Associates generate 1 billion documents/year
Get rid of paper!
9,000 tons of paper = 200,000 trees
➢ Total savings per year:
Paper elimination1,5M euros
Operative efficiency 2M euros45K hours
2002… 2014-2015
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
Research
UAM→ CECABANK
Researchat UAM
18/10/2018
6
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Development and Transfer of signature biometric technology:
– Office-like scenarios (Wacom devices)
– Small number of signatures per user
– Signature Verification System
• Main core: Dynamic Time Warping (DTW)
2002… 2014-2015
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
Research
UAM→ CECABANK
Researchat UAM
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• R&D: device interoperability scenarios
– e-BioSign database (Wacom and Samsung general purpose devices)
– Publically available to the research community
• In use +25 research groups (USA, China, India, Italy, etc.)
15 / 39
Available at: https://atvs.ii.uam.es/atvs/eBioSign-DS1.html
2002… 2014-2015 2015-2016
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
R&DResearch
UAM→ CECABANK
Researchat UAM
• R. Tolosana, R. Vera-Rodriguez, J. Ortega-Garcia and J. Fierrez, "Preprocessing and Feature Selection for Improved Sensor
Interoperability in Online Biometric Signature Verification", IEEE Access, May 2015.
https://atvs.ii.uam.es/atvs/eBioSign-DS1.html
18/10/2018
7
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• R&D: template and system configuration update strategies
– ATVS Signature Long-Term Extended database (Publically available*)
– Large number of enrolment signatures per user (46 samples/user)
– Signature Verification Systems
• Hidden Markov Models (HMM), Gaussian Mixture Models (GMMs)
• Dynamic Time Warping (DTW)
3 weeksQuery
9 months 2 weeks 4 months
Session 1 Session 2 Session 5 Session 6
2002… 2014-2015 2015-2016
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
R&DResearch
UAM→ CECABANK
Researchat UAM
* J. Galbally, M. Martinez-Diaz and J. Fierrez, "Aging in Biometrics:Experimental Analysis on On-line Signature", PLOS ONE, July 2013.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• 2nd Technology Transfer: signature biometrics technology
– ATVS Signature Long-Term Extended database
Enrolment Signatures
EER (%) 4 signatures 16 signatures 31 signatures 41 signatures
Random Forgery 2.8 0.7 0.1 0.1
Skilled Forgery 11.7 6.9 4.1 0.7
2002… 2014-2015 2015-2016 2016-2017
1st Tech. Transfer
UAM→ CECABANK
Signature Biometrics: From Lab to Market
R&DResearchat UAM
2nd Tech. Transfer
18/10/2018
8
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• R&D: improving traditional signature verification approaches
– Recurrent Neural Networks (RNNs) deep learning techniques
• LSTM, GRU, Siamese architectures, Bidirectional approaches, etc
•
System Performance , EER (%)
1 enrol. sign. 4 enrol. sign.
DTW 10.17 7.75
RNNs 5.60 4.75
Case: Skilled forgery
2002… 2014-2015 2015-2016 2016-2017 2017…
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
R&DResearch2nd Tech. Transfer R&D
UAM→ CECABANK
Researchat UAM
R. Tolosana, R. Vera-Rodriguez, J. Fierrez and J. Ortega-Garcia, "Exploring Recurrent Neural Networks for On-Line
Handwritten Signature Biometrics", IEEE Access, Vol. 6, pp. 5128-5138, February 2018.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Deployment by Cecabank: marketed as BIOTRACE100 technology
BIOTRACE100 technology:▪ > 18K branches▪ > 46K sensors ▪ > 500K transactions/year
2002… 2014-2015 2015-2016 2016-2017 2017…
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
R&DResearch2nd Tech. Transfer R&D
UAM→ CECABANK
Researchat UAM
Deployment byCECABANK
18/10/2018
9
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Users spend more than 2x time on mobile scenarios compared to desktop scenarios
Source: “Mobile Global Report 2018”, May 2018
Mobile
Desktop
Desktop Signature > Mobile: Touchscreen Biometrics
Motivation
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Users spend more than 2x time on mobile scenarios compared to desktop scenarios
• “In March 2018, 48% of digital consumers were accessing banking services through
smartphones or tablets almost exclusively”
Source: “Online Banking in Europe Report”, June 2018
Mobile
Desktop
Desktop Signature > Mobile: Touchscreen Biometrics
Motivation
18/10/2018
10
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• User authentication difficult to accomplish using only traditional approaches
Personal Identification Number (PIN) One-Time Password (OTP)
Desktop Signature > Mobile: Touchscreen Biometrics
Motivation
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• User authentication difficult to accomplish using only traditional approaches
• Security threats [Bonneau et al., 2012]
Personal Identification Number (PIN) One-Time Password (OTP)
Related to personal details, typical words or sequential numbers
Desktop Signature > Mobile: Touchscreen Biometrics
Motivation
18/10/2018
11
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• User authentication difficult to accomplish using only traditional approaches
• Security threats [Bonneau et al., 2012]
Personal Identification Number (PIN) One-Time Password (OTP)
Shoulder surfing (visual access to passwords)
Desktop Signature > Mobile: Touchscreen Biometrics
Motivation
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• User authentication difficult to accomplish using only traditional approaches
• Security threats [Bonneau et al., 2012]
Personal Identification Number (PIN) One-Time Password (OTP)
Smudge attack (finger grease traces on screen )
Desktop Signature > Mobile: Touchscreen Biometrics
Motivation
18/10/2018
12
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Password-Based Mobile Authentication Incorporating Handwritten Touch Biometrics
• Two-factor authentication approach:
1) Check the password
2) Check the biometric information
Password
5 7 6 1 8
Desktop Signature > Mobile: Touchscreen Biometrics
Example Application
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Password-Based Mobile Authentication Incorporating Handwritten Touch Biometrics
• Two-factor authentication approach:
1) Check the password
2) Check the biometric information
Password
5 7 6 1 8
Desktop Signature > Mobile: Touchscreen Biometrics
Example Application
18/10/2018
13
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• R&D: touchscreen biometrics
22 / 39
2002… 2014-2015 2015-2016 2016-2017 2017-2018 2018…
1st Tech. Transfer
UAM → CECABANK
Signature Biometrics: From Lab to Market
R&DResearch2nd Tech. Transfer R&D
UAM→ CECABANK
Researchat UAM
Deployment byCECABANK
R&D
• M. Martinez-Diaz, J. Fierrez and J. Galbally, "Graphical Password-based User Authentication with Free-Form Doodles", IEEE
Trans. on Human-Machine Systems, August 2016.
• R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, J. Ortega-Garcia, “Benchmarking Desktop and Mobile Handwriting
across COTS Devices: the e-BioSign Biometric Database”, PLOS ONE, May 2017.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Acquisition: one handwritten digit at a time. Only X and Y time information is considered
* Available at: https://atvs.ii.uam.es/atvs/e-BioDigit.html
SAMSUNG GALAXY NOTE 10.1
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”,
Proc. of CVPR, Workshop on Biometrics, Salt Lake City, USA, June 2018.
Example
e-BioDigit Database*
https://atvs.ii.uam.es/atvs/e-BioDigit.html
18/10/2018
14
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
3 weeks
Session 1 Session 2
# Users: 93
10 numerical digits (0,1,..,9) 10 numerical digits (0,1,..,9)
4 samples/numerical digit 4 samples/numerical digit
e-BioDigit Database*
* Available at: https://atvs.ii.uam.es/atvs/e-BioDigit.html
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”,
Proc. of CVPR, Workshop on Biometrics, Salt Lake City, USA, June 2018.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
System Architecture
• Two-factor authentication scheme: OTP system
Password
7 9 2 8
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”, Proc. of CVPR,
Workshop on Biometrics, Salt Lake City, USA, June 2018.
https://atvs.ii.uam.es/atvs/e-BioDigit.html
18/10/2018
15
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Comparison with Related Works
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”, Proc. of CVPR,
Workshop on Biometrics, Salt Lake City, USA, June 2018.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Comparison with Related Works
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”, Proc. of CVPR,
Workshop on Biometrics, Salt Lake City, USA, June 2018.
18/10/2018
16
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Comparison with Related Works
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”, Proc. of CVPR,
Workshop on Biometrics, Salt Lake City, USA, June 2018.
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Comparison with Related Works
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”, Proc. of CVPR,
Workshop on Biometrics, Salt Lake City, USA, June 2018.
18/10/2018
17
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Comparison with Related Works
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
• Proposed approach outperforms other touch biometric approaches for skilled forgeries
– User-friendly interface
– Small number of enrolment samples
Comparison with Related Works
* R. Tolosana, R. Vera, J. Fierrez, et al. “Incorporating Touch Biometrics to Mobile One-Time Passwords”, Proc. of CVPR,
Workshop on Biometrics, Salt Lake City, USA, June 2018.
18/10/2018
18
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
The Future of Touchscreen Biometrics
36 / 39
• J. Fierrez, et al., "Benchmarking Touchscreen Biometrics for Mobile Authentication", IEEE Trans. on IFS, November 2018.
• A. Morales, J. Fierrez, et al, "Keystroke Biometrics Ongoing Competition", IEEE Access, Vol. 4, Nov. 2016.
• J. Hernandez-Ortega, A. Morales, J. Fierrez, et al., "Detecting Age Groups using Touch Interaction", IET Elect. Letters, Sept. 2017.
• J. Fierrez, A. Morales, et al, "Multiple Classifiers in Biometrics. Part 2: Trends and Challenges", Information Fusion, Nov. 2018.
• M. Gomez-Barrero, et al, "Multi-Biometric Template Protection Based on Homomorphic Encryption", Pattern Rec., July 2017.
Session 2
Julian Fierrez – Biometrics Congress, London, UK, Oct. 17, 2018
Touchscreen Biometrics:Desktop Signature goes Mobile
Prof. Julian Fierrez
http://atvs.ii.uam.es/fierrez
BiDA Lab - Biometrics and Data Pattern Analytics
Universidad Autonoma de Madrid (UAM), SPAIN
(Based on PhD material
from Dr. Ruben Tolosana)
http://atvs.ii.uam.es/fierrezTop Related