A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
-
Upload
saurabh-bhardwaj -
Category
Documents
-
view
220 -
download
0
Transcript of A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 1/10
Pergamon
Pattern Recognition Vol. 29, No. 3, pp. 415 424, 1996
Elsevier Science Ltd
Copyright © 1996 Pattern Recognition Society
Printed in Great Britain. All rights reserved
0031 3203/96 15.00+.00
0031-3203 95)00092-5
NEURAL NETWORK APPROACH TO OFF LINE
SIGNATURE VERIFICATION USING DIRECTIONAL PD F
J .- P. D R O U H A R D , R . S A B O U R I N a n d M . G O D B O U T
Labo ratoire d 'Im agerie , de Vision et d 'Intelligence Artific ielle (LIVIA), Drpartem ent de g~nie de la
produc tion au tomatisre , l~cole de technologie sup+rieure, 4750 Henri-Julien, M ontrr al (Quebec), Canada
H2T 2C8
Received 24 February 1994; revised 11 April 1995; received or publication 3 July 1995)
Ab str aet --A neural network a pproac h is propo sed to build the firs t s tage of an A utom atic Handwritten
Signature Verification System. The directional Pro bability Density Functio n was used as a g lobal shape
factor and its discriminating power w as enhanced b y reducing its cardin ality via filtering. Various
experimental proto cols were used to implement the ba ckpro pagation network (BPN ) classifier. A com pari-
son, on the same da tabase and with the same decis ion rule, shows that the B PN classifier s c learly better than
the thresh old classifier and co mp ares favourably with the k-Nearest-Neighbour classifier.
Pattern recognition Classifiers Neural networks Backp ropagation
Autom atic s ignature verification Directional proba bility density function
1. INTRODUCTION
T h e d e s i gn o f a c o m p l e t e A u t o m a t i c H a n d w r i t t e n
S i g n a t u r e V e r i f i c a t i o n S y s t e m ( A H S V S ) t h a t w i l l b e
a b le to t a k e in to a c c o u n t a l l c l a s s e s o f fo rg e r i e s i s a v e ry
d i f f i c u lt t a s k . ~1) In d e e d , a c o m p le te A H S V S s h o u ld b e
a b l e t o d i s c r i m i n a t e b e t w e e n g e n u i n e s i g n a t u r e s a n d
th e fo l lo w in g fo rg e r ie s :
r a n d o m f o r g e r i e s ,
c h a r a c t e r i z e d
b y a d i f f er e n t se m a n t i c m e a n i n g a n d c o n s e q u e n t l y b y
a d i f fe r e n t o v e r a l l s h a p e w h e n c o m p a r e d t o g e n u i n e
s ig n a tu re s ; s i mp l e f o r g e r i e s , w i t h t h e s a m e s e m a n t i c
m e a n i n g a s g e n u i n e s i g n a t u r e s b u t a n o v e r a l l s h a p e
th a t d i f f e r s g re a t ly ;
f r e e h a n d a n d s i mu l a t e d f o r g e r i e s ,
p r o d u c e d w i t h t h e a pr ior i k n o w l e d g e o f b o t h t h e
s e m a n t i c m e a n i n g a n d t h e g r a p h i c a l m o d e l o f a t a r g e t
s i g n a t u r e b y a s k i l le d o r a n o c c a s i o n a l f o r g e r r e s p e ct -
ive ly ; f ina l ly , t r a c in g f o r g e r i e s a n d p h o t o c o p i e s , w i t h
a l m o s t t h e s a m e g r a p h i c a l a s p e c t a s g e n u i n e s i g n a -
t u r e s , b u t w i t h d i f f e r e n t p s e u d o - d y n a m i c p r o p e r t i e s
s u c h a s d i s s im i l a r i t i e s n g re y - l e v e l - r e l a t e d e a tu re s l i k e
t e x tu re , c o n t ra s t .
I n s u c h a s y s te m , in o r d e r t o t a k e i n t o a c c o u n t a l l
c l a s s e s o f fo rg e r ie s , t h e d e c i s io n i s m a d e o n ly a t t h e e n d
o f th e v e r i f i c a t io n p ro c e s s . C o n s e q u e n t ly , t h i s a p -
p r o a c h i s a v e r y c o s t l y s o l u t i o n i n t e r m s o f c o m p u t a -
t i o n a l r e s o u r c e s a n d i n t e r m s o f r e l a t e d a l g o r i t h m i c
c o m p l e x i t y .12) S in c e r a n d o m a n d s im p le fo rg e r i e s r e p -
r e s e n t a l m o s t 9 5 o f t h e c a s es g e n e r a l l y e n c o u n t e r e d
in p ra c t i c e , 13'41 a b e t t e r s o lu t io n m ig h t b e to s u b d iv id e
th e v e r i f i c a t io n p ro c e s s in s u c h a w a y to r a p id ly e l im i -
n a t e g r o s s f o r g e ri e s . T h u s , a t w o - s t a g e A H S V S s e e m s
to b e a m o re p ra c t i c a l s o lu t io n , w h e re th e f i r s t s t a g e
w o u l d b e r e s p o n s i b l e f o r t h is r a p i d e l i m i n a t i o n a n d t h e
s e c o n d s t a g e u s e d o n l y i n c o m p l i c a t e d c a s e s . T h e
d e s i g n o f t h i s f i r s t s t a g e w a s m a d e w i t h r a n d o m
fo rg e r i e s b a s e d o n th e f a c t t h a t a v e r i f i c a t io n s y s t e m
a b l e t o c o p e w i t h r a n d o m f o r g e r i e s w i l l b e a b l e t o
s u c c e s s fu l ly d i s c r im in a te s im p le fo rg e r i e s .
T h e f i r s t s t a g e o f t h i s c o m p l e t e A H S V S t h u s h a s t w o
m a i n o b j e ct i v e s: f i rs t ly , t o c o n s i d e r o n l y r a n d o m a n d
s im p le fo rg e r i e s a n d , s e c o n d ly , t o m a k e a r a p id d e c i -
s io n . T o m e e t t h e f i r s t o b je c t iv e , a c h a ra c te r i s t i c d e a l -
i n g w i t h t h e o v e r a l l sh a p e o f h a n d w r i t t e n s i g n a t u r e s
s e e m s a p p r o p r i a t e . A c c o r d i n g l y , w e h a v e c h o s e n t o u s e
t h e d i r e c ti o n a l P r o b a b i l i t y D e n si t y F u n c t i o n ( P D F ) a s
a g lo b a l s h a p e f a c to r . ~51 I t s d i s c r im in a t in g p o w e r i s n o t
o p t i m u m b e c a u se , e ve n t h o u g h i t is i n v a r i a n t i n t r a n s -
l a t io n a n d in s c a l e , i t i s n o t i n v a r i a n t i n ro t a t io n . O n
t h e o t h e r h a n d , i t d o e s n o t r e q u i r e t o o m u c h c o m p u t e r
t im e , t h e re b y s a t i s fy in g th e s e c o n d o b je c t iv e .
T o m e e t t h e s e c o n d o b je c t iv e , w e h a v e c h o s e n to u s e
a B a c k P r o p a g a t i o n N e t w o r k ( B P N ) a s a si g n a tu r e
c l a s si f i e r . In d e e d , o n c e t r a in e d , u n l ik e c o n v e n t io n a l
c l a ss i fi e rs s u c h a s t h e k N e a r e s t N e i g h b o u r ( k N N )
c la s s if i e r , i t h a s a v e ry f a s t r e s p o n s e t im e s in c e i t d o e s
n o t h a v e to m e m o r iz e in fu l l a l l s ig n a tu re s p e c im e n s .
H o w e v e r , t h e l e a rn in g p h a s e o f t h e s e c la s s i f i e rs i s
a r e l a t iv e ly d if f i c u l t t a s k in th i s a p p l i c a t io n . A s a m a t -
t e r o f f a c t , w i th th i s t y p e o f c l a s s if i e r , w e m u s t k n o w
a pr ior i a l l t h e f a l s e s ig n a tu re s a n d h a v e m a n y
e x a m p l e s p r i o r t o t r a i n i n g . I n a d d i t i o n , d u e t o t h e v e r y
h i g h v a r i a b i l i t y o f h a n d w r i t t e n s i g n a t ur e s , t h e s e p a r -
a t i o n b e t w e e n t r u e a n d f a ls e si g n a t u r e s is n o t a s h a r p
o n e . T h i s r e s u l t s i n v e ry lo n g c o n v e rg e n c e t im e s a n d
th e o v e ra l l p e r fo rm a n c e w i l l n e v e r b e p e r fe c t . N e v e r -
th e l e s s, a s s h o w n in a f e a s ib i l i t y s tu d y , ~5) the se diffic ul-
t i es c a n b e d i m i n i s h e d i f s o m e p r e c a u t i o n s a r e t a k e n
a n d t h e r e s u l t s o b t a i n e d c o r r e s p o n d t o t h e m a i n
415
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 2/10
416 J .-P . DR OU HA RD et al
o b je c t iv e o f t h e f i r s t s t a g e o f t h e d e c i s io n , w h ic h i s t h e
ra p id e l im i n a t io n o f g ro s s fo rg e r ie s .
T h i s p a p e r a d d r e s s e s t h e p r o b l e m s r e l a t e d t o t h e
d e s i g n o f th e f i r st s t a g e o f a c o m p l e t e A H S V S . I n
S e c t io n 2 , w e g iv e a l l t h e d e f in i t i o n s th a t a re u s e d
t h r o u g h o u t t h i s p a p e r . T h e c h o i c e o f t h e p r e - t r e a t m e n t
p e r f o r m e d o n t h e fu l l d i r e c t i o n a l P D F i n o r d e r t o
e n h a n c e i t s p e r fo rm a n c e s i s b r i e fly p re s e n te d in S e c -
t io n 3 . T h e d e s ig n o f t h e B P N u s e d a s a s ig n a tu re
c l a s s i f ie r i s s u m m a r iz e d in S e c t io n 4 . F in a l ly , in S e c t io n
5 th e B P N c l a ss i f ie r p e r f o r m a n c e s a r e c o m p a r e d , o n
t h e s a m e d a t a b a s e , t o t h o s e o b t a i n e d w i t h t h e c o n v e n -
t io n a l k N N a n d T c l a s s i f i e r s .
2. DEFINITIONS
2 1 D a t a s e ts
A s t a n d a r d s i g n a t u r e d a t a b a s e o f 4 0 s i g n a t u r e s w r i t -
t e n b y 2 0 in d iv id u a l s 8 0 0 im a g e s ) i s u s e d in th i s s tu d y .
F o r th e c h o ic e o f p re - t r e a t m e n t S e c t io n 3 ), t h e f i r st 2 0
a n d th e l a s t 2 0 s ig n a tu re s o f e a c h w r i t e r w e re u s e d to
b u i ld th e r e fe re n c e s e t a n d th e t e s t s e t , r e s p e c t iv e ly . F o r
th e d e s ig n o f t h e B P N S e c t io n 4) , t h e f i r st 20 s ig n a tu re s
w e re u s e d to b u i ld th e t r a in in g s e t , b u t t h e l a s t 2 0
s i g n a t u r e s w e r e d i v i d e d i n t o t w o g r o u p s t o b u i l d t h e
tes t se t f i rs t 10) an d the va l i da t ion se t la s t 10) .
2.2. R e f e r e n c e a n d t r a i n i n o s e ts
T h e re fe re n c e s e t fo r t h e k N N c la s s i f ie r a n d th e
t r a i n i n g s e t fo r t h e B P N c l a s si f i er w e r e c o m p o s e d o f
2 8 0 e x a m p le s : 1 4 0 r e l a t e d to th e g e n u in e s ig n a tu re s o f
a n in d i v id u a l c l a s s 0 )1 ) a n d a n o th e r 1 4 0 r e l a t e d to
r a n d o m f o r g e ri e s d ef i n e d a s a s u b s e t o f s i g n a t u r e s f r o m
a l l t h e o th e r i n d iv id u a l s c l a s s 0 )2 ). F o r e a c h w r i t e r , t h e
1 4 0 e x a m p le s in c l a s s 0 )1 w e re o b ta i n e d b y ro t a t in g a l l
t h e f u l l d i r e c t i o n a l P D F i s s u e d f r o m t h e 2 0 r e f e r e nc e
s ig n a tu re s in a c i r c u la r f a s h io n f ro m - 6 to + 6 ° i n 2 °
in c re m e n t s . In th e c a s e o f c l a s s 0 )2 , t h e 1 4 0 e x a m p le s
w e r e o b t a i n e d b y c h o o s i n g s e v e n o r e i g h t r e f e r e n c e
s i g n a t u r e s at r a n d o m f r o m 1 9 o t h e r i n d i v i d u a l s . T h e
c a rd i n a l i ty o f c l a s se s 0)1 a h d 0 )2 is e q u a l i n o rd e r n o t t o
fa v o u r o n e c l a s s o v e r th e o th e r .
2.3.
T e s t s e t f o r t h e k N N c l as s if ie r
T h e t e s t s e t fo r t h e k N N c la s s i f ie r u s e d in S e c t io n
3 c o n ta in s 1 6 0 e x a m p le s : 2 0 g e n u in e t e s t s ig n a tu re s
c l a s s 0 )1 ) a n d 1 4 0 s ig n a tu re s t a k e n a t r a n d o m f ro m a l l
t h e o th e r w r i t e r s c l a s s 0 )2 ), i n th e s a m e w a y a s fo r t h e
re fe re n c e a n d t r a in in g s e t s . In th i s c a s e , w e d o n o t
in t ro d u c e a n y ro t a t io n in c l a s s 0)1, s in c e i t i s n e c e s s a ry
t o c h e c k t h e c l a s si f i er p e r f o r m a n c e w i t h s t a t i s ti c a l l y
i n d e p e n d e n t s i g n a tu r e s .
2.4. T e s t s e t f o r t h e B P N c l a s s if i e r
T h e t e s t s e t fo r t h e B P N c la s s i f i e r u s e d in S e c t io n
4 c o n ta in s 1 0 5 e x a m p le s : 1 0 g e n u in e t e s t s ig n a tu re s
c l a s s 0 )1 ) a n d 9 5 s ig n a tu re s t a k e n a t r a n d o m f ro m a l l
t h e o th e r w r i t e r s c l a s s
0 2 .
T h e 1 0 e x a m p le s o f c l a s s 0) 1
a re th e f i r st 1 0 s ig n a tu re s o f t h e t e s t s e t o f o n e in d iv id -
u a l . T h e 9 5 e x a m p le s o f c l a s s 0 )2 a re f iv e r a n d o m
e x a m p le s o f t h e l a s t 1 0 s ig n a tu re s o f th e t e s t s e t o f t h e
o th e r 1 9 in d iv id u a l s . T h e c a rd in a l i ty o f c l a ss 0)2 w a s
r e d u c e d t o o f fs e t t h e d i m i n u t i o n i n t h e c a r d i n a l i t y o f
c l a s s 0 )1 , b u t n o t t o o m u c h in o rd e r n o t t o a f f e ct t h e
a c c u r a c y o f t h e p e r f o r m a n c e m e a s u r e .
2.5.
Val ida t ion se t fo r a l l c lass i fi e rs
T h e v a l id a t io n s e t fo r a l l c l a s si f i e r s u s e d in S e c t io n
5 c o n ta in s 1 0 5 e x a m p le s : 1 0 g e n u in e t e s t s ig n a tu re s
c l a s s 0 )1 ) a n d 9 5 s ig n a tu re s t a k e n a t r a n d o m f ro m a l l
t h e o th e r w r i t e r s c l a s s 0 )2 ). T h e 1 0 e x a m p le s o f c l a s s 0) 1
a re th e l a s t 1 0 s ig n a tu re s o f t h e t e s t s e t o f o n e in d iv id -
u a l . T h e 9 5 e x a m p le s o f c l a s s 0 )2 a re f iv e r a n d o m
e x a m p le s o f t h e l a s t 1 0 s ig n a tu r e s o f t h e t e s t s e t o f t h e
o th e r 1 9 in d iv id u a l s .
2.6. P e r f o r m a n c e m e a s u r e s
T h e p e r f o r m a n c e o f e a c h c l a s si f i er is e v a l u a t e d g l o -
b a l ly fo r t h e 2 0 w r i t e r s a n d fo r 2 5 e x p e r im e n t s fo r
w h ic h th e r e fe re n c e , t r a in in g a n d t e s t s e ts a re c h a n g e d
e a c h t im e . T h u s , i n e a c h e x p e r im e n t c l a s s 0 ) t w i l l
a lw a y s b e th e s a m e , b u t c l a s s 0 )2 w i l l b e a lw a y s d i f f e r-
e n t . In th i s w a y , i t i s p o s s ib l e to r e d u c e th e b i a s th a t
c o u l d h a v e b e e n i n t r o d u c e d b y p a r t i c u l a r r a n d o m
f o r g er i e s. C l a s si f i e r p e r f o r m a n c e i s m e a s u r e d b y m e a n s
o f th e to t a l e r r o r r a t e e e x p re s s e d in t e rm s o f e 1 ty p e
I e r r o r r a t e , t h e f a l s e c l a s s i f i c a t io n o f g e n u in e s ig n h -
tu re s) , e : t y p e I I e r ro r r a t e , t h e f a l s e c l a s s i f i c a t io n o f
r a n d o m f o r g er i es ) a n d P [ 0 ) i ] , th e a pr ior i p r o b a b i l i t y
for c lasses 0 ) i , wh ich is se t a t 0 .5 in o ur case :
e t = E 1 X P [0 )1 ] ) + e 2 x P [0 )2 ] ) ). W h e n a p p ro p r i a t e ,
w e u s e th e to t a l r e j e c t io n r a t e R t , w h i c h i s o b t a i n e d
w i t h a n e q u a t i o n s i m i l a r t o t h e l a t t e r o n e w h e r e e r r o r
ra t e s e i a r e s u b s t i t u t e d w i th r e j e c t io n r a t e s R i R 1
r e j e c ti o n o f g e n u i n e s i g n a t u r e s a n d R 2 r e j e c ti o n o f
r a n d o m f o rg e r ie s ). F i n a l l y , i n o r d e r t o t a k e i n t o a c -
c o u n t i n a s in g l e p a r a m e t e r t h e t o t a l e r r o r a n d r e je c -
t ion ra tes , a re l ia b i l i t y fac t o r 161 de f ine d as fo l lows :
R F = 100 - e t - R t ) / 1 0 0 - R t ) i s u s e d to f in d th e b e s t
c o n f i g u r a t i o n o f t h e B P N c l as s if i er .
3 C H O I C E O F PRE-TREATMENTON THE FULL
D I R E C T I O N A L P D F
T h e g o a l o f p r e - t r e a t m e n t o n t h e f u l l d i r e c t i o n a l
P D F i s t o e n h a n c e i t s d i s c r i m i n a t i n g p o w e r , i n o t h e r
w o r d s , t o i m p r o v e t h e p e r f o r m a n c e s o f a c l a s si f i er th a t
u s e s th e P D F a s a n in p u t v e c to r . A c l a s s i f i e r w i l l b e
m o r e e f f ic i e nt if t h e d i m e n s i o n o f t h e i n p u t v e c t o r i s n o t
t o o b i g u n n e c e s s a ry d a t a ) a n d i f t h e v a r i a t i o n s i n t h e
i n p u t v e c t o r a r e n o t t o o a b r u p t n o i s y d a t a ) . U n f o r t u -
n a t e l y , t h e f u l l d i r e c t i o n a l P D F d o e s n o t h a v e t h e s e
c h a ra c te r i s t i c s F ig . 1). T h u s , p re - t r e a tm e n t , i n c lu d in g
f i l te r i n g a n d c o m p r e s s i o n , m u s t b e c a r r i e d o u t . T h e
p u r p o s e o f t h i s s e c t i o n is t o d e t e r m i n e w h i c h i s t h e b e s t
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 3/10
Off-line signature verification 417
0 0 3 0
0 0 2 5
0 0 2 0
0 0 1 5
0 0 1 0
0 0 0 6
0 0 0 0
i
I F u l l P D F
P r e t r e a t e d P D F
i L L
0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0
d e g r e e s
Fig. 1. Full and pretreated (with a 6B7 filter) directional PDF of the handwritten signature shown in the
upper part of this figure.
comb ina tion based on the following heuristic: the
best performance having the higher compression rate
and the shorter computer time .
3.1.
Description of the pre-treatments
The treatments that we have chosen to evaluate
can be classified in three categories. The first, called
the integration filter (I) , consists of making a summa-
tion of the full directional PDF samples inside a win-
dow, whose width is equal to the compression step
which can take the values 1, 2, 3, 4, 5, 6, 9, 10 (A) or
12 (B). The second, call ed the undersamplin f i l ter
(U), consists of making a simple undersampling of
the full directional PD F with the value of the compres-
sion step. The third, called weighted filter, consists
of first smoothing the full directional PDF with
three types of filters: rectangular (R), triangular (T)
and binom ial (B), whose window width can take vari-
ous values: 3, 5, 7, 9 or I I(A), and then perfo rming
a simple undersa mplin g of this smoothed curve. In
each case the process is the same and only the coeffi-
cient number varies with the window width and its
value will depend on the filter used. In addition, to
facilitate the comparisons, the s um of the coefficients is
always equal to one to maintain the window area
constant.
3.2. Experimental protocol
To find the pre-treatment that improves the dis-
criminatingpower of the directional PD F the most, we
used the kN N classifier because it permits the evalu-
ation of a lower limit of the total error rate when the
maxim um available information is kept in memory. 7t
The reference set and the test set needed by a kNN
classifier have been described in Secton 2.2 and 2.3,
respectively. The total error rate e,, as defined in Sec-
tion 2.6, is evaluated for each writer in the signature
database. The mean of these e, is then determined to
obtain the globa l total error (eg,) of the verificat ion
system. This procedure is repeated 25 times with 25
different reference and test sets. Finally , it is the mean
of these 25 egt that is used to choose the best pre-
treatment. In this part of the study, no rejection is
permitted and it is the first mi nimu m distance met that
is taken in to consideration.
3.3. Results analysis
We note a two-fold reduction in the error as soon as
a smoothing is performed. In addition, up to a com-
pression step of 6, the width of the win dow does not
affect the result, while above this the performance is
slightly lower when the width diminishes. Except for
the integration filter, the performance seems to reach
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 4/10
4 18 J .- P. D R O U H A R D et al
a m a x i m u m a t t h i s c o m p r e s s i o n v a l u e . T h e s i m p l e
u n d e r s a m p l i n g , a t t h e o t h e r h a n d , r e d u c e s t h e e r r o r
l e ss a n d e v e n a f t e r a c o m p r e s s i o n s t e p o f 6 t h e e r r o r
i n c r ea s e s. C o n s e q u e n t l y , w e a l r e a d y k n o w t h a t s i m p l e
u n d e r s a m p l i n g is n o t a g o o d p r e - t r e a t m e n t t o a p p l y t o
t h e f u l l d i r e c t i o n a l P D F . F i n a l l y , i n a l l c a se s t h e s t a n -
d a rd d e v ia t io n is l o w e n o u g h (b e tw e e n 0 . 2 a n d 0 .3 5) s o
t h a t w e m a y c o n c l u d e t h a t t h e c l a s s if i c a t io n r e s u lt s a r e
s t a b l e f o r e ac h p r e - t r e a t m e n t s t u d i e d .
F r o m o u r s t a t i s ti c a l a n a l y s i s w e r e t a i n e d t h e i n t e -
g ra t io n f i l t e r w i th a c o m p re s s io n s t e p o f 1 0 , w h ic h
c o r r e s p o n d s v e r y w e l l t o o u r h e u r i st i c . C o n s e q u e n t l y ,
b y a p p l y i n g th i s p r e - t r e a t m e n t o n t h e f u ll d i r e c t io n a l
P D F , w e i n c re a s e i t s d i s c r i m i n a t i n g p o w e r r o u g h l y
tw o fo ld (e t d ro p s f ro m 5 .9 4 to 2 . 6 9 ) a n d , o v e ra l l , w e
re d u c e th e in p u t v e c to r d im e n s io n f ro m 1 8 0 to 1 8. T h i s
i s v e ry im p o r t a n t s in c e i t w i l l g re a t ly f a c i l i t a t e t h e
d e s i g n a n d t r a i n i n g o f t h e B P N c l as s if i e r.
4 . B AC K PR OPA GAT ION NE T WOR K C L ASSIFIE R
4.1.
Backpropagation network BP N )
N e u r a l n e t w o r k s p r e s en t a c o m p u t a t i o n a l p a r a d i g m
f o r c o n s t r u c t i n g c l a s s if i e rs t h a t c a n p e r f o r m a s a c c u -
ra t e ly a s c o n v e n t io n a l t e c h n iq u e s , ca) F o r th e f i r s t st a g e
o f th e A H S V S w e u s e d a c o m p l e t e l y c o n n e c t e d f e e d-
f o r w a r d n e u r a l n e t w o r k w i t h t h e c l a s s ic a l b a c k p r o p a -
g a t i o n l e a r n i n g a l g o r i t h m , 9) m o r e s i m p l y k n o w n a s
t h e B a c k p r o p a g a t i o n N e t w o r k ( B P N ) w h i c h i s d e -
s c r i b e d i n d e t a i l i n m a n y t e x t b o o k s , l° - t 3 ) T o b u i l d
a B P N , t h e r e a r e m a n y p a r a m e t e r s t o c h o o s e f r o m
d e a l i n g w i t h t h e n e t w o r k s i ze o r t h e l e a r n i n g l a w .
U n f o r t u n a t e l y , t h e r e i s n o w a y t o d e t e r m i n e t h e m
r i g o r o u s l y s i n ce t h e y a r e s t r o n g l y d e p e n d e n t o n t h e
a p p l i c a t i o n . T h e f i r s t is t h e n u m b e r o f h i d d e n l a y e r s ,
w h ic h h a s b e e n s e t t l e d to o n e (1 ) s in c e m a n y
a u th o r s l a° -1 3 ) c o n s id e r t h a t a s in g le h id d e n l a y e r i s
s u f fi c ie n t f o r m o s t a p p l i c a t i o n s . T h e n u m b e r o f n e u -
r o n s o n t h e i n p u t l a y e r (N i ) i s 1 8, w h i c h c o r r e s p o n d s t o
t h e d i m e n s i o n o f t h e v e c t o r
F Oi)
f t e r a p p l y i n g t h e b e s t
p r e - t r e a t m e n t f o u n d i n S e c t i o n 3 . 3 . T h e n u m b e r o f
n e u r o n s o f t h e o u t p u t l a y e r ( N o) is t w o , s i n ce w e h a v e
tw o c l a s s e s (to 1 a n d ~ o2 ) a n d w e w a n t to u s e v a r io u s
r e j e c t io n m e t h o d s . I t i s n o t s o e a s y t o f i n d t h e n u m b e r
o f n e u r o n s o n t h e h i d d e n l a y e r ( Nh ) w h o s e u p p e r l i m i t
i s t h e o re t i c a l ly 2 N i + 1 . 12) U p t o n o w , o n ly ru l e s o f
t h u m b h a v e b e e n p r o p o s e d t o d e t e r m i n e N h, a n d w e
h a v e a r b i t r a r i l y s e t t l e d N h = 1 2, a n u m b e r i n c l u d e d
b e t w e en t h e m a x i m u m a n d t h e m i n i m u m p r o p o s e d b y
t h e v a r i o u s m e t h o d s . L a t e r , w e h a v e a d j u s t e d t h i s
n u m b e r b y e x a m i n i n g i t s i nf l u e n ce o n t h e g l o b a l p e r -
f o r m a n c e o f t h e A H S V S ( s e e S e c t i o n 4.4 ). C o n c e r n i n g
t h e l e a r n i n g la w , t h e r e a r e t w o p a r a m e t e r s t o c h o o s e :
t h e l e ar n i n g ra t e r / a n d t h e s m o o t h i n g r a t e o r m o m e n -
tu m 0t. A g a in , t h e re i s n o w a y to f in d a r ig o ro u s v a lu e
f o r t h e s e p a r a m e t e r s . M o r e o v e r , t h e e m p i r i c a l r u l es
p r o p o s e d a r e o f t e n c o n t r a d i c t o r y . C o n s e q u e n t l y , a f t e r
a f e w p r e l i m i n a r y t r i al s , w e a r b i t r a r i l y d e c i d e d t o s e t tl e
= 0 . 6 a n d ct = 0 .0 . In o rd e r to f a c i l i t a t e t h e s t a r t o f t h e
t r a i n i n g p h a s e , t h e w e i g h t s s h o u l d b e i n i t i a l i z e d t o
s m a l l r a n d o m v a lu e s , . 3' 14 ) a n d th e b i a s t e rm s h o u ld
b e u s e d t o a v o i d a s a t u r a t i o n o f th e o u t p u t o f t h e
n e u ro n s 3 s '1 3 ) W e th e re fo re u s e d a b i a s t e rm fo r t h e
h i d d e n a n d o u t p u t l a y e r s a n d a l l w e i g h t s w e re i n -
i t i a l iz e d r a n d o m l y i n t h e - 0 . 1 t o 0 .1 r a n g e . O n e l a s t
w a y t o i m p r o v e t h e c o n v e r g e n c e t i m e o f t h e B P N
d u r i n g t h e t r a i n i n g p h a s e i s t o n o r m a l i z e t h e i n p u t
v e c to r s b e tw e e n 0 a n d 1 . ~ 5) T h i s i s p a r t i c u la r ly t ru e
w h e n th e d a ta v a lu e s a re v e ry s im i l a r . S in c e th i s i s s o in
o u r c a s e , w e h a v e n o r m a l i z e d a l l t r a i n i n g , t e s t a n d
p e r f o r m a n c e s e ts u s e d b y t h e B P N .
W i t h t h e B P N , t h e t r a i n i n g p h a s e i s c r i t ic a l , e s -
p e c ia l ly w h e n th e d a t a to b e c l a s s if i e d a re n o t c l e a r ly
d i s t i n g u i s h a b l e a n d w h e n t h e r e a r e n o t e n o u g h
e x a m p l e s t o c o n d u c t t r a i n i n g . I n t h i s c a se , t h e t r a i n i n g
p h a s e c a n b e v e r y lo n g a n d i t m a y e v e n b e i m p o s s i b l e
t o o b t a i n a n a c c e p t a b l e p e r f o r m a n c e . S i n c e t h i s is t h e
c a s e fo r o u r a p p l i c a t io n (f ew s ig n a tu re s w i th h ig h
v a r i a b i l i t y ) : f i r s t, w e h a v e d e f in e d a c r i t e r io n fo r s to p -
p in g th e t r a in in g p h a s e (S e c t io n 4 .2 ); s e c o n d , w e h a v e
e v a l u a t e d s e v e r a l r e j e c t i o n m e t h o d s t o i m p r o v e t h e
d e c i s io n t a k e n b y th i s t y p e o f c l a s s i fi e r (S e c t io n 4 .3 );
f i n al l y, w e h a v e a d j u s t e d t h e n u m b e r o f n e u r o n s i n
t h e h i d d e n l a y e r o f t h e B P N i n o r d e r t o i n c r e a s e t h e
g l o b a l p e r f o r m a n c e o f t h e f i r s t s t a g e o f t h e A H S V S
(Sec t io n 4 .4) .
4.2. Stopping criterion
I t i s w e ll k n o w n t h a t t h e t r a i n i n g p h a s e i s c r u c i al i n
t h e d e s i g n o f a B P N c l as s if i er . T h e m a j o r d i f f i c u lt y s t o
d e c i d e o n w h a t b a s i s t o s t o p t r a i n i n g . F o r t h e B P N w e
c a n u s e th e e r ro r v a lu e e ( th e d i f f e re n c e b e tw e e n th e
d e s i r e d o u t p u t a n d t h e a c t u a l o u t p u t. ) t o s t o p t r a i n i n g
w h e n t h i s i s l o w e r t h a n a p r e - e s t a b l i s h e d l i m i t ( id e a l l y
0 ) f o r a l l e x a m p l e s i n c l u d e d i n t h e t r a i n i n g s e t. H o w -
e v e r , i t is n o t a lw a y s p o s s ib l e to r e a c h th i s s to p p in g
c r i t e r io n , e sp e c i a l ly w h e n t h e i n p u t d a t a a r e n o t c l e a r l y
s e p a r a t e d . A v a r i a n t o f t h i s m e t h o d , w h i c h t a k e s t h i s
p r o b l e m i n t o a c c o u n t , i s t o s t o p t r a i n i n g w h e n t h e
R o o t M e a n S q u a r e ( R M S ) e r r o r o n t h e t r a i n i n g s e t i s
lo w e r th a n a f ix e d th re s h o ld ( id e a l ly 0 ). T h i s v e ry
p o p u l a r m e t h o d w a s n o t s e l e ct e d b e c a u s e i t i s n o t w e l l
s u i t e d t o o u r a p p l i c a t i o n . In d e e d , i t is n o t a b s o l u t e l y
n e c e s s a r y f o r u s t o h a v e a s m a l l R M S e r r o r t o o b t a i n
a n a c c e p t a b l e p e r f o r m a n c e o f t h e c l a ss i fi e r. I n o t h e r
w o r d s , w e d o n o t w a n t t h e B P N t o l e a r n t o g i ve t h e
e x a c t o u t p u t l e ve l b u t t o m a k e a g o o d d e c i s i o n . T h i s
m a y b e d o n e w e l l b e f o r e t h e R M S e r r o r b e c o m e s w e a k .
C o n s e q u e n t l y , w e h a v e b a s e d o u r s t o p p i n g c r i t e r i o n
o n th e p e r f o rm a n c e m e a s u re e d e f in e d in S e c t io n 2 .6 .
T h u s , t h e n e t w o r k w e i g h t s a r e a d j u s t e d f o r e a c h
e x a m p l e o f t h e t r a i n i n g s e t a n d o n c e a l l e x a m p l e s h a v e
b e e n p r e s e n t e d t o t h e n e t w o r k ( l a t e r r e f e r r e d t o a s
a p re s e n ta t io n ) , w e f r e e z e th e w e ig h t s a n d w e e v a lu a te
e o n th i s t r a in in g s e t ( l a t e r r e fe r r e d to a s etm . In th i s
w a y , w e m e a s u r e t h e m e m o r i z a t i o n p e r f o r m a n c e o f th e
B P N c l as s if i er . H o w e v e r , t h e f i r s t o b j e c t iv e o f th e B P N
c l a ss i f ie r i s t o h a v e a g o o d g e n e r a l i z a t o n p e r f o r m a n c e ,
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 5/10
Off-line signature verification 419
v
a
i
. Q
O
r -
ig
I M e a n
11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M e a n + 1- S D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
/
8 . . . . . . . . . t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
n
7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . l . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
i. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . n e r a z a ii o n . . . . . . . . . .. . . . .
2
0 . . . .. . . .. . . . .. . . . .. . . .. . . . .. . . . .. . . .. . . . .. . . . .. . . . .. . . .. . . . .. . . . .. . . .. . . . .. . . . Y . . . .
I I I I t I I I I I I I I I I I i I I I i I ]
0 1 0 0 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0 8 0 0 9 0 0 1 0 0 0
N u m b e r o f p r e s e n t a t i o n s ( N B P )
Fig . 2 . M ean g loba l er ror i n m emo r i za t i on e tm) and g enera l i za t i on e ta) as a func t i on o f the numb er o f
presenta t i ons NBP) . T he bo ld l i ne shows the mean va lue and the fi ne li nes on both s ides show the s tanda rd
dev ia t i on SD) .
that is, on examples not seen duri ng the train ing phase.
It is thus logical to stop traini ng on the ge neralization
perfo rmance measure of the BPN classifier. To do this,
we evaluate e on the test set (late r referred to as etg).
The drawback of the BPN, which is known as the
overtraining phenomenon ,~12) is that we must stop
training when
8 t g
goes through a global min imum. We
have not retained this criterion since preliminary re-
suits showed that for our data et, was quite noisy and
the curve was relatively fiat after a few hundre d presen-
tations, and that the overtraining phenom enon was
often imperceptible even after man y tho usand s of pres-
entations. Und er these conditions , it would be best to
stop training as soon as 8tm and etg are both almost
stable (i.e. when the slope of the curve is very weak).
However, this method needs a great deal of compu ting
time and is very unstabl e when the data are noisy. For
these reasons, we decided to stop training when ~,g is
stable for all 20 writers. This consists of finding the
num ber of pres entations (NBP = Tp) after which the
generalizatio n performance for each writer is not sig-
nificantly mproved with l onger training. Nevertheless,
in some cases the training phase could be greatly
reduced if we stopped it when the memorization and
generalization performances are acceptable, that is,
when etm < T m and etg < Tg. The thresholds T m and Tg
must be settled to 0, since we do no t wan t to impose
a limit on the BPN classifier's performance in either
memori zatio n or in generaliza tion. Tp, the maxim um
numb er of presentations used in the train ing phase, is
determined by means of an experimental protocol.
Figure 2 illustrates the results found with this experi-
ment al protocol. Since the ex pone ntia l etg curve is very
noisy, we smooth ed it before calculating its slope by
curve-fitting . We have tried the two following slope
values 10- a and 10 _4 and the resulting Tp values were
250 and 650, respectively. As we can see in Fig. 2, the
first value is quite low and the second is a little too
high. Thus, we have set Tp at 500 to o bta in a sufficient-
ly stable curve without excessive co mput atio n time. In
conclusion, the st opping criterion used in this study is
stated as follows: the tra ining phase will be stopped
when
gtm=Tm=0) a n d
/ ; t g : T g = 0 ) ) o r when
(NBP > Tp = 500) .
4.3.
ejection criterion
For systems not requiring an immediate decision,
the addit ion of a rejection criterion to the decision rule
allows significant improvement in classifier perform-
ance by re fusing to classify doubtful cases, t16'17) How-
ever, even if the cost of a rejection is lower tha n tha t of
an error, 16~ the rejection rate mu st be as weak as
possible for two reasons. The first is that we can also
reject good decisions and if the rejection rate of good
decisions becomes higher than that of bad decisions,
the classifier's performance will be decreased ins tead of
increased. The second con cerns classifier utility. In-
deed, in an extreme case the classifier can have an error
rate of 0% but a re jection rate of 100%. In this case, the
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 6/10
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 7/10
Off-line signature verification 421
p e r f o r m a n c e o f th e B P N c l a s si f i er w h e n t h e t h r e s h o l d
v a l u e s i n cr e a se . T h i s b e t t e r p e r f o r m a n c e , c o r r e s p o n d -
i n g t o a h i g h e r r e l i a b i l i t y ( R F ) a n d a l o w e r t o t a l e r r o r
ra t e ( e, ), i s o b ta in e d a t t h e e x p e n s e o f a t o t a l r e j e c t io n
ra t e (R 0 th a t i s c l e a r ly h ig h e r . A s s e e n in F ig . 3 , u n l ik e
R C 2 a n d R C 3 , R C 1 b e c o m e s e f f e ct iv e o n l y w h e n t h e
th re s h o ld T M i s h ig h e r th a n 0 . 5 . F ro m F ig . 3 w e c a n
a l s o c o n c l u d e t h a t R C 1 , R C 2 a n d R C 3 a r e e q u i v a le n t
w h e n a p p l i e d t o o u r d a t a . T h e r e s u l ts d e a l i n g w i t h
R C 5 a n d R C 6 s i m p l y s h o w t h a t t h e a d d i t i o n o f R C 2 o r
R C 3 to R C 1 i s o n ly e ff e c tiv e w h e n th e th re s h o ld T A o r
T R~ i s h ig h e r t h a n a v a lu e fo r w h ic h i t a l lo w s a b e t t e r
p e r f o r m a n c e t h a n t h a t a l r e a d y o b t a i n e d w i t h t h e
th re s h o ld TM ( se e, fo r e x a m p le , R C 5 in F ig . 3 ). T h i s
m e a n s t h a t b e l o w t h i s p a r t i c u l a r t h r e s h o l d T A o r T R y,
t h e r e j e c ti o n c r i t e r i a R C 1 a n d R C 2 o r R C 3 r e j ec t
e x a c t ly th e s a m e e x a m p le s . A l l t h e s e r e s u l t s c a n e a s i ly
b e e x p l a i n e d b y t h e f a c t t h a t o u r o u t p u t l e v el s a r e
c o m p l e m e n t a r y .
I n c o n c l u s i o n , w e c a n s a y t h a t , d u e t o t h e o u t p u t
c o m p l e m e n t a r i t y , t h e r e j e c ti o n c ri t e r i a R C 1 , R C 2 a n d
R C 3 a r e s t r i c t l y e q u i v a l e n t a n d t h e r e j e c t i o n c r i te r i a
R C 5 a n d R t ~ 6 a r e o n l y u s e fu l w h e n w e w a n t t o i m p o s e
a l o w e r l i m i t o n t h e B P N c l a ss i f ie r p e r f o r m a n c e v i a
R C 1 . C o n s e q u e n t l y , t h e r e j e c t io n c r i t e r io n R C 1 h a s
b e e n c h o s e n o n ly b e c a u s e o f it s p ra c t i c a l a s p e c t ( l e ss
c o m p u t a t i o n t i m e ) . W e n o w h a v e t o d e t e r m i n e t h e
t h r e s h o l d T M , b u t t h i s d e t e r m i n a t i o n m u s t b e m a d e
w i t h c a u t i o n . A c a r e f u l e x a m i n a t i o n o f o u r r e s u lt s
p r o v e s t h a t t h e a p p l i c a t i o n o f a r e j e c t i o n cr i t e r i o n m a y
d e g r a d e B P N c l a ss i fi e r p e r f o r m a n c e a n d a l s o i n d i c at e s
t h a t t h e t h r e s h o l d v a l u e m u s t b e h i g h en o u g h , b u t n o t
t o o h i g h . A s a l r e a d y m e n t i o n e d , B P N c l a ss i f ie r p e r -
f o r m a n c e i n c r e a se s a t t h e e x p e n s e o f a n i n c r e a s e i n t h e
r e j e c ti o n r a t e . C o n s e q u e n t l y , w e c h o s e t h e t h r e s h o l d
T M a c c o r d i n g t o t h e f o l l o w i n g h e u ri s t ic : o b t a i n t h e
b e s t p e r f o r m a n c e w i t h a n a c c e p t a b l e r e j e c t i o n r a t e .
F o r o u r A H S V S a p p l i c a t i o n a r e j e ct i o n ra t e o f 5 %
c o u l d b e c o n s i d e r e d a s a c c e p t a b l e . I n t h i s c a s e f r o m
F ig . 3 w e f in d a v a lu e o f 0 . 9 4 fo r t h e th re s h o ld T u . In
a d d i t i o n , o u r r e s u l t s s h o w t h a t t h e i n t r o d u c t i o n o f
a r e j e c ti o n c r i t e r i o n i m p r o v e s t h e r e l i a b i l i ty ( R F ) o f t h e
A H S V S s l i g h t ly , b u t s h a r p l y d e c r e a s e s it s t o t a l e r r o r
ra t e ( e , ) v i a th e ty p e I e r ro r r a t e ( e l ) .
4.4. Backpropagation network BPN) optimization
E s s e n t i al l y , t h e r e a r e t w o w a y s t o o p t i m i z e a B P N :
(1) w e c a n a d ju s t t h e l e a rn i n g a lg o r i th m l s -2 ~ ) a n d (2)
w e c a n m o d i fy i t s s t ru c tu r e b e fo re z 2-25 ~ o r d u r -
i n g 26 -2 8) th e t r a i n i n g p h a s e . W e o n l y p e r f o r m e d t h e
B P N o p t i m i z a t i o n o n t h e s t r u c t u r a l a s p e c t o f t h e
n e t w o r k . A t t h e t i m e t h e B P N w a s d e f in e d , w e m e n -
t i o n e d t he i m p o r t a n c e o n B P N p e r f o r m a n c e o f t h e
h id d e n n e u ro n s ( s e e S e c t io n 4 .2 ). U n fo r t u n a te ly , t h e re
a r e n o t h e o r e t i c a l m e a n s f o r f in d i n g th e o p t i m u m
n u m b e r o f h id d e n n e u r o n s ( N h o) . H o w e v e r , w e k n o w
i t s u p p e r l im i t , w h ic h i s
N h m =
2 N i + 1~2~ = 37, in our
c a s e . C o n s e q u e n t ly ,
Nho
m u s t b e f o u n d b y e x p e r i m e n -
t a t i o n i n w h i c h N hv a r i e s g r a d u a l l y b e t w e e n 0 a n d Nhm
T h e r e s u l ts o f th i s e x p e r i m e n t a t i o n a r e s h o w n
g ra p h ic a l ly in F ig . 4. F ro m th e s e r e s u l t s , w e c a n s e e
t h a t t h e B P N p e r f o r m a n c e s c l e a r l y i m p r o v e a s s o o n a s
w e h a v e a h i d d e n l a y e r a n d t h a t , f o r o u r d a t a b a s e , t h e
e ff ec t o f t he n u m b e r o f h i d d e n n e u r o n s i s n o t v e r y
p r o n o u n c e d . A s t a t i s t i c a l a n a l y s i s s h o w s t h a t t h e c o n -
f i g u r a t i o n w i t h e i g h t h i d d e n n e u r o n s i s t h e b e s t b e -
c a u s e i t h a s a l o w e r e r ro r r a t e (~ t= 1 -2 4%) a n d
re j e c t io n r a t e (R = 4 . 5 0 6 %) a n d a h ig h e r r e l i a b i l i t y
fa c to r (R F = 0 .9 8 7) . T h u s , t h e e ig h t -h id d e n -n e u ro n
c o n f i g u r a t i o n w a s u s e d f o r t h e c o m p a r a t i v e s t u d y
d e s c r ib e d in th e d i s c u s s io n .
5 D I S C U S S I O N
I n o r d e r t o e v a l u a t e t h e b e h a v i o u r o f t h e B P N
c l a ss i f ie r m o r e a c c u r a t e l y , w e c a r r i e d o u t a c o m p a r a -
t iv e s tu d y o f t h e k N N c la s s if i e r , t h e T c l a s s i f i e r a n d th e
B P N c l a ss i f ie r o n t h e s a m e d a t a s e t s. T h e k N N a n d
B P N c la s s if i e r s u s e a n e w d e f in i t i o n o f t h e d a ta s e t s , a s
def ined in Sec t ions 2 .2 , 2 .4 and 2 .5 . There fore , the re is
n o c o m p a r i s o n b e t w e e n p e r f o r m a n c e s o b t a i n e d n o w
i n t h i s c o m p a r a t i v e s t u d y a n d t h o s e o b t a i n e d p r e v i -
o u s ly in S e c t io n 3 fo r t h e k N N c la s s i f ie r a n d in S e c t io n
4 fo r t h e B P N c la s si f i e r. T h e T c l a s s i f i e r o n th e o th e r
h a n d , e x c e p t f o r t h e v a l i d a t i o n s e t w h i c h i s r i g o r o u s l y
th e s a m e fo r a l l c l a s s if i e r s , u s e s th e fo l lo w in g r e fe re n c e
a n d t e s t s e ts . T o b u i ld th e T r e fe re n c e se t , w e s e l e c t a t
r a n d o m a f ew e x a m p l e s f r o m t h e f i rs t 20 g e n u i n e
s i g n a t u r e s a n d a p p l y a r o t a t i o n a s i n S e c t io n 2 .2 . F r r
t h i s st u d y , t h e n u m b e r o f e x a m p l e s b e f o r e r o t a t i o n f o r
th e T r e fe re n c e se t w a s v a r i e d b e tw e e n 1 a n d 1 0 w i th a n
in c re m e n t o f 1 , t h u s fo rm in g 1 0 T c l a s s i f i e rs . A s fo r t h e
T t e s t s e t , w h ic h i s u s e d o n ly to f in d th e th re s h o ld
va lue , c las s ~o1 c o m p r i s e s t h e r e m a i n i n g e x a m p l e s o f
th e f i r st 2 0 g e n u in e s ig n a tu re s a n d c l a s s ~ 2 c o m p r i s e s
f iv e r a n d o m e x a m p le s o f t h e f i r s t 2 0 s ig n a tu re s o f t h e
1 9 o th e r i n d iv id u a l s . C o n s e q u e n t ly , c l a s s ~ o2 a l w a y s
h a s th e s a m e d im e n s io n (9 5 ), b u t c l a s s c o1 h a s a d i m e n -
s i o n t h a t v a r i e s a c c o r d i n g t o t h e n u m b e r o f e x a m p l e s
a l r e a d y p ic k e d u p fo r t h e T r e fe re n c e s e t.
B e f o re d o i n g t h e c o m p a r a t i v e s t u d y , w e h a v e t o f i n d
th e b e s t T c l a s s i f ie r a m o n g th e 1 0 w e h a v e s im u la t e d .
B e tw e e n th re e a n d s e v e n e x a m p le s , t h e d i f f e ren c e i s n o t
v e r y p r o n o u n c e d a n d w e h a v e c h o se n t h a t w h i c h g i v e s
t h e b e s t p e r f o r m a n c e , t h a t i s, fi v e e x a m p l e s t o b u i l d t h e
T re fe re n c e se t . N o w , i t i s t h i s b e s t T c l a s s i f i e r t h a t w i l l
b e u s e d f o r t h e c o m p a r i s o n . A s e x p e c t e d , t h e B P N
c la s s i f i e r g iv e s a r e s u l t ( et = 3 . 2 2 %) th a t l i e s b e tw e e n
th e b e s t g iv e n b y th e k N N c la s s i f ie r ( et = 1 . 6 8 %) a n d
th e w o rs t g iv e n b y th e T c l a s s i f ie r ( et = 5 . 6 1 %) . T h i s i s
a l s o t ru e fo r t h e s t a b i l i t y o f t h e d e c i s io n w h ic h i s
r e f l ec t e d i n t h e s t a n d a r d d e v i a t i o n . O u r r e s u l ts c l e a r l y
s h o w t h a t a l l cl a ss i f ie r s a r e m o r e u n s t a b l e b e t w e e n
w r i t e r s t h a n b e t w e e n e x p e r i m e n t s . T h is h i g h s t a n d a r d
d e v i a t io n v a lu e in d ic a t e s th e d i f f i c u l ty th a t t h e c l a s s i -
f i e r s h a v e in id e n t i fy in g s o m e w r i t e r s . In th e tw o c a s e s ,
t h e B P N c l a ss i fi e r i s c l o s e r t o t h e k N N c l a s si f i er t h a n
to the T c lass i f ie r .
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 8/10
4 J .-P . D R O U H A R D e t a l
10
i th reject ion
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 . 0 0
. . . . . . . . . . . . . . . . . . . .
• RF 0 96
. . . . . . . . . . . . . . . . . . ' . . . . . . . . . . . . . . . . . . . . . . .
0 94
I i i i I I i i I I i i 0 . 9 0
0 4 8 12 16 20 24 28 32 36
Nh
u l
10
i thout reject ion
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . 1 . 0 0
~ I I L T I I I
l : / r i 2 1 1
± z
1 9 2
i , I , I , . . . . . • • 1 . 9 0
0 4 8 12 16 20 24 28 32 36
N h
Fig. 4. Effect of the numb er of hidden n u r o n s Nh) on the performance E, R, and R F) of the BP N evaluated
without a rejection criterion or with the rejection criterion RC 1 with a threshold level T , of 0.94.
6. CONCLUSIONS
T h e m a i n o b j e c t i v e o f t h i s w o r k w a s t o d e t e r m i n e
w h e t h e r o r n o t a B P N c l a ss i f ie r c o u l d b e u s e d i n t h e
d e s i g n o f t h e f i r s t s t a g e o f a c o m p l e t e A u t o m a t i c
H a n d w r i t t e n S i g n a t u r e V e r i f i c a t i o n S y s t e m A H S V S ) .
T o d o t h i s, w e h a v e c h o s e n t h e d i r e c t i o n a l P r o b a b i l i t y
D e n s i t y F u n c t i o n P D F ) a s a g l o b a l s h a p e f a ct o r a n d
t h e c o m p l e t e l y c o n n e c t e d f e e d - f o r w a r d n e u r a l n e t -
w o r k w i t h t h e c l a s si c a l b a c k p r o p a g a t i o n l e a r n i n g a l -
g o r i t h m r e f e r re d t o a s t h e B a c k p r o p a g a t i o n N e t w o r k
B P N ) .
H o w e v e r , th e d i m e n s i o n 1 80 ) o f t h e P D F i s t o o
l a r g e t o b e p r o p e r l y m a n i p u l a t e d b y a B P N c l as s if i er .
C o n s e q u e n t l y , i n a f ir s t a t t e m p t a n d b y m e a n s o f th e
k N e a r e s t N e i g h b o u r k N N ) c l as s if i e r, w e h a v e d e te r -
m i n e d t h e p r e - t r e a t m e n t t h a t i m p r o v e s t h e c l a s si f ic a -
t i o n w h i l e d e c r e a s i n g t h e d i m e n s i o n o f t h e i n p u t
v e c to r . T h e r e s u l t s i n S e c t io n 3 s h o w th a t t h e p re -
t r e a t m e n t w i t h a n i n t e g r a t i o n f i l te r w i th a s t e p o f t en
g a v e th e b e s t r e s u l t .
I n t h e c a s e o f t h e B P N c l a ss i fi e r, t h e t r a i n i n g p h a s e i s
c ru c ia l a n d d i f f i c u l t t o c o n t ro l . T h e m a jo r d i f f i c u l ty i s
to d e c id e w h e n to s to p t r a in in g . In S e c t io n 4 . 2 , w e
d e f i n e d a s t o p p i n g c r i t e r i o n b a s e d o n a m e a s u r e o f t h e
p e r f o r m a n c e o f t h e B P N c l a ss i f ie r b o t h i n m e m o r -
i z a t io n e m ) a n d in g e n e ra l i z a t io n e t, ), a s w e l l a s o n
a m a x i m u m n u m b e r o f p re s e n t a ti o n s N B P ) o f t h e
t r a i n i n g s e t. U s i n g a n e x p e r i m e n t a l p r o t o c o l , w e d e -
c i d e d t h a t t h e t r a i n i n g p h a s e w o u l d b e s t o p p e d w h e n
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 9/10
Off-line signature verif icat ion 423
(e tm = 0 a n d e ts = 0 ) o r w h e n ( N B P > 5 0 0) . S t a t i s t i c a l
a n a l y s i s c a r r i e d o u t l a t e r , d u r i n g t h e e v a l u a t i o n o f t h e
p e r f o r m a n c e o f t h e B P N , s h o w e d t h a t t h e m a j o r i t y
( 8 1 ) o f t h e n e t w o r k s s t o p p e d t h e i r tr a i n i n g o n t h e
s e c o n d c o n d i t i o n a n d 1 4 . 2 o f t h e m s t o p p e d th e i r
t r a i n i n g o n t h e f ir s t c o n d i t i o n w i t h f ew e r t h a n 1 0 0
p r e s e n t a t i o n s .
I n o r d e r t o i m p r o v e t h e r e l ia b i l it y o f th e B P N
c l a ss i fi e r, w e h a v e i n t r o d u c e d a r e j e c t i o n c r i t e r i o n i n
t h e d e c i s i o n r u l e . I n S e c t i o n 4 .3 , w e s h o w e d t h a t i n a l l
c a s e s t h e a d d i t i o n o f a r e j e c t io n c r i t e r i o n s h a r p l y
d e c r e a s e d t h e t o t a l e r r o r r a t e a n d s l i g h t l y i n c r e a s e d t h e
r e l i a b i l i t y a t t h e e x p e n s e o f a r e j e c t i o n r a t e t h a t i s
s o m e t i m e s h i g h . D u e t o t h e f a c t t h a t o u r d a t a a r e
c o m p l e m e n t a r y , al l t h e r e j e c t i o n c r i t e r ia p r o p o s e d a r e
s i m i l a r . T h u s , w e h a v e c h o s e n t h e e a s i e s t o n e ( R C 1 )
w i t h a t h r e s h o l d ( T u --- 0 . 9 4 ) g i v i n g a n a c c e p t a b l e r e j e c -
t i o n ra t e ( 5 ) f o r o u r a p p l i c a t i o n .
I1: s w e ll k n o w n t h a t t h e n u m b e r o f n e u r o n s o n t h e
h i d d e n l a y e r p l a y s a n i m p o r t a n t r o l e o n B P N p e r f o r m -
a n c e . I n S e c t i o n 4 .4 , w e e v a l u a t e d t h i s r o l e b y m e a s u r -
i n g B P N p e r f o r m a n c e w i t h a n d w i t h o u t t h e r e j e c ti o n
c r i t e r i o n p r e v i o u s l y d e f i n e d w h e n t h e n u m b e r o f
h i d d e n n e u r o n s i s v a r i e d . I n o u r c a s e , t h e a d d i t i o n o r
n o t o f th e r e j e c t i o n c r i t e r i o n d o e s n o t s i g n i f i c a n t l y
a f fe c t t h e n u m b e r o f h i d d e n n e u r o n s t h a t g i v e s t h e b e s t
p e r f o r m a n c e . A s t a t i s t i c a l a n a l y s i s o n o u r d a t a h a s
s h o w n t h a t t h e b e s t c o n f i g u r a t i o n w a s w i t h e i g h t
h i d d e n n e u r o n s .
F i n a l l y , i n o r d e r t o b e t t e r a s s es s t h e p o t e n t i a l o f t h e
B P N c l a ss i fi e r, w e m a d e a c o m p a r a t i v e s t u d y o f t h is
o n e ', t h e k N N c l as s if i er , w h i c h s h o u l d g i v e a n u p p e r
l i m i t f or t h e p e r f o r m a n c e , a n d t h e T c l a ss i fi e r, w h i c h i s
v e r y p o p u l a r i n s p i t e o f i t s f ai l in g s . S i n ce t h e s e l a t t e r d o
n o t u s e a r e j e c t i o n c r i t e r i o n i n m a k i n g t h e i r d e c i s i o n ,
w e h a v e u s e d , in S e c t i o n 5 , a B P N c l a ss i fi e r w i t h o u t t h e
r e j e c t i o n c r i t e r i o n p r e v i o u s l y d e f i n e d . I n e f f e c t , t h e
B P N c l a ss i fi e r b e h a v e s m u c h b e t t e r t h a n t h e T c l a s si -
f ie r a n d a l i t t le le s s w e ll t h a n t h e k N N c l a s s i f i e r .
I n c o n c l u s i o n , w e c a n s a y t h a t , o n c e t r a i n e d , t h e
B P N c la s si fi e r c o m p a r e s f a v o u r a b l y w i t h t h e k N N
c l a ss i fi e r s i n c e i t h a s a l m o s t t h e s a m e p e r f o r m a n c e b u t
w i t h a s h o r t e r r e s p o n s e t i m e a t g e n e r a l i z a t i o n , e s -
p e c ia l ly i f w e u s e a h a r d w a r e i m p l e m e n t a t i o n o f t h e
B P N c l as s if ie r . H o w e v e r , f o r o u r a p p l i c a t i o n t h e d r a w -
b a c k o f t h e B P N c l a ss i fi e r i s t h e w a y i n w h i c h i t d e a l s
w i t h t h e t r a i n i n g p h a s e a n d t h is c o u l d u n d e r m i n e t hi s
c o n c l u s i o n s o m e w h a t .
A c k n o w l e d g e m e n t s - - T h i s work was suppor ted in pa r t by
a PSIR gran t f rom the Eco le de t echno log ie supr r ieu re to
Jean-P ie r re Drouhard and Rober t Sabour in , and by g ran t
O G P 0 1 0 6 4 5 6 t o R o b e r t S a b o u r i n f ro m t h e N S E R C o f C a n a -
da . Mas te r ' s s tuden t M ar io Go dbou t , who took pa r t in th i s
research project , a lso received a scholarsh ip from the E cole de
technologic sup~rieure.
R E F E R E N E S
1 . R . Sabour in , Une Approache de Type Comp rrhens ion de
Scrne Appl iqure au Prob l rm e de l a Vr r i f i ca tion Auto-
mat ique de l ' Iden t i t6 pa r l ' Imag e de l a S igna tu re Ma nus-
crite . Ph.D. Thesis , l~cole Polytec hniqu e de M ontr ral
(1990).
2. R. Sabou rin, R. Pla m on do n an d G. Loret te , Off- line
identif icat ion with ha ndw rit ten signa ture images: survey
and perspectives,
Structured Document Image Analysis.
H. S. Baird, H. B unke an d K. Y ama mo to, eds., pp. 219
234, Springer-Verlag, Ge rm any (1992).
3. W. R. Harrison,
Suspect Docum ents Their Scientific
Examination.
Nelso n-Ha ll Publishers , Ch icago ( 1981).
4 . G . Pou l in , Pe rsona l Com mun ica ton , Labora to i re Jud i -
caire de la GRC, 4229 boul . Dorchester guest , Wes-
tmount , QC, H3 Z 2T4 .
5 . R . Sabour in , J. -P . Drou hard , L . Gagn6 and N. Paque t ,
Autom at ic handw r i t t en s igna tu re ve r i f ica t ion us ing d i -
rect ional PDF and neural networks: a feasibi l i ty s tudy,
Proc. Conf. Exhibit. lndust. Automat.
Montr ra l , 20 . 17-
20. 20 (1992).
6 . L . Xu , A . Krzyzak and C . Y . Suen , Methods o f combin ing
mu lt iple classifiers and their applicat io ns to hand writ in g
recognit ion,
IEEE Trans. Syst . Man Cybernet.
22, 418
435 (1992).
7 . K . Fuku naga ,
Introduction to Statistical Pattern Recogni-
tion 2nd ed n. Academic Press Inc., New Y ork (1990).
8. D. J. Burr , Experim ents on neu ral net recognit ion of
spoke n and w ri t ten text ,
I EE E Trans. Acoust. Speech Sig.
Process.
36, 1162-1168 (1988).
9. D. E. Rumelhart , G. E. Hinton and R. J . Will iams,
Learn ing in te rna l r ep resen ta tion by e r ro r p ropaga t ion , in
Parallel Distributed Processing
D. E . Rum elhar t and J . L .
McC lelland, eds, Vol. 1 , pp. 318 362. M IT Press, M ass-
achussetts (1986).
10. R . P . L ippm ann , An in t roduc t ion to com put ing wi th
neura l ne t s ,
I E E E A S S P M a g .
4-22 (April 1987).
11. P. D. Wasserman,
Neural Computing: Theory and Prac-
tice. Van N os t rand Re inho ld , New York (1989).
12. R. Hecht-Nielsen,
Neurocomputing.
Addison-Wesley,
New York (1990).
13. J . A. Freeman and D. M. Skapura,
Neural Networks:
Algorithms Applications and Program ming Techniques.
Addison-Wesley, New York (1991).
14. J . Hertz , A Krogh and R. G. Palmer,
Introduction to the
Theory o f Neural Computation. Addison-Wesley, New
York (1991).
15. A . J . M aren , D . Jones and F . F ran k l in , Conf igur ing and
op t imiz ing the ba ck-propaga t ion ne twork ,
Handbook of
Neural Computing Applications.
Academic Press, San-
tiago (1990).
16. M. Nadler and E. P. Smith,
Pattern Recognition Enoi-
neering Joh n W iley & Son, New York (1993).
17. B. Dubuisson,
Diagnostic et reconnaissance des formes.
Hermes, Paris (1990).
18. A .V. Ooyen and B . Ne inhu is , Improv in g he convergence
of the backpropag a t ion a lgor i thm,
Neura l Ne tworks
5,
465-471 (1992).
19. T. P. Vogl, J . K. Mangis, A. K. Rigler , W. T. Zink and
D. L. Alkon, Accelerat ing the convergence of the
back p ropaga t ion method ,
Biol. Cybernet.
59, 257 263
(1988).
20. R. A. Jacobs, Increased rates of convergence throug h
lea rn ing ra te adap ta t ion ,
Neura l Ne tworks
l , 295-307
(1988).
21 . N . Baba , A new approach fo r f ind ing he g loba l min imu m
of error fu nction of neur al networks,
Neural Networks 2
367 373 (1989).
22 . S . J . Marsha l l and R . F . Har r i son , Opt im iza t ion and
t ra in ing o f the feedforward neura l ne tworks by gene t ic
algori thms, 2nd Int. Conf. Artific. Neural Networks 349
39 43 (1991).
23. S. Akaho and S. Amari , On the capacity of three-layer
ne twork ,
Proc. Int . Joint. Conf. Neural Networks
3, 1-6
(1990).
24. E. B. Baum and D. Haussier , What s ize net gives val id
general izat ion?
Neural Comput.
l, 151 160 (1989).
8/20/2019 A NEURAL NETWORK APPROACH TO OFF-LINE SIGNATURE VERIFICATION USING DIRECTIONAL PDF
http://slidepdf.com/reader/full/a-neural-network-approach-to-off-line-signature-verification-using-directional 10/10
4 24 J .- P. D R O U H A R D
e t al
25. H. Akaike , A new look at th e statist ical mod el identif ica-
tion,
I E E E T r an s A u t o m a t C o n t r o l
19, 716-7 23 1974).
26 . T . Ash , Dynam ic node c rea t ion in backprop aga t ion ne t -
works ,
P r o c I n t J o i n t C o n f N e u r a l N e t w o r k s
2, 623 628
1989).
27 . M . Hag iwara , Nove l backpropag a t ion a lgor i thm fo r re -
duc t ion o f h idden un i t s and acce le ra t ion o f convergence
using artif icial selection,
P r o c I n t J o i n t C o n f N e u r a l
N e t w o r k s
2, 625 630 1990).
28 . O. Fu j i ta , Op t im isa t ion o f the h idden un i t func t ion in
feedfo rward neura l ne tworks ,
N e u r a l N e t w o r k s
5, 755
764 1992).
bout the
A u t h o r - - J E A N - P I E R R E D R O U H A R D r e c e iv e d t h e M . S c. d eg r e e in e le c t ro n i c , e l e ct r ic a l a n d
con t ro l eng ineer ing f rom the U nivers i ty o f Caen , F rance , in 1972 and the M.Sc .A. and Ph .D. degrees in
b iomed ica l eng ineer ing f rom the Eco le Po ly techn iqu e de Mo ntr6a l in 1975 and 1979, re spect ive ly . F rom 1980
to 1989 he was a resea rch assoc ia te fo r the B iomedica l Eng ineer ing Ins t i tu te o f the E co le Po ly techn ique de
Mo ntrea l . In 1989 , he jo ine d the s ta f f o f the l~co le de Tec hno log ie Sup6rieure , Un ivers i t6 du Q uebec ,
Montr6a l , P .Q. , Canada , where he i s cu r ren t ly P ro fesso r in the D6par temen t de G6n ie de la P roduc t ion
Autom at ism. His cu r ren t re sea rch in te res ts a re in the app l ica t ion o f the a r t i f ic ia l in te l l igence techn ics such as
exper t systems , a r t i f ic ia l neura l ne tworks and fuzzy systems in the f ields o f pa t te rn recogn i t ion and com pute r
vision.
bout the
A u t h o r - - R O B E R T S A B O U R I N r e c e i v e d B .i ng ., M . S c. A . an d P h . D . d e g r e es i n e le c t ri c a l
eng ineer ing f rom the l~co le Po ly techn ique de Montr6a l in 1977 , 1980 and 1991 respec t ive ly . In 1977 he jo ined
the physics depar tm en t o f the Univers i t6 de M ontr6a l where he was respons ib le fo r the des ign and
dev e lo pm ent o f sc ien ti fic in s t rumenta t ion fo r the Observ a to i re du M ont M6gan tic . In 1983 , he jo ined the
s ta f f o f the Eco le de Techno log ie Sup~rieure , Un ivers i t~ Qu6bec , M ontr6a l , P .Q , Cana da , w here he i s
cu r ren t ly P ro fesso r in the D6par tem en t de G~ nie de la P rodu c t ion Automatis6e . His resea rch in te res ts a re in
the a reas o f compu te r v is ion , scene unders tand ing , segmenta t ion , s t ruc tu ra l pa t te rn recogn i t ion , neura l
ne two rks and fuzzy systems , charac te r recogn i t ion and s ign a tu re ve r i f ica tion .
bout the
A u t h o r - - M A R I O G O D B O U T r e ce i ve d t h e B A ng . d e g re e in P r o d u c t io n A u t o m a t i s ~ f r o m t h e
l~cole de Techno log ie Sup6r ieu re de Montr6a l in 1992 . He i s cu r ren t ly mak ing a m as te r degree a t tha t same
school. H is researc h intere sts are in the f ield o f artif icial intell igence.