Martinet z 1994

download Martinet z 1994

of 16

Transcript of Martinet z 1994

  • 7/25/2019 Martinet z 1994

    1/16

    Pergamon

    Neural Networks, Vol. 7, No. 3, pp. 507-522, 1994

    Copyright 1994 ElsevierScience Ltd

    Printed in the USA. All rights reserved

    0893-6080/94 6.00 + .00

    C O N T R I B U T E D R T I C L E

    Topology Representing etworks

    THOMA S MARTIN ETZ 1'2 AND K_LAUS SCHULTEN l

    tUniversityof Illinoisand 2Siemens AG

    Received 4 March

    1993;

    accepted

    20

    September

    1993 )

    Abstract--A

    Hebbian adap ta t ion ru le w i th w inner - take -a l l l i ke compe t i t ion i s in t roduced . I t i s shown tha t th i s

    compe t i t i ve Hebb ian ru le for ms so -ca l led De lau nay t r iangulat ions, wh ich p lay an imp or tan t ro le in computa t iona l

    geom etry o r e f f ic ien tly so lv ing prox imi ty prob lems . Given a se t o f neura l un i t s i, i = 1 . . . . N , the synap t ic we igh ts

    o f wh ich can be in terpre ted as po in ters w i, i = 1 . . . . N in n , the compe t i t i ve Hebb ian ru le leads to a connec tiv i ty

    s t ruc ture be tween the un i t s i tha t corresponds to the De lau nay t r iangu la t ion o f the se t o f po in ters w i . Such compe t i t i ve

    Heb bian rule develops connec tions C o > O) betw een neura l units i , j with neighbo ring receptive f ie lds Voronoi

    po lygons) V i , V j , whereas be tween a l l o ther un i t s i, j n o connec t ions evolve Ci j = 0 ) . Co mb ined wi th a procedure

    tha t d i s t r ibu tes the po in ters w i over a g iven f ea tu re m ani fo ld M, for ex ample , a subm ani fo ld M C n , the compe t i t i ve

    Hebbian ru le prov ides a nove l approach to the prob lem o f cons t ruct ing topo logy preserv ing f ea tu re map s and rep-

    resenting intricately s tructure d manifolds. Th e compe ti t ive Hebbian ru le connects only neu ral units , the receptive

    f i e lds Vorono i po lygons) V i , Vy o f wh ich are ad jacen t on the g iven mani fo ld M . Th is l eads to a connec t iv i ty s t ruc ture

    tha t de f ines a per fec t l y topo logy preserv ing ma p and for ms a d i screte, pa th preserv ing represen ta t ion o f M, a l so in

    cases where M has an intricate topology. This m ak es this novel approach part icularly useful in al l applications

    where ne ighborhood re la t ions have to be exp lo i t ed or the shap e and topo logy o f subma ni fo lds have to be taken in to

    account.

    Keywords--Proximity problems, Delaunay triangulation, Vbronoi polyhedron, Hebb rule, Winner-take-all com-

    petition, Topologypreserving feature map, Topology epresentation, Path preservation, Path planning.

    1. DELAUNAY TRIANGULATION AND

    PROXIMITY PRO LEMS

    In a number of information processing tasks the prob-

    lem that has to be solved efficiently can be reduced to

    a geometric problem that deals with the proximi ty of

    points in a metric space. The most promine nt example

    of such a

    p r o x i m i t y p r o b l e m

    is the

    n e a r e s t - n e i g h b o r

    or,

    more generally, the

    k - n e a r e s t - n e i g h b o r s e a r c h :

    given N

    points in a metric space, which is (are) the nearest (k

    nearest) neighbor (s) to a new given query point (Duda

    & Hart 1973). This

    b e s t m a t c h r e t r ie v a l

    has to be per-

    formed in classification and in terpolation tasks with

    applications in areas ranging from speech and image

    processing over robotics to efficient storage and transfer

    Acknowledgements: We than k Benoi t Noel and Phi lippe Dalger

    for pointing out th e connection between the competitive Hebb rule

    and second-order Voronoi polyhedra and for formulating Theorem

    1. This work has been support ed by the Carver Charitabl e Trust, the

    BMFT under Grant No. 01IN102A7, and by a fellowship of the

    olkswagen

    Foundation to T.M.

    Requests for reprints should be sent t o Klaus Schulten, Beckman-

    Institute and Depart ment of Physics, University of Illinois at Urb ana-

    Champaign, 405 North Mathews Ave., Urbana, IL 61801.

    5 7

    of data (Makhoul, Roucos, & Gish, 1985; Kohonen,

    M/ikisara, & Saram~iki, 1984; Naylor & Li, 1988; Gray,

    1984; Nasrabadi & King, 1988; Nasrabadi & Feng,

    1988; Ritter & Schulten, 1986). Another example of

    a proximity problem is the construction o f the

    E u c l i d -

    e a n m i n i m u m s p a n n i n g tr ee :

    given Npoi nts in a metric

    space, what is the graph of minimum total length whose

    vertices are the given points (Kruskal, 1956; Prim,

    1957; Dijkstra, 1959). Constructing the Euclidean

    minimum spanning tree is a common task in appli-

    cations requir ing optimally designed networks, for ex-

    ample, communication systems that have minimal in-

    terconnection length. Other applications of the Euclid-

    ean mi nimum spanning tree are in clustering (Gower

    & Ross, 1969; Zahn , 1971 ), patt ern recognition (Os-

    teen & Lin, 1974), and in searching for (approximate )

    solutions of the

    t r av e l in g s a l e s m a n p r o b l e m R o s e n -

    krantz, Stearns, & Lewis, 1974). A third example is

    the

    t r i a n g u l a t i o n p r o b l e m :

    given N points in a plane,

    connect them by nonintersecting straight lines so that

    every region inside the convex hull of the N points is a

    triangle. The triangulation problem occurs in the finite-

    element method (Strang & Fix, 1973) and in function

    interpolation on the basis of N data points, where the

  • 7/25/2019 Martinet z 1994

    2/16

    508

    7 .

    Martinetz and K Schulten

    f u n c t i o n s u r f a c e is a p p r o x i m a t e d b y a n e t w o r k o f tr i -

    angu la r face ts (Georg e , 1971 ) . A com p rehens iv e ove r -

    v i e w o f t h e a b o v e a n d f u r t h e r p r o x i m i t y p r o b l e m s c a n

    b e f o u n d i n P r e p a r a t a a n d S h a m o s ( 1 98 5 ) .

    A p o w e r f u l s t r u c t u r e f r o m c o m p u t a t i o n a l g e o m e t r y

    that solves or , a t leas t , yie lds a s tart ing point for eff i -

    c ien t ly so lv ing the above and o the r p rox im i ty p rob lem s

    is the so-cal led

    V o r o n o i d i a g r a m

    and i t s dua l , the

    D e -

    l a u n a y t r ia n g u l a t i o n .

    The Voro noi d iagra m CVs of a s e t

    S = { wl . . . . . wx } o f po in t s wi E ~D i s g iven by N D-

    d i m e n s i o n a l p o l y h e d r a , t h e V o r o n o i p o l y h e d r a V i ,

    which a re de f ined a s fo l lows : the Voronoi po lyhedron

    l~ of a po in t w~ E S i s g iven by the s e t o f po in t s v E

    ,~D tha t a re c lose r to w, tha n to a ny o the r wj @ S , j #

    i , that is ,

    J , = { v ~ ~ 1 I l v - w , II - < I t v -

    w j l l

    = 1 . . . . N }

    i = 1 . . . . N . ( 1 )

    The V oronoi po lyh edra p rov ide a com ple te pa r t i t ion ing

    of the em bed ding space ~D, tha t i s, ~D = U U_l

    V i .

    In

    t h e c o n t e x t o f

    w i n n e r - t a k e - a l l

    t y p e n e u r a l n e t w o r k s th e

    Voro noi po lyhe dro n V~ is often cal led the recept ive f ie ld

    of neura l u n i t i , w i th wi be ing the syna p t ic we igh t vec tor

    of th i s neura l u n i t . F or a l l inp u t v ec tors v E Vi the

    e l e m e n t i i s t h e b e s t m a t c h i n g u n i t o f t h e n e t w o r k a n d

    i s e m p l o y e d t o r e p r e s e n t ( e n c o d e ) t h e se i n p u t p a t t e r n s

    v . F igure l a i l lus t ra te s the de f in i t ion o f Vi by show ing

    the V oronoi d iagram , tha t i s, a ll the Vo ronoi po lygons ,

    o f a s am p le s e t o f po in t s in a p lane . In ~2 the V oronoi

    d i a g r a m f o r m s a g r a p h , t h e v e r ti c e s o f w h i c h a r e g i v e n

    by all the v ~ 9~2 t h a t a r e s i m u l t a n e o u s l y e l e m e n t o f

    th ree Voronoi po lygons Vi . The edges a re g iven by a l l

    t h e v E ~ 2 t h a t a r e s i m u l t a n e o u s l y e l e m e n t o f t w o V o -

    r o n o i p o l y g o n s . T h e V o r o n o i d i a g r a m a s t h e s e t o f

    a l l

    V o r o n o i p o l y h e d r a c o n t a i n s a l l t h e p r o x i m i t y i n f o r -

    m a t i o n a b o u t t h e s e t o f p o i n t s S .

    T h e s t r a ig h t l i n e d u a l o f th e V o r o n o i d i a g r a m i s t h e

    so-cal led

    D e l a u n a y t r i a n g u l a t i o n

    (De la unay , 1934 ) . In

    a p l a n e , t h e D e l a u n a y t r i a n g u l a t i o n i s o b t a i n e d i f w e

    con nec t a l l pa i r s wi , wj E S , the Voro noi po lygon s V~,

    I ~ o f w h i c h s h a r e a n e d g e. I n g e n e r a l , f o r e m b e d d i n g

    s p a ce s ~ D o f a r b i t r a r y d i m e n s i o n D , t h e D e l a u n a y

    t r i angu la t ion ~0s o f a s e t S = { Wl . . . . . WN } of po in t s

    (a)

    (b)

    F I G U R E 1 . ( a ) T h e V o r o n o i d i a g r a m o f a s e t o f p o i n t s. ( b ) T h e

    D e l a u n a y t r ia n g u l a t i o n t h a t c o r r e s p o n d s t o t h e V o r o n o i d i a g r a m

    i n ( a ) .

    Wi ~ ~ j ~ D i s de f ined by the g raph who se ve r t i ces a re the

    w i a n d w h o s e

    a d j a c e n c y m a t r i x A A i j ~ { O 1 } i j =

    1 . . . N ca r r i e s the va lue one i f f Vi fq ~ 4 : ~ . Two

    ver ti ces wi , wj a re connec ted by an edge i f f the i r Voronoi

    po ly hed ra V~, ~ a re ad jacen t . An i l lus t ra t ion o f the

    p lana r case is g iven in F igure 1b , w h e r e t h e D e l a u n a y

    t r i a n g u l a t i o n t h a t c o r r e s p o n d s t o t h e V o r o n o i d i a g r a m

    of F igure l a i s shown. In a p lane , each edge o f the

    D e l a u n a y t r i a n g u l a t i o n c o r r e s p o n d s t o a n e d g e o f t h e

    V o r o n o i d i a g r a m , a n d t h e t w o c o r r e s p o n d i n g e d g e s a re

    a lways pe rpe ndicu la r to each o the r .

    A n u m b e r o f t h e o r e m s a b o u t p r o p e r t i e s o f t h e V o -

    r o n o i d i a g r a m a n d t h e D e l a u n a y t r i a n g u l a t i o n a r e

    kn ow n (see , e .g. , Prep arata & Sha mo s, 1 985). However,

    m o s t o f t h e m a r e v a l i d o r a t le a s t c a n b e p r o v e n o n l y

    in the p lana r case , fo r D = 2 . In h ighe r -d im ens iona l

    em bed ding spaces ~D, D > 2 , on ly l i t tl e i s kn ow n so

    fa r . One reason i s tha t on ly fo r D = 2 the Voronoi

    d i a g r a m a n d t h e D e l a u n a y t r i a n g u l a t i o n a r e p l a n a r

    g r a p h s a n d , t h e r e f o r e , o n l y f o r D = 2 E u l e r ' s f o r m u l a

    can be ap p l ied (Bol lob~s , 1979 ) . Eu le r ' s fo rm ula p ro-

    v i d e s t h e i m p o r t a n t r e s u l t t h a t i n t h e p l a n a r c a s e t h e

    n u m b e r o f ed g e s o f th e V o r o n o i d i a g r a m a s w e ll a s o f

    t h e D e l a u n a y t r i a n g u l a t i o n d o es n o t ex c e e d 3 N - 6

    a n d , h e n c e , t h e V o r o n o i d i a g r a m a n d t h e D e l a u n a y

    t r i a n g u l a t i o n c a n b e s t o r e d i n o n l y l i n e a r s p a c e ( l i n e a r

    in the nu m b er o f ver t i ces N) . F ur the r , due to th i s re su l t ,

    b o t h s t r u c t u r e s a r e t r a n s f o r m a b l e i n t o e a c h o t h e r i n

    on ly l inea r t im e .

    C o n s t r u c t i n g t h e D e l a u n a y t r i a n g u l a t i o n i n a p r e -

    proces s ing s tage y ie lds a s t a r t ing po in t fo r e f f i c ien t ly

    s o l v in g p r o x i m i t y p r o b l e m s . I t c a n b e s h o w n t h a t i f t h e

    D e l a u n a y t r i a n g u l a t i o n o f a g i v en s e t o f p o i n t s S i s

    k n o w n , t h e a b o v e s t a t ed a n d o t h e r p r o x i m i t y p r o b l e m s

    c a n b e s o l v e d w i t h a t m o s t l i n e a r l y i n c r e a s i n g c o m -

    p u t a t i o n a l e f f o r t . T h e t r i a n g u l a t i o n p r o b l e m , f o r e x -

    am ple , i s obv ious ly a l ready so lved wi th the cons t ru c t ion

    o f th e D e l a u n a y t r i a n g u la t i o n a n d d o e s n o t n e e d f u r t h e r

    c o m p u t a t i o n , l T h e c o m p u t a t i o n t i m e n e e d e d f o r f i n d -

    i n g t h e E u c l i d e a n m i n i m u m s p a n n i n g t r e e is r e d u c e d

    s ign i f i can t ly because the edges o f the E uc l idea n m in i -

    m u m s p a n n i n g t r e e a r e a s u b s e t o f th e e d g e s o f t h e

    D e l a u n a y t r ia n g u l a t i o n ( S h a m o s , 1 97 8 ) . K n o w i n g th e

    D e l a u n a y t r i a n g u l a t i o n , i t o n l y r e q u i re s O ( N ) i n s t e a d

    o f O ( N l o g N ) t i m e f o r i ts c o n s t r u c t i o n . T h e n e a r e s t-

    n e i g h b o r a n d k - n e a r e s t - n e i g h b o r s e a r c h c a n b e p e r -

    f o r m e d i n o n l y O (l o g N ) i n s t e a d o f O ( N ) t i m e b y e x -

    p l o i ti n g t h e D e l a u n a y t r i a n g u l a t i o n ( K n u t h , 1 9 7 3 ) .

    I n t h e f o l lo w i n g s e c t io n w e w i l l i n t r o d u c e a H e b b i a n

    a d a p t a t i o n r u l e t h a t , w i t h i n a n e t w o r k o f n e u r a l u n i t s ,

    y i e l d s i n t e r n e u r a l c o n n e c t i o n s c o r r e s p o n d i n g t o t h e

    l S o l v i n g t h e t r i a n g u l a t i o n p r o b l e m b y m e a n s o f t h e D e l a u n a y

    t r i a n g u l a t i o n h a s a d v a n t a g e s p a r t ic u l a r l y i n f u n c t i o n i n t e r p o l a t i o n .

    W h e n a f u n c t i o n i s a p p r o x i m a t e d p i e c e w i s e - l i n e a r o v e r t h e f a c e t s o f

    a t r i a n g u l a t i o n , t h e D e l a u n a y t r i a n g u l a t i o n y i e l d s a s m a l l e r w o r s t c a s e

    e r r o r t h a n a n y o t he r t ri a n g u la t i o n ( O m o h u n d r o , 1 9 9 0 ) .

  • 7/25/2019 Martinet z 1994

    3/16

    Topology Representing Networks 509

    D e l a u n a y t r i a n g u l a t i o n . I n S e c t i o n 3 w e w i l l e x p l a i n

    t h e c o n c e p t o f t o p o l o g y p r e se r v i n g f e a t u r e m a p s . W e

    w i l l i n t r o d u c e t h e t e r m

    m a s k e d V o ro n oi p o l y h e d r o n

    a n d s h o w h o w t h e s e m a s k e d V o r o n o i p o l y h e d r a p r o v id e

    a r i g o r o u s d e f i n i t i o n f o r th e t e r m s

    n e i g h b o r h o o d a n d

    t o p o l o g y p r e s e r v i n g m a p s .

    T h i s l e a d s t o

    i n d u c e d D e -

    l a u n a y t r i a n g u l a t i o n s

    a s t h e g r a p h i c a l s t r u c t u r e t h a t

    p r o v i d e s p e r f e c t t o p o l o g y p r e s e r v a t i o n . W e w i l l s h o w

    t h a t t h e H e b b i a n a d a p t a t i o n r u l e o f t h e f o l l o w i n g s ec -

    t i o n f o r m s i n d u c e d D e l a u n a y t r i a n g u l a t io n s a n d , h e n c e ,

    i s a b le t o f o r m p e r f e c t ly t o p o l o g y p r e se r v i n g m a p s a l s o

    o f t o p o l o g i c a l l y i n t r i c a t e l y s t r u c t u r e d m a n i f o l d s , f o r

    e x a m p l e , o f m a n i fo l d s t h a t a r e d i s c o n n e c t e d a n d / o r

    p iecewise 0 -, 1 -, 2- , e t c ., d im en s iona l . In S ec t ion 4 w e

    w i l l s h o w t h a t t h e i n d u c e d D e l a u n a y t r i a n g u l a t i o n s

    p r o v i d e d b y t h e H e b b i a n a d a p t a t i o n r u l e y i e l d p a t h

    p r e s e r v in g r e p r e s e n t a t i o n s a n d a l l o w o n e t o d e s c r ib e ,

    c la s si fy , a n d p l a n p a t h s o n m a n i f o l d s . T h e s e i n d u c e d

    De launa y t r i angula t ions , which re f l ec t the topo log y and ,

    a t t h e s a m e t i m e , f o r m p e r f e c t ly t o p o l o g y p re s e r v i n g

    m a p s a n d p a t h p r e s e r v i n g r e p r e s e n t a t io n s o f g i v en

    m ani fo lds , we ca l l t o p o l o g y r e p r e s e n t i n g n e t w o r k s

    ( T R N ) . I n t h e l a st se c t i o n w e w i ll p r e s e n t a c o m p a c t

    a l g o r i t h m , d e r i v e d f r o m t h e H e b b i a n a d a p t a t i o n r u l e

    o f t h e n e x t s e c t io n , w h i c h c o n s t r u c t s T R N s .

    2 . H E B B I A N A D A P T A TI O N R U L E F O R M S

    D E L A U N A Y T R I A N G U L A T I O N S

    I n t h e f o l lo w i n g w e a s s u m e a s e t o f n e u r a l u n i t s i , i =

    1 . . . . N t h a t c a n d e v el o p

    l a t e ra l connec t i ons

    b e t w e e n

    e a c h o t h e r . A n e u r a l u n i t c o n n e c t s i t s e l f w i t h a n o t h e r

    u n i t b y d e v e l o p i n g a

    s y n a p t i c l i n k

    t o t h i s u n i t . T h e

    l a t e r a l c o n n e c t i o n s a r e d e s c r i b e d b y a c o n n e c t i o n

    s t r e n g t h m a t r i x C w i t h e l e m e n t s

    C o E ~ .

    The la rge r

    a m a t r i x e l e m e n t C 0 , t h e s t r o n g e r i s t h e s y n a p t i c l i n k

    f r o m u n i t i t o u n i t j . O n l y i f C o > 0 , w e r e g a r d n e u r a l

    u n i t i a s b e i n g c o n n e c t e d w i t h u n i t j . I f C 0 = 0 , n e u r a l

    un i t i i s

    n o t

    c o n n e c t e d w i t h u n i t j . N e g a t i v e v al u e s fo r

    C o

    do no t a r i s e .

    T h e b a s i c p r in c i p l e t h a t g o v e r n s t h e c h a n g e o f i n -

    t e r n e u r a l c o n n e c t i o n s t r e n g th , a t l e a s t in p a r t s o f t h e

    h i p p o c a m p u s ( K e l s o , G a n o n g , & B r o w n , 1 9 8 6 ) , w a s

    f i rs t f o r m u l a t e d b y H e b b ( 1 9 4 9 ) . A c c o r d i n g to H e b b ' s

    p o s t u l a t e , a p r e s y n a p t i c u n i t i in c r e a s e s t h e s t r e n g t h o f

    i t s s y n a p t ic l i n k t o a p o s t s y n a p t i c u n i t j i f b o t h u n i t s

    a re co ncu r ren t ly ac t ive , tha t i s, i f bo th ac t iv i t i e s do cor -

    r e la t e . A v a r i e t y o f q u a n t i t a t i v e f o r m u l a t i o n s o f t h i s

    c o n j u n c t i v e m e c h a n i s m h a v e b e e n p r o p o s e d , f o r e x -

    a m p l e , f o r m o d e l i n g P a v l o v i a n c o n d i t i o n i n g ( G r o s s -

    be rg , 1974) , m o tor l ea rn ing (Marr , 1969) , o r a s soc ia -

    t iv e m e m o r y ( P a l m , 1 98 2 ; K o h o n e n , O j a , & L e h t i o ,

    1 9 8 5 ). I n i t s s im p l e s t m a t h e m a t i c a l f o r m u l a t i o n H e b b ' s

    r u l e i s d e s c r i b e d b y t h e e q u a t i o n

    Co oc y i . yj, (2 )

    i n w h i c h t h e c h a n g e o f t h e s t r e n g t h C o o f t h e s y n a p t i c

    l i n k f r o m u n i t i t o u n i t j i s l i n e a r l y p r o p o r t i o n a l t o t h e

    p r e s y n a p t i c a c t i v i t y

    Yi

    a n d t o t h e p o s t s y n a p t i c a c t i v i ty

    yj (see , e .g. , Coo per, 1973 ). T he qu an ti t ie s Yi , i = 1,

    . . . . N d e n o t e t h e o u t p u t a c t iv i t ie s o f t h e n e u r a l u n i t s .

    M o r e r e a li s ti c a n d d e t a i l e d m a t h e m a t i c a l d e s c r i p t i o n s

    o f H e b b ' s p o s t u l a t e t a k e t e m p o r a l a s p e c t s a n d t h e s t o -

    c h a s t ic n a t u r e o f t h e o u t p u t a c t iv i t ie s o f n e u r o n s i n t o

    a c c o u n t . A l s o , it m a y b e n e c e s s a r y t o a d d a d e c a y t e r m

    t o k e e p t h e s t r e n g t h o f t h e s y n a p t i c l i n k s b o u n d e d .

    D i f f e r e n t n e u r a l n e t w o r k m o d e l s h a v e e m p l o y e d d if -

    f e r e n t r e a li z a t io n s o f H e b b ' s p o s t u l a t e , d e p e n d i n g o n

    t h e a d a p t a t i o n r u l e s r e q u i r e d t o a c h i e v e t h e d e s i r e d

    i n f o r m a t i o n p r o c e s s in g t as k . W e w i l l e m p l o y t h e H e b b

    r u l e i n a f o r m t h a t i n c o r p o r a t e s t h e n o v e l a s p e c t o f

    c o m p e t i t i o n

    a m o n g t h e s y n o p t i c l i n k s. W e a s s u m e t h a t

    to each neura l u n i t i a we igh t vec tor w~ E ~ t is assigned.

    F u r t h e r , w e a s s u m e t h a t e a c h n e u r a l u n i t i , i = 1 . . . .

    N r e c e iv e s t h e s a m e e x t e r n a l i n p u t p a t t e r n s v E ~ o .

    T h e w e i g h t v e c t o r w i d e t e r m i n e s t h e c e n t e r o f t h e r e -

    cep t ive f i e ld o f un i t i in the s ense tha t w i th the recep t ion

    o f a n i n p u t p a t t e r n v t h e o u t p u t a c t i v it y y ; o f u n i t i i s

    the larger the c loser i ts w

    is

    to V. In m a them at ica l t e rm s ,

    w e a s s u m e t h a t

    Y i = R (

    I I v - w i l l )

    i s v a li d , w i t h R ( . )

    b e i n g a p o s i t i v e a n d c o n t i n u o u s l y m o n o t o n i c a l l y d e -

    c r e a s in g f u n c t i o n ( e . g ., a G a u s s i a n ) .

    I f w e a p p l y t h e H e b b r u l e i n t h e s i m p l e f o r m g i v e n

    b y e q n ( 2 ) , w e o b t a i n f o r t h e c h a n g e o f t h e s t r e n g t h o f

    t h e l a t e r a l c o n n e c t i o n s

    C o

    w i t h t h e p r e s e n t a t i o n o f a n

    i n p u t p a t t e r n v

    A f o o c R l l v - - w , l l ) . R l l v - w i l l ) . 3 )

    Asym pto t ica l ly , tha t i s , wi th the s equen t ia l p re sen ta t ion

    o f m a n y i n p u t p a t t e r n s v ~ 3 to , t h e

    Co's are

    d e t e r m i n e d

    b y t h e i n t e g ra l o f e q n ( 3 ) o v e r t h e g i v e n p a t t e r n d i s t r i -

    b u t i o n P ( v ) . I f w e a s s u m e a p a t t e r n d i s t r i b u t i o n t h a t

    i s h o m o g e n e o u s o v e r ~ u , w e o b t a i n

    A C i j ( t ~ o o) oc f ~ D

    g(l lv - wil l)" g(l lv - wil l) dv. (4)

    H e n c e , e m p l o y i n g t h e H e b b r u l e i n t h e s i m p l e f o r m a s

    g i v e n i n e q n ( 2 ) y i e l d s t h e r a t h e r t r i v i a l r e s u l t t h a t e a c h

    n e u r a l u n i t i de v e lo p s c o n n e c t i o n s t o a l l t h e o t h e r u n i t s

    j ~ i , wi th l a te ra l con nec t ion s t reng ths C o.t h a t a r e s i m -

    p ly p ropo r t ion a l to the ov e r laps o f the recep t ive f i e lds

    R ( l l v -

    w ; l l )

    a n d R ( l l v -

    w i l l ) .

    T h e s t re n g t h o f t h e

    s y n a p t i c li n k b e t w e e n t w o u n i t s i a n d j i s s i m p l y m o n o -

    t o n i c a ll y a n d c o n t i n u o u s l y d e c r e a si n g w i t h t h e d i s t an c e

    b e t w e e n w i a n d w j .

    A s i n m a n y s y s t e m s g o v e r n e d b y s e l f - o r g a n i z i n g

    p r o c es s es , t h e c o n n e c t i v i t y p a t t e r n t h a t e v o l v es o n t h e

    s e t o f n e u r a l u n i t s b e c o m e s s i g n i f i ca n t l y m o r e s t r u c -

    t u r e d i f w e i n t r o d u c e c o m p e t i t i o n . I n a w i n n e r - t a k e -

    a l l n e t w o r k , f o r e x a m p l e , t h e u n i t s c o m p e t e w i t h e a c h

    othe r based on the i r ou tpu t ac t iv i ti e s , wh ich f ina l ly l eads

    t o a n a d a p t a t i o n o n l y o f t h e w e i g h ts o f th e u n i t w i t h

    the h ighes t ou t pu t ac t iv i ty ( s ee , e. g ., Gros sbe rg , 1976 ) .

  • 7/25/2019 Martinet z 1994

    4/16

    5 1 0 T . M a r t i n e t z a n d K . S c h u l t e n

    W i t h o u t c o m p e t i t i o n , a l l t h e u n i t s w o u l d b e h a v e a l i k e

    a n d n o s p e c i a l iz a t i o n o f t h e u n i t s , a s i t is c h a r a c t e r i s t ic

    f o r w i n n e r - t a k e - a l l n e t w o r k s , w o u l d e v o l v e .

    A n a l o g t o t h e c o m p e t i t i o n a m o n g t h e u n i t s in a w i n -

    n e r -t a k e- a ll n e t w o r k w e i n t r o d u c e c o m p e t i t i o n a m o n g

    t h e s y n a p t i c l in k s . I n s t e a d o f b e i n g b a s e d o n t h e o u t p u t

    ac t i v i t i e s o f t he neu r a l u n i t s i t s e l f, a s i n a w i n ne r - t ak e -

    a ll n e tw o r k , i n o u r m o d e l t h e c o m p e t i t io n a m o n g t h e

    s y n a p t i c l i n k s i s d e t e r m i n e d b y w h a t w e w a n t t o c a l l

    t h e c o r r e l a t e d o u t p u t a c t i v i t ie s Y 0, t h e c o r r e l a t i o n s o f

    t h e o u t p u t a c t iv i ti e s o f al l p a ir s o f p r e - a n d p o s t s y n a p t i c

    u n i t s . I n t h e q u a n t i t a t i v e f o r m u l a t i o n g i v e n b e lo w , t h e

    c o r r e l a t e d o u t p u t a c t i v i t i e s a r e d e t e r m i n e d b y

    Y~j =

    y g . y j ,

    a c c o r d i n g t o t h e H e b b r u l e ( 2 ) . K e e p i n g th e

    a n a l o g y t o w i n n e r - t a k e - a l l n e t w o r k s , w i t h t h e p r e s e n -

    t a t i o n o f a n i n p u t p a t t e r n v w e o n l y m o d i f y t h e s y n a p t i c

    l i nk i - j who se ac t i v i t y Y o = Y i Y j i s h i ghes t . I ns t ead

    o f c h a n g i ng t h e c o n n e c t i o n s t re n g t h s

    C o

    a c c o r d i n g t o

    t h e H e b b r u l e ( 2 ) , i n t h e f o l lo w i n g w e w i ll e m p l o y a

    w i n n e r - t a k e - a l l o r c o m p e t i t i v e v e rs i o n o f e q n ( 2 ) , d e -

    t e r m i n e d b y

    A C ~ j O C { o i . Y j

    i f

    y i . y j > - y k . y t

    ot he r wi se .

    V k , l = 1 . . . . N

    ( 5 )

    W e w i ll s h o w t h a t , i n s t e a d o f c o n n e c t i n g e a c h u n i t

    w i t h a ll th e o t h e r u n i t s , t h e c o m p e t i t i v e H e b b r u l e ( 5 )

    f o r m s a c o n n e c t i v i t y s t r u c t u r e a m o n g t h e n e u r a l u n i t s

    i , i = 1 . . . . N t h a t c o r r e s p o n d s t o t h e D e l a u n a y t r ia n -

    g u l a t i o n o f t h e w e i g h t v e c t o rs w~ . . . . . W N. M o r e p r e -

    c is el y, we wi l l show t ha t i f we p r esen t s equen t i a l l y i npu t

    p a t t e r n s v w i t h a d i s tr i b u t i o n P ( v ) t h a t h a s s u p p o r t ( i s

    n o n z e r o ) e v e r y w h e r e o n ~ D , t h e n t h e e l e m e n t s

    C o

    o f

    t h e c o n n e c t i o n s t r e n g th m a t r i x C o b e y a s y m p t o t i c a l ly

    O [Ci j ( t - -~

    oo)] =

    A o i , j = 1 . . . . N

    ( 6 )

    w h e r e 0 ( . ) i s t h e H e a v y s i d e s t e p f u n c t i o n a n d w h e r e

    A o a r e t h e e l e m e n t s o f t h e a d j a c e n c y m a t r i x A o f th e

    D e l a u n a y t r i a n g u l a t i o n o f t h e p o i n t s w ~ . . . . . W u , f o r

    w h i c h

    i f

    Vi f -) Vj # ~ ( Vi ,

    V j a r e a d j a c e n t )

    i f

    V, f ] V j = Z ( V i , V j

    a r e n o t a d j a c e n t )

    ( 7 )

    i s v a l id . V g, ~ a g a i n d e n o t e th e V o r o n o i p o l y h e d r a o f

    w~, w , t h a t i s , i n a w i nn e r - t ake - a l l ne t w or k t he r ecep t i ve

    f i e lds o f t he un i t s i , j .

    B e f o r e w e p r o v e e q n ( 6 ) , w e f i rs t s h o w h o w t h e c o m -

    p e t it iv e H e b b r u l e f o r c o n s t r u c t in g t h e D e l a u n a y t r ia n -

    g u l a t io n c a n b e f o r m u l a t e d i n a v e r y c o m p a c t a l g o -

    r i t h m . T h e c o r r e l a t e d a c t i v i t y

    Y o = Y i Y j

    i s h i ghes t i f

    i ( o r j ) d e n o t e s t h e n e u r a l u n i t w i t h t h e l a r g e st o u t p u t

    a n d j ( o r i , r e s p e c t i v e l y ) d e n o t e s t h e u n i t w i t h t h e s e c -

    o n d l a r g e s t o u t p u t . B e c a u s e

    Y o = Y j i

    i s va l i d , t he r e a r e

    a l w a y s t w o

    w i n n i n g

    l in k s , t h e s t r e n g th s o f w h i c h a r e

    c h a n g e d b y t h e s a m e a m o u n t , t h a t i s, t h e s t r e n g t h o f

    t he l i nk f r om i t o j a s we l l a s o f t he l i nk f r om j t o i .

    H e n c e , t h e c o n n e c t i o n s t r e n g t h m a t r i x C o i s s y m m e t r i c ,

    a n d i f w e s a y " u n i t i is c o n n e c t e d w i t h u n i t j , " t h i s

    a l w a y s i m p l i e s t h a t u n i t j i s a l s o c o n n e c t e d w i t h u n i t

    i . B e c a u s e w e a r e i n t e r e s t e d i n t h e g r a p h d e f i n e d b y

    t h e c o n n e c t i o n s b e t w e e n t h e u n i t s , a n d b e c a u s e t h i s

    g r a p h is c o m p l e t e l y d e t e r m i n e d b y t h e a d j a c e n c y m a -

    t r i x A i j = O C i j ) , i n t h e f o l l o w i n g a l g o r i t h m w e w i l l n o t

    c a r e a b o u t t h e a b s o l u t e v a l u e s o f th e

    C~j's,

    b u t w i ll o n l y

    r e g i s te r w h e t h e r a Ci j b e c o m e s n o n z e r o .

    T h e p r o c e d u r e f o r c o n s t ru c t i n g t h e c o n n e c t i o n s b e -

    t w e e n t h e u n i t s i , j ( i , j = 1 . . . . N ) c a n n o w b e f o r -

    mul a t ed a s f o l l ows :

    ( i ) S e t a l l c o n n e c t i o n s t r e n g th s

    C~j

    t o z e r o ;

    ( i i ) P r e s e n t a n i n p u t p a t t e r n v @ ~ z~ w i t h p r o b a b i l i t y

    P ( v ) ;

    ( i i i ) D e t e r m i n e u n i t io f o r w h i c h

    IIv - w~oll -< IIv - will v j = l . . . . . N

    a n d u n i t i l f o r w h i c h

    IIv - w,, II -< IIv - w, II Vj # io, j = 1 . . . . N;

    ( iv ) I fCioi, = 0, se t Cioi > 0, that i s , co nn ec t io an d i , .

    I f C~oi > 0 , l eave C~ oi,, unc hang ed . Co nt i n ue w i t h

    ( i i ) .

    I n F i g u r e 2 a l g o r i t h m ( i ) - ( i v ) i s i l l u s t r a t e d . T w o

    u n i t s i , j b e c o m e c o n n e c t e d i f f t h e i r V o r o n o i p o l y h e d r a

    V~, ~ a r e ad j acen t . T o show r i gor o us l y t ha t t he ad j a -

    c e n c y m a t r i x A ,j = 0 ( C u ) o f t h e c o n n e c t i v i t y s t r u c t u r e

    b e c o m e s e q u i v a l e n t to t h e a d j a c e n c y m a t r i x o f t h e D e -

    l a u n a y t r i a n g u l a t i o n ~ gs o f th e s e t o f p o i n t s S = ( w ~,

    F I G U R E 2 . I ll u s t r a ti o n o f th e u p d a t e r u l e f o r th e c o n n e c t i o n s

    b e t w e e n t h e n e u r a l u n i t s . E a c h t i m e a n i n p u t p a t t e r n v i s p r e -

    s e n t e d w i t h in t h e s h a d e d a r e a , w h i c h d e p i c t s t h e V o r o n o i p o l y -

    g o n V l o f u n i t i , a c o n n e c t i o n f r o m i t o t h e u n i t j w i t h i ts w e i g h t

    v e c t o r w s e c o n d c l o s e s t t o th e i n p u t p a t t e r n is e s t a b l i s h e d .

    T h e n u m b e r s 1 . . . . 6 d e n o t e t h e s u b re g i o n s w i th i n t h e s h a d e d

    a r e a f o r w h i c h th e r e s p e c t i v e n e ig h b o r i n g u n i t i s s e c o n d c l o s e s t

    t o a n i n p u t s i g n a l v E V I . U n i t i d e v e l o p s c o n n e c t i o n s o n l y t o

    t h o s e u n i t s, th e V o r e n o i p o l y g o n s o f w h i c h s h a r e a n e d g e w i th

    i t s o w n V o r o n o i p o l y g o n .

  • 7/25/2019 Martinet z 1994

    5/16

    Topology Representing Networks 511

    . . . . w u ) , w e i n t r o d u c e t h e

    s e c o n d - o r d e r V o r o n o i p o l y -

    h e d r a V o , i , j = 1 . . . . N .

    T h e s e c o n d - o r d e r V o r o n o i

    p o l y h e d r o n V0 is given b y a l l th e v G ~RD for wh ich wi

    a n d w j a r e t h e t w o c l o s e s t p o i n t s o f S ; t h a t i s,

    V o

    is

    d e f i n e d b y

    vo

    = {v e ~111v - will -< IIv - wkll A IIv - will

    -< IIv - Wkll

    V k + i , j } .

    (8)

    AS 1I,-, als o Vo f o r m s a c o n v e x p o l y h e d r o n . W e s e e f r o m

    e q n ( 8 ) t h a t a l g o r i t h m ( i ) - ( i v ) c o n n e c t s t w o u n i t s i,

    j on ly i f K , j J~ i s va l id . We wi l l p rove th a t

    V o q: $25

    i s v a l i d i f f t h e c o r r e s p o n d i n g f i r st - o r d er V o r o n o i p o l y -

    hed ra Vi , Vj a re ad jace n t , tha t i s , i f f

    V i f l V j ~ ~ .

    T h e n , i n c a se f z P ( v ) d v 0 h o l d s f o r e a c h VU 4= ~5 ,

    . ~ t J . . .

    t h e c o n n e c t io n s g e n er a t ed b y a l g o r i th m ( 1 ) - ( w ) f o r m

    t h e D e l a u n a y t r i a n g u l a t i o n o f t h e p o i n t s w~ . . . . . W N.2

    THEOREM 1.

    F o r a s e t

    S = { w~ . . . . w~}

    o f p o i n t s w i

    E ~ D t h e r e l a t i o n

    V/

    f l ~ ,~=~ ~ j ~ .~

    (9)

    i s v a l i d f o r e a c h p a i r i , j . V i d e n o t e s t h e f i r s t - o r d e r

    V o r o n oi p o ly h e d r o n o f p o i n t

    wi ,

    a n d V o. d e n o t e s t h e s e c -

    o n d - o r d e r V o r o no i p o l y h e d r o n o f th e p o i n t s w i , w j .

    P r o o f .

    If V/f-) Vj 4: O is val id, there is a v E ~ o w ith

    v ~ V i

    an d v E Vj. Th en w e obta in [Iv - wil l = [Iv -

    wi l l -< I l v - w~ ll fo r al l Wk E S a nd , the re fore , v ~

    Vo,

    that is ,

    V i i 4 : ~ ,

    is val id.

    I f

    Vii ~ ~

    is val id, there is a v E ~tD f o r w h i c h t h e

    p o i n t s w i a n d w j a r e t h e t w o n e a r e s t n e i g h b o r s . W i t h o u t

    los s o f gene ra l i ty we as sum e th a t w~ i s the nea res t

    ne ighbor . Because fo r each u ~ ~ the po in t wj i s e i the r

    t h e n e a r e s t o r t h e s e c o n d n e a r e s t n e i g h b o r o f u , a n d

    for u = v the p o in t wi i s c loses t and fo r u = wj the po in t

    w~ is c loses t to u, there is a u* G V ~ for wh ich [[u* -

    w i l l - - I l u *

    - wjl l is val id. Hen ce, we ob tai n u* ~ 1I,.

    a n d u * ~ ~ , a n d , t h e r e f o r e , u * ~

    Vi f l V j ,

    that is , Vj

    fq Vj # ~5, is valid .

    In the fo l lowing we wi l l cons ide r pa t t e rn d i s t r ibu t ions

    P ( v ) t h a t h a v e s u p p o r t n o t o n t h e e n t i r e e m b e d d i n g

    s p a c e ~ D , b u t o n l y o n a s u b m a n i f o l d M . I n t h e s e ca s es ,

    f o r s o m e

    V o 4= ~

    t h e i n t e g r a l

    f v o

    P ( v ) d v m i g h t v a n i sh ,

    wi th the re su l t tha t the edge i - j wi l l no t be e s tab l i shed

    b y t h e c o m p e t i t i v e H e b b r u l e . I n t h e s e c as e s, t h e c o m -

    p e t i t i v e H e b b r u l e d o e s n o t f o r m t h e e n t i r e D e l a u n a y

    t r i a n g u l a t i o n , b u t o n l y s u b g r a p h s o f it . W e w i l l s h o w

    t h a t t h e s e s u b g r a p h s d e f i n e t o p o l o g y p r e se r v i n g m a p s .

    3 . D E L A U N A Y T R I A N G U L A T I O N A N D

    T O P O L O G Y P R E S E R V I N G M A P S

    T h e d e f i n i t i o n o f t h e D e l a u n a y t r i a n g u l a t i o n a s b e i n g

    t h e g r a p h t h a t c o n n e c t s t h o s e p o i n t s w i , w j t h a t h a v e

    2 The following heo rem was formulated ogether with Ph ilippe

    Dalger and B ennoit Noel (Dalger et a l., 1 992).

    ad jace n t Voronoi po lyhe dra V, , Vj m ak es th i s s t ruc tu re

    i d e a l ly s u it e d f o r a p r o x i m i t y p r o b l e m o f a d i f f e re n t

    t y p e t h a n t h e o n e s m e n t i o n e d i n t h e f i r s t s e c t i o n ,

    n a m e l y , f o r t h e f o r m a t i o n o f s o - c a l l e d t o p o l o g y p r e -

    se rv ing fea tu re m aps . Topology prese rv ing fea tu re m ap s

    p l a y a n i m p o r t a n t r o le a s c o m p o n e n t s i n a v a r i e ty o f

    n a t u r a l a s w e l l a s a r t i f i c i a l n e u r a l i n f o r m a t i o n p r o -

    ces s ing sys tem s (Knudsen , de Lac , & Es te r ly , 1987;

    Kohonen , 1989; R i t t e r , Mar t ine tz , & S chul ten , 1992) .

    B y p r o je c t i n g i n p u t p a t t e r n s o n t o a n e t w o r k o f n e u r a l

    u n i t s s u c h t h a t s i m i l a r p a t t e r n s a r e p r o j e c t e d o n t o u n i t s

    a d j a c e n t i n t h e n e t w o r k a n d , v i c e v er sa , s u c h t h a t u n i t s

    a d j a c e n t i n t h e n e t w o r k c o d e s i m i l a r p a t t e r n s , a r e p -

    r e s e n t a t i o n o f t h e i n p u t p a t t e r n s i s a c h i e v e d t h a t i n

    pos tproces s ing s tages a l lows one to exp lo i t the s im i la r i ty

    r e l a t io n s o f th e i n p u t p a t t e r n s . E x a m p l e s o f t o p o l o g y

    p r e s e r v in g f e a t u r e m a p s i n t h e n e r v o u s s y s t e m a r e t h e

    r e t i n o t o p i c m a p i n t h e v i s u a l c o r t e x ( H u b e l & W i e s e l,

    1 97 4; B l a sd e l & S a l a m a , 1 9 8 6 ) , t h e m a p p i n g f r o m t h e

    b o d y s u r f ac e o n t o t h e s o m a t o s e n s o r y c o r t e x ( K a a s e t

    a l. , 1 9 7 9 ) , o r t h e t o n o t o p i c m a p s i n t h e a u d i t o r y c o r t e x

    ( S u g a & O ' N e i ll , 1 9 7 9 ) . A s c o m p o n e n t s o f a rt i fi c i a l

    n e u r a l i n f o r m a t i o n p r o c e s s i n g s y s t em s , t o p o l o g y p r e-

    s e r v i n g f e a tu r e m a p s h a v e b e e n a p p l i e d s u c c e ss f u l ly in

    s p e e c h p r o ce s s i n g ( K o h o n e n e t a l ., 1 98 4; K o h o n e n ,

    1990; Naylor & L i , 1988; Bran dt , Beh m e , & S t rube ,

    1991 ) , im age proces s ing (Nas r abad i & F eng , 19 88) ,

    a n d r o b o t i c s ( R i t t e r & S c h u l t e n , 1 9 86 ; M a r t i n e t z , R i t -

    t e r , & S chul ten , 1990) .

    A t o p o l o g y p r es e r v i n g f e a tu r e m a p i s d e t e r m i n e d b y

    a m a p p i n g f r o m a m a n i f o l d M _ ~ D o n t o t h e v e r ti c e s

    ( n e u r a l u n i t s ) i , i = 1 . . . . N o f a g r a p h ( n e t w o r k ) G .

    T h e m a p p i n g f r o m M t o G i s d e t e r m i n e d b y p o i n t e r s

    wi E ~D, i = 1 . . . . N a t t ach ed to the ve r t i ces i . A

    f e a t u r e v e c to r v E M i s m a p p e d t o t h e v e r t e x i * ( v ) ,

    the p o in te r wi . ( , ) o f wh ich i s c losest to v ; th a t i s , v i s

    m a p p e d t o t h e v e r te x i * ( v ) w h o s e V o r o n o i p o l y h e d r o n

    V i . ( , )

    e n c l o se s v . T h e m a p p i n g ~ i s c o m p l e t e l y d e t e r-

    m i n e d b y t h e p o in t e r se t S = { w l . . . . . W N } , a n d w e

    c a n w r i t e

    s :M ~ - - ,G , v E M - - - , i * ( v ) @ G

    (10)

    w h e r e i * ( v ) i s d e t e r m i n e d b y

    IIw,.~v) - vii ~ II w /- vii v i E G. (11 )

    T h e m a p p i n g ~ s f r o m M t o G i s n e i g h b o r h o o d p r e -

    se rv ing , i f s im i la r fe a tu re vec tors , tha t i s , v ' s tha t a re

    c l o se w i t h i n t h e m a n i f o l d M , a r e m a p p e d t o v e r t ic e s i

    t h a t a r e c l o s e w i t h i n t h e g r a p h . T h i s r e q u i r e s t h a t

    p o i n t e r s w i , w j t h a t a r e n e i g h b o r i n g o n t h e f e a t u r e

    m a n i f o l d a r e a s s i g n ed t o v e r t i c es i , j t h a t a r e a d j a c e n t

    i n t h e g r a p h . 3

    T h e i n ve r se m a p p i n g ~ 1 f r o m th e g ra p h G o n t o

    t h e m a n i f o l d M i s d e t e r m i n e d b y

    ~ s l : G ~ - ' ~ M , i E G ~ - ' ~ w i E M

    (12)

  • 7/25/2019 Martinet z 1994

    6/16

    512 T Martinetz and K Schuhen

    (a) (b) (c)

    F I G U R E 3 . A s q u a r e -s h a p e d m a n i f o l d M m a p p e d o n t o t h r e e

    d i f fe r e n t g r a p h s G . Ea c h o f t h e g r a p h s c o n s is t s o f n in e v e r t ic e s

    i , a n d t h e a s s o c i a t e d p o i n t e r s w l a r e d i s t r ib u t e d r e g u l a rl y o v e r

    t h e s q u a r e . T h e t h r e e d i f f e re n t f e a t u r e m a p s a r e v i s u a l i z e d b y

    m a r k in g t h e p o i n t e r p o s it io n s w l o n t h e m a n i f o l d M b y d o t s a n d

    c o n n e c t i n g b y l i n e s t h o s e p o i n t e r p o s i ti o n s w ~ , w j t h e c o r r e -

    s p o n d i n g v e r t ic e s i , j o f w h i c h a r e a d j a c e n t i n t h e g r a p h G . I n

    ( a ) t h e g r a p h i s a s t ri n g t h a t o n l y a l l o w s a n e i g h b o r h o o d p r e -

    s e r vi n g i n v e r s e m a p p i n g f r o m G t o M , b u t n o t a n e i g h b o r h o o d

    p r e s e rv in g m a p p i n g f r o m M t o G . A d j a c e n t v e r t i c e s a r e a s -

    s i g n e d t o n e i g h b o r in g l o c a ti o n s , b u t n o t a ll t h e p o i n t e r s w ~ t h a t

    a r e n e i g h b o r i n g o n M b e l o n g t o a d j a c e n t v e r ti c e s . W i t h t h i s

    g r a p h i c a l s t r u c tu r e i t i s n o t p o s s i b l e t o fo r m a t o p o l o g y p r e -

    s e rv in g m a p o f M . I n ( b ) t h e g r a p h G h a s a t h r e e - d i m e n s i o n a l

    s t ru c t u re . I n t h i s c a s e t h e m a p p i n g f r o m M t o G , b u t n o t t h e

    i n v e r s e m a p p i n g f r o m G t o M , i s n e i g h b o r h o o d p r e s e r vi n g .

    P o i n te r s t h a t a r e n e ig h b o r i n g o n M a r e a s s i g n e d t o a d j a c e n t

    v e r ti c e s , b u t n o t a l l th e v e r t i c e s t h a t a r e a d j a c e n t i n G a r e a s -

    s i g n e d t o n e i g h b o ri n g l o c a ti o n s o n M . H e n c e , a s i n ( a ) , t h e

    g i v e n g r a p h i c a l s t r u c tu r e d o e s n o t a l l o w o n e t o fo r m a t o p o lo g y

    p r e se r vi n g m a p o f M . O n l y i n ( c ) w h e r e t h e g r a p h G i s a s q u a r e

    l a t t ic e a n d m a t c h e s t h e t o p o l o g y o f M , b o t h th e m a p p i n g f r o m

    M t o G a n d t h e i n v e r s e m a p p i n g f ro m G t o M a r e n e i g h b o r h o o d

    p r e s e r v in g . On ly in t h is c a s e d o e s G f o r m a t o p o lo g y p r e s e r v in g

    m a p o f M .

    ( w e a s s u m e t h a t a l l p o i n t e r s w , , i = 1 . . . . N , l i e o n

    M ) . A n a l o g t o n e i g h b o r h o o d p r e s e rv a t i o n o f ~ s ,

    n e i g h b o r h o o d p r e s e r v a t i o n o f t h e i n ve r se m a p p i n g

    s ~ i s g iven i f po i n t e r s w i , w j o f ad j a cen t ve r t i ces i , j

    a r e n e i g h b o r i n g o n t h e m a n i f o l d M .

    T he g r aph G wi t h i t s ve r t i ces i a s s i gned t o t he l o -

    c a t i o n s w i f o r m s a t o p o l o g y p r e s e r v i n g m a p o f M , i f

    t he m app i ng ~ i' s f r o m M t o G as we l l a s t he i nve r se

    m a p p i n g ~ f r o m G t o M i s n e i g h b o r h o o d p re s e rv i n g .

    O n l y t h e n G f o r m s a m a p o n w h i c h a d j a c e n t l o c a t io n s

    ( v e r t ic e s ) c o r r e s p o n d t o f e a t u r e s n e i g h b o r i n g o n M a n d ,

    v i c e v e r s a , w h i c h r e p r e s e n t s f e a t u r e s n e i g h b o r i n g o n

    M b y a d j a c e n t l o c a t i o n s .

    T h e q u e s t i o n a r i s es w h i c h t y p e o f g r a p h G c a n p r o -

    v i d e a t o p o l o g y p re s e r v in g m a p o f a g iv e n m a n i f o l d M .

    C r u c i a l i s G ' s c o n n e c t i v i t y s t r u c t u r e o r t o p o l o g y c o m -

    p a r e d t o t h e t o p o l o g y o f M . T h i s i s d e m o n s t r a t e d s c he -

    3 A t t h i s p o i n t w e l e a v e th e d e f i n i t i o n o f " a d j a c e n c y o f p o i n t e r s

    w i o n a m a n i f o l d M " t o t h e r e a d e r ' s i n t u i ti o n , l i ke i n m o s t c o n t r i -

    b u t i o n s o n t o p o l o g y p r e s e r v i n g f e a t u r e m a p s . A r i g o r o u s d e f i n i t i o n

    b a s e d o n V o r o n o i p o l y h e d r a i s g i v e n in t h e n e x t s e c t i o n .

    m a t i c a l l y i n F i g u r e 3 . I n e a c h o f t h e t h r e e c a s e s t h a t

    a r e d e p i c t e d th e m a n i f o l d M i s s i m p l y a s q u a r e , a n d

    t h e g r a p h G c o n s i s ts o f n i n e v e r t i c e s i, t h e a s s o c i a t e d

    p o i n t e r s w i o f w h i c h a r e d i s t r i b u t e d r e g u l a r l y o v e r t h e

    s q u a r e - s h a p e d m a n i f o l d . I n F i g u r e 3 a t h e g r a p h G h a s

    a o n e - d i m e n s i o n a l t o p o l o g y a n d c o n s is t s o f n i n e v e r t i ce s

    a r r a n g e d a s a st r in g , c o m p a r e d t o t h e t w o - d i m e n s i o n a l

    m a n i f o l d M . I n t h i s c a s e G c a n n o t f o r m a t o p o l o g y

    p r e s e r v i n g m a p o f M . A l l i t c a n a c h i e v e is a n e i g h b o r -

    h o o d p r e s e r v i n g i n v er s e m a p p i n g ~ - 1 f r o m G t o M .

    I n F i g u r e 3 b w e h a v e t h e o p p o s i t e c a s e . T h e d i m e n -

    s i o n a l it y o f G is h i g h e r t h a n t h e d i m e n s i o n a l i t y o f M .

    T h e r e s u lt is th a t a n e i g h b o r h o o d p r e s e r v i n g m a p p i n g

    f r o m M t o G is p o s s ib l e , b u t n o t a n e i g h b o r h o o d

    p r e s e r v i n g i n v e r s e m a p p i n g ~ f r o m G t o M . H e n c e ,

    a s in ( a ) , G i s n o t a b l e t o f o r m a t o p o l o g y p r e s e r v i n g

    m a p o f M . I n F i g u r e 3 c t h e t o p o l o g y o f t h e g r a p h G i s

    a s q u a r e l a t ti c e a n d c o r r e s p o n d s t o t h e t o p o l o g y o f t h e

    m a n i f o l d M . O n l y i n t h is c a s e d o e s G f o r m a t o p o l o g y

    p r e s e rv i n g m a p o f M ; t h a t i s, t h e m a p p i n g f r o m M

    t o G a s w e ll a s th e i n v e r s e m a p p i n g ~ - 1 f r o m G t o M

    i s n e i g h b o r h o o d p r e s e r v i n g .

    A n u m b e r o f n e u r a l n e t w o r k m o d e l s f o r a d a p t iv e l y

    f o r m i n g t o p o l o g y p r e se r v i n g f e a tu r e m a p s h a v e b e e n

    p r o p o s e d ( W i l l s h a w & y o n d e r M a l s b u r g , 1 9 7 6 ; y o n

    d e r M a l s b u r g & W i ll s ha w , 1 9 7 7 ; T a k e u c h i & A m a r i ,

    1 97 9 ; K o h o n e n , 1 9 8 2 a, b; D u r b i n & M i t c h i s o n , 1 9 9 0 ) .

    A m o d e l t h a t p r o v i d es a v e r y c o m p a c t p r o c e d u r e a n d ,

    t he r e f o r e , has f ound w i despr ead app l i ca t i on i n a r t if i c ia l

    n e u r a l i n f o r m a t i o n p r o c e s s i n g s y s t e m s i s K o h o n e n ' s

    s e l f- o r g a n i zi n g f e a t u r e m a p ( K o h o n e n , 1 9 8 2 a , b , 1 9 8 9 ,

    1 9 9 0 ) . T h i s a l g o r i t h m r e q u i r e s t h a t o n e f i r s t c h o o s e s

    a g r a p h G , u s u a l l y a o n e - , t w o - , o r t h r e e - d i m e n s i o n a l

    l a t t i c e . I n a s u b s e q u e n t a d a p t a t i o n p r o c e d u r e , t h e

    p o i n t e r s W g a r e d i s t r i b u t e d o v e r t h e g i v e n f e a t u r e m a n -

    i f o ld M i n s u c h a w a y, t h a t ( i ) p o i n t e r s l ie o n l y o n M ,

    a n d ( i i ) p o i n t e r s o f v e r ti c e s a d j a c e n t i n G a r e a s s i g n e d

    t o l o c a ti o n s cl o se o n M . H e n c e , K o h o n e n ' s a l g o r i t h m

    t r i e s t o f o r m a n e i g h b o r h o o d p r e s e r v i n g i n v e r s e m a p -

    p i n g -~ f r o m G t o M , b u t n o t n e c e s s a r i l y a n e i g h -

    b o r h o o d p r e s e r v in g m a p p i n g f r o m M t o G . T o o b t a i n

    a t o p o l o g y p r e s e r v i n g m a p , t h e t o p o l o g i c a l s t r u c t u r e o f

    t h e p r e s e t g r a p h G h a s t o m a t c h t h e t o p o l o g i c a l s t r u c -

    t u r e o f t h e g i v en m a n i f o l d M .

    F i g u r e 4 i l l u s t ra t e s t h r e e e x a m p l e s o f d i f f e r e n t m a p s

    t h a t e v o l v e w i t h K o h o n e n ' s a l g o r it h m , d e p e n d i n g o n

    t h e d e g r e e o f m i s m a t c h b e t w e e n t h e t o p o l o g i c a l s tr u c -

    t u r e o f th e p r e s e t g ra p h G a n d t h e g i v en m a n i f o l d M .

    I n a l l t h r e e e x a m p l e s , t h e m a n i f o l d M i s a s u b m a n i f o l d

    o f ~ 2 . I n F i gur es 4a , b , t he ma ni f o l d M aga i n i s a squa r e ,

    l i ke i n F i gur e 3 . I n F i gur e 4c t he mani f o l d M i s d i s -

    c o n n e c t e d a n d p i ec e w is e o n e - a n d t w o - d i m e n s i o n a l . I n

    F i g u r e 4 a t h e p r e s e t g r a p h G i s a t w o - d i m e n s i o n a l l a t t ic e

    a n d , h e n c e , t h e t o p o l o g y o f t h e g i v e n m a n i f o l d a n d t h e

    g r a p h d o m a t c h . A s w e c a n s e e , i n th i s ca s e K o h o n e n ' s

    s e l f- o r g a n i z in g f e a t u r e m a p a l g o r i t h m p r o v i d e s a t o -

    p o l o g y p r e s e r v i n g m a p . I n F i g u r e 4 b t h e t o p o l o g y o f

  • 7/25/2019 Martinet z 1994

    7/16

    Topology Represen t ing Networks 5 3

    (a) (b) (c)

    F I G U R E 4 . M a p s o f m a n i f o l d s f o rm e d b y K o h o n e n s s e l f- o r -

    g a n i z i n g f e a t u r e m a p a l g o r i t h m . I n ( a ) t h e g r a p h G , a s q u a r e

    l a t ti c e , h a s a t o p o l o g y t h a t c o r r e s p o n d s t o t h e t o p o l o g y o f t h e

    s q u a r e - s h a p e d f e a t u r e m a n i f o l d M . H e n c e , t h e m a p t h a t e v o l v e s

    i s to p o l o g y p r e s e r v i n g . I n ( b ) t h e g i v e n m a n i f o l d M a g a i n i s a

    s q u a r e ; h o w e v e r , t h is t i m e t h e g r a p h i s a s t ri n g a n d d o e s n o t

    c o r r e s p o n d t o p o l o g i c a l l y to M . W i t h t h i s g r a p h i t is n o t p o s s i b l e

    t o a c h i e v e a t o p o l o g y p r e s e r v in g m a p , a n d t h e K o h o n e n a l -

    g o r i t h m c a n o n l y fo r m a n e i g h b o r h o o d p r e s e r v in g i n v e r s e

    m a p p i n g ~ - 1 f r o m G t o M , b u t n o t a n e i g h b o r h o o d p r e s e r v i n g

    m a p p i n g (]D f r o m M t o G . P a r t ( c ) s h o w s a n e x a m p l e o f v e r y

    h i g h t o p o l o g i c a l m i s m a t c h b e t w e e n M a n d G . T h e g i v e n m a n -

    i f o ld M i s d i s c o n n e c t e d a n d c o n s i s t s o f a t w o - d i m e n s i o n a l ( a

    r e c t a n g l e a n d a s q u a r e ) a n d a o n e - d i m e n s i o n a l ( a l i n e a n d a

    c i r c l e ) s u b m a n i f o l d ; t h e g r a p h G i s a s q u a r e - la t t i c e . A g a i n n o

    t o p o l o g y p r e s e r v a t io n i s p o s s i b l e . I n t h is c a s e t h e K o h o n e n

    a l g o r it h m n e i t h e r a c h i e v e s a n e i g h b o r h o o d p r e s e rv i n g m a p p i n g

    f r o m M t o G n o r a n e i g h b o r h o o d p r e s e r v i n g i n v e r s e m a p p i n g

    (]D 1 f r o m G to M .

    the given manifold and of the graph do no t match. The

    graph is a one-dimensional string of vertices, in contrast

    to the two-dimensional feature manifold. As shown

    schematically in Figure 3a, because of this mismatch

    topology preservation is not possible, and Kohonen's

    algorithm can only provide a neighborhood preserving

    inverse mapping ~-1 from G to M. Figure 4c finally

    shows an example where the topological structure of

    the graph G and the given manifold M mismatch to an

    extent that neither a neighborhood preserving inverse

    mapping ~-t from G to M nor a neighborhood pre-

    serving mapping from M to G can be achieved by

    Kohonen's algorithm. The graph G is again a two-di-

    mensional lattice. The given manifold M, however, has

    a relatively intrica te topological structure. I t consists of

    a combination of two-dimensional and one-dimen-

    sional partially disconnected submanifolds. The two-

    dimensional submanifold is formed by a rectangle and

    a square. The one-dimensional submanifold consists of

    a circle and a connecting line between the rectangle

    and the circle. Figure 4c shows that in this case some

    vertices that are adjacent in G are assigned to locations

    remote on the manifold M, and, vice versa, feature vec-

    tors adjacent on Mm ay be mapped onto vertices remote

    in G. Obviously, the topology preservation of the map

    is highly disturbed.

    In Figure 3 we have demonstrated schematically hat

    topology preserving maps are only possible if the to-

    pological structure of the graph G matches the topo-

    logical structure of he given manifold M. The Kohonen

    algorithm starts from a prespecified graph G. The three

    examples of Figure 4 demonstrate that Kohonen 's al-

    gorithm tries to form a neighborhood preserving inverse

    mapping from G to M and indeed succeeds in con-

    structing a topology preserving map, if the topological

    structure o f the preset graph G matches the topological

    structure o f the manifold M. In cases, however, where

    it is not possible to a p r io r i determine an appropriate

    graph G, for example, in cases where the topological

    structure of M is not known

    a p r io r i

    or is too compli-

    cated to be specified, Kohonen's algorithm necessarily

    fails in providing perfectly topology preserving maps.

    An application where the topological structure o f M

    is not known and, at the same time, a perfectly topology

    preserving map of M is essential for optimal perfor-

    mance, is in speech recognit ion as described in Brandt

    et al. (1991). In their word recognition scheme each

    word is described by a sequence of 19-dimensional fea-

    ture vectors, with each feature vector describing the

    frequency spectrum at a different time step. Hence, a

    word is a trajectory in a 19-dimensional input space,

    given typically by a sequence o f 40-50 feature vectors.

    The feature vectors that occur with the spoken words

    form a lower-dimensional submanifold M within the

    19-dimensional input space. By constructing a topology

    preserving map of the manifold M, as it was first de-

    scribed by Kohonen et al. (1984), each word can be

    represented by a trajectory within the graph G, that is,

    by a sequence o f a c t i v e nodes. Interestingly, it turns out

    that a classification of the words based on their trajec-

    tories in G yields a significantly ncreased performance

    compared to a classification based on their trajectories

    on M. To obtain optimal performance in this word

    recognition scheme, the topological structure o f G has

    to match the topology of the feature manifold M. This

    manifold, however, has a topological struc ture that is

    complicated and not known. Its average dimension, for

    example, seems to lie between two and three (Bauer &

    Pawelzik, 1992).

    Another application where only a neighborhood

    preserving mapping from M to G together with a

    neighborhood preserving inverse mapping from G to

    M, that is, a perfectly topology preserving map of M,

    provides optimal results is in representing and learning

    input-o utput relations with topology preserving maps,

    for example, in robotics (Ritter & Schulten, 1986;

    Martinetz et al., 1990). For representing input-output

    relations, to each vertex i of G an output quantity ai is

    assigned. The ai's determine the output o f the map for

    a given input vector v. It can be shown that a robust

    and fast learning procedure for the ai's can only be

    obtained if the graph G preserves the neighborhood

    relations of the input manifold, which then allows a

    concerted adaptation of the output quantities of adja-

    cent vertices (Ritter et al., 1992). Only if adjacent ver-

    tices are assigned to locations neighboring on the input

    manifold, the learning process will be successful. The

    learning procedure is optimized if, also, all the pointers

  • 7/25/2019 Martinet z 1994

    8/16

    5 1 4 I . M a r t i n e t z a n d K . S c h u l t e n

    W t h a t a r e n e i g h b o r i n g o n t h e i n p u t m a n i f o l d c o r r e -

    s p o n d t o a d j a c e n t v e r t ic e s. T h i s i s v al i d i f f t h e m a p o f

    t h e i n p u t m a n i f o l d i s p e r fe c t l y t o p o l o g y p r e se r v i n g. I n

    m a n y a p p l i c a t i o n s , h o w e v e r , t h e i n p u t m a n i f o l d i s a

    s u b m a n i f o l d o f a h i g h - d i m e n s i o n a l i n p u t s p a ce a n d m a y

    n e i t h e r b e k n o w n a p r i o r y n o r t o p o l o g ic a l l y s im p l e

    e n o u g h f o r p r e sp e c i f y in g a c o r r e s p o n d i n g l y s t r u c t u r e d

    graph .

    F o r t h e a p p l i c a t i o n s d e s c r i b e d a b o v e i t w o u l d b e

    h i g h l y d e s ir a b le t o h a v e a p r o c e d u r e t h a t a d a p t s t h e

    t o p o l o g y o f t h e g r a p h G t o t h e t o p o l o g y o f t h e g i v e n

    m a n i f o l d M o r , a t l e as t , t o h a v e a m e a n s b y w h i c h o n e

    c a n d e c i d e w h e t h e r a c h o s e n g r a p h i s a p p r o p r i a t e f o r

    fo rm in g a topo logy prese rv ing m ap of a g iven m ani fo ld .

    A n a p p r o a c h o f t h e f i rs t k i n d h a s b e e n p r o p o s e d b y

    K o h o n e n a n d c o w o r k e rs ( K a n g a s, K o h o n e n , & L a a k -

    s o n e n , 1 9 9 0 ) . T h e i d e a o f t h i s a p p r o a c h i s ( i ) t o d e -

    t e r m i n e t h e m i n i m u m s p a n n i n g t re e b e tw e e n t h e p o i n ts

    w~ a t d i f fe ren t s t ages o f the i r adap ta t ion p roces s and

    ( i i ) t o t a k e th e r e s u l t in g m i n i m u m s p a n n i n g t r e e a s th e

    graph G tha t i s approp r ia te a t th e re spec t ive st age . Be-

    c a u s e t h e w ~ 's a t t h e e n d o f t h e a d a p t a t i o n p r o c e d u r e

    a r e d i s t r i b u t e d o n l y o v e r t h e m a n i f o l d M , t h e f i n a l

    m i n i m u m s p a n n i n g t r e e w i l l r e f l e c t a t l e a s t s o m e o f

    t h e t o p o l o gi c a l p r o p e r t ie s o f M . T h e d e f i n i t i o n o f t h e

    m i n i m u m s p a n n i n g t r e e , h ow e ve r, o n l y r e q u i r es a d j a -

    cen t ve r t i ces i to be a s s igned to ne ighbor ing loca t ions

    w~ , no t v ice ve rsa . P o in te rs wi tha t a re ad jacen t on M

    m ight ve ry we l l be long to rem ote ve r t i ces . The re fore ,

    excep t fo r spec ia l cases , th i s approach i s no t ab le to

    prov ide a pe r fec t ly topo logy prese rv ing m ap . In gene ra l ,

    o n l y t h e i n v e rs e m a p p i n g ~ - 1 f r o m G t o M b u t n o t t h e

    m a ppin g ~ I, f rom M to G wi l l be ne igh bor hoo d pre -

    serving.

    A n o t h e r a p p r o a c h o f t h e f i rs t k i n d h a s b e e n i n t r o -

    d u c e d b y F r i t z k e ( 1 99 1 ) . H i s a p p r o a c h e m p l o y s t w o -

    d i m e n s i o n a l , t r i a n g u l a r c e l l s t r u c t u r e s t h a t a r e d i s t r ib -

    u t e d o v e r t h e m a n i f o l d M . B y s e le c ti v e ly a d d i n g a n d

    rem o ving ce l ls , based on a h eur i s t i c c r i t e r ion tha t t akes

    a ce l l' s ave rage desc r ip t ion e r ro r in t o acc oun t , a g raph

    i s f o r m e d t h a t r e s e m b l e s t h e s h a p e o f th e m a n i f o l d M .

    A n a p p r o a c h o f t h e s e c o n d k i n d f o r o b t a i n i n g a n

    a p p r o p r i a t e g r a p h G h a s b e e n p r o p o s e d b y B a u e r a n d

    P a w e l z i k ( 1 9 9 2 ) . T h e y e m p l o y t h e s o - c a l l e d t o p o -

    g r a p h i c p r o d u c t a s a m e a n s t o d e t e r m i n e t h e d e g r e e t o

    which a pa r t i cu la r g raph ica l s t ruc tu re i s ab le to p rese rve

    t h e n e i g h b o r h o o d r e l a t i o n s o f a g i v en m a n i f o l d . B y

    tes t ing , fo r exam ple , one - , two- , o r th ree -d im ens iona l

    l a t t ices , one can f ina l ly choose the one th a t p rese rves

    t h e n e i g h b o r h o o d r e l a ti o n s b e st .

    In the nex t s ec t ion we wi l l in t roduce a nove l ap-

    p r o a c h t o t h e p r o b l e m o f c o n s t r u c t i n g t o p o l o g y p r e -

    s e r v in g m a p s , a n a p p r o a c h o f th e f i r st k i n d b a s e d o n

    t h e D e l a u n a y t r i a n g u l a t i o n . B e c a u s e t h e D e l a u n a y

    t r i angula t ion connec t s those ve r t i ces i the a s s igned

    p o i n t e r s w i o f w h i c h a r e a d j a c e n t b y h a v i n g n e ig h b o r i n g

    V o r o n o i p o l y h e d r a V i , th i s s t ruc tu re i s idea l ly su i t ed

    f o r t h i s p u r p o se . T h e a p p r o a c h t a k e n i s o p p o s i te t o t h a t

    o f K o h o n e n ' s s e l f - o rg a n i z in g f ea t u r e m a p . N o t t h e

    graph G bu t the po in te rs w~ a re p respec i f i ed , fo r ex-

    a m p l e , b y u s i n g a v e c t o r q u a n t i z a t i o n p r o c e d u r e t h a t

    d i s t r ibu te s them over M . S ubsequent ly , the approp r ia te

    e d g e s b e tw e e n t h e w ~ 's a r e c o n s t r u c t e d f o r f o r m i n g a

    graph G tha t de f ines a pe r fec t ly topo logy prese rv ing

    m a p o f t h e g i ve n m a n i f o l d M .

    3 1 Definit ion of Topology Preserving M aps

    I n t h e p r e v i o u s s e c t i o n th e d e f i n i t i o n o f n e i g h b o r h o o d

    p r e s e r v a t io n o f a m a p p i n g ~ f r o m M t o G o r o f a n

    inve rse m a ppin g ~ l f rom G to M was , l ike in m o s t

    con t r ibu t ions on topo logy prese rv ing fea tu re m aps , no t

    r i g o ro u s . T h e p r o b l e m i s t h a t a d j a c e n c y o f v e r t ic e s i

    in a g raph G i s c lea r ly de f ined , a p rope r de f in i t ion fo r

    ad jacency o f po in te rs w~ on M , however, i s no t obv ious .

    T h i s i s w h y t h e d e f i n i t io n o f n e i g h b o r h o o d p r e s e r v at i o n

    o f f e a t u r e m a p s h a s u s u a l l y b e e n l e f t to t h e r e a d e r ' s

    i n t u i t i o n .

    An excep t ion i s the t r iv ia l one -d im ens iona l case .

    Obvious ly , two po in t s w~ , wj in ~ t a re ne ighbor ing i f

    t h e r e i s n o p o i n t w k i n b e t w e e n . E x p r e s s e d i n t e r m s o f

    Voronoi po lyhedra , an equ iva len t de f in i t ion i s : two

    poin t s w~ wj in ~ a re ne igh bor ing i f the i r Voro noi po ly-

    hedr a a re ad jacen t , tha t i s , i f V~ f ') ~ ~ . In these

    t e r m s a g e n e r a l i z a t i o n t o h i g h e r - d i m e n s i o n a l e m b e d -

    d ing spaces ~o i s s t ra igh t fo rw ard . However , because we

    a r e i n t e r e s t e d i n a d e f i n i t i o n o f n e i g h b o r h o o d o f p o i n t s

    o n a m a n i f o l d M ,

    w e i n t r o d u c e t h e

    m a s k e d F o r o n o i

    p o l y h e d r o n . T h e m a s k e d V o r o n o i p o l y h e d r o n F IM) is

    the pa r t o f V~ tha t i s a l so pa r t o f M, tha t i s , V IM) =

    F~ f~ M . T he sup e rsc r ip t ind ica te s the dep enden ce o f

    t h e m a s k e d V o r o n o i p o l y h e d r o n o n t h e g i v e n m a n i f o l d

    M . B y u s in g th e n e i g h b o r h o o d o f th e m a s k e d V o r o n o i

    V M) V~ M)

    p o l y h e d r a _ i , i n s t e a d o f t h e n e i g h b o r h o o d o f

    t h e V o r o n o i p o l y h e d r a V i, V j f o r d e t e r m i n i n g n e i g h -

    b o r h o o d o f p o i n t s w ~, w j o n M , w e e n s u r e t h a t t w o

    poin t s wi , wj a re ca l l ed a d j a c e n t o n M o n l y i f t h e y d o

    n o t b e l o n g t o d i s c o n n e c t e d r e g i o n s o f M . T h i s l e a d s to

    the fo l lowing de f in i t ion :

    DEFINITION 1.

    L e t M ~_ .~ D b e a g i v e n m a n i f o l d a n d S

    = {w~ . . . . . WN} b e a s e t o f p o i n t s w ~ E M . T h e V o ro no i

    p o l y h e d r a o f S a r e d e n o t e d b y V ~ , i = 1 . . . . N . T w o

    p o i n t s w ~ , w j ~ M ~_ ~ D a r e a d j a c e n t o n M i f t h e i r

    M) M)

    m a s k e d V o r o n o i p o l y h e d r a V ~ = Vi f q M , V ~ =

    V . . . . M) M) j

    0 M a r e a d ) a c e n t , t h a t i s , i J V ~ a n d V . s h a r e

    a n e l e m e n t v E M o r , e q u i v a l e n t l y , i f V ~ M) f ~ v J M )

    ; ~ i s v a l i d .

    E a c h m a s k e d o r o n o i p o l y h e d r o n i s a p a r t o f t h e m a n -

    i f o ld M , a n d , i n s t e ad o f f o r m i n g a c o m p l e t e p a r t i t i o n i n g

    o f th e e m b e d d i n g s p a c e ~ D l ik e t h e V o r o n o i p o l y h e d r a ,

    t h e m a s k e d V o r o n o i p o l y h e d r a f o r m a c o m p l e t e

    p a r t i t i o n i n g o n l y o f t h e m a n i f o l d M , t h a t i s , M - -

    v (M) is valid.

    N-I -- i

    A n a l o g t o t h e d e f i n i t i o n o f t h e D e l a u n a y t r i a n g u -

    la t ion ~gs , which i s based on the Voronoi po lyhedra ,

  • 7/25/2019 Martinet z 1994

    9/16

    Topology Representing Networks 515

    we define the

    induced De la una y t r iangula t ion ~9 sM)

    through the masked Voronoi polyhedra:

    DEFINITION 2.

    L e t M ~_ ~o be a g iven man i fo ld and S

    = {wz . . . . . WN}

    b e a s e t o f p o i n t s w i E M . T h e i n d u c e d

    Delaunay tr iangulat ion ~9~sM) o f S , g iven M , i s de te r -

    m i n e d b y t h e g r ap h t h a t c o n n e c ts t w o p o i n t s

    wz, wj/ff

    their m as ke d Voronoi polyh edra V ~M), V~.M) are ad-

    j a c e n t , t h a t i s , b y t h e g r a p h w h o s e a d ja c e n c y m a t r i x

    A , A o E { O, 1 } , i , j = 1 . . . . N has the proper ty

    A~j = 1 =~ V

  • 7/25/2019 Martinet z 1994

    10/16

    5 16 7". Ma rtinetz an d K. Schulten

    m a p p i n g f r o m M t o G n o r t h e in v e rs e m a p p i n g f r o m

    G to M i s n e i g h b o r h o o d p r e s e rv i n g . I n F i g u r e 5 d t h e

    g r a p h G i s t h e i n d u c e d D e l a u n a y t r ia n g u l a t i o n o f t h e

    s e t o f p o i n t s w i a n d i s a g a i n a s u b g r a p h o f t h e D e l a u n a y

    t r ia n g u l a t i o n i n ( a ) . W i t h t h e in d u c e d D e l a u n a y t r i a n -

    g u l a t i o n t h e m a p p i n g f r o m M t o G a s w el l a s t h e i n v e rs e

    m a p p i n g f r o m G t o M i s n e i g h b o r h o o d p r e s er v i n g . T w o

    v e r t ic e s a r e c o n n e c t e d b y a n e d g e i f f t h e i r m a s k e d V o -

    r o n o i p o l y g o n s a r e a d j a c e n t . O n l y i n F i g u r e 5 d d o e s

    t h e g r a p h G f o r m a p e r f e c tl y t o p o l o g y p r e s e r v in g m a p

    o f t h e g i ve n m a n i f o l d M .

    I t i s n o t s u r p r i s i n g t h a t t h e D e l a u n a y t r i a n g u l a t i o n

    a n d t h e m i n i m u m s p a n n i n g t r e e in g e n e ra l d o n o t d e -

    f i n e a p e r f e c t l y t o p o l o g y p r e s e r v i n g m a p o f M . B o t h

    g r a p h s a r e d e t e r m i n e d o n l y b y t h e s e t o f p o in t s S =

    { w l . . . . . W N } . T h e t o p o l o g y o f th e m a n i f o l d M i n f lu -

    e n c e s b o t h g r a p h i c a l s t r u c t u r e s o n l y v e r y i n d i r e c t l y .

    O n l y t h r o u g h t h e c o n d i t i o n t h a t e a c h w i l ie s o n M d o e s

    t h e g i v e n m a n i f o l d s h a p e t h e D e l a u n a y t r i a n g u l a t i o n

    o r t h e m i n i m u m s p a n n i n g t r e e . T h i s i s d i f f e re n t fo r t h e

    i n d u c e d D e l a u n a y t r ia n g u l a ti o n . T h e i n d u c e d D e l a u -

    n a y t r ia n g u l a t io n i s d e f i n e d th r o u g h t h e a d j a c e n c y o f

    t he m aske d Vo r ono i po l yh edr a V ~M ) = Vj 71 M . Be-

    c a u s e e a c h m a s k e d V o r o n o i p o l y h e d r o n i s c o m p l e t e l y

    a p a r t o f M , t h e a d j a c e n c y o f t h e s e p a r t s c a r r i e s im -

    m e d i a t e i n f o r m a t i o n a b o u t t h e t o p o lo g i c al s t ru c t u r e o f

    t h e m a n i f o l d M .

    T h e a b o v e c o n s i d e r a t i o n s a n d d e f i n i t io n s l e a d t o th e

    f o l lo w i n g t h e o r e m :

    THEOREM 2. L e t G be a graph (ne t work ) w i t h ver t i ces"

    ( n e u r a l u n i ts ) i , i = 1 . . . . N a n d e d g e s ( c o n n e c t i o n s )

    de f i ned b y an ad j acen cy m at r i x A , A o E { O , 1 } . L e t

    M c_ ~ D b e a g i v e n m a n i f o l d o f a D - d i m e n s i o n a l

    e m b e d d i n g s p a c e a n d S = {W l . . . . . W N} b e a s e t o f

    po i n t e r s w i E M each o f wh i ch i s a t t ac hed t o a ver t ex

    i o f t h egr ap h G . T heg rap h G w i t h i t s vert ices i as s i gned

    t o t he l oca t i ons w~ on M f or m s a per f ec t ly t opo l ogy pre -

    s e r v i n g m a p o f M , i f f t h e g r a p h G i s th e i n d u c e d D e -

    lau na y t r iang ulat ion 2D (sM) o f S .

    P roo f . T h e p r o o f is s t r a ig h t f o r w a r d . I f G i s t h e i n d u c e d

    D e l a u n a y t r i a n g u l a t i o n ) (sM ) o f S , ve r t i ces i , j t ha t a r e

    a d j a c e n t i n G ( A 0 = 1 ) a re a s s i g n e d t o m a s k e d V o r o n o i

    p o l y h e d r a V IM), V } M) t h a t a r e n e ig h b o r i n g o n M , a n d ,

    V(M) V~M ) ha t

    i c e v e r sa , m a s k e d V o r o n o i p o l y h e d r a _ i ,

    a r e n e i g h b o r i n g o n M b e l o n g t o v e r t i c e s i , j t h a t a r e

    a d j a c e n t in G . T h e n t h e m a p p i n g q ' s f r o m M t o G a s

    w e l l a s t h e i n v e r s e m a p p i n g q , ~ l f r o m G t o M a r e

    n e x g h b o r h o o d p r es e r v in g , a n d , h e n c e , t h e i n d u c e d D e -

    " Z ~M r m

    a u n a y t r i a n g u l a t i o n s o f S f o s a p e r f e c t l y t o -

    p o l o g y p r es e r v in g m a p o f M .

    I f t h e g r a p h G f o r m s a p e r f e c t l y t o p o l o g y p r e s e r v i n g

    m a p o f M , t h e n th e m a p p i n g ~ s f r o m M to G as we l l

    a s t h e i n v e r s e m a p p i n g ~ f r o m G t o M a r e n e i g h -

    b o r h o o d p r e s e r v i n g . T h i s i s v a l i d i f v e r t i c e s i , j t h a t a r e

    a d j a c e n t i n G (A ~j = 1 ) a r e a s s i g n e d t o m a s k e d V o r o n o i

    p o l y h e d r a V IM), V~M ) t h a t a r e n e i g h b o r i n g o n M , a n d

    (M) (g)

    i f m a s k e d V o r o n o i p o l y h e d r a V~ , V j t h a t a r e

    n e i g h b o r i n g o n M b e l o n g t o v e r t ic e s i, j t h a t a r e a d -

    j a c e n t i n G . A c c o r d i n g t o t h e d e f i n i t i o n , G i s t h e i n -

    d u c e d D e l a u n a y tr i a n g u l a t i o n )( sM) o f S .

    3.2 . Competitive Heb bian Rule Forms Perfect

    Topology Preserving Maps

    T h e c o m p e t i t i v e H e b b r u l e c o n s t r u c t s t h e f u l l D e -

    l a u n a y t r i a n g u l a t i o n o f a s e t o f p o i n t s W l . . . . , W N o n l y

    i f e a c h V o r o n o i p o l y h e d r o n o f s e c o n d - o r d e r V0 is, at

    l eas t pa r t i a ll y , cove r ed by t he dens i t y d i s t r i bu t i on P ( v ) .

    I f w e d e f i n e t h e m a n i f o l d M a s b e in g t h e m a n i f o l d o f

    3~D o n w h i c h P ( v ) i s n o n z e r o , t w o u n i ts i , j b e c o m e

    co nn ec ted i f /V~j 71 M 4= ~ . H e n c e , i f t h e m a n i f o ld M

    f o r m s a s u b m a n i f o ld t h a t d o e s n o t c o v e r e a c h V o r o n o i

    p o l y h e d r o n o f s e c o n d o r d er , th e D e l a u n a y t r i a n g u l a t i o n

    w i ll e v o l v e o n l y p a r t l y . I f th e d i s t r i b u t i o n o f t h e p o i n t s

    wi i s dense on M i n a s ense we wi l l de f i ne be l ow, t he

    p a r t o f t h e D e l a u n a y t r ia n g u l a t io n t h a t i s f o r m e d b y

    t h e c o m p e t i t iv e H e b b r u l e is th e i n d u c e d D e l a u n a y

    t r i a n g u l a t i o n a n d , h e n c e , p r o v i d e s a p e r fe c t l y t o p o l o g y

    p r e s e rv i n g m a p o f M .

    DEFINITION 6. L e t S = { Wl . . . . . W N } b e a s e t o / p o i n t s

    w i t ha t are d i s t r i bu t ed over a g iven m an i f o l d M c_ ~ " .

    T h e d is t r ib u t i o n o f t h e p o i n t s w i E M , i = 1 . . . . N i s

    d e n s e o n M , i f f o r e a c h v E M t h e t ri a n g l e A(v, W,o,

    w ~, ) f o r m e d b y t h e p o i n t w~ t ha t i s c l osest t o v , t he po i n t

    w ~, t ha t i s s econd c l oses t t o v , an d v i t s e l f l i es co mpl e t e l y

    o n M , t h a t is , i fA(v, w~ ' Wil

    ~

    M is val id.

    A d i s t r i b u t i o n o f p o i n t s w i is d e n s e o n M a c c o r d i n g

    t o t h e a b o v e d e f i n i t io n , i f t h e d i s t r i b u t i o n i s d e n s e c o m -

    p a r e d t o t h e s t r u c t u r a l d e t a i l s o f M . A d e n s e d i s t r i -

    bu t i o n o f t he po i n t s w i r e so lves t he t opo l og i ca l s t r uc t u r e

    o f M . I f f o r e a c h s a m p l e p o i n t v E M t h e r e i s a c l o s e st

    p o i n t w i0 and a s eco nd c l oses t po i n t w~Ls u c h , t h a t t h e

    t r i ang l e A( v , W ~o, w i, ) l ie s com pl e t e l y on M , t he d i s t r i -

    b u t i o n o f th e w ~ ' s i s d e n s e o n M a c c o r d i n g t o D e f i n i t io n

    6 . I f t h e d i s t r i b u t i o n is h o m o g e n e o u s , t h e d i s t r i b u t i o n

    b e c o m e s d e n s e s i m p l y b y i n c r e a si n g th e n u m b e r N o f

    points w~.

    W i t h D e f i n i t i o n 6 w e o b t a i n t h e m a i n t h e o r e m o f

    t h i s s ec t i on :

    THEOREM 3. L e t S = { w~ . . . . . W N} b e a s e t o / p o i n t s

    w i t h a t a r e d i s tr i b u t e d o v e r a g i ve n m a n i f o l d M . I f t h e

    d i s t ri b u t io n o f t h e p o i n t s w i E M i s d e n s e o n M , t h e n

    t h e g r a p h G t h a t i s o r m e d b y t h e c o m p e t i ti v e H e b b r u l e

    i s t he i nduced De l aunay t r i angu l a t i on ) ( sM) o f S and ,

    h e n c e , f o r m s a p e rf e c tl y t o p o l o g y p r e s e r v i n g m a p o f M .

    P roo f . A n a l o g t o T h e o r e m 1 w e p r o v e th e a b o v e t h e o -

    r e m b y s h o w i n g th a t

    vC~t) " (M) 4: ~23 ,: , v M ) -~ S2~ ( 16)

    0 Vj - -u

    M)

    is valid , w ith V ~j = V o (-1 M a s t h e m a s k e d V o r o n o i

    p o l y h e d r o n o f s e c o n d o r d e r.

    If v-"~M)i V/ _iVM) 4= ~ i s val id, the re i s a v E M wi th

    v E Vi

    an d v ~ ~ . Th en we obt ain I[v - w~[[ = [ Iv -

    wi ll -< [Iv - wk[[ for a l l wk E S , and , th ere fore , v

    V ~y) is v alid.

  • 7/25/2019 Martinet z 1994

    11/16

    Topology Representing Networks

    v M) q= ~ i s v a l id , t h e r e i s a v * ~ V ~ ) w i t h

    f - i j

    a ( v * , w i , w j ) __ M , b e c a u s e t h e d i s t r i b u t i o n o f t h e

    p o i n t e r s w ] . . . . . W N i s d e n s e o n M . F o r e a c h v E

    V {/jM) he po i n t s w i and wj a r e t he t wo nea r es t ne i ghbor s .

    W i t h o u t l o s s o f g e n e r a li t y w e a s s u m e t h a t f o r v * t h e

    p o i n t w i is t h e n e a r e s t n e i g h b o r . B e c a u s e f o r e a c h u E

    v * w j t h e p o i n t w j i s e i t h e r t h e n e a r e s t o r t h e s e c o n d

    n e a r e s t n e i g h b o r o f u , a n d b e c a u s e f o r u = v * t h e p o i n t

    w i is c l oses t an d f o r u = w j the p o i n t w j is c loses t t o u ,

    t he r e i s a u* E v*w j f o r wh i ch [ [u* - w i ll = I I n * -

    w i l l

    i s v a l i d . H e n c e , w e o b t a i n u * E Vi , u* E Vj, and

    u* E A( v* , w i , w j ) c_ M , wh i ch y i e l ds M f q ( V i fq V j )

    . V M ) : ~ .

    ~: ~ or , equ ivale nt ly, V IM ) A _ j

    W i t h t h e t h e o r e m a b o v e w e h av e s h o w n t h a t t h e

    c o m p e t i ti v e H e b b r u l e f o r m s p e r f e c t ly to p o l o g y p r e -

    s e r v i n g m a p s , s u p p o s e d t h e d i s t r i b u t i o n o f t h e p o i n t s

    w i i s d e n s e o n t h e g i v e n m a n i f o l d . I n F i g u r e s 6 a n d 7

    w e s h o w t w o s i m u l a t i o n e x a m p l e s . I n F i g u r e 6 t h e

    m a n i f o l d M c o n s i s ts o f a t h r e e - d i m e n s i o n a l ( r i g h t p a r -

    a l l e l e p ip e d ) , a t w o - d i m e n s i o n a l ( r e c t a n g l e ) , a n d a o n e -

    d i m e n s i o n a l ( c i rc l e a n d c o n n e c t i n g l i n e ) s u b m a n i f o l d .

    I n it ia l ly , th e p o i n t e r s w i a r e d i s t r i b u t e d r a n d o m l y w i t h i n

    t h e s p a c e t h a t e m b e d s M , t h a t i s , w i E M i s n o t v a l i d

    n e c e s s a ri ly . T o o b t a i n w i E M f o r e a c h i , i = 1 . . . .

    N t h e p o i n t e r s a r e d i s t r ib u t e d b y e m p l o y i n g t h e n e u r a l

    g a s a l g o r i t h m . T h e n e u r a l g a s a l g o r i t h m i s a n e f fi c ie n t ,

    p a t t e r n d r i v e n v e c t o r q u a n t i z a t i o n p r o c e d u r e a n d l e a d s

    t o a h o m o g e n e o u s d i s t ri b u t i o n o f t h e p o i n t e r s wi o n M

    ( M a r t i n e t z & S c h u l t e n , 1 9 9 1 ; M a r t i n e t z , B e r k o v i c h , &

    S c h u l t e n , 1 9 9 3 ) .

    S i m u l t a n e o u s l y t o d i s t r i b u t i n g t h e p o i n t e r s t h e c o n -

    n e c t i o n s a r e f o r m e d . W i t h e a c h p r e s e n t a t i o n o f a n i n p u t

    p a t t e r n v ~ M a c y c le o f t h e c o m p e t i t i v e H e b b r u l e a s

    w e l l a s o f t h e n e u r a l g a s a l g o r i t h m i s p e r f o r m e d . F i g u r e

    6 shows t he i n i t ia l s t a te , t he ne t w or k a f t e r 5000 , 10 ,000 ,

    1 5 , 0 0 0 , 2 5 , 0 0 0 , a n d a t t h e f i n a l s t a t e a f t e r 4 0 , 0 0 0 a d -

    a p t a t i o n s t e p s . T h e p o i n t e r s w i c h a n g e t h e i r l o c a t i o n s

    s l o w ly b u t p e r m a n e n t l y a n d , t h e r e f o r e , p o i n t e r s t h a t

    a r e n e i g h b o r i n g a t a n e a r l y s ta g e o f t h e a d a p t a t i o n p r o -

    c e d u r e m i g h t n o t b e n e i g h b o r i n g a n y m o r e a t a m o r e

    a d v a n c e d s ta g e. T h o s e c o n n e c t i o n s t h a t w e r e f o r m e d

    i n t h e e a r l y s t ag e a n d a r e n o t v a l i d a n y m o r e i n a l a t e r

    s t ag e s h o u l d d i e o u t . F o r t h a t r e a s o n t h e c o m p e t i t i v e

    H e b b r u l e n o t o n l y e s ta b l i s h e s b u t a l s o r e f r e s h e s c o n -

    n e c t io n s . C o n n e c t i o n s t h a t h a v e n o t b e e n r e f r e sh e d f o r

    a w h i l e d i e o u t a n d a r e r e m o v e d . T h e a l g o r i t h m t h a t

    r e s u lt s f r o m c o m b i n i n g t h e c o m p e t i ti v e H e b b r u l e w i t h

    t h e n e u r a l g a s a l g o r i t h m i s l i st e d i n S e c t i o n 5 .

    A t t h e e n d o f t h e a d a p t a t i o n p r o c e d u r e , a f t e r t h e

    r a n d o m p r e s e n t a t i o n o f 4 0 , 0 0 0 i n p m p a t t e r n s v E M ,

    t h e c o n n e c t i v i t y s t r u c t u r e c o r r e s p o n d s t o t h e i n d u c e d

    D e l a u n a y t r i a n g u l a ti o n o f t h e f i n al p o i n t e r d i s t ri b u t i o n

    a n d f o r m s a p e r f e c t ly t o p o l o g y p re s e r v i n g m a p t h a t r e -

    f le c t s t h e d i m e n s i o n a l i t y a n d t o p o l o g i c a l s t r u c t u r e o f

    t h e g i v en m a n i f o l d M .

    I n F i g u r e 7 t h e m a n i f o l d M i s a t o r u s . I n t h i s s i m -

    u l a t io n e x a m p l e t h e p o i n t e r s w e r e d i s tr i b u t e d o v e r M

    i n a p r e p r o c e s s i n g s t ag e , a g a i n b y t h e n e u r a l g a s a lg o -

    r i t h m . A f t e r h a v i n g d i s t r i b u t e d t h e p o i n t e r s , t h e c o n -

    517

    F I G U R E 6 . T h e c o m p e t i t iv e H e b b r u l e t o g e t h e r w i t h t h e n e u r a l

    g a s a l g o r i t h m f o r m i n g a t o p o l o g y p r e s e r v i n g m a p o f a t o p o -

    l o g i c al l y h e t e r o g e n e o u s l y s t r u c t u re d m a n i f o ld . T h e g i v e n m a n -

    i f ol d M c o n s i s t s o f a t h r e e - d i m e n s i o n a l ( r i g h t p a r a l l e l e p i p e d ) ,

    a t w o - d im e n s i o n a l ( r e c t a n g l e ) , a n d a o n e - d i m e n s i o n a l ( c i r c le

    a n d c o n n e c t i n g l i n e ) s u b s e t . T h e n e u r a l g a s a l g o r i t h m a s a n

    e f f i c i e n t i n p u t d r i v e n v e c t o r q u a n t i