CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond -...

130
J|jk}ias+ca~jo Sj}sj~jetatd`e~ b`s @cgjnt Ojtjntd`e) A~~`ndatd`e aeo Cj`eo _`ka~w Kaid~djvdnw NKZ+SD+_S+11+>3 ^zckdttjo de }astdai bziiikjet `b tlj sjqzdsjkjet~ b`s tlj ojfsjj `b  O`nt`s `b ]ldi`~`}l de S`c`tdn~ _lj S`c`tdn~ De~tdtztj Nasejfdj Kjii`e Zedrjs~dt ]dtt~czsfl) ]jee~iraeda 1:31>  Azfz~t 3=11 _lj~d~ N`kkdttjj7 ]s`bj~~`s Aij|jd A% Jbs`~) Nlads ]s`bj~~`s Kastdai Ljcjst ]s`bj~~`s _ahj` Haeaoj ]s`bj~~`s ]djts` ]js`ea) Naitjnl n 3=11 c _`ka~w Kaid~djvdnw% Aii sdflt~ sj~jsrjo%

Transcript of CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond -...

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 1/130

J|jk}ias+ca~jo Sj}sj~jetatd`e~ b`s@cgjnt Ojtjntd`e) A~~`ndatd`e aeo

Cj`eo

_`ka~w Kaid~djvdnw

NKZ+SD+_S+11+>3

^zckdttjo de }astdai bzifiiikjet `b tljsjqzdsjkjet~ b`s tlj ojfsjj `b 

 O`nt`s `b ]ldi`~`}l de S`c`tdn~

_lj S`c`tdn~ De~tdtztjNasejfdj Kjii`e Zedrjs~dt 

]dtt~czsfl) ]jee~iraeda 1:31>

 Azfz~t 3=11

_lj~d~ N`kkdttjj7]s`bj~~`s Aij|jd A% Jbs`~) Nlads

]s`bj~~`s Kastdai Ljcjst

]s`bj~~`s _ahj` Haeaoj]s`bj~~`s ]djts` ]js`ea) Naitjnl

n 3=11 c _`ka~w Kaid~djvdnw% Aii sdflt~ sj~jsrjo%

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 2/130

3

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 3/130

 B`s tlj rjnt`s~)

 b`s ~z}}`stdef k sj~jasnl aii tlj~j jas~%

>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 4/130

 Ac~tsant

Sjn`fedwdef aeo sja~`edef ac`zt tlj `cgjnt~ b`zeo de ae dkafj d~ `ej `b tlj hj 

}s`cijk~ de n`k}ztjs rd~d`e% _ld~ tlj~d~ d~ ca~jo `e tlj doja tlat de `sojs t` zeojs~taeo

a e`rji `cgjnt) dt d~ `btje e`t je`zfl t` sjn`fedwj tlj `cgjnt natjf`s dt cji`ef~ t` ,d%j%)

ae~vjsdef ’Vlat d~ tld~0‛$% Vj asfzj tlat a k`sj kjaedefbzi detjs}sjtatd`e nae cj

`ctadejo c idehdef tlj de}zt `cgjnt vdtl a ~dkdias sj}sj~jetatd`e de kjk`s ,d%j%) a~hdef

’Vlat d~ tld~ idhj0‛$% De tld~ tlj~d~) vj }sj~jet a kjk`s+ca~jo ~~tjk b`s sjn`fedwdef

aeo detjs}sjtdef `cgjnt~ de dkafj~ c j~tacid~ldef rd~zai a~~`ndatd`e~ cjtvjje ae de}zt

dkafj aeo a iasfj oataca~j `b `cgjnt j|jk}ias~% _lj~j rd~zai a~~`ndatd`e~ nae tlje

cj z~jo t` }sjodnt }s`}jstdj~ `b tlj e`rji `cgjnt vldnl naee`t cj ojoznjo ~`iji bs`k

natjf`s kjkcjs~ld} ,j%f%) vldnl va d~ dt bandef0 vlat d~ dt~ ~jfkjetatd`e0 d~ tljsj a

}js~`e ~dttdef `e dt0$%

]ast D `b tld~ tlj~d~ d~ ojodnatjo t` j|jk}ias sj}sj~jetatd`e~ aeo aif`sdtlk~ b`s

nsjatdef rd~zai a~~`ndatd`e~% Vj }s`}`~j I`nai Od~taenj Bzentd`e~ aeo J|jk}ias+^RK~)

 vldnl asj tsadejo ~j}asatji b`s janl j|jk}ias aeo aii`v ae de~taenj+~}jndfin e`td`e `b 

 rd~zai ~dkdiasdt% Vj ~l`v tlat ae je~jkcij `b J|jk}ias+^RK~ }jsb`sk~ n`k}jtdtdrji 

t` ~tatj+`b+tlj+ast `e tlj ]A^NAI R@N `cgjnt ojtjntd`e ta~h% De ]ast DD) vj b`nz~ `e

tlj aoraetafj~ `b z~def j|jk}ias~ `rjs a }zsji natjf`s+ca~jo a}}s`anl% Cjnaz~j

J|jk}ias+^RK~ ~l`v f``o aidfekjet cjtvjje ojtjntd`e vdeo`v~ aeo tljds a~~`ndatjo

j|jk}ias~) vj ~l`v tlat dt d~ }`~~dcij t` tsae~bjs ae aradiacij j|jk}ias kjta+oata

,~jfkjetatd`e) fj`kjtsdn ~tszntzsj) >O k`oji) jtn%$ odsjnti `et` tlj ojtjntd`e~) vldnl

nae tlje cj z~jo a~ }ast `b `rjsaii ~njej zeojs~taeodef% Bdeaii) vj n`e~tsznt a Rd~zai

Kjkj|) a ra~t fsa}l `rjs j|jk}ias~ jen`odef c`tl rd~zai a~ vjii a~ ~}atdai sjiatd`e~ld}~)

aeo a}}i dt t` ae `cgjnt }sjodntd`e ta~h% @zs sj~zit~ ~l`v tlat j|jk}ias~ }s`rdoj a

cjttjs e`td`e `b `cgjnt n`etj|t tlae natjf`s+ca~jo a}}s`anlj~%

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 5/130

Id~t `b Bdfzsj~

1%1 Rd~zai ^e`ek~ aeo ] i~jk  % % % % % % % % % % % % % % % % % % % % % % % 3=

3%1 Ijasedef a od~taenj bzentd`e de od~taenj+t`+j|jk}ias rjnt`s ~}anj % % % % 34

3%3 J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 1/3 % % % % % % % % % % >6

3%> J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 3/3 % % % % % % % % % % ><

3%? IacjiKj ^jfkjet Iacjidef7 Jraizatdef Ijasejo Od~taenj Bzentd`e~ % % ?1

3%: J|jk}ias A~~ ndatd`e de tlj IacjiKj tj~t ~jt % % % % % % % % % % % % % % % % ??

3%2 Ojtjntdef aeo ^jfkjetdef `cgjnt~ de IacjiKj% % % % % % % % % % % % % % % % ?:

3%6 Dkafj ]as~def rda Kzitd}ij ^jfkjetatd`e~ aeo I`nai Od~taenj Bzentd`e~ ?6

>%1 @cgjnt Natjf`s Ojtjnt`s r~% Je~jkcij b J|jk}ias Ojtjnt`s~% % % % % % % ?4

>%3 Natjf`s ^RK r~% J|jk}ias+^RK~ % % % % % % % % % % % % % % % % % % % % % :=

>%> L@F katnldef n`k}asd~`e % % % % % % % % % % % % % % % % % % % % % % % % % :1

>%? Ijasedef ae J|jk}ias+~}jndfin L@F _jk}iatj % % % % % % % % % % % % % % % :3

>%: A bjv ’tsade‛ J|jk}ias+^RK~ % % % % % % % % % % % % % % % % % % % % % % % % :?

>%2 J|jk}ias+^RK naidcsatd`e % % % % % % % % % % % % % % % % % % % % % % % % % % :4

>%6 @cgjnt Ojtjntd e aeo A}}jasaenj _sae~bjs % % % % % % % % % % % % % % % % % 2>

>%< Rasdef ejfatdrj ~jt ~dwj~ % % % % % % % % % % % % % % % % % % % % % % % % % % 24

:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 6/130

>%4 ]A^NAI R@N Sj~zit~ r~% Ezkcjs `b J|jk}ias~ % % % % % % % % % % % % % % 63

>%1= ]A^NAI R@N Sj~zit~ b`s Rasdef L@F _jk}iatj ^dwj~ % % % % % % % % % % % 6?

?%1 ^jfkjetatd`e _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % % % % % 64

?%3 Qzaidtatdrj Fj`kjts _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % <=

?%> >O K`oji _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % <1

?%? Sjiatjo @cgjnt ]sdkdef % % % % % % % % % % % % % % % % % % % % % % % % % % % % <3

?%: ^znnj~~bzi J|jk}ias _sae~bjs~ 1 % % % % % % % % % % % % % % % % % % % % % % % <?

?%2 ^znnj~~bzi J|jk}ias _sae~bjs~ 3 % % % % % % % % % % % % % % % % % % % % % % % <:

?%6 J|jk}ias _sae~bjs Badizsj~ 1 % % % % % % % % % % % % % % % % % % % % % % % % % <2

?%< J|jk}ias _sae~bjs Badizsj~ 3 % % % % % % % % % % % % % % % % % % % % % % % % % <6

?%4 _sade N`enj}t Rd~zai Kjkj| % % % % % % % % % % % % % % % % % % % % % % % % % 4=

?%1= K`t`scdhj N`enj}t Rd~zai Kjkj| % % % % % % % % % % % % % % % % % % % % % % 41

?%11 Ajs`}iaej N`enj}t Rd~zai Kjkj| % % % % % % % % % % % % % % % % % % % % % % 43

:%1 _lj Rd~zai Kjkj| fsa}l % % % % % % % % % % % % % % % % % % % % % % % % % % % 4:

:%3 _ ssaica—~ N`etj|t Nlaiijefj % % % % % % % % % % % % % % % % % % % % % % % % % 42

:%> N`etj|t Nlaiijefj Jraizatd`e % % % % % % % % % % % % % % % % % % % % % % % % 1=3

:%? Qzaidtatdrj Sj~zit~ `e tlj N`etj|t Nlaiijefj % % % % % % % % % % % % % % % % 1=>

  A%1 Ns`~~+O`kade Dkafj Katnldef % % % % % % % % % % % % % % % % % % % % % % % 1=4

  A%3 ^hjtnl aeo ]adetdef Katnldef % % % % % % % % % % % % % % % % % % % % % % % % 111

  A%> ^hjtnl3Dkafj % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 113

  A%? ]adetdef3Dkafj % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 113

  A%: ]adetdef3F]^ Qzaidtatdrj J|ak}ij~ % % % % % % % % % % % % % % % % % % % % % 113

2

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 7/130

Id~t `b _acij~

3%1 ^jfkjet Oj~nsd}t`s7 1? Jijkjetas Bjatzsj~ % % % % % % % % % % % % % % % % >2

3%3 Iacji n`ebz~d`e abtjs od~taenj bzentd e ijasedef % % % % % % % % % % % % % % >4

>%1 J|jk}ias+^RK @cgjnt Ojtjntd`e Sj~zit~ `e ]A^NAI R@N 3==6% % % % % % 2?

>%3 J|jk}ias+^RK sj~zit~ vlje denizodef de+nia~~ dkafj~ ozsdef ejfatdrj

kdedef % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 26

>%> J|jk}ias+^RK sj~zit~ vdtl 3: ejfatdrj~ % % % % % % % % % % % % % % % % % % 6=

>%? J|jk}ias+^RK sj~zit~ vdtl 3:= ejfatdrj~ % % % % % % % % % % % % % % % % % 6=

>%: J|jk}ias+^RK sj~zit~ vdtl 1=== ejfatdrj~ % % % % % % % % % % % % % % % % % 6=

>%2 J|jk}ias+^RK sj~zit~ vdtl 3:== ejfatdrj~ % % % % % % % % % % % % % % % % % 6=

>%6 J|jk}ias+^RK sj~zit~ vdtl tde L@F tjk}iatj~ % % % % % % % % % % % % % % 6:

>%< J|jk}ias+^RK sj~zit~ vdtl ~kaii L@F tjk}iatj~ % % % % % % % % % % % % % 6:

>%4 J|jk}ias+^RK sj~zit~ vdtl ojbazit+~dwjo L@F tjk}iatj~ % % % % % % % % % 6:

>%1= J|jk}ias+^RK sj~zit~ vdtl iasfj L@F tjk}iatj~ % % % % % % % % % % % % % 6:

?%1 D~ tljsj a }js~`e sdodef tld~ l`s~j0 % % % % % % % % % % % % % % % % % % % % % <>

6

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 8/130

N`etjet~

1 Dets`ozntd`e 1>

1%1 Canhfs`zeo `e Natjf sdwatd e _lj`sdj~ % % % % % % % % % % % % % % % % % % 12

1%1%1 ]s`cijk~ Vdtl Rd~zai Natjf`sdj~ % % % % % % % % % % % % % % % % % % 14

1%1%3 Vl Natjf`sdwj0 % % % % % % % % % % % % % % % % % % % % % % % % % % % 3=

1%3 Sjiatjo V`sh  % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 31

1%> _lj~d~ @rjsrdjv % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 3?

]ast D7 Ijasedef J|jk}ias Sj}sj~jetatd`e~ 3:

3 I`nai Od~taenj Bzentd`e~ 36

3%1 Ijasedef Od~taenj Bzentd`e~ Vdtl Bsdjeo~ % % % % % % % % % % % % % % % % % 34

3%3 Sjiatjo V`sh  % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % >>

3%> Jraizatdef I`nai Od~taenj Bzentd`e~ `e IacjiKj % % % % % % % % % % % % % % >?

3%>%1 ^jfkjet+ca~jo Bjatzsj~ % % % % % % % % % % % % % % % % % % % % % % % >:

3%>%3 Od~taenj Bzentd`e Ijasedef Ojtadi~ % % % % % % % % % % % % % % % % % >:

3%>%> ^jfkjet Iacjidef % % % % % % % % % % % % % % % % % % % % % % % % % % ?=

3%>%? ]as~def Dkafj~ rda Kzitd}ij ^jfkjetatd`e~ % % % % % % % % % % % % ?1

3%>%: B`ii`v+z} V`sh  % % % % % % % % % % % % % % % % % % % % % % % % % % % % ?:

<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 9/130

> J|jk}ias+^RK~ ?<

>%1 Vl` Ejjo~ bsdjeo~0 Ijasedef Vdtl a ^defij ] ~dtdrj De~taenj % % % % % % :3

>%1%1 Sjiatd`e~ld} t` I`nai Od~taenj Bzentd`e~ % % % % % % % % % % % % % % ::

>%1%3 Detjs}sjtdef ^RK~ tsadejo vdtl a ~defij }`~dtdrj de~taenj % % % % % :2

>%3 Naidcsatd e % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % :6

>%> J|jk}ias N`+`nnzssjenj Katsdnj~ % % % % % % % % % % % % % % % % % % % % % % :4

>%? Jraizatd`e `b J|jk}ias+^RK~ % % % % % % % % % % % % % % % % % % % % % % % % 21

>%?%1 ]A^NAI R@N @cgjnt Ojtjntd e _a~h  % % % % % % % % % % % % % % % % % 23

>%: J|jk}ias+^RK Aeai~d~ % % % % % % % % % % % % % % % % % % % % % % % % % % % 22

>%:%1 Denizodef ^akj+Natjf`s De~taenj~ de tlj Ejfatdrj ^jt % % % % % % 22

>%:%3 ^dwj `b Ejfatdrj ^jt % % % % % % % % % % % % % % % % % % % % % % % % % 2<

>%:%> Oj}jeojenj @e tlj Ezkcjs `b J|jk}ias~ % % % % % % % % % % % % % 61

>%:%? _lj S ij b Bjatzsj Odkje~d`eaidt  % % % % % % % % % % % % % % % % % 6>

]ast DD7 Cj`eo @cgjnt Ojtjntd e 62

? A~~`ndatd`e aeo Kjta+oata _sae~bjs 6<

?%1 ^jfkjetatd`e aeo Fj`kjts _sae~bjs % % % % % % % % % % % % % % % % % % % % 64

?%3 >O K`oji _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % <=

?%> Sjiatjo @cgjnt ]sdkdef % % % % % % % % % % % % % % % % % % % % % % % % % % % % <3

?%? J|jk}ias De+}adetdef % % % % % % % % % % % % % % % % % % % % % % % % % % % % % <>

?%: Rd~zaidwdef J|jk}ias+J|jk}ias A~~`ndatd`e~ % % % % % % % % % % % % % % % % <<

: N`etj|tzai @cgjnt ]sjodntd`e 4>

:%1 Czdiodef a IacjiKj Rd~zai Kjkj| Fsa}l % % % % % % % % % % % % % % % % % % 4:

4

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 10/130

:%3 _ ssaica—~ N`etj|t Nlaiijefj % % % % % % % % % % % % % % % % % % % % % % % % % 42

:%> Debjsjenj de tlj Rd~zai Kjkj| K`oji % % % % % % % % % % % % % % % % % % % % 4<

:%? N`etj|t Nlaiijefj Jraizatd`e7 Rd~zai Kjkj| rjs~z~ Natjf`sdj~ % % % % % % 1 = 3

:%?%1 B`ii`v+z} V`sh  % % % % % % % % % % % % % % % % % % % % % % % % % % % % 1=:

2 N`eniz~d`e7 _ vaso tlj Rd~zai Kjkj| 1=2

  A Ns`~~+O`kade Dkafj Katnldef 1=4

C J|jk}ias+^RK~7 Ae @}je ^`zsnj Dk}ijkjetatd e 11>

C%1 J|jk}ias+^RK _sadedef ]asakjtjs~ % % % % % % % % % % % % % % % % % % % % % 11?

C%3 ^idodef Vdeo`v~ aeo Bjatzsj ]sakdo~ % % % % % % % % % % % % % % % % % % % 11?

Kzitd~naij L@F ]sakdo % % % % % % % % % % % % % % % % % % % % % % % 112

I`naidwdef kae j|jk}ias~ rda a ~defij katsd| kzitd}idnatd`e % % 114

C%> J|jk}ias+^RK sze+tdkj n`k}ij|dt % % % % % % % % % % % % % % % % % % % % % 13=

1=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 11/130

11

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 12/130

 Anhe`vijofjkjet~

D v`zio fis~t idhj t` tlaeh k aord~`s) Aij|jd ,Ai`~la$ Jbs`~) b`s }s`rdodef kj vdtl tlj

jen`zsafjkjet t` }zs~zj tlj cdf }s`cijk~ de n`k}ztjs rd~d`e% E`t `ei la~ lj cjje a

kjet`s) czt lj ai~` }iajo ~jrjsai `tljs hj s`ij~ de k ~ndjetdfin ojrji`}kjet7 a bsdjeo)

a nsdtdn) a rd~d`eas) a }ldi`~`}ljs) aeo k`~t dk}`staeti) a n`iiac`sat`s%

Dt v`zio cj ~dii t` tldeh tlat k osjak `b czdiodef detjiidfjet kanldej~ ~tastjo

 vlje D jetjsjo fsaozatj ~nl``i% Cjb`sj n`kdef t` NKZ) k bakdi jen`zsafjo kj

t` }zs~zj k osjak~) aeo tlz~ e`ej `b tld~ v`sh v`zio larj cjje }`~~dcij vdtl`zt

tlj ~z}}`st `b k bakdi) j~}jndaii k k`tljs W`fia Kaid~djvdnw aeo k cs`tljs Katt

Kaid~djvdnw%

D v`zio ai~` idhj t` tlaeh k bjii`v S`c`fsao~ b`s kahdef fsaozatj ~nl``i tszi 

akawdef% De }astdnzias) D v`zio idhj t` tlaeh Fj`bb L`iidefjs) C`sd~ ^`bkae) G`eatlae

Lzaef) Kash ]aiatznnd) aeo Laee~ _a}}jdejs b`s lji}def kj sjaidwj tlat tlj 3==:

S`c`fsao~ vjsj ae akawdef czenl% _lj `tljs rd~d`easdj~ ,^aet`~l Odrraia) ^azsacl

^defl) Acldear ^lsdra~tara$ lji}jo kahj VAS] lanhdef jeg`acij aeo vdtl`zt tljds

lji} aeo vdiidefej~~ t` id~tje t` k }s`cijk~) tl`~j detje~j iatj+edflt ojczffdef

~j~~d`e~ v`zio e`t larj cjje }`~~dcij% Aeo fieaii) D v`zio idhj t` tlaeh Gjeedbjs Cs`ve

 vdtl`zt vl`k k tdkj at NKZ v`zio larj cjje ~dfedfinaeti ij~~ jeg`acij%

Sjfasodef bzeodef) D v`zio idhj t` tlaeh tlj Eatd`eai ^ndjenj B`zeoatd`e) b`s

avasodef kj tlj E^B Fsaozatj Sj~jasnl Bjii`v~ld}) vldnl lji}jo bzeo k sj~jasnl

bs`k 3==2 zetdi 3==4% _lj fieai jas `b k sj~jasnl at NKZ va~ kaoj }`~~dcij c a

sj~jasnl avaso bs`k tlj K^S+NKZ Njetjs b`s N`k}ztatd`eai _ldehdef%

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 13/130

Nla}tjs 1

Dets`ozntd`e

’@zs dej}tdtzoj de fjttdef at tlj sjn`so d~ iasfji naz~jo c tlj astdfindaidt `b 

~~tjk~ `b deoj|def% Vlje oata `b ae ~`st asj }ianjo de ~t`safj) tlj asj fiijo

ai}lacjtdnaii `s ezkjsdnaii) aeo deb`skatd`e d~ b`zeo ,vlje dt d~$ c tsandef dt

o`ve bs`k ~zcnia~~ t` ~zcnia~~%%% _lj lzkae kdeo o`j~ e`t v`sh tlat va% Dt

`}jsatj~ c a~~`ndatd`e% Vdtl `ej dtjk de dt~ fsa~}) dt ~ea}~ de~taeti t` tlj ej|t

tlat d~ ~zffj~tjo c tlj a~~`ndatd`e `b tl`zflt~) de ann`soaenj vdtl ~`kj detsdnatj

 vjc `b tsadi~ nassdjo c tlj njii~ `b tlj csade%%% ^jijntd`e c a~~`ndatd`e) satljs tlae

c deoj|def) ka jt cj kjnlaedwjo% @ej naee`t l`}j tlz~ t` jqzai tlj ~}jjo aeo

ffij|dcdidt vdtl vldnl tlj kdeo b`ii`v~ ae a~~`ndatdrj tsadi) czt dt ~l`zio cj }`~~dcij

t` cjat tlj kdeo ojnd~drji de sjfaso t` tlj }jskaejenj aeo niasdt `b tlj dtjk~

sj~zssjntjo bs`k ~t`safj% N`e~dojs a bztzsj ojrdnj b`s deodrdozai z~j) vldnl d~ a

~`st `b kjnlaedwjo }sdratj fiij aeo idcsas% Dt ejjo~ a eakj) aeo t` n`de `ej at

saeo`k) ’kjkj|‛ vdii o`% A kjkj| d~ a ojrdnj de vldnl ae deodrdozai ~t`sj~ aii

ld~ c``h~) sjn`so~) aeo n`kkzednatd`e~) aeo vldnl d~ kjnlaedwjo ~` tlat dt ka 

cj n`e~zitjo vdtl j|njjodef ~}jjo aeo ffij|dcdidt% Dt d~ ae jeiasfjo detdkatj~z}}ijkjet t` ld~ kjk`s%

 Raeejras Cz~l) A~ Vj Ka _ldeh ,14?:$

1>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 14/130

 Raeejras Cz~l—~ rd~d`e) a~ j|}`zeojo de tlj ac`rj j|njs}t bs`k ld~ ~jkdeai 14?:

j~~a) ’A~ vj ka tldeh‛ XCz~l) 14?:[) ~tjk~ bs`k tlj sjaidwatd`e tlat tljsj d~ a fsjat

kd~katnl cjtvjje tlj va de vldnl deb`skatd`e d~ t}dnaii `sfaedwjo aeo tlj va de

 vldnl tlj lzkae kdeo v`sh~ ,d%j%) natjf`sdnai deoj|def rjs~z~ a~~`ndatd`e$% N`enjsejo

 vdtl tlj tsae~kd~~d`e aeo annj~~dcdidt `b ~ndjetdfin doja~) Cz~l bazitjo tlj ’astdfindaidt `b 

~~tjk~ `b deoj|def‛ aeo }s`}`~jo tlj Kjkj|1) a }l~dnai ojrdnj vldnl v`zio cj z~jo

t` sjtsdjrj deb`skatd`e ca~jo `e a~~`ndatdrj ideh~ de~tjao `b ~tsdnt natjf`sdnai deoj|def%

_lj a~~`ndatdrj ideh~ vjsj t` cj jetjsjo kaezaii c tlj z~js aeo n`zio cj `b ~jrjsai

odbbjsjet t}j~% Nlade~ `b ideh~ v`zio b`sk det` i`efjs a~~`ndatdrj tsadi~) nsjatdef ejv

eassatdrj~ de tlj n`enj}t ~}anj% Cz~l cjidjrjo tlat tlj Kjkj| v`zio cj deraizacij t` a

~ndjetd~t) antdef a~ ae j|tjeojo ~kast kjk`s vldnl v`zio e`t `ei ~t`sj tlj a~~`ndatdrj

tsadi~ j~tacid~ljo ozsdef s`ztdej sj~jasnl) czt n`zio ai~` azt`katdnaii ~zffj~t e`rji

a~~`ndatd`e~ vldnl kdflt e`t cj `crd`z~ t` a lzkae `}jsat`s%

K`tdratjo c tlj doja tlat ’tlj }s`nj~~ `b tdef tv` dtjk~ t`fjtljs d~ tlj dk}`staet

tldef)‛ Cz~l cjidjrjo tlj Kjkj| v`zio sjr`iztd`edwj tlj va vj `sfaedwj aeo sjtsdjrj

deb`skatd`e% Cz~l—~ Kjkj|) kznl idhj Aiae _zsdef—~ _zsdef Kanldej) va~ ae attjk}t t`

k`oji vlat dt kjae~ t` cj a n`fedtdrj afjet8 l`vjrjs) vldij _zsdef }s`}`~jo tlj _zsdef

Kanldej de ae attjk}t t` kjnlaed~j ascdtsas sja~`edef ,aif`sdtlk j|jnztd`e$) Cz~l

tsdjo t` kjnlaed~j tlj a~~`ndatdrj }s`nj~~j~ vldnl asj aeai`f`z~ t` z~def kjk`s X^h+

afj~tao) 1442[% Vldij tlj _zsdef Kanldej cjnakj tlj ca~d~ `b k`ojse n`k}ztatd`e)

tlj Kjkj| va~ ~jje ojnaoj~ iatjs a~ }d`ejjsdef l}jstj|t aeo tlj V`sio Vdoj Vjc%

N`e~dojsdef tlj hj s`ij `b tlj detjsejt de k`ojse idbj) dt d~ ~abj t` ~a tlat kae `b 

Cz~l—~ doja~ asj qzdtj aidrj t`oa% _lj va de vldnl vj fieo deb`skatd`e `e tlj detjsejt

t`oa d~ k`sj ~dkdias t` Cz~l—~ Kjkj| tlae tlj va de vldnl a ~ndjetd~t z~jo t` fieo

sjijraet deb`skatd`e at tlj idcsas ,canh de 14?:$%

1 A }`stkaetjaz `b kjk`s aeo deoj|) `s kjk`s j|tjeojs%

1?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 15/130

_lj ~znnj~~ `b tlj detjsejt la~ osakatdnaii nlaefjo tlj va vj tldeh ac`zt

he`vijofj% De ld~ c``h) ’Jrjstldef d~ Kd~njiiaej`z~7 _lj ]`vjs `b tlj Ejv Odfdtai

Od~`sojs‛ XVjdecjsfjs) 3==6[) Oardo Vjdecjsfjs j|}`zeo~ tlj rdjv tlat vj) a~ a ~`ndjt)

~l`zio jkcsanj odfdtai he`vijofj ,j%f%) Vdhd}joda) F``fij$ cjnaz~j `enj deb`skatd`e

d~ }ianjo `eidej) dt cjn`kj~ ~dfedfinaeti k`sj z~jbzi% Zeidhj c``h~ aeo `tljs }l~dnai

jetdtdj~ vldnl kz~t cj }l~dnaii `sfaedwjo de >O ~}anj ,j%f%) `e c``h~ljirj~$) odfdtai

aeo l}js+idehjo he`vijofj d~ kznl k`sj rjs~atdij cjnaz~j tljsj d~ e` ejjo t` j|}idndti 

a~~dfe dtjk~ t` a ~defij ~zcgjnt/natjf`s% Idhj Vjdecjsfjs) Nia ^ldsh) a }s`kdejet

tldehjs vl` vsdtj~ ac`zt tlj detjsejt aeo `tljs ojnjetsaidwjo tjnle`i`fdj~) asfzj~

jijfaeti afade~t natjf`sdj~ de ld~ astdnij) ’@et`i`f d~ @rjssatjo7 Natjf`sdj~) Ideh~)

aeo _af~‛ X^ldsh) 3==:[% ^ldsh cjidjrj~ de tlj }`vjs `b tv` }`vjsbzi czdiodef ci`nh~

 vldnl nae cj z~jo t` `sfaedwj deb`skatd`e7 1$ ideh~) vldnl nae }`det t` aetldef) aeo

3$ taf~) vldnl asj a va `b attanldef iacji~ t` ideh~% ]s`rdodef kzitd}ij asfzkjet~

afade~t tsaodtd`eai natjf`sdwatd`e ~nljkj~) ^ldsh asfzj~ tlat ’vlat vj—sj ~jjdef vlje

 vj ~jj tlj Vjc d~ antzaii a saodnai csjah vdtl }sjrd`z~ natjf`sdwatd`e ~tsatjfdj~) satljs

tlae ae j|tje~d`e `b tljk%‛ Ideh~ dk}i tlat tljsj vdii fjejsaii j|d~t kzitd}ij }atl~

 vldnl vj n`zio tsarjs~j t` fieo tlat `ej nszndai cdt `b deb`skatd`e vj asj i``hdef b`s%

E`t aii csaenlj~ `b ~ndjenj larj bzii bjit tlj vsatl `b tld~ ejv odfdtai od~`sojs%

@ej kdflt asfzj tlat tlj }s`cijk `b `cgjnt sjn`fedtd`e) a~ t}dnaii b`zeo de tlj fijio

`b n`k}ztjs rd~d`e aeo s`c`tdn~) banj~ kae }s`cijk~ vldnl) z~def Cz~l—~ v`so~) asj

’iasfji naz~jo c tlj astdfindaidt `b ~~tjk~ `b deoj|def%‛ _lj z~j `b natjf`sdj~ ,nia~~j~$

t` sj}sj~jet rd~zai n`enj}t~ d~ ~` }sjraijet de n`k}ztjs rd~d`e aeo kanldej ijasedef

tlat k`~t sj~jasnljs~ o` e`t fdrj dt a ~jn`eo tl`zflt% Banjo vdtl a ejv ta~h) `ej ~dk}i 

nasrj~ z} tlj ~`iztd`e ~}anj det` nia~~j~ ,j%f%) nas~) }j`}ij) czdiodef~$) a~~dfe~ nia~~

iacji~ t` tsadedef j|ak}ij~ aeo a}}idj~ `ej `b tlj kae }`}zias kanldej ijasedef t``i~

t` `ctade a nia~~dfijs%

1:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 16/130

De tld~ v`sh) vj asj aor`natdef a odbbjsjet va `b tldehdef ac`zt sjn`fedtd`e ‟ e`t

a~ `cgjnt eakdef) czt satljs a~ `cgjnt a~~`ndatd`e% _lj doja) ~zffj~tjo c jrdojenj bs`k

n`fedtdrj ~ndjenj) d~ tlat tlj njetsai qzj~td`e `b sjn`fedtd`e kdflt e`t cj ’Vlat d~ dt0‛

czt satljs ’Vlat d~ dt idhj0‛ XCas) 3==6[% _lj jtk`i`f `b tlj rjs v`so ’sj+n`fedwj‛

,t` he`v afade$ ~z}}`st~ tlj rdjv tlat a~~`ndatd`e }ia~ a hj s`ij de sjn`fedtd`e%

Zeojs tld~ k`oji) vlje banjo vdtl a e`rji `cgjnt) tlj ta~h d~ t` a~~`ndatj dt vdtl tlj

k`~t ~dkdias `cgjnt~ de `ej—~ kjk`s% _lj~j sjkjkcjsjo `cgjnt~) de tzse) }s`rdoj tlj

kjta+oata ,j%f%) `cgjnt eakj) fj`kjts) a~~`ndatjo antd`e~$ ejjojo t` detjs}sjt tlj

e`rji `cgjnt% _ld~ aii`v~ b`s a oeakdn ojfiedtd`e `b natjf`sdj~ ca~jo `e oata aradiacdidt 

aeo ta~h ,j%f%) ae `cgjnt nae cj a rjldnij) a nas) a R`ir`) `s C`c—~ R`ir`$%

 Ae dk}`staet cjejfit `b `cgjnt a~~`ndatd`e `rjs `cgjnt eakdef d~ tlat tljsj d~ e` ejjo

t` odrdoj tlj v`sio z} det` sdfdo) }sj+ojfiejo natjf`sdj~ a }sd`sd% De~tjao) janl `cgjnt

de~taenj z~j~ dt~ ejasj~t ejdflc`s~ ,de ~`kj bjatzsj ~}anj$ t` debjs dt~ `ve dojetdt) a~

fjejsai `s a~ ~}jndfin a~ tlj aradiacij oata aii`v~% B`s j|ak}ij) db `zs oata~jt o`j~ e`t

n`etade kae nas~) tlje tlj cj~t tlat vj nae ~a ac`zt a ejv nas de~taenj d~ tlat dt nae

cj katnljo t` ae`tljs ’nas‛% Czt a~ tlj ezkcjs `b odbbjsjet nas~ de tlj oata~jt fs`v~)

 vj ~l`zio cj acij t` fieo rjs ~}jndfin nas katnlj~ vldnl vdii aii`v z~ t` sjn`fedwj tlj

~akj `cgjnt de~taenj a~ ’sjo L`eoa Ann`so‛%

1%1 Canhfs`zeo `e Natjf`sdwatd`e _lj`sdj~

’Sj~tsdntdef tlj sj}sj~jetatd`e~ ojsdrjo bs`k ~njej~ t` cjdef n`enj}tzai ak`zet~

t` dk}`~def a ~jrjsj laeodna} `e tlj rd~zai ~~tjk‛

^ldk`e Jojikae

_` zeojs~taeo ae dkafj d~ t` sjn`fedwj dt~ n`e~tdtzjet~ `cgjnt~ vljsj `cgjnt ’sjn`f+

edtd`e‛ d~ z~zaii a~~zkjo t` kjae `cgjnt eakdef ‟ fdrje ae dkafj) tlj f`ai d~ t`

12

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 17/130

eakj tlj oj}dntjo `cgjnt~ ,aeo }`~~dci ~l`v tlj `cgjnt~— ~}atdai j|tjet$% Czt ~denj `zs

iaefzafj o`j~ e`t larj a eakj b`s jrjs }`~~dcij `cgjnt de~taenj) tld~ sjqzdsj~ tlat

`cgjnt natjf`sdj~ cj z~jo b`s eakdef }zs}`~j~% L`vjrjs) f`def bs`k `cgjnt~ t` `cgjnt

natjf`sdj~ d~ ae j|tsjkji e`d~ aeo i`~~ }s`nj~~7 ’a }dntzsj d~ v`stl a tl`z~aeo v`so~‛

‟ e`t `ej `s tv` t}dnaii z~jo b`s natjf`sdwatd`e% Cjb`sj od~nz~~def tlj n`etsdcztd`e~ `b 

tld~ tlj~d~) dt d~ v`stlvldij f`def canh de tdkj t` cjttjs zeojs~taeo tlj s`ij natjf`sdj~

larj }iajo de rd~d`e aeo sjiatjo od~nd}idej~%

_lj`sdj~ `b natjf sdwatd e oatj canh t` tlj aendjet Fsjjh~% Asd~t tij ojfiejo nat+

jf`sdj~ a~ od~nsjtj jetdtdj~ nlasantjsdwjo c a ~jt `b }s`}jstdj~ ~lasjo c aii tljds

kjkcjs~ X Asd~t`tij) [% Ld~ natjf`sdj~ asj kztzaii j|niz~drj) aeo jrjs kjkcjs `b 

a natjf`s d~ jqzai% _ld~ nia~~dnai rdjv d~ ~tdii tlj k`~t vdoji annj}tjo va `b sja+

~`edef ac`zt natjf`sdj~ aeo ta|`e`kdj~ de laso ~ndjenj~%3 L`vjrjs) a~ }`detjo `zt

c Izovdf Vdttfje~tjde) tld~ d~ aik`~t njstadei e`t tlj va k`~t `b `zs jrjsoa 

n`enj}t~ v`sh ,j%f%) vlat d~ tlj ~jt `b }s`}jstdj~ tlat ojfiej tlj n`enj}t ’fakj‛ aeo

e`tldef ji~j0 XVdttfje~tjde) 14:>[$% Jk}dsdnai jrdojenj b`s t}dnaidt ,j%f%) a s`cde

d~ a k`sj n`kk`ei ndtjo j|ak}ij `b ’cdso‛ tlae a nldnhje$ aeo kzitd}ij natjf`s 

kjkcjs~ld}~ ,j%f%) nldnhje d~ c`tl ’cdso‛ aeo ’b``o‛$ bzstljs n`k}idnatj tlj Asd~t`tjidae

 rdjv% Vdttfje~tjde asfzjo tlat tldef~ sjbjssjo t` c tlj ~akj eakj asj e`t ejnj~~asdi 

n`eejntjo c `ej j~~jetdai bjatzsj ‟ de~taenj~ cji`efdef t` a ~defij n`enj}t kdflt cj

n`eejntjo c `rjsia}}def ~dkdiasdtdj~) zeojs tlj doja he`ve a~ bakdi sj~jkciaenj~%

K`tdratjo c Vdttfje~tjde—~ e`td`e `b bakdi sj~jkciaenj~) tlj fs`zeo+csjahdef

 v`sh `b n`fedtdrj }~nl`i`fd~t Jijae`s S`~nl XS`~nl aeo Kjsrd~) 146:) S`~nl) 146<[

ojk`e~tsatjo tlat lzkae~ o` e`t nzt z} tlj v`sio det` ejat natjf`sdj~ ojfiejo c 

~lasjo }s`}jstdj~) czt de~tjao z~j ~dkdiasdt  a~ tlj ca~d~ `b natjf`sdwatd`e% S`~nl—~

}s`t`t}j tlj`s }`~tziatj~ tlat ae `cgjnt—~ natjf`s d~ ojtjskdejo c dt~ ~dkdiasdt t`

3_ld~ d~ tlj sja~`e vl tlj ojcatj sjfasodef ]izt`—~ ~tatz~ a~ a }iaejt la~ cjje ~` rdnd`z~%

16

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 18/130

,a ~jt `b$ }s`t`t}j~ vldnl ojfiej janl natjf`s) aii`vdef b`s rasdef ,fsaojo$ ojfsjj

`b kjkcjs~ld}% ^znl }s`t`t}j k`oji~ larj cjje ~znnj~~bzii z~jo b`s `cgjnt sjn`fed+

td`e XCa~sd) 1443) Jojikae) 144:[% F`def jrje bzstljs) j|jk}ias tlj`s XKjode aeo

^nlabbjs) 146<) E`~`b~h) 14<2[ sjgjnt~ tlj ejjo b`s ae j|}idndt natjf`s sj}sj~jetatd`e)

asfzdef de~tjao tlat a n`enj}t nae cj dk}idndti b`skjo rda aii dt~ `c~jsrjo de~taenj~% De

j|jk}ias tlj`s) tljsj d~ e` ejjo t` b`sk ae j|}idndt ac~tsantd`e ~znl a~ a }s`t`t}j ‟

natjf`sdwatd`e d~ }jsb`skjo c katnldef a e`rji de}zt t` a iasfj ~jt `b tsadedef de~taenj~%

B`s a fsjat dets`ozntd`e t` tlj`sdj~ natjf`sdwatd`e ,denizodef }s`t`t}j aeo j|jk}ias

tlj`sdj~$) vj sjbjs tlj sjaojs t` Fsjf`s Kzs}l—~ Cdf C``h `b N`enj}t~ XKzs}l)

3==3[%

_ljsj d~ ai~` aczeoaet n`etjk}`sas jrdojenj de tlj fijio `b n`fedtdrj ~ndjenj vldnl

~zffj~t~ tlat tljsj d~ k`sj t` rd~zai zeojs~taeodef tlae nae cj na}tzsjo c sdfdo

natjf`sdj~% C ~tzodef lzkae ~zcgjnt~) Azoj @idra aeo n`iijafzj~ XCsao jt ai%) 3==4[

larj ~l`ve tlat rd~zai kjk`s la~ a ka~~drj ~t`safj na}andt b`s `cgjnt ojtadi~% _ljds

fieodef~ ~zffj~t tlat rd~zai i`ef+tjsk sj}sj~jetatd`e~ asj k`sj ojtadijo tlae }sjrd`z~i 

tl`zflt) aeo ~l`v tlat ’rd~zai i`ef+tjsk kjk`s sj}sj~jetatd e~ nae n`etade e`t `ei 

fd~t deb`skatd`e c ai~` ojtadi~ ~zbfindjet t` od~nsdkdeatj cjtvjje j|jk}ias~%‛ XCsao 

jt ai%) 3==4[ _ld~ ~zffj~t~ tlat lzkae~ o` e`t ~dk}i ac~tsant+ava `cgjnt ojtadi~ c 

}iandef tljk det` ejat aeo od~tdent natjf`sdj~%

Fzdojo c detzdtd`e bs`k tlj`sdj~ `b lzkae rd~zai }jsnj}td`e aeo jrdojenj bs`k

n`fedtdrj ejzs`~ndjenj) K`~lj Cas `ztidej~ tlj dk}`staenj `b aeai`fdj~) a~~`ndatd`e~)

aeo }sjodntd`e~ de tlj lzkae csade XCas) 3==4[% Lj asfzj~ tlat tlj f`ai `b rd~zai

}jsnj}td`e d~ e`t t` sjn`fedwj ae `cgjnt de tlj tsaodtd`eai ~je~j `b natjf`sdwdef dt ,d%j%)

a~hdef ’Vlat d~ tld~0‛$) czt de~tjao idehdef tlj de}zt vdtl ae aeai`f`z~ sj}sj~jetatd`e

de kjk`s ,d%j)% a~hdef ’Vlat d~ tld~ idhj0‛$%> @enj a e`rji de}zt d~ idehjo vdtl

>Cas z~j~ tlj tjsk ’aeai`f‛ b`s tlj }s`nj~~ `b j~tacid~ldef a ideh cjtvjje a }sjrd`z~i ~jje `cgjntaeo a e`rji `ej) vldij vj ~dk}i sjbjs t` tlj~j ideh~ a~ rd~zai a~~`ndatd`e~%

1<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 19/130

aeai`f`z~ sj}sj~jetatd`e~) a~~`ndatjo sj}sj~jetatd`e~ asj antdratjo sa}doi aeo }sjodnt

tlj sj}sj~jetatd`e~ `b vlat d~ k`~t idhji t` `nnzs ej|t% Cas ~zffj~t~ tlat tlj }sdend}ij

`b }sjodntd`e d~ a ~ts`ef naeodoatj b`s a zedrjs~ai }sdend}ij tlat nae j|}iade a kag`sdt `b 

tlj csade—~ `}jsatd`e XCas) 3==4[% Czt }sjodntd`e) a~ asfzjo c Cas) sjidj~ `e tlj acdidt 

t` j~tacid~l kjaedefbzi aeai`fdj~ ,`s z~def `zs tjskde`i`f) rd~zai a~~`ndatd`e~$%

1%1%1 ]s`cijk~ Vdtl Rd~zai Natjf`sdj~

Vj sjbjs t` `cgjnt~ c tlj natjf`sdj~ tlj cji`ef t` ~` jbb`stij~~i de `zs oadi idrj~ tlat

dt d~ e`t nijas vl kanldej~ ~l`zio larj ~znl a laso tdkj }jsb`skdef tlj ~akj ta~h%

L`vjrjs) dt d~ zenijas vljtljs natjf`sdwatd`e d~ z~jbzi b`s n`k}ztjs rd~d`e% B`s j|ak}ij)

kae `cgjnt natjf`sdj~ asj bzentd`eai% _lj~j natjf`sdj~ `btje j|ldcdt rd~zai }`i~jk 

‟ `cgjnt de~taenj~ tlat larj rd~zaii e`tldef t` o` vdtl janl `tljs ,j%f%) ’nlads‛$%

K`sj`rjs) natjf`sdj~ asj iaefzafj oj}jeojet ‟ ae `cgjnt natjf`s de `ej iaefzafj

kdflt e`t j|d~t de ae`tljs XIah`bb) 14<6[% Tjt ae`tljs ~`zsnj `b rd~zai }`i~jk 

}astdnzias t` 3O dkafj ~jt~ d~ rdjv+oj}jeojenj% _ahje `e dt~ `ve) a ~doj+rdjv `b a nas

la~ rd~zaii e`tldef de n`kk`e vdtl a bs`etai rdjv `b a nas ,Bdfzsj 1%1 sdflt$% _ljsjb`sj)

tsdef laso t` kahj a ~defij nas n`enj}t ojtjnt`s fisj `e c`tl ~jjk~ n`zetjs}s`ozntdrj

‟ c ijasedef natjf`s+~}jndfin k`oji~) vj kdflt antzaii cj tsdef t` ~`irj a k`sj

odbfinzit }s`cijk tlae d~ ejnj~~as%

De aoodtd`e) rd~zai `cgjnt natjf`sdwatd`e d~ e`t jrje n`e~d~tjet ans`~~ deodrdozai~%

N`e~dojs) b`s de~taenj) tlj IacjiKj oata~jt XSz~~jii jt ai%) 3==<[ vljsj lzkae iacjijs~

nae nl``~j ae Jefid~l v`so/}lsa~j tlj idhj b`s `cgjnt aee`tatd`e% Bdfzsj 1%1 ,ijbt$~l`v~ a t}dnai j|ak}ij `b rd~zai ~e`ek~ ‟ tv` rd~zaii ~dkdias `cgjnt~ tlat larj

cjje ascdtsasdi a~~dfejo odbbjsjet iacji~ ,’czdiodef‛ r~% ’l`z~j‛$%

14

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 20/130

L`z~j Nas NasCzdiodef

Bdfzsj 1%17 _}dnai j|ak}ij~ `b rd~zai ~e`ek~ aeo rd~zai }`i~jk tlat asj n`kk`e de IacjiKj X Sz~~jii

jt ai%) 3==<[ aee`tatd`e~% Rd~zai ~e`ek~7 tv` `cgjnt~ tlat asj rd~zaii qzdtj ~dkdias czt larj odbbjsjetnia~~ iacji~ ,ijbt$% Rd~zai }`i~jk7 tv` `cgjnt~ tlat larj e`tldef de n`kk`e rd~zaii czt asj iacjijo t`cj tlj ~akj nia~~ ,sdflt$%

1%1%3 Vl Natjf`sdwj0

Czt dt kdflt e`t cj t`` }s`ozntdrj t` n`enjetsatj `e tlj rasd`z~ natjf`sdwatd`e tlj`sdj~

 vdtl`zt n`e~dojsdef tlj fieai adk ‟ vlat o` vj ejjo natjf`sdj~ b`s0 @ej asfzkjet d~

tlat natjf`sdwatd e d~ a t``i t` bandidtatj he`vijofj tsae~bjs% J%f%) lardef cjje attanhjo

`enj c a tdfjs) dt d~ nsdtdnaii dk}`staet t` ojtjskdej db a ejvi `c~jsrjo `cgjnt cji`ef~

t` tlj tdfjs natjf`s ~` a~ t` ztdidwj tlj deb`skatd`e bs`k tlj }sjrd`z~ jen`zetjs% E`tj

tlat ljsj sjn`fedwdef tlj j|}idndt natjf`s d~ zedk}`staet) a~ i`ef a~ tlj tv` tdfjs~n`zio cj a~~`ndatjo vdtl janl `tljs% Ae`tljs asfzkjet b`s z~def natjf`sdj~ d~ tlj 

jeacij n`kkzednatd`e% Db tv` afjet~ ~lasj a n`kk`e idefzd~tdn r`naczias) tlje tlj 

nae jbbjntdrji j|nlaefj deb`skatd`e ac`zt tlj v`sio%

Czt vlat db tljsj d~ e` ~defij ldjsasnldnai `sfaedwatd`eai ~nljkj tlat d~ ~zdtacij

b`s `sfaedwdef tlj rd~zai v`sio0 A~ asfzjo c Ai`e Laijr de ’_lj Zesja~`eacij

Jbbjntdrjej~~ `b Oata)‛ XLaijr jt ai%) 3==4[) a }ast `b tlj v`sio d~ j|}iadejo vjii c 

}asakjtsdn/katljkatdnai k`oji~) aeo a }ast `b tlj v`sio d~ cjttjs j|}iadejo c oata%

Vj asfzj tlat tlj rd~zai v`sio nae cj ~zbfindjeti j|}iadejo c oata) aeo tlj ~jasnl b`s

a ~defij `sfaedwatd`eai ~nljkj b`s tlj rd~zai v`sio kdflt cj bztdij%

3=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 21/130

1%3 Sjiatjo V`sh 

@b n`zs~j) tlj doja `b a~~`ndatdef a ejv de~taenj vdtl ~`kjtldef ~jje de tlj }a~t la~

a i`ef aeo sdnl ld~t`s) ~tastdef vdtl tlj Csdtd~l Jk}dsdnd~t~ XI`nhj) 12<4) Cjshjij)

161=) Lzkj) 16>4[) aeo n`etdezdef a~ j|jk}ias tlj`s de n`fedtdrj }~nl`i`f XKjode

aeo ^nlabbjs) 146<)E`~`b~h) 14<2[) na~j+ca~jo sja~`edef de AD X^nlaeh) 14<>[) de~taenj+

ca~jo kjtl`o~ de kanldej ijasedef X Ala jt ai%) 1441[) oata+osdrje tsae~bjs de fsa}l+

dn~ XSje jt ai%) 3==:[) jtn% De n`k}ztjs rd~d`e) tld~ t}j `b e`e+}asakjtsdn tjnledqzj

la~ cjje qzdtj ~znnj~~bzi at a rasdjt `b ta~h~ denizodef7 `cgjnt aidfekjet XCji`efdj

jt ai%) 3==3) Cjsf jt ai%) 3==:[) ~njej sjn`fedtd`e X_`ssaica jt ai%) 3==<)  \da` jt ai%)

3=1=[) dkafj }as~def XIdz jt ai%) 3==4[) ak`ef `tljs~% L`vjrjs) b`s `cgjnt ojtjntd`e)

oata+osdrje kjtl`o~ ~znl a~ XSz~~jii jt ai%) 3==6) Kaid~djvdnw aeo Jbs`~) 3==<[) larj

e`t cjje n`k}jtdtdrj afade~t od~nsdkdeatdrj a}}s`anlj~ ,tl`zfl tlj lcsdo kjtl`o

`b XNlzk aeo Wd~~jskae) 3==6[ n`kj~ ni`~j$% Vl d~ tld~0 De `zs rdjv) tlj }sdkas 

odbfinzit ~tjk~ bs`k tlj ka~~drj ak`zet~ `b ejfatdrj oata tlat kz~t cj n`e~dojsjo de

tlj ojtjntd`e }s`cijk% De dkafj nia~~dfinatd`e) vljsj oata~jt ~dwj~ t}dnaii saefj bs`k a

bjv tl`z~aeo~ t` a kdiid`e) z~def tlj h Ejasj~t Ejdflc`s~ ,h+EE$ aif`sdtlk t` n`k}ztj

od~taenj~ t` aii tsadedef dkafj~ d~ ~tdii qzdtj bja~dcij% De `cgjnt ojtjntd`e) l`vjrjs) tlj

ezkcjs `b ejfatdrj vdeo`v~ nae f` a~ ldfl a~ lzeosjo~ `b kdiid`e~) }jsb`skdef h+EE

z~def c`tl }`~dtdrj~ aeo ejfatdrj~ d~ }s`ldcdtdrji j|}je~drj% Z~def ljzsd~tdn~) ~znl a~

~zc~ak}idef `s dfe`sdef tlj ejfatdrj ~jt) sj~zit~ de a ~zc~taetdai os`} de }jsb`skaenj%

De n`etsa~t) nzssjet ~tatj+`b+tlj+ast kjtl`o~ de `cgjnt ojtjntd`e , XOaiai aeo _sdff~)

3==:[) XBjiwje~wvaic jt ai%) 3=1=[) aeo tljds ojsdratdrj~$ asj }astdnziasi vjii+~zdtjo b`s

laeoidef iasfj ak`zet~ `b ejfatdrj oata% _lj jk}i` ’oata+kdedef‛ t` dtjsatdrji ~dbt

tls`zfl kdiid`e~ `b ejfatdrj~ aeo fieo tlj ’laso‛ `ej~ vldnl asj tlje z~jo t` tsade a

od~nsdkdeatdrj nia~~dfijs% Cjnaz~j tlj nia~~dfijs d~ a idejas ^RK) jrje tlj laso ejfatdrj~

31

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 22/130

o` e`t ejjo t` cj j|}idndti ~t`sjo czt asj sj}sj~jetjo }asakjtsdnaii) de tjsk~ `b a

ojnd~d`e c`zeoas%

L`vjrjs) tlj }asakjtsdn eatzsj `b tlj~j nia~~dfijs~) vldij a cij~~def b`s laeoidef

ejfatdrj oata) cjn`kj~ k`sj }s`cijkatdn vlje sj}sj~jetdef tlj }`~dtdrj~% _}dnaii)

aii }`~dtdrj j|ak}ij~ `b a fdrje `cgjnt natjf`s asj sj}sj~jetjo a~ a vl`ij) dk}idn+

dti a~~zkdef tlat tlj asj aii sjiatjo t` janl `tljs rd~zaii % Zeb`stzeatji) k`~t

~taeoaso ~jkaetdn natjf`sdj~ ,j%f%) ’nas‛) ’nlads‛) ’tsade‛$ o` e`t b`sk n`ljsjet rd~zai

natjf`sdj~ XKaid~djvdnw aeo Jbs`~) 3==<[) tlz~ tsjatdef tljk }asakjtsdnaii sj~zit~ de

 vjah aeo `rjsi+fjejsdn ojtjnt`s~% _` aoosj~~ tld~ }s`cijk) a ezkcjs `b a}}s`anlj~

larj z~jo ~jkd+}asakjtsdn kd|tzsj k`oji~) fs`z}def tlj }`~dtdrj~ det` niz~tjs~ ca~jo

`e kjta+oata ~znl a~ c`zeodef c`| a~}jnt satd` XBjiwje~wvaic jt ai%) 3=1=[) `cgjnt

~naij X]ash jt ai%) 3=1=[) `cgjnt rdjv}`det XFz aeo Sje) 3=1=[) }ast iacji~ XC`zsojr 

jt ai%) 3=1=[) jtn% Czt tlj i`v ezkcjs `b kd|tzsj n`k}`ejet~ z~jo de }santdnj kjae~

tlat tljsj d~ ~tdii n`e~dojsacij rasdatd`e vdtlde janl niz~tjs% A~ a sj~zit) tlj aidfekjet) `s

 rd~zai n`ssj~}`eojenj) cjtvjje tlj ijasejo k`oji aeo a ojtjntjo de~taenj d~ t`` n`as~j

t` cj z~acij b`s `cgjnt a~~`ndatd`e aeo iacji tsae~bjs% Vldij }ast+ca~jo k`oji~ XBjiwje+

~wvaic jt ai%) 3=1=[ aii`v odbbjsjet i`naidwatd`e~ `b }ast~ vdtlde od~tdent ojtjntd`e~) tlj

sjqzdsjkjet tlat tlj kz~t cj ~lasjo ans`~~ aii kjkcjs~ `b a natjf`s kjae~ tlat tlj~j

’}ast~‛ asj ai~` j|tsjkji rafzj aeo tlj sj~zitdef n`ssj~}`eojenj~ asj zedetzdtdrj% De

fjejsai) dt kdflt cj cjttjs t` tldeh `b tlj~j }ast~ a~ ~`bt) ojb`skacij ~zc+tjk}iatj~% _lj

]`~jijt~ a}}s`anl XC`zsojr jt ai%) 3=1=[ attjk}t~ t` aoosj~~ tld~ }s`cijk c kaezaii 

iacjidef }ast~ aeo z~def tljk t` tsade a ~jt `b }`~j+~}jndfin }ast ojtjnt`s~% Vldij rjs 

jen`zsafdef) tlj ljar kaezai iacjidef czsoje d~ a cdf idkdtatd`e `b tld~ kjtl`o%

Kjta+oata tsae~bjs la~ ai~` cjje z~jo de a ’sjn`fedwj+tlje+tsae~bjs‛ ~jttdef ,j%f%)

X_l`ka~ jt ai%) 3==4) Id jt ai%) 3=11[$% _lj a}}s`anl `b X_l`ka~ jt ai%) 3==4[ aoo~ a

kjta+oata sja~`edef ~tafj t` tlj sjn`fedtd`e ~~tjk `b XIjdcj jt ai%) 3==?[) czt ~denj

33

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 23/130

`cgjnt natjf`sdj~ asj sj}sj~jetjo }asakjtsdnaii) a e`e+tsdrdai kjta+oata }s`}afatd`e

~tj} kz~t cj denizojo% Dt d~ ai~` v`stlvldij kjetd`edef tlj a}}s`anl `b X^arasj~j

aeo Bjd+Bjd) 3==6[) vldnl `zt}jsb`sk~ X_l`ka~ jt ai%) 3==4[ aeo }s`rdoj~ a k`sj

n`k}ant sj}sj~jetatd e `b tlj `cgjnt natjf`s% L`vjrjs c`tl X_l`ka~ jt ai%) 3==4[

aeo X^arasj~j aeo Bjd+Bjd) 3==6[ asj natjf`s+ca~jo aeo kz~t n`}j vdtl tlj }s`cijk~

deljsjet de ojaidef vdtl rd~zai natjf`sdj~ ,~jj od~nz~~d`e de ^jntd`e 1%1%1$% _lj a}}s`anl

`b XId jt ai%) 3=11[ fis~t z~j~ a n`iijntd`e `b }`~j+~je~dtdrj `cgjnt ojtjnt`s~ t` }s`oznj

ojtjntd`e~ de tlj dkafj aeo tlje }jsb sk~ a iaeokash+ca~jo aidfekjet z~def ae dtjsatdrj

aif`sdtlk% A~ vdii cj ~jje de Nla}tjs ?) `zs a}}s`anl o`j~ e`t sjqzdsj ae aoodtd`eai

aidfekjet ~tj}~ abtjs ojtjntd`e) e`s o`j~ dt sjqzdsj tsadedef ~j}asatj natjf`s+ca~jo

`cgjnt ojtjnt`s~ t` nsjatj dedtdai ojtjntd`e~%

3>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 24/130

1%> _lj~d~ @rjsrdjv 

]ast D `b tld~ tlj~d~ d~ ojodnatjo t` ijasedef j|jk}ias sj}sj~jetatd`e~ b`s `cgjnt natjf`s 

ojtjntd e% Nla}tjs 3 k`tdratj~ tlj }s`cijk `b ijasedef j|jk}ias+~}jndfin ~dkdiasdt 

kja~zsj~ aeo }sj~jet~ I`nai Od~taenj Bzentd`e~ aeo l`v tlj nae cj z~jo vdtl Kzitd}ij

^jfkjetatd`e aif`sdtlk~ b`s i`naidwdef j|jk}ias~ de dkafj~ XKaid~djvdnw aeo Jbs`~)

3==<[% Nla}tjs > d~ ac`zt J|jk}ias+^RK~ XKaid~djvdnw jt ai%) 3=11[) vldnl asj a

kassdafj `b tlj j|jk}ias+ca~jo kjtl`o`i`f vdtl od~nsdkdeatdrj tsadedef b`zeo de

a}}s`anlj~ ~znl a~ XBjiwje~wvaic jt ai%) 3=1=[% ]ast D ~l`v~ tlat c j|}i`dtdef iasfj

ak`zet~ `b rd~zai oata) vj nae ijase j|jk}ias+~}jndfin ~dkdiasdt kja~zsj~) vldnl asj

~z}jsd`s t` ijasedef+bsjj ejasj~t+ejdflc`s a}}s`anlj~% Vj n`enizoj ]ast D c ~l`vdef

tlat vdtl ae je~jkcij `b J|jk}ias+^RK~) vj nae anldjrj }jsb`skaenj `e tlj ]A^NAI

 R@N ojtjntd`e ta~h tlat d~ `e }as vdtl tlj kznl k`sj n`k}ij| iatjet }ast+ca~jo k`oji

`b XBjiwje~wvaic jt ai%) 3=1=[) at `ei a k`oj~t n`k}ztatd`eai n`~t densja~j%

]ast DD d~ ojodnatjo t` tlj }`vjs `b j|jk}ias a~~`ndatd`e~ ‟ vj ~l`v sj~zit~ `e ta~h~

 vldnl f` cj`eo `cgjnt natjf`s ojtjntd`e% Vj jraizatj tlj J|jk}ias+^RK a}}s`anl

`e a odrjs~j ~jt `b kjta+oata tsae~bjs ta~h~ XKaid~djvdnw jt ai%) 3=11[ ,~jfkjetatd`e

tsae~bjs) qzaidtatdrj fj`kjts tsae~bjs) aeo sjiatjo `cgjnt }sdkdef$ a~ vjii a~ rd~zaidwj

tlj J|jk}ias+^RK—~ deoznjo j|jk}ias+j|jk}ias ~dkdiasdt ~tszntzsj `e ]A^NAI% Vj

n`enizoj vdtl a n`etj|tzai `cgjnt }sjodntd`e ta~h XKaid~djvdnw aeo Jbs`~) 3==4[) vldnl

sjqzdsj~ tlj b`skatd`e `b a fsa}l `rjs j|jk}ias~) vldnl vj naii tlj Rd~zai Kjkj| ) aeo

~l`v ae dk}s`rjkjet `rjs natjf`s+ca~jo ca~jidej~% De tlj A}}jeod|) vj `rjsrdjv

ae a}}idnatd`e `b tlj J|jk}ias+^RK aif`sdtlk t` tlj }s`cijk `b ns`~~+o`kade dkafj

katnldef X^lsdra~tara jt ai%) 3=11[ ,j%f%) katnldef }adetdef~ t` }l`t`~ aeo katnldef

~hjtnlj~ t` }l`t`~$ a~ vjii a~ od~nz~~ `zs n`etsdcztd`e~ de tlj b`sk `b ae `}je ~`zsnj

J|jk}ias+^RK `cgjnt ojtjntd`e idcsas%

3?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 25/130

]ast D7 Ijasedef J|jk}ias

Sj}sj~jetatd`e~

’Sj}sj~jet aii tlj oata vdtl a e`e}asakjtsdn k`oji satljs tlae tsdef t`

~zkkasdwj dt vdtl a }asakjtsdn k`oji) cjnaz~j vdtl rjs iasfj oata ~`zsnj~) tlj

oata l`io~ a i`t `b ojtadi%%% E`v f` `zt aeo fatljs ~`kj oata) aeo ~jj vlat dt nae

o`%‛

 Ai`e Laijr) ]jtjs E`srdf) aeo Bjseaeo` ]jsjdsa)

’_lj Zesja~`eacij Jbbjntdrjej~~ `b Oata‛ ,F``fij) 3==4$

3:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 26/130

Oj~}dtj tlj cjejfit~) }`~def sjn`fedtd`e a~ j|jk}ias a~~`ndatd`e d~ e`t ae ja~ 

ta~h% @ej sjqzdsjkjet d~ a rjs iasfj oata~jt) sdnl je`zfl t` n`etade kae odbbjsjet

`cgjnt~ aeo kae de~taenj~% Sjnjeti) vdtl tlj a}}jasaenj `b iasfj dkafj n`iijntd`e~)

~jrjsai ~~tjk~ larj ~l`ve tlat ~dk}ij h+ejasj~t+ejdflc`s ,h+EE$ a}}s`anlj~ nae `btje

}jsb`sk ~zs}sd~defi vjii X_`ssaica jt ai%) 3==<) La~ aeo Jbs`~) 3==6) Sz~~jii jt ai%)

3==6[% L`vjrjs) aii tlj~j kjtl`o~ katnl tlj dkafj a~ a vl`ij) vldnl jbbjntdrji idkdt~

tljk t` `}jsatdef `e tlj n`as~j ~njej ijrji ,tljsj d~ ~dk}i e`t je`zfl oata de tlj

 v`sio t` `c~jsrj aii }`~~dcij `cgjnt~ de aii }`~~dcij n`efifzsatd`e~$% _` katnl deodrdozai

`cgjnt~ vdtlde ~njej~) vj kz~t }astdtd`e tlj dkafj det` nlzeh~ vldnl asj ~kaii je`zfl

t` cj katnlacij de a sja~`eaci+~dwjo oataca~j) czt iasfj je`zfl t` jen`oj ~}jndfin

`cgjnt~) e`t fjejsdn ’rd~zai v`so~‛ X^drdn aeo Wd~~jskae) 3==>[% @ej a}}s`anl d~ t`

aii`v }astdai ~njej katnlj~ de~tjao `b katnldef jetdsj ~njej~ XSz~~jii jt ai%) 3==4[8

l`vjrjs) ~denj vj asj detjsj~tjo de ojtjntdef jetdsj `cgjnt~) vj b`nz~ `e katnldef

`cgjnt+~dwjo dkafj nlzeh~ ,b`s vldnl vj z~j jdtljs kzitd}ij ~jfkjetatd e~ aeo ~idodef

 vdeo`v~$%

De aoodtd`e t` a tsadedef oata~jt) j|jk}ias+ca~jo a}}s`anlj~ sjqzdsj a ~zdtacij

e`td`e `b rd~zai ~dkdiasdt% De `sojs t` kahj `zs a}}s`anl v`sh `e a k`ojsatji+~dwjo

tsadedef ~jt) vj ~l`v tlat ijasedef rd~zai ~dkdiasdt bzentd`e~ d~ ~z}jsd`s t` z~def a

~defij laeo+nsabtjo kjtsdn) a~ d~ n`kk`e de ejasj~t+ejdflc`s a}}s`anlj~% Vj }s`}`~j

tv` t}j~ `b ~dkdiasdt bzentd`e~) I`nai Od~taenj Bzentd`e~ aeo J|jk}ias+^RK~) a~ vjii

a~ aif`sdtlk~ b`s ijasedef tljk bs`k oata% Vldij tlj ezkcjs `b ijasedef }s`cijk~ vj

~`irj ~naij~ idejasi vdtl tlj ezkcjs `b j|jk}ias~) }js+j|jk}ias ijasedef }s`cijk~ asj

kznl ja~djs t` ojai vdtl cjnaz~j janl j|jk}ias—~ ijasedef }s`cijk d~ ~dk}ijs tlae a

fi`cai natjf`s+vd~j }s`cijk% K`sj dk}`staeti) }js+j|jk}ias ~dkdiasdt bzentd`e~ fdrj

z~ a rasdacij aeo `cgjnt+oj}jeojet e`td`e `b ~dkdiasdt) aii`vdef z~ t` sja~`e ac`zt a

odrjs~j ~jt `b `cgjnt t}j~%

32

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 27/130

Nla}tjs 3

I`nai Od~taenj Bzentd`e~

Vlje vj i``h at tlj v`sio as`zeo z~) vj e`tdnj tlat ~dkdiasdt d~ ojfiejo odbbjsjeti 

b`s odbbjsjet t}j~ `b `cgjnt~ de tlj v`sio% V`s~j jt) `cgjnt~ nae j|ldcdt ~dkdiasdt 

`e kae odbbjsjet) `btje n`etsaodnt`s) ijrji~7 ~la}j) ~dwj) n`i`s) tj|tzsj) jtn% B`s

j|ak}ij) Aoji~`e odrdoj~ tlj v`sio det` ’tldef~‛ ,~znl a~ nas~) }j`}ij$ aeo ’~tzbb‛

,fsa~~) }arjkjet) dnj nsjak) jtn%$ X Aoji~`e) 3==1[% B`s ’tldef~‛) idhj nas~) `cgjnt ~la}j

d~ ae dk}`staet nzj vljsja~ `cgjnt n`i`s d~ z~zaii e`t% Czt b`s ’~tzbb‛) idhj fsa~~) vldnl

o`j~ e`t `ve dt~ c`zeoasdj~) ~la}j d~ z~jij~~ ,de bant) ojtsdkjetai$ czt n`i`s aeo tj|tzsj

asj j|tsjkji dk}`staet% _ljsjb sj) t` fieo vlat a fdrje `cgjnt de~taenj d~ ~dkdias

t`) dt d~ dk}jsatdrj tlat tlj sdflt od~taenj kjtsdn b`s tlat de~taenj cj z~jo% Czt) `b 

n`zs~j) t` he`v tlj sdflt od~taenj kjtsdn sjqzdsj~ he`vdef vlat tlat `cgjnt d~! A~ d~

`btje tlj na~j de rd~d`e) vj asj banjo vdtl a odbfinzit nldnhje aeo jff }s`cijk% De tlj

na~j `b od~taenj bzentd`e~) vj tanhij tld~ }s`cijk c ijasedef a ~j}asatj n`kcdeatd`e

`b jijkjetas od~taenj~ ,~znl a~ n`i`s) tj|tzsj aeo ~la}j$ b`s janl j|jk}ias de `zs

oataca~j% _` kahj tldef~ odbfinzit) t}dnai lzkae `cgjnt iacji~ ,vldnl asj fjejsaii tlj

ca~dn+ijrji natjf`sdj~ t` vldnl ae `cgjnt cji`ef~ t`$ asj e`t f``o je`zfl t` kahj tlj

a~~zk}td`e tlat ae j|jk}ias ~l`zio cj ~dkdias t` aii `tljs j|jk}ias~ vdtl tlj ~akj

36

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 28/130

iacji ,~jj Bdfzsj 1%1 aeo od~nz~~d`e de ~jntd`e 1%1%1$% Vj de~tjao }s`}`~j a iasfji 

oata+osdrje a}}s`anl vldnl vjahi z~j~ tlj `cgjnt iacji~ t` azt`katdnaii ijase b`s janl

j|jk}ias a od~taenj bzentd`e aeo a ~kaii ~jt `b j|jk}ias~ vldnl asj rd~zaii ~dkdias t`

tlj j|jk}ias%

_lj od~taenj bzentd`e~ ojfiejo de tld~ ~jntd`e asj }`~dtdrj idejas n`kcdeatd`e~ `b 

jijkjetas od~taenj~% Vj z~j `ej n`kk`e od~taenj bzentd`e }asakjtjsdwatd e b`s

aii j|jk}ias~) czt ijase de~taenj+~}jndfin }asakjtjs~ c ~`irdef a odbbjsjet ~z}jsrd~jo

ijasedef }s`cijk b`s janl j|jk}ias% Janl j|jk}ias la~ dt~ `ve od~taenj bzentd`e ‟ vj

oje`tj j|jk}ias j—~ od~taenj bzentd`e a~ Oj,|d$ ; O,|dy|j)vj$) vldnl tahj~ a~ de}zt

~`kj qzjs bjatzsj~ |d ∌ B  aeo sjtzse~ tlj ~dkdiasdt cjtvjje tlj j|jk}ias) |j ∌ B 

aeo tlj de}zt |d ca~jo `e tlj e`td`e `b ~dkdiasdt jen`ojo c vj ∌ B (%

Vj z~j a }`}zias nia~~ `b od~taenj bzentd`e~ vldnl asj ai~` naiijo odaf`eai Kala+

iae`cd~ od~taenj kjtsdn~% _lj ojnd~d`e c`zeoas `b janl od~taenj bzentd`e d~ ae jiid}~`do

njetjsjo at |j) vljsj janl n`k}`ejet `b  vj oje`tj~ tlj ji`efatd`e `b tlj jiid}~j

ai`ef tlat }astdnzias odkje~d`e% ^denj vj hjj} janl od~taenj bzentd`e aenl`sjo at tlj

j|jk}ias) vj nae tldeh `b ae ejv }`det |d) e`t de dt~ sav bjatzsj ~}anj) czt de tljod~taenj+t`+j|jk}ias+j ~}anj ,~jj Bdfzsj 3%1$% Ijt z~ ojfiej ojd t` cj tlj B  odkje~d`eai

e`e+ejfatdrj ’od~taenj ~qzasjo rjnt`s‛ cjtvjje |j aeo tlj de}zt |d) aeo ojdXh[ tlj

h+tl n`k}`ejet `b tld~ rjnt`s% Vj vj nae tlz~ vsdtj j|jk}ias j—~ od~taenj bzentd`e a~

b`ii`v~7

Oj,|d$ ; ,|j ∓ |d$_ V j,|j ∓ |d$ ,3%1$

Oj,|d$ ; v_ j ojd ,3%3$

vj ; odaf,V j$ ,3%>$

ojdXh[ ; ,|jXh[ ∓ |dXh[$3 ,3%?$

3<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 29/130

n

a

c

+

+

+

j

a

c

n

+

+

Sav Bjatzsj ^}anj ’Od~taenj+t`+j|jk}ias‛

rjnt`s ~}anj

r~%

+

j

+

+

Bdfzsj 3%17 Ijasedef a od~taenj+bzentd`e de ’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj% De~tjao `b }jsb`sk+def ijasedef de tlj sav bjatzsj ~}anj ,vljsj tlj ojnd~d`e c`zeoas d~ e`e+idejas$) vj ka} tlj }s`cijkt` a od~taenj+t`+j|jk}ias ~}anj ,vljsj tlj ojnd~d`e c`zeoas d~ idejas$% _lj j|jk}ias) deodnatjo c  j)d~ e`v }ianjo at tlj `sdfde) cjnaz~j dt~ `ve od~taenj+t +j|jk}ias vdii cj =% Ae oata }`det |d nae cjka}}jo t` tlj }`~dtdrj }`det ojd de tlj od~taenj+t`+j|jk}ias ~}anj a~ deodnatjo c Jqzatd`e 3%?%

B`skziatdef tlj }s`cijk de a ’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj) vj nae ~jj

tlat ae a}}idnatd`e `b tlj od~taenj bzentd`e d~ ~dk}i a o`t }s`oznt de tlj ejv ~}anj

,Jqzatd`e 3%3$% Ae dkkjodatj aoraetafj d~ tlat dt aii`v~ z~ t` z~j `ej `b tlj kae 

 vjii+zeojs~t``o t``i~ fjasjo t`vaso~ ijasedef idejas ojnd~d`e c`zeoasdj~%

3%1 Ijasedef Od~taenj Bzentd`e~ Vdtl Bsdjeo~

B`s janl j|jk}ias vj ijase vj c b`skdef a ~z}jsrd~jo+ijasedef }s`cijk bs`k tlj

iacji~ a~~`ndatjo vdtl tsadedef j|ak}ij~% L`vjrjs) ~denj tljsj d~ a fsjat ojai `b rd~zai

odrjs~dt vdtlde a ~defij `cgjnt natjf`s ,a~ asfzjo de ^jntd`e 1%1%1$) vj vaet t` ar`do

b`sndef ae j|jk}ias t` cj ~dkdias t` aii de+nia~~ de~taenj~% De~tjao `b z~def aii `tljs

34

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 30/130

`cgjnt de~taenj~ vdtl tlj ~kaii nia~~ a~ j—~ }`~dtdrj~) vj ijt tlj ijasedef }s`nj~~ nl``~j

 vldnl j|ak}ij~ j d~ antzaii ~dkdias t` ,sjbjssjo t` a~ j—~ bsdjeo~$% B`s tld~ }zs}`~j)

 vj z~j ae j|jk}ias+~}jndfin cdeas rjnt`s αj ,vl`~j ijeftl d~ jqzai t` tlj ezkcjs `b 

j|jk}ias~ vdtl tlj ~akj iacji a~ j$) vljsj tlj e`e+wjs` jijkjet~ `b αj deodnatj vldnl

j|ak}ij~ asj j—~ bsdjeo~% ^denj vj o` e`t he`v vldnl j|jk}ias~ asj rd~zaii ~dkdias

t` janl `tljs) vj asj banjo vdtl a nldnhje+aeo+jff }s`cijk aeo kz~t ijase vj aeo αj

 g`deti% Ai~`) a~ d~ n`kk`e de dkcaiaenjo nia~~dfinatd`e }s`cijk~) vj ijase a cda~ tjsk

cj de aoodtd`e t` tlj `sdjetatd`e `b tlj l}js+}iaej vj%

B`s ijasedef) vj ~tast `zt vdtl a iasfj aee`tatjo ~jt `b `cgjnt j|jk}ias~ x,|d) td${E d;1)

 vljsj td ∌ x1) % % % ) K  { deodnatj~ tlj nia~~ tlat janl j|ak}ij cji`ef~ t`% Vj deoj}je+

ojeti ijase a ~j}asatj od~taenj bzentd`e b`s janl j|jk}ias) tlz~ `zs a}}s`anl ~naij~

idejasi vdtl tlj ezkcjs `b j|jk}ias~ E % Cjnaz~j b`s janl j|jk}ias—~ ijasedef }s`cijk)

 vj ~`irj a tv`+nia~~ }s`cijk) `zs a}}s`anl d~ e`t ~je~dtdrj t` tlj `rjsaii ezkcjs `b 

`cgjnt nia~~j~) K % Fdrje a ~defij j|ak}ij) vj }astdtd`e aii `tljs `cgjnt~ det` tlj de+nia~~

j|ak}ij~ ~jt) N j ; x g 7 t g ; tj{) aeo dt~ n`k}ijkjet) tlj `zt+nia~~ ~jt%

_lj `zt}zt `b }js+j|jk}ias ijasedef d~ a ~defij vjdflt rjnt`s) a ~naias cda~) aeo tljj|jk}ias—~ bsdjeo deodnat`s rjnt`s7

Xv∖j) c∖j)α∖

j[ ; asfkdev)c)α

™,v) c)α$ ,3%:$

Ijasedef tlj od~taenj bzentd`e d~ }jsb`skjo c `}tdkdwdef tlj b`ii`vdef `cgjntdrj

bzentd`e b`s janl j|jk}ias ,~denj vj deoj}jeojeti ijase janl j|jk}ias—~ od~taenj

bzentd`e) vj os`} tlj j ~zc~nsd}t b`s niasdt$7

™,v) c)α$ ;θ

3yyvyy3 (

d∌N 

αd I,∓,v_ od ( c$$ (d/∌N 

I,v_ od ( c$ ∓ ώyyα yy3 ,3%2$

>=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 31/130

v ≪ =

α ∌ x=) 1{yN y

De `zs b`skziatd`e) ijt I,|$ cj a i`~~ bzentd`e) ~znl a~ tlj ldefj+i`~~ Ildefj,|$ ;

ka|,1 ∓ |) =$ `s tlj ~qzasjo ldefj+i`~~ Ildefj∓~q,|$ ; ka|,1 ∓ |) =$3) aeo ijt N  cj tlj

~jt `b j|jk}ias~ tlat larj tlj ~akj iacji a~ tlj b`nai j|jk}ias ,d%j%) tlj j|jk}ias vl`~j

od~taenj bzentd`e vj asj ijasedef$% Vj dedtdaii z~jo tlj n`e~tsadet tlat

d αd ; H )

 vljsj H  d~ tlj tasfjt ezkcjs `b j|jk}ias~ vj b`snj t` cj ~dkdias t` j XKaid~djvdnw aeo

Jbs`~) 3==<[) czt iatjs b`zeo tlat z~def a tjsk vldnl aii`v~ janl j|jk}ias t` ~jijnt

a rasdacij ezkcjs `b bsdjeo~ kahj~ k`sj ~je~j ,tld~ d~ tlj ia~t tjsk) k`oziatjo c 

ώ$ XKaid~djvdnw aeo Jbs`~) 3==4[% @e `ej laeo) c `}tdkdwdef tlj od~taenj bzentd`e

ijasedef `cgjntdrj bzentd`e vj ~tsdrj t` kadetade a ka|+kasfde ~j}asatd`e cjtvjje tlj

j|jk}ias }iz~ dt~ ~jt `b bsdjeo~ aeo tlj ~jt `b ejfatdrj j|ak}ij~% @ej tlj `tljs laeo)

 vj ts t` ~jijnt tlj bsdjeo~ ~znl tlat tlj aii`v b`s a iasfj ~j}asatd`e) }sjbjssdef t`

nl``~j a iasfj ezkcjs `b bsdjeo~%

Vdtl`zt tlj α }asakjtjs aeo vdtl e` n`e~tsadet `e v) Jqzatd`e 3%2 d~ gz~t tlj

}sdkai b`sk `b kae n`erj| ~tatd~tdnai ijasedef tjnledqzj~ ,~znl a~ I`fd~tdn Sjfsj~~d`e

aeo ^z}}`st Rjnt`s Kanldej~$% De `zs na~j vj z~j tlj n`erjetd`e tlat ~kaiijs raizj~ `b 

v_ od dk}i a fsjatjs ojfsjj `b ~dkdiasdt t` j|jk}ias j ,tlj ~jkaetdn~ `b a od~taenj$)

~` tlj }`~dtdrdt n`e~tsadet `e v d~ kjaet t` je~zsj tlat a iasfj jijkjetas od~taenj

nae ejrjs dk}i a rjs ~kaii ,}`tjetdaii ejfatdrj$ od~taenj% _ld~ ai~` je~zsj~ tlat

tlj j|jk}ias la~ ka|dkai ~dkdiasdt vdtl dt~jib% ^denj tlj }sj~jenj `b tlj cdeas α—~

sjeojs~ tlj }s`cijk e`e+n`erj| ,a~ d~ n`kk`e de iatjet rasdacij k`oji~$) vj }s`njjo

dtjsatdrji j~tdkatdef α fdrje v aeo j~tdkatdef v fdrje α% Ozsdef janl dtjsatd`e) vj

>1

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 32/130

asj fzasaetjjo t` ejrjs densja~j tlj raizj `b `zs `cgjntdrj bzentd`e ,Jqzatd`e 3%2$ aeo

tlz~ jbfindjeti fieo a i`nai kdedkzk% Vj ~tast vdtl ae dedtdai od~taenj bzentd`e v= aeo

csjah o`ve Jqzatd`e 3%2 c dtjsatdrji ~`irdef tlj b`ii`vdef tv` n`erj| ~zc+}s`cijk~7

αh ; asfkde

α

d∌N 

αdI,∓vh ¶ od$ ∓ ώyyα yy3 ,3%6$

Xvh(1) ch(1[ ; asfkde,v)c$

θ

3yyvyy3 (

d7αhd;1

I,∓,v ¶ od$ ( c$ (d/∌N 

I,v ¶ od ( c$ ,3%<$

Fdrje vh) dt d~ tsdrdai t` ~`irj b`s αh ,Jqzatd`e 3%6$ cjnaz~j janl αhd nae cj ~`irjo

b`s deoj}jeojeti7

αh ; asfkde

α

d∌N 

αdI,∓vh ¶ od$ ∓ ώyyα yy3 ,3%4$

αh ; asfkde

α

d∌N 

αd,I,∓vh ¶ od$ ∓ ώ$ ,3%1=$

αhd ; asfkde

αd

αd,I,∓vh ¶ od$ ∓ ώ$ ,3%11$

αhd ;

=) db I,∓vh ¶ od$ ≪ ώ

1) db I,∓vh ¶ od$ 9 ώ

,3%13$

_lj ~zc+}s`cijk `b ijasedef vh(1 fdrje αh ,Jqzatd`e 3%<$ tahj~ tlj b`sk `b a

nia~~dnai n`erj| ~z}jsrd~jo ijasedef }s`cijk% Cjnaz~j vj z~j a ldefj+i`~~) vj z~j tlj

}zcidni aradiacij }sdkai ^RK ~`irjs bs`k XNla}jiij) 3==6[ ,vdtl a ~idflt k`odfinatd`e

t` laeoij tlj }`~dtdrdt n`e~tsadet$% _lj jetdsj }s`njozsj n`erjsfj~ vljeα

h(1

h

)~denj tld~ dk}idj~ tlat vh(1 ; vh% Abtjs ~`irdef janl j|jk}ias—~ ijasedef }s`cijk)

 vj ~naij tlj sj~zitdef od~taenj bzentd`e ~znl tlat a od~taenj raizj `b  1 ,de~tjao `b  =$

n`ssj~}`eo~ t` tlj ojnd~d`e c`zeoas aeo a raizj `b  = ,de~tjao `b ∓vE B $ n`ssj~}`eo~

t` }jsbjnt ~dkdiasdt%

>3

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 33/130

3%3 Sjiatjo V`sh 

I`nai ijasedef la~ a i`ef ld~t`s ‟ tjnledqzj~ ca~jo `e i`naii vjdfltjo idejas sjfsj~+

~d`e X Athj~`e jt ai%) 1446[ larj cjje ~znnj~~bzii a}}idjo t` `tljs o`kade~ ~znl a~

s`c`t n`ets`i% De rd~d`e) tlj v`sh `b Bs`kj jt ai% XBs`kj aeo Kaidh) 3==2) Bs`kj jt ai%)

3==6[ ojai~ vdtl a iasfj ezkcjs `b `cgjnt natjf`sdj~ aeo }jsb`sk~ j|jk}ias+katnldef

b`s tlj ta~h `b dkafj nia~~dfinatd`e ,`e Naitjnl 1=1$% Bs`kj—~ v`sh) vldnl va~ tlj fis~t

t` }s`}`~j ijasedef }js+j|jk}ias ~dkdiasdt kja~zsj~) la~ cjje rjs deffizjetdai de tlj

ojrji`}kjet `b `zs v`sh% Dt d~ }jsla}~ e`t ~zs}sd~def tlat Bs`kj—~ v`sh `e ijasedef

i`nai+od~taenj bzentd`e~ ,idhj `zs~$ d~ ai~` ljardi deffizjenjo c Vdttfje~tjde—~ doja `b 

bakdi sj~jkciaenj~ XBs`kj) 3==6[%

L`vjrjs) dt d~ v`stl ldflidfltdef tlj cdffj~t odbbjsjenj cjtvjje `zs v`sh aeo tlat

`b XBs`kj jt ai%) 3==6[7 vldij vj aoosj~~ tlj }s`cijk `b `cgjnt i`naidwatd`e) XBs`kj

jt ai%) 3==6[ `ei n`e~dojsjo dkafj nia~~dfinatd`e% At tlj sj}sj~jetatd`eai ijrji) XBs`kj

jt ai%) 3==6[ z~jo i`nai bjatzsj katnldef) aii`vdef odbbjsjet rdjv~ `b tlj ~akj `cgjnt

t` }`tjetdaii cj katnljo) vldij vj z~jo kznl k`sj sdfdo a~~jkcidj~ `b bjatzsj~

je~zsdef tlat osa~tdnaii odbbjsjet rdjv~ `b ae `cgjnt vdii e`t cj n`ebz~jo ,jrje db ~`kj

tsae~b`skatd`e `b i`nai bjatzsj~ n`zio }ianj tlje de n`ssj~}`eojenj$% Aoodtd`eaii) vj

z~j tlj cj~t katnldef j|jk}ias~ b`s detjs}sjtatd`e vldnl kjae~ tlat dt d~ kznl k`sj

dk}`staet de `zs bsakjv`sh t` `ctade rd~zaii ~dkdias katnlj~) vldij XBs`kj jt ai%)

3==6[ `ei jraizatjo tljds a}}s`anl `e natjf`s }sjodntd`e%

N`eejntd`e t` e`e+n`erj| ijasedef% _ljsj d~ ai~` ae dk}`staet n`eejntd`e cj+

tvjje `zs z~j `b bsdjeo deodnat`s rasdacij~ de Jqzatd`e 3%2 aeo tlj S`cz~t ^z}}`st

 Rjnt`s Kanldej tsadedef aif`sdtlk `b X \z jt ai%) 3==2[% _lj ~l`v tlat tlj z~j `b 

deodnat`s rasdacij~ d~ jqzdraijet t` z~def tlj Sak}+I`~~) a tszenatjo rasdaet `b tlj ldefj

i`~~ vldnl sjtzse~ a n`e~taet i`~~ b`s oata }`det~ ~zbfindjeti bas bs`k tlj kasfde%

>>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 34/130

Isak},|$ ; kde,1) Ildefj,|$$

Ildefj,|$ ; ka|,=) 1 ∓ |$

Vlat tld~ kjae~ d~ `zs od~taenj bzentd`e b`skziatd`e n`zio cj sj+vsdttje vdtl`zt

ae cdeas rasdacij~ ‟ tlj ~akj jbbjnt n`zio cj `ctadejo c sj}iandef tlj n`erj| i`~~

bzentd`e `e tlj }`~dtdrj~ vdtl tlj e`e+n`erj| Isak}% X \z jt ai%) 3==2[ ai~` ~l`vjo tlat

 vlje z~def ae aitjseatdef `}tdkdwatd`e aif`sdtlk) dt o`j~ e`t kattjs vljtljs vj b`snj

αd t` cj cdeas `s de tlj saefj X=) 1[ ~denj tlj ~`iztd`e vdii aiva~ cj jdtljs = `s 1%

3%> Jraizatdef I`nai Od~taenj Bzentd`e~ `e IacjiKj

Vj asj zitdkatji detjsj~tjo de }as~def a ~njej det` dt~ n`e~tdtzjet `cgjnt~ ‟ zeojs~taeo+

def a~ kznl a~ vj nae ac`zt ae de}zt dkafj% O`def ~` b`s a sja~`eaci fjejsai nia~~

`b dkafj~ sjqzdsj~ laeoidef a iasfj ezkcjs `b odbbjsjet `cgjnt~ tlat `nnzs de jrjsoa 

idbj% _ljsjb`sj) tlj nl`dnj `b tlj sdflt tsadedef oata d~ `b tlj ztk`~t dk}`staenj% @b aii

tlj nzssjeti aradiacij oata~jt~) tlj `ei `ej n`etadedef a iasfj ezkcjs `b sjai+v`sio

~njej~) vdtl a vdoj rasdjt `b jrjsoa `cgjnt~ tlat asj e`t `ei iacjijo czt ai~` ~jf+

kjetjo) d~ tlj IacjiKj oata~jt XSz~~jii jt ai%) 3==<[% IacjiKj d~ ae `ef`def `eidej

dkafj+aee`tatd`e jbb`st der`irdef kae iacjijs~% A~ a sj~zit) e`t `ei asj tlj dkafj~

z~js+n`etsdcztjo) ~}aeedef a vdoj saefj `b ~njej~) czt z~js~ asj bsjj t` iacji janl `cgjnt

 vdtl ae Jefid~l tj|t ~tsdef tlj idhj) }s`rdodef a f``o ~ak}idef `b tlj od~tsdcztd`e `b 

`cgjnt eakj~ ’de tlj vdio‛%

 A~ a ~`zsnj `b j|jk}ias~) vj z~j a ~zc~jt `b IacjiKj vldnl n`e~d~t~ `b `rjs :===

dkafj~% Abtjs dfe`sdef tde `cgjnt~) vj nijae z} tlj `cgjnt aee`tatd`e~ c od~nasodef

>?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 35/130

az|didas v`so~ bs`k tlj iacji~ ,z~def tlj bzentd`e }s`rdojo de IacjiKj t``ihdt$) aeo

hjj} aii `cgjnt~ vl`~j zedqzj iacji `nnzs~ at ija~t : tdkj~% _ld~ fdrj~ z~ a t`tai `b 

13) 4=: `cgjnt~ ~}aeedef 161 zedqzj iacji~%

3%>%1 ^jfkjet+ca~jo Bjatzsj~

Vj sj}sj~jet janl `cgjnt c E B  ; 1? odbbjsjet bjatzsj~% _lj bjatzsj~ na}tzsj odbbjsjet

a~}jnt~ `b ~la}j) tj|tzsj) n`i`s) aeo dkafj i`natd`e b`s ae dkafj ~jfkjet ,~jj _acij

3%1$% _ na}tzsj deb`skatd`e ac`zt ~la}j vj n`k}ztj7 tlj njetjsjo `cgjnt ka~h de

a nae`ednai >3 Ö >3 bsakj) tlj ~dwj `b tlj sjfd`e) aeo tlj ~dwj `b sjfd`e—~ c`zeodef

c`|% _` na}tzsj tj|tzsj vj n`k}ztj e`skaidwjo tj|t`e ld~t`fsak~ de tlj detjsd`s `b 

tlj `cgjnt) aeo) ~j}asatji) ai`ef tlj c`zeoasdj~ `b tlj `cgjnt% B`s n`i`s vj n`k}ztj

tlj kjae SFC+raizj) dt~ ~taeoaso ojrdatd`e) a~ vjii a~ a n`i`s ld~t`fsak% Bdeaii) t`

na}tzsj he`vijofj ac`zt tlj }`~dtd`e `b tlj ~jfkjet de ae dkafj) vj n`k}ztj a n`as~j

,cizssjo$ < Ö < ac~`iztj ~jfkjetatd`e ka~h a~ vjii a~ tlj e`skaidwjo ljdflt `b tlj

t`}+k`~t aeo c`tt`k+k`~t }d|ji de tlj sjfd`e%

3%>%3 Od~taenj Bzentd`e Ijasedef Ojtadi~

Vj ojfiej janl `b `zs od~taenj bzentd`e~ a~ a idejas n`kcdeatd`e `b jijkjetas od~taenj~%

Vj z~j 1? n`i`s) ~la}j) tj|tzsj) aeo i`natd`e bjatzsj~ t` sj}sj~jet j|jk}ias~% De~tjao `b 

}jsb`skdef ijasedef de `ej g`det bjatzsj ~}anj nsjatjo c n`enatjeatdef tlj 1? odbbjsjet

bjatzsj~) vj }jsb`sk ijasedef de tlj sjoznjo 1?O jijkjetas+od~taenj ~}anj% _lz~

tlj od de Jqzatd`e 3%2 d~ tlj rjnt`s `b  1? Jznidojae od~taenj~ cjtvjje tlj j|jk}ias

 vl`~j ~dkdiasdt vj asj ijasedef ,tlj b`nai j|jk}ias$ aeo tlj d+tl j|ak}ij% Vj z~j

I,|$ ; ka|,1 ∓ |) =$3) tlj ldefj+~qzasjo i`~~ bzentd`e) b`s ijasedef vldnl aii`v~ z~ t`

z~j a ~jn`eo `sojs ^RK ~`irjs XNla}jiij) 3==6[%

>:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 36/130

_}j Eakj Odkje~d`e

^la}jNjetjsjo Ka~h >3|>3;1=3?CC J|tjet 3

]d|ji Asja 1

_j|tzsj

Sdflt C`zeoas _j|+Ld~t 1==_ } C`zeoas _j|+Ld~t 1==

Ijbt C`zeoas _j|+Ld~t 1==C`tt`k C`zeoas _j|+Ld~t 1==

Detjsd`s _j|+Ld~t 1==

N`i`s

Kjae N`i`s >

N`i`s ~to >N`i`s Ld~t`fsak >>

I`natd`e  Ac~`iztj Ka~h <|<;2?_ } Ljdflt 1C`t Ljdflt 1

_acij 3%17 ^jfkjet Oj~nsd}t`s7 1? Jijkjetas Bjatzsj~ z~jo t` sj}sj~jet `cgjnt~% Jijkjetas od~taenj~ asj ~dk}i tlj I3 od~taenj~ cjtvjje n`ssj~}`eodef bjatzsj rjnt`s~%

B`s jrjs j|jk}ias vj dedtdaidwj tlj od~taenj bzentd`e ijasedef }s`nj~~ vdtl v= ;

vtj|t`e) vljsj vtj|t`e la~ wjs`~ ai`ef aii jijkjetas od~taenj odkje~d`e~ j|nj}t tlj de+

tjsd`s tj|t`e ld~t`fsak bjatzsj ,a caf+`b+v`so~ bjatzsj$% Bs`k tlj odrjs~j n`kcdeatd`e~

`b ljzsd~tdnaii ojfiejo od~taenj~/vjdflt~ vj j|}jsdkjetjo vdtl) vtj|t`e }jsb`skjo

tlj cj~t b`s a vdoj assa `b `cgjnt t}j~) aeo vj z~j dt a~ tlj ca~jidej ’ijasedef+bsjj‛

ejasj~t+ejdflc`s kjtl`o%

>2

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 37/130

nas

nas nas ~zr nas nas

nas nas nas nas

nas

nas nas~ nas nas

nas nas nas nas

nas

nas nas nas nas

nas nas nas nas

tsjj b`sj~t tsjj rjfjtatd`e

}js~`e }js~`e }js~`e~taeodef v`kae }js~`e}js~`e

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C   

 J   |  t   j  e  t   

_   `   }  

 _   j  |  t     e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N  `  i   `  s   ~  t   o   

K   j  a  e   N    i   `  s  

D   e  t   j  

s  d     s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t     e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  

j  i    A  s  j  a  

_   `   }  

 L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N    i   `  s   ~  t   o   

K   j  a  e   N  `  i   `  s  

D   e  t   j  s  d   `  s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N  `  i     s   ~  t   o   

K   j  a  e   N  `  i     s  

D   e  t   j  s  d   `  s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t     e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N    i   `  s   ~  t   o   

K   j  a  e   N    i   `  s  

D   e  t   j  s  d   `  s   _   j  |  t     e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N  `  i     s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

Bdfzsj 3%37 J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 1/3% Fdrje ae j|jk}ias `e tlj t`}ijbt) tlj sjkadedef s`v ~l`v~ tlj t`} ? k`~t ~dkdias `cgjnt~ abtjs ijasedef a od~taenj bzentd`e% _lj

od~taenj bzentd`e d~ rd~zaidwjo a~ a od~tsdcztd`e `rjs jijkjetas od~taenj~ aeo ~l`ve de tlj c`tt`k ijbt%_lj ? j|jk}ias~ `e tlj c`tt`k sdflt asj tlj ? k`~t ~dkdias `cgjnt~ vdtl sj~}jnt t` tlj tj|t`e ld~t`fsakod~taenj%

>6

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 38/130

vdeo`v

czdiodef vdeo`v czdiodef vdeo`v

vdeo`v vdeo`v vdeo`v vdeo`v

vljji

k`t`scdhj tsjj kdss`s vljji

vljji vljji vljji vljji

czdiodef

~dojvaih nas nas nas

czdiodef czdiodef czdiodef czdiodef

czdiodef

vaii czdiodef }adetdef czdiodef

czdiodef czdiodef czdiodef czdiodef

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N    i   `  s   ~  t   o   

K   j  a  e   N    i   `  s  

D   e  t   j  s  d   `  s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N    i   `  s   ~  t   o   

K   j  a  e   N    i   `  s  

D   e  t   j  s  d   `  s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N    i   `  s   ~  t   o   

K   j  a  e   N    i   `  s  

D   e  t   j  s  d   `  s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

N  j  e  t   j  s  j  o    K   a  ~  h  

C   C    J   |  t   j  e  t   

_   `   }   _   j  |  t   `  e  ~  

S   d    f  l  t    _   j  |  t   `  e  ~  

N    i   `  s   ~  t   o   

K   j  a  e   N    i   `  s  

D   e  t   j  s  d   `  s   _   j  |  t   `  e  ~  

C   `  t    _   j  |  t   `  e  ~  

I  j  b   t    _   j  |  t   `  e  ~  

]   d   |  j  i    A  s  j  a  

_   `   }   L   j  d    f  l  t   

N    i   `  s   L   d   ~  t   

A  c  ~    i   z  t   j   K   a  ~  h  

C   `  t    L   j  d    f  l  t   

Bdfzsj 3%>7 J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 3/3% Fdrje ae j|jk}ias `e tlj t`}ijbt) tlj sjkadedef s`v ~l`v~ tlj t`} ? k`~t ~dkdias `cgjnt~ abtjs ijasedef a od~taenj bzentd`e% _lj

od~taenj bzentd`e d~ rd~zaidwjo a~ a od~tsdcztd`e `rjs jijkjetas od~taenj~ aeo ~l`ve de tlj c`tt`k ijbt%_lj ? j|jk}ias~ `e tlj c`tt`k sdflt asj tlj ? k`~t ~dkdias `cgjnt~ vdtl sj~}jnt t` tlj tj|t`e ld~t`fsakod~taenj%

><

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 39/130

Detjs}sjtdef Ijasejo Od~taenj Bzentd`e~% Abtjs ijasedef tlj od~taenj bzentd`e~)

 vj a}}i janl j|jk}ias—~ od~taenj bzentd`e t` aii `b tlj `tljs j|jk}ias~ ~jje ozsdef

tsadedef aeo ojfiej tlj ~z}}`st ~jt `b janl j|jk}ias a~ tlj j|jk}ias~ vldnl sjtzsejo a

~n`sj cji`v 1%=%

w ∌ ^z}},j$ ↟ Oj,|w$ 9 1%= ,3%1>$

De }santdnj tlj sj~zitdef ~z}}`st ~jt~ vdioi ras de ~dwj% B`s j|jk}ias~ bs`k fjejsdn

nia~~j~ ~znl a~ ’~h‛ vljsj vj j|}jnt kae ~hdj~ t` cj satljs ~dkdias) tlj ~z}}`st ~jt d~

iasfj% _lj ~z}}`st ~jt nae ai~` cj rjs ~kaii ‟ vldnl la}}je~ vlje dt~ n`ssj~}`eodef

j|jk}ias d~ jdtljs e`t rd~zaii od~tdentdrj `s akcdfz`z~i/den`ssjnti iacjijo% Vj }szej

ava tlj j|jk}ias~ vdtl ae jk}t ~z}}`st ~jt% ^jrjsai ijasejo od~taenj bzentd`e~

aeo tlj t`} jijkjet~ de tljds ~z}}`st ~jt~ asj ~l`ve aeo n`k}asjo t` tlj ejdflc`s~

fdrje a ~dk}ij tj|t`e+ld~t`fsak od~taenj de Bdfzsj 3%3 aeo Bdfzsj 3%>% _lj ijasejo

od~taenj bzentd`e~ asj o`def a f``o g`c at n`kcdedef jijkjetas od~taenj~ t` kja~zsj

~dkdiasdt% E`tdnj tlat ae j|jk}ias—~ ~z}}`st o`j~ e`t aiva~ n`etade j|jk}ias~ vdtl

tlj ~akj iacji% De }astdnzias ae j|jk}ias vdtl tlj iacji ’~taeodef }js~`e v`kae‛ va~

ojjkjo ~dkdias t` tlj tasfjt j|jk}ias vdtl iacji ’}js~`e‛ jrje tl`zfl tlj asj od~tdent

iacji~% Vj kja~zsj l`v `btje tld~ la}}je~ aeo ~l`v tlj t`} bjv jijkjet~ `b tlj iacji

n`ebz~d`e katsd| de _acij 3%3% E`tdnj tlat k`~t `b tlj~j n`ebz~d`e~ n`ssj~}`eo t` rd~zai

~t`} ~dfe ~dfe 6%<* s`ao ldflva s`ao >%?*}`ij ~tsjjtidflt 2%6* }adetdef }dntzsj >%?*

k`t`snnij k`t`scdhj 2%3* ~dojvaih s`ao >%3*

k`zetade~ k`zetade 2%3* ni`zo ~h >%1*fs`zeo fsa~~ ~dojvaih >%6* fsa~~ fs`zeo fsa~~ >%1*fsa~~ iave >%2* k`zetade k`zetade~ 3%6*

_acij 3%37 Iacji n`ebz~d`e abtjs od~taenj bzentd`e ijasedef% Vj ~l`v tlj t`} 13 iacji n`ebz~d`e~od~n`rjsjo abtjs od~taenj bzentd e ijasedef% B`s j|ak}ij) 6%<* `b tlj tdkj a ’~t`} ~dfe‛ j|jk}ias

a~~`ndatjo vdtl a ’~dfe‛ j|jk}ias~% _ld~ ~zffj~t~ tlat rd~zai ~dkdiasdt nae aiijrdatj ~`kj `b tlj }s`cijk~ vdtl akcdfz`z~ iacji~%

>4

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 40/130

~e`ek~%

3%>%> ^jfkjet Iacjidef

De `sojs t` ojtjskdej db tlj od~taenj bzentd`e~ asj `rjs+fittdef) vj n`e~dojs a ~jfkjet+

iacjidef ta~h `e a ljio+`zt ~zc~jt `b IacjiKj% @zs jraizatd e z~j~ a tj~t ~jt `b  1?6

`zto``s dkafj~ aii n`kdef bs`k `ej ~}jndfin ~zcb`iojs de IacjiKj ,t` kdedkdwj tlj

nlaenj~ `b ~dkdias oata cjdef z~jo b`s tsadedef aeo tj~tdef$% _ld~ tj~tdef ~zc~jt n`etade~

a t`tai `b  1) 1?2 `cgjnt~% B`s tlj ~jfkjet+iacjidef ta~h) vj sj}sj~jet c`tl j|jk}ias~

aeo tj~tdef sjfd`e~ c ~jfkjet+ca~jo bjatzsj~ ,a~ oj~nsdcjo de ^jntd`e 3%>%1$ j|tsantjo

bs`k tlj fs`zeo+tsztl ~jfkjetatd`e ka~h~% _lj k`sj odbfinzit }s`cijk `b azt`katdnaii 

j|tsantdef `cgjnt~ bs`k dkafj~ vlje e` fs`zeo+tsztl ~jfkjetatd`e~ asj fdrje d~ ojiajo

zetdi ^jntd`e 3%>%?% Vj iacji tlj tj~tdef ~jfkjet c tlj iacji bs`k tlj ~defij k`~t

 rd~zaii ~dkdias j|jk}ias z~def tlj ijasejo od~taenj~% Vj n`e~dojs a ~jt `b od~taenj

tlsj~l`io~ aeo n`k}ztj tlj }sjnd~d`e rjs~z~ sjnaii nzsrj% ]sjnd~d`e kja~zsj~ tlj

}s`cacdidt tlat a sjtzsejo iacji d~ dojetdnai t` tlj fs`zeo tsztl iacji aeo de `zs na~j

sjnaii kja~zsj~ tlj bsantd`e `b ~jfkjet~ tlat fjt iacjijo% A~ a ca~jidej) vj n`k}asj tlj

}jsb`skaenj `b `zs ijasejo od~taenj bzentd`e~ t` a ejasj~t+ejdflc`s nia~~dfijs z~def a

tj|t`e ld~t`fsak od~taenj% _lj }sjnd~d`e+sjnaii nzsrj nae cj ~jje de Bdfzsj 3%?% Db vj

`ei detjs}sjt tlj od~taenj bzentd`e~ tlat sjtzse a raizj cji`v 1%= ,vlat `zs ijasedef

b`skziatd`e ~zffj~t~ d~ tlj cj~t tldef t` o`$) vj `ctade iacji~ b`s 2=* `b `cgjnt~ vdtl a

}sjnd~d`e `b 41*% _ld~ ~zffj~t~ tlat tlj ijasejo od~taenj bzentd`e~ asj }s`rdodef a rjs 

kjaedefbzi od~taenj b`s sjn`fedtd`e%

?=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 41/130

= =%3 =%? =%2 =%< 1=

=%3

=%?

=%2

=%<

1

Sjnaii'

       ]     s     j     n       d     ~       d     `     e

Ijasejo Od~taenj Bzentd`e~ b`s ^jfkjet Iacjidef

 

Ijasejo Od~taenj Bzentd`e~

O91 tlsj~l`io }`det

_j|t`e Ld~t`fsak

Bdfzsj 3%?7 IacjiKj ~jfkjet iacjidef7 ]js+J|jk}ias Od~taenj Bzentd e~ r~% _j|t`e Ld~t`fsak~%Vj ijase a odbbjsjet od~taenj bzentd`e b`s janl j|jk}ias aeo a}}i tljk t` a ljio+`zt ~jt `b `cgjnt~

bs`k IacjiKj% ^jfkjet bjatzsj~ asj `ctadejo bs`k fs`zeo+tsztl ~jfkjetatd`e~% E`tj tlat tlj Od~taenjBzentd`e~ asj ~dfedfinaeti cjttjs tlae _j|t`e ld~t`fsak~ de tlj ldfl+}sjnd~d`e) i`v+sjnaii sjfdkj%

3%>%? ]as~def Dkafj~ rda Kzitd}ij ^jfkjetatd`e~

Vj aisjao ~av de ^jntd`e 3%> tlat j|jk}ias+~}jndfin od~taenj bzentd`e~ nae cj z~jo t`

ojtjskdej tlj dojetdt `b a fs`zeo+tsztl ~jfkjet) czt l`v nae vj z~j tljk t` ~jfkjet

`zt `cgjnt~ de~doj e`rji) zeiacjijo dkafj~0 Vj tanhij tld~ }s`cijk de tv` ~tj}~% Vj z~j

tlj kzitd}ij ~jfkjetatd`e a}}s`anl XL`djk jt ai%) 3==:)Sz~~jii jt ai%) 3==2)Kaid~djvdnw

aeo Jbs`~) 3==6[) vldnl va~ ~l`ve t` }s`rdoj f``o ~}atdai ~z}}`st b`s kae odbbjsjet

`cgjnt t}j~) t` fjejsatj a iasfj n`iijntd`e `b naeodoatj ~jfkjet~% Vj tlje hjj} tlj

e`e+sjozeoaet ~zc~jt `b ~jfkjet~ vldnl fjejsatjo vjii+~n`sdef rd~zai a~~`ndatd`e~

 vdtl tlj ~jt `b j|jk}ias~%

De XKaid~djvdnw aeo Jbs`~) 3==6[) vj asfzjo at fsjat ijeftl tlat z~def kzitd}ij

~jfkjetatd`e~ d~ cjttjs tlae `ej ‟ tlj hj de~dflt d~ tlat vldij e` ~defij ~jfkjetatd`e d~

idhji t` }s`oznj aii f``o ~jfkjet~) ~`kj `b tlj ~jfkjet~ de ~`kj `b tlj ~jfkjetatd`e~

asj rjs f``o% Vj z~j a rasdaet `b tlj kzitd}ij ~jfkjetatd`e a}}s`anl XSz~~jii jt ai%)

3==2)Kaid~djvdnw aeo Jbs`~) 3==6[) t` fjejsatj a ’~`z} `b ~jfkjet~‛ de a }zsji c`tt`k+

?1

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 42/130

z} ba~ld`e ,e` `cgjnt he`vijofj d~ }s`rdojo$% De }astdnzias) vj ras tlj }asakjtjs~ `b 

tv` ~jfkjetatd`e jefdej~ ‟ Kjae+^ldbt ca~jo JOD^@E XN`kaedndz aeo Kjjs) 3==3[

aeo E`skaidwjo Nzt~ X^ld aeo Kaidh) 3===[ ‟ t` fjejsatj kzitd}ij dkafj ~jfkjetatd`e~

b`s jrjs de}zt dkafj% ^denj vj larj ~l`ve de XKaid~djvdnw aeo Jbs`~) 3==6[ tlat ~`kj

n`k}`~dtj `cgjnt~ asj rjs zeidhji t` n`kj `zt a~ a ~defij ~jfkjet de ae ~jfkjetatd`e)

czt nae cj vjii a}}s`|dkatjo c a kjsfj `b a bjv aoganjet ~jfkjet~% _ljsjb`sj) vj

azfkjet `zs dedtdai ~`z} `b ~jfkjet~ c n`e~dojsdef tlj kjsfj~ `b  3 `s > aoganjet

~jfkjet~ a~ od~nz~~jo de XKaid~djvdnw aeo Jbs`~) 3==6[% _lj sj~zitdef c`tt k+z}

~jfkjet sj}sj~jetatd`e nae }s`rdoj sjfd`e~ vdtl f``o ~}atdai ~z}}`st b`s c`tl ~la}j+

bsjj ’~tzbb‛ `cgjnt~ ~znl a~ fsa~~) s`ao) aeo ~h a~ vjii a~ fi|jo+j|tjet ’tldef~‛ ~znl a~

nas~) cdnnij~) aeo }j`}ij% Ae aoodtd`eai aoraetafj `b z~def a c`tt k+z} kjnlaed~k

t` fjejsatj naeodoatj sjfd`e~ d~ tlat dt d~ deoj}jeojet `b tlj ezkcjs `b j|jk}ias~ `s

`cgjnt natjf`sdj~ z~jo de bzstljs }s`nj~~def ~tafj~%

_lj od~taenj bzentd`e~ ijasejo ~` bas asj e`t rjs f``o at sjn`fedwdef cao ~jfkjet~

‟ tlj ejrjs ~av ae de tsadedef! Vj tlz~ azfkjet tlj oata z~jo de od~taenj bzentd`e

ijasedef t` n`etade a iasfj ezkcjs ,`rjs >=) ===$ `b cao ~jfkjet~ vldnl na}tzsj tlj

a}}jasaenj `b }attjse~ tlat o` e`t n`ssj~}`eo t` ae `cgjnt~%

 Abtjs vj fjejsatj tlj ~`z} `b c`tt`k+z} ~jfkjet~) vj n`k}ztj tlj bzii katsd| `b 

od~taenj~ cjtvjje aii j|jk}ias~ aeo ~jfkjet~% Vj `ei n`e~dojs tlj od~taenj~ cji`v

1%= aeo tlj sj~zitdef a~~`ndatd`e~ asj rjs ~}as~j% @e arjsafj) ij~~ tlae %3* `b tlj

}`tjetdai a~~`ndatd`e~ asj antdrj% Qzaidtatdrj j|ak}ij~ `b j|jk}ias a~~`ndatd`e de tlj

~`z} `b c`tt`k+z} ~jfkjet~ nae cj ~jje de Bdfzsj 3%:% Qzdtj `btje) a ~defij ~jfkjet

 vdii a~~`ndatj vdtl kae j|jk}ias~ aeo vj n`e~tsznt a sjn`fedtd`e ~n`sj `zt `b tlj id~t

`b a~~`ndatdef od~taenj~% Ijttdef J  cj tlj id~t `b j|jk}ias~ a~~`ndatdef vdtl ~jfkjet ^ 

?3

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 43/130

tlj sjn`fedtd`e n`efiojenj ~,^) J $ d~ n`e~tszntjo a~ b`ii`v~7

~,^) J $ ; 1/j∌J 

1

Oj,^ $,3%1?$

B`s jraizatd`e }zs}`~j~) vj z~j tlj ljio+`zt ~zc~jt `b IacjiKj ojfiejo de ^jntd`e 3%>%

Vj iacji janl `cgjnt l}`tlj~d~ vdtl tlj k`~t bsjqzjeti `nnzssdef iacji ak`ef dt~

a~~`ndatd`e~% Vj ai~` sjtade aii ~jfkjet~ tlat a~~`ndatj vdtl at ija~t `ej j|jk}ias) aeo

tlz~ larj kzitd}ij ,}`tjetdaii aii n`ssjnt$ `rjsia}}def `cgjnt l}`tlj~j~% ^denj vj o`

e`t vaet t` }jeaidwj b`s tlj~j aitjseatdrj a~~`ndatd`e~ vj ojfiej ojtjntd`e }sjnd~d`e

a~ b`ii`v~7 vj n`e~dojs ae `cgjnt l}`tlj~d~ t` cj n`ssjnt db dt la~ a ~jfkjet `rjsia}~n`sj ,ojfiejo a~ de XKaid~djvdnw aeo Jbs`~) 3==6[$ `b at ija~t =%: vdtl a fs`zeo tsztl

sjfd`e tlat la~ tlj ~akj dojetdnai iacji a~ tlj l}`tlj~d~% Vj n`e~dojs aii `cgjnt~ de

taeojk aeo o` e`t }jeaidwj b`s kzitd}ij n`ssjnt `rjsia}}def a~~`ndatd`e~% Vj ras tlj

sjn`fedtd`e n`efiojenj t` nsjatj tlj }sjnd~d`e rjs~z~ sjnaii nzsrj de Bdfzsj 3%2%

De `sojs t` qzaetdb l`v vjii vj ~jfkjet `cgjnt~) b`s janl n`ssjnt ojtjntd`e vj

kja~zsj tlj `rjsia} ~n`sj cjtvjje tlj a~~`ndatjo fs`zeo tsztl sjfd`e~ aeo tlj `cgjnt

l}`tlj~j~% Vj ~l`v tlj arjsafj `rjsia} ~n`sj a~ vj ras tlj sjn`fedtd`e n`efiojenj

aeo n`k}asj tlat t` tlj arjsafj `rjsia} ~n`sj `b tlj cj~t ~jfkjet de `zs ~`z} `b 

~jfkjet~% _lj n`ssj~}`eodef }i`t nae cj ~jje `e tlj sdflt ~doj `b Bdfzsj 3%2%

_lj acdidt t` sjtzse a ~kaii ezkcjs `b `cgjnt l}`tlj~j~ vdtl ldfl qzaidt ~jfkje+

tatd`e ka~h~ d~ nszndai b`s dkafj zeojs~taeodef% Jrje tl`zfl tlj detjs}ia cjtvjje

`cgjnt~ ,j%f%) XSacde`rdnl jt ai%) 3==6[$ d~ njstadei a nszndai n`k}`ejet b`s ojtjskdedef

tlj dojetdt `b aii tlj ~njej—~ rd~zai jijkjet~) vj nae ~tdii nsjatj kjaedefbzi ,}astdai$

}as~j~ z~def `zs i`nai od~taenj bzentd`e~ ai`ej% Vj nsjatj ae dkafj }as~j bs`k `zs

`rjsia}}def `cgjnt l} tlj~j~ a~ b`ii`v~7 fdrje a id~t `b `cgjnt l}`tlj~j~ de a ~defij

dkafj ~`stjo c tljds sjn`fedtd`e n`efiojenj aeo ae dedtdaii jk}t id~t `b `cgjnt~ de tlj

?>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 44/130

czdiodef czdiodef czdiodef czdiodef

czdiodef czdiodef czdiodef czdiodef

wjcsa ns`~~def wjcsa ns`~~def wjcsa ns`~~def wjcsa ns`~~def

nas

nas

nas

nas

nas

nas

nas

tsjj tsjjtsjjtsjj

~h ~h~h ~h

~dojvaih ~dojvaih~dojvaih ~dojvaih

^jfkjet IacjiKj J|jk}ias~

nas

Bdfzsj 3%:7 J|jk}ias A~~`ndatd`e de tlj IacjiKj tj~t ~jt% Janl j|ak}ij ~l`v~ a ~jfkjet fjejsatjobs`k kzitd}ij ~jfkjetatd`e~ ai`ef vdtl dt~ t`} ? j|jk}ias a~~`ndatd`e~%

??

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 45/130

@cgjnt Ojtjntd`e ]jsb`skaenj @cgjnt ^jfkjetatd`e Annzsan

Sjnaii Sjnaii

   K  j  a  e   @  r  j  s   i  a  }   ^  n  `  s  j

   ]

  s  j  n   d  ~   d  `  e

Kjae @^

== ===%3 =%3

=%3=%3

=%?=%?

=%? =%?

=%2=%2

=%2 =%2=%< =%<

=%<=%<

1 1

11

Bdfzsj 3%27 Ojtjntdef aeo ^jfkjetdef `cgjnt~ de IacjiKj% C`tl nzsrj~ asj nsjatjo c rasdef tljsjn`fedtd`e n`efiojenj ,Jqzatd`e 3%1?$% _lj fis~t }i`t ~l`v~ tlj }sjnd~d`e+sjnaii nzsrj b`s tlj ta~h `b `cgjnt ojtjntd`e% A ojtjntd`e d~ ojjkjo n`ssjnt db dt sjtzse~ tlj ~akj iacji a~ vjii a~ la~ ae `rjsia} ~n`sj,@^$ fsjatjs tlae %: vdtl a fs`zeo+tsztl ~jfkjet% _lj ~jn`eo }i`t ~l`v~ tlj arjsafj ~jfkjetatd`eqzaidt `b n`ssjnt ojtjntd`e~ aeo n`k}asj~ tlat t` tlj kjae cj~t `rjsia} ~n`sj `b tlj de}zt kzitd}ij~jfkjetatd`e~%

}as~j) vj fsjjodi }ianj tlj nzssjet cj~t `cgjnt l}`tlj~j~ det` tlj id~t `b `cgjnt~ de tlj

}as~j vldij sjk`rdef aii l}`tlj~j~ tlat `rjsia} vdtl a ~n`sj `b =%: `s k`sj% _ld~ d~

jqzdraijet t` tlj t}dnai e`e+ka|dkzk ~z}}sj~~d`e aif`sdtlk ,a~ z~jo de XBjiwje~wvaic

jt ai%) 3==<[$) czt a}}idjo t` bsjj+b`sk ~jfkjet~ de~tjao `b c`zeodef c`|j~% _v`

sj~zitdef dkafj }as~def j|ak}ij~ nae cj ~jje de Bdfzsj 3%6%

_ld~ nla}tjs la~ ojk`e~tsatjo tlat kzitd}ij ~jfkjetatd`e~ aeo I`nai Od~taenj

Bzentd`e~ aii`v z~ t` }jsb`sk ojtjntd`e aeo ~jfkjetatd`e b`s a iasfj ezkcjs `b 

odbbjsjet `cgjnt~% De aoodtd`e) vj larj ~l`ve tlat tlj detjfsai n`k}`ejet `b ~znl a

~jfkjet+tlje+sjn`fedwj a}}s`anl d~ tlj ijasedef `b j|jk}ias+~}jndfin od~taenj bzentd`e~%

3%>%: B`ii`v+z} V`sh 

^denj dt~ }zcidnatd`e de 3==<) ~jrjsai sjiatjo a}}s`anlj~ larj cjje }zcid~ljo aeo dtd~ v`stlvldij kjetd`ejo tljk de tld~ tlj~d~% Sjfasodef tlj ’~jfkjet+tlje+sjn`fedwj‛

a}}s`anl) tlj a}}s`anl `b XId jt ai%) 3=1=[ la~ cjn`kj qzdtj }`}zias% XId jt ai%) 3=1=[

z~j fsa}l+nzt~ t` fjejsatj a ~`z} `b fifzsj+fs`zeo ~jfkjetatd`e~ aeo saeh tljk t`

}jsb`sk `cgjnt sjn`fedtd`e% @zs i`nai od~taenj bzentd`e bsakjv`sh la~ cjje j|tjeojo

?:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 46/130

t` tlj }s`cijk `b sjn`fedtd`e bs`k c`tl n`i`s aeo oj}tl ~je~`s~ XIad jt ai%) 3=11[% De

aoodtd`e) tljsj la~ cjje sd~def detjsj~t de i`nai od~taenj bzentd`e~ aeo kae odbbjsjet

b`skziatd`e~ larj cjje j|tje~drji aeaiwjo c Sakaeae aeo Cahjs XSakaeae aeo

Cahjs) 3=11[%

?2

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 47/130

czdiodef czdiodef czdiodef czdiodef

s`ao s`ao s`ao s`ao

nas nas nas nas

s`ao s`ao s`ao s`ao

czdiodef czdiodef czdiodef czdiodef

s`ao s`ao s`aos`ao

nas nasnasnas

nas nas nas

nas nas nas

~dojvaih

De}zt Dkafj ^jfkjet~ IacjiKj J|jk}ias~

Bdfzsj 3%67 Dkafj ]as~def rda Kzitd}ij ^jfkjetatd`e~ aeo I`nai Od~taenj Bzentd`e~% A }as~j d~nsjatjo c }jsb`skdef ~jfkjet+ca~jo e`e+ka|dkzk ~z}}sj~~d`e abtjs ~n`sdef ~jfkjet~ vdtl I`nai

Od~taenj Bzentd`e~%

?6

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 48/130

Nla}tjs >

J|jk}ias+^RK~

 A kjsj ojnaoj af`) azt`katdnaii sjn`fedwdef jrjsoa `cgjnt~ de dkafj~ ,~znl a~ tlj

cz~ de Bdfzsj >%1$ va~ tl`zflt t` cj ae aik`~t ze~`iracij ta~h% Tjt t`oa) a ezkcjs `b 

kjtl`o~ nae o` gz~t tlat vdtl sja~`eacij annzsan% Czt ijt z~ n`e~dojs tlj `zt}zt `b a

t}dnai `cgjnt ojtjnt`s ‟ a s`zfl c`zeodef c`| as`zeo tlj `cgjnt aeo a natjf`s iacji

,Bdfzsj >%1 ijbt$% Vldij tld~ kdflt cj ~zbfindjet b`s a sjtsdjrai ta~h ,’fieo aii cz~j~ de

tlj oataca~j‛$) dt ~jjk~ satljs ianhdef b`s ae ~`st `b ojj}js sja~`edef ac`zt tlj ~njej%

L`v d~ tlj cz~ `sdjetjo0 D~ dt a kded+cz~ `s a o`zcij+ojnhjs0 Vldnl }d|ji~ antzaii 

cji`ef t` tlj cz~0 Vlat d~ dt~ s`zfl fj`kjts0 _lj~j asj aii rjs laso qzj~td`e~ b`s

a t}dnai `cgjnt ojtjnt`s% Czt vlat db) de aoodtd`e t` tlj c`zeodef c`|) vj asj acij t`

`ctade ae a~~`ndatd`e vdtl a rjs ~dkdias j|jk}ias bs`k tlj tsadedef ~jt ,Bdfzsj >%1

sdflt$) vldnl nae }s`rdoj a ldfl ojfsjj `b n`ssj~}`eojenj% ^zoojei) ae hdeo `b 

kjta+oata }s`rdojo vdtl tlj tsadedef ~ak}ij ,a }d|ji+vd~j aee`tatd`e `s iacji ~znl a~

 rdjv}`det) ~jfkjetatd`e) n`as~j fj`kjts) a >O k`oji) attsdcztj~) jtn%$ nae cj ~dk}i 

tsae~bjssjo t` tlj ejv de~taenj%

?<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 49/130

Bdfzsj >%17 @cgjnt Natjf`s Ojtjnt`s r~% Je~jkcij `b J|jk}ias Ojtjnt`s~% @zt}zt `b a t}dnai `cgjntojtjnt`s d~ gz~t a c`zeodef c`| aeo a natjf`s iacji ,ijbt$% Czt `zs je~jkcij `b J|jk}ias+^RK~ d~ acij t`

a~~`ndatj janl ojtjntd`e vdtl a rd~zaii ~dkdias tsadedef j|jk}ias ,sdflt$) aii`vdef b`s odsjnt tsae~bjs `b kjta+oata ~znl a~ ~jfkjetatd`e) fj`kjts) jrje a >O k`oji ,c`tt`k$%

?4

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 50/130

Bdfzsj >%37 Natjf`s ^RK r~% J|jk}ias+^RK~% De~tjao `b tsadedef a ~defij }js+natjf`s nia~~dfijs) vj

tsade a ~j}asatj idejas ^RK nia~~dfijs b`s janl j|jk}ias de `zs oata~jt vdtl a ~defij }`~dtdrj j|ak}ij aeokdiid`e~ `b ejfatdrj vdeo`v~% Ejfatdrj~ n`kj bs`k dkafj~ e`t n`etadedef ae de~taenj~ `b tlj j|jk}ias—~natjf`s%

Vlat ~jjk~ oj~dsacij d~ ae a}}s`anl tlat la~ aii tlj ~tsjeftl~ `b a Oaiai+_sdff~ X Oaiai

aeo _sdff~) 3==:[ / Bjiwje~wvaic jt ai% XBjiwje~wvaic jt ai%) 3=1=[ ojtjnt`s „ }`vjsbzi

oj~nsd}t`s) jbfindjet od~nsdkdeatdrj bsakjv`sh) nijrjs kdedef `b laso+ejfatdrj~) jtn% „czt vdtl`zt tlj osavcanh~ dk}`~jo c a sdfdo) natjf`s+ca~jo sj}sj~jetatd`e `b tlj

}`~dtdrj~% _ }zt dt ae`tljs va) vlat vj vaet d~ a kjtl`o tlat d~ e`e+}asakjtsdn

 vlje sj}sj~jetdef tlj }`~dtdrj~) czt }asakjtsdn ,`s at ija~t ~jkd+}asakjtsdn$ vlje

sj}sj~jetdef tlj ejfatdrj~% _ld~ d~ tlj hj k`tdratd`e cjldeo `zs J|jk}ias+ RK

a}}s`anl% Vlat vj }s`}`~j d~ a kassdafj `b tlj j|jk}ias+ca~jo kjtl`o`i`f) vldnl

aii`v~ z~ t` }s`}afatj sdnl aee`tatd`e~ bs`k j|jk}ias~ `et` ojtjntd`e vdeo`v~) vdtl

od~nsdkdeatdrj tsadedef) vldnl aii`v~ z~ t` ijase }`vjsbzi j|jk}ias+ca~jo nia~~dfijs~

bs`k ra~t ak`zet~ `b }`~dtdrj aeo ejfatdrj oata%

@zs `cgjnt ojtjnt`s d~ ca~jo `e a rjs ~dk}ij doja7 t` ijase a ~j}asatj nia~~dfijs b`s

janl j|jk}ias de tlj oata~jt ,~jj Bdfzsj >%3$% Vj sj}sj~jet janl j|jk}ias z~def a sdfdo

L@F tjk}iatj XOaiai aeo _sdff~) 3==:[% ^denj vj z~j a idejas ^RK) janl nia~~dfijs

nae cj detjs}sjtjo a~ a ijasejo j|jk}ias+~}jndfin L@F vjdflt rjnt`s% A~ a sj~zit)

de~tjao `b a ~defij n`k}ij| natjf`s ojtjnt`s) vj larj a iasfj n`iijntd`e `b ~dk}ijs

deodrdozai J|jk}ias+^RK  ojtjnt`s~ `b rasd`z~ ~la}j~ aeo ~dwj~) janl ldfli tzejo t` tlj

j|jk}ias—~ a}}jasaenj% Czt) zeidhj a ~taeoaso ejasj~t+ejdflc`s ~nljkj) janl ojtjnt`s

d~ od~nsdkdeatdrji tsadejo% ^` vj asj acij t` fjejsaidwj kznl cjttjs vdtl`zt sjqzdsdef

:=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 51/130

Bdfzsj >%>7 L@F+katnldef n`k}asd~`e% Fdrje a cdnnij tsadedef ~ak}ij bs`k ]A^NAI ,sj}sj~jetjo vdtla L@F vjdflt rjnt`s v$) vj ~l`v tlj t`} 2 katnlj~ bs`k tlj ]A^NAI tj~t+~jt z~def tlsjj kjtl`o~% S`v 17 eadrj ejasj~t ejdflc`s ,z~def sav e`skaidwjo L@F$% S`v 37 _sadejo J|jk}ias+^RK ,e`tdnj l`v v

b`nz~j~ `e cdhj+~}jndfin jofj~$% S`v >7 Ijasejo od~taenj bzentd`e „ ae J|jk}ias+^RK czt tsadejo de tlj’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj) vdtl tlj j|jk}ias cjdef }ianjo at tlj `sdfde ,i``~ji n`ssj~}`eodef

t` XBs`kj aeo Kaidh) 3==2) Kaid~djvdnw aeo Jbs`~) 3==<[$%

ae je`sk`z~ oata~jt `b j|jk}ias~) aii`vdef z~ t` }jsb`sk ~zs}sd~defi vjii jrje `e a

k`ojsatji+~dwjo tsadedef oata~jt ~znl a~ tlj ]A^NAI R@N 3==6 XJrjsdeflak jt ai%)

3=1=[%

_lj ~dkdiasdt bzentd`e~ vj }s`}`~j de tld~ Nla}tjs tahj tlj b`sk `b a o`t }s`oznt

cjtvjje ae j|jk}ias—~ ijasejo vjdflt rjnt`s vj ,ai~` sjbjssjo t` a~ ae j|jk}ias+~}jndfin

tjk}iatj$ aeo |d) tlj sav bjatzsj~ sj}sj~jetdef ae de}zt `cgjnt ,}iz~ a cda~ cj$%

Oj,|d$ ; v_ j |d ( cj ,>%1$

_lj `zt}zt `b ~znl a ~dkdiasdt bzentd`e nae cj detjs}sjtjo a~ tlj ~dfejo od~taenj

bs`k tlj l}js}iaej sj}sj~jetjo c  v_ j | ( cj ; =% _lj z~j `b a l}js}iaej }a+

sakjtjsdwatd`e d~ k`tdratjo c tlj fsjat ~znnj~~ `b tjk}iatj~ de tlj `cgjnt ojtjntd`e

n`kkzedt ,j%f%) kae `b tlj cj~t+}jsb`skdef a}}s`anlj~ asj tlj L@F+ca~jo Oaiai+

_sdff~/Bjiwje~wvaic jt ai% bakdi `b k`e`idtldn ojtjnt`s~$% L`vjrjs) zeidhj k`e`idtldn

a}}s`anlj~ vldnl ijase natjf`s+~}jndfin tjk}iatj~ ,j%f%) vo`f) vnas) vnlads) jtn$) `zs

tjk}iatj~ asj j|jk}ias+~}jndfin ,j%f%) vj1)vj3) % % % )vjE $ aeo tsadejo de a }js+j|jk}ias

ba~ld`e% De `sojs t` kadetade tlj ~}jndfindt `b tlj ijasejo tjk}iatj t` a ~defij j|jk}ias

:1

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 52/130

J|jk}ias Ijasejo v

Bdfzsj >%?7 Ijasedef ae J|jk}ias+~}jndfin L@F _jk}iatj% ^denj vj z~j idejas nia~~dfijs~) tlj sj~zitdef

ojnd~d`e c`zeoas nae cj detjs}sjtjo a~ ae j|jk}ias+~}jndfin L@F tjk}iatj% _lj rd~zaidwatd`e d~ tlj}`~dtdrj }ast `b tlj ijasejo l}js}iaej%

|j) vj ijase tlj tjk}iatj vj vdtl a ~defij }`~dtdrj de~taenj% _lj sj~zitdef vjdflt

 rjnt`s nae tlje cj detjs}sjtjo a~ a tjk}iatj ldfli tzejo t` tlj de}zt j|jk}ias ,~jj

Bdfzsj >%?$% ^denj vj z~j tlj ~akj idejas ^z}}`st Rjnt`s Kanldej ,^RK$ a}}s`anl

a~ XOaiai aeo _sdff~) 3==:[ t` ijase vj) czt asj acij t` }s`oznj a tjk}iatj ldfli tzejo

t` tlj j|jk}ias |j) vj ozc tld~ a}}s`anl tlj J|jk}ias+^RK%

>%1 Vl` Ejjo~ bsdjeo~0 Ijasedef Vdtl a ^defij ]`~dtdrj

De~taenj

@ej v`zio dkafdej tlat tsadedef ae ^RK vdtl a ~defij }`~dtdrj j|ak}ij vdii caoi 

`rjs+fit% Czt e`tj tlat vj sjqzdsj bas ij~~ bs`k a }js+j|jk}ias nia~~dfijs a~ n`k}asjo t`

a }js+natjf`s nia~~dfijs „ janl `b `zs ojtjnt`s~ `ei ejjo~ t` }jsb`sk vjii `e rd~zaii 

~dkdias j|ak}ij~% ^denj janl nia~~dfijs d~ ~`irdef a kznl ~dk}ijs }s`cijk tlae de tlj

bzii+natjf`s na~j) vj nae z~j a ~dk}ij sjfziasdwjo idejas ^RK t` }sjrjet `rjs+fittdef%

 Ae`tljs nszndai n`k}`ejet d~ tlat) vldij vj `ei larj a ~defij }`~dtdrj j|ak}ij) vj

:3

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 53/130

larj kdiid`e~ `b ejfatdrj j|ak}ij~ tlat vj kdej bs`k tlj tsadedef ~jt ,d%j%) bs`k dkafj~

tlat o` e`t n`etade ae de~taenj~ `b tlj j|jk}ias—~ natjf`s$% A~ a sj~zit) tlj j|jk}ias—~

ojnd~d`e c`zeoas d~ ojfiejo) de iasfj }ast) c vlat dt d~ e`t% @ej `b tlj hj n`etsdcztd`e~

`b `zs a}}s`anl d~ tlat vj ~l`v fjejsaidwatd`e d~ }`~~dcij bs`k a ~defij }`~dtdrj j|ak}ij

aeo a ra~t ~jt `b ejfatdrj~%

 At tj~t+tdkj) vj deoj}jeojeti sze janl nia~~dfijs `e tlj de}zt dkafj ,~jj tlj ~idodef

 vdeo`v od~nz~~d`e de A}}jeod| ~jntd`e C%3$ aeo z~j ~dk}ij e`e+ka|dkzk ~z}}sj~~d`e

t` nsjatj a fieai ~jt `b ojtjntd`e~) vljsj janl ojtjntd`e d~ a~~`ndatjo vdtl a ~defij

j|jk}ias% L`vjrjs) ~denj `zs deoj}jeojeti+tsadejo nia~~dfijs~ kdflt e`t `zt}zt odsjnti 

n`k}asacij ~n`sj~) vj kz~t }jsb`sk naidcsatd e `e a raidoatd`e ~jt% _lj detzdtd`e

na}tzsjo c tld~ naidcsatd`e ~tj} d~ tlat odbbjsjet j|jk}ias~ vdii `bbjs osa~tdnaii odbbjsjet

fjejsaidwatd`e }`tjetdai% A ljardi `nnizojo `s tszenatjo `cgjnt de~taenj vdii larj

}``sjs fjejsaidwatd e tlae a nijaejs j|jk}ias) tlz~ s`cz~tej~~ afade~t jrje a ~defij

cao nia~~dfijs d~ dk}jsatdrj t` `ctadedef f``o `rjsaii }jsb`skaenj% ^denj `zs nia~~dfijs~

asj tsadejo vdtl`zt ~jjdef ae `tljs }`~dtdrj de~taenj~ czt dt~jib) vj nae z~j tljk b`s

naidcsatd`e de a ’ijarj+aii+czt+`ej+`zt‛ ba~ld`e%

Dt d~ v`stlvldij }`detdef `zt ~`kj `b tlj hj odbbjsjenj~ cjtvjje `zs a}}s`anl aeo

`tljs sjiatjo ^RK+ca~jo tjnledqzj~ ~znl a~ `ej+nia~~ ^RK~ X^nlih`}b jt ai%) 3==1) Nlje

jt ai%) 3==1[) kzitd+nia~~ hjseji ^RK~) hjseji+ijasedef a}}s`anlj~ X Rjoaiod jt ai%) 3==4[)

aeo tlj hEE+^RK aif`sdtlk XWlaef jt ai%) 3==2[% Aii `b tlj~j a}}s anlj~ sjqzdsj

ka}}def tlj j|jk}ias~ det` a n`kk`e bjatzsj ~}anj `rjs vldnl a ~dkdiasdt hjseji

nae cj n`k}ztjo ,vldnl vj ar`do$) czt k`sj dk}`staeti) hjseji kjtl`o~ i`~j tlj

~jkaetdn~ `b ~defij+j|jk}ias a~~`ndatd`e~ vldnl asj ejnj~~as b`s ldfl qzaidt kjta+oata

tsae~bjs%

Fdrje a ~jt `b tsadedef j|jk}ias~) vj sj}sj~jet janl j|jk}ias J  rda a sdfdo L@F

tjk}iatj) |J % Vj nsjatj a oj~nsd}t`s bs`k tlj fs`zeo+tsztl c`zeodef c`| `b janl

:>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 54/130

Bdfzsj >%:7 J|jk}ias+^RK~% A bjv ’tsade‛ j|jk}ias~ vdtl tljds t`} ojtjntd`e~ `e tlj ]A^NAI R@N

tj~t+~jt% E`tj tlat janl j|jk}ias—~ L@F la~ dt~ `ve odkje~d`e~% E`tj ai~` l`v janl ojtjnt`s d~ ~}jndfine`t gz~t t` tlj tsade—~ `sdjetatd`e) czt jrje t` tlj t}j `b tsade%

j|jk}ias vdtl a njii ~dwj `b  < }d|ji~ z~def a ~dwdef ljzsd~tdn vldnl attjk}t~ t` sj}sj~jet

janl j|jk}ias vdtl s`zfli 1== njii~% De~tjao `b vas}def janl j|jk}ias t` a nae`ednai

bsakj) vj ijt janl j|jk}ias ojfiej dt~ `ve L@F odkje~d`e~ sj~}jntdef tlj a~}jnt satd`

`b dt~ c`zeodef c`| ,~jj tlj oj~nsd}td`e `b tld~ }s`nj~~ de A}}jeod| ~jntd`e C%3$% Vj

nsjatj ejfatdrj ~ak}ij~ `b tlj ~akj odkje~d`e~ a~ |J  c j|tsantdef ejfatdrj vdeo`v~)

 E J ) bs`k dkafj~ e`t n`etadedef ae `cgjnt~ bs`k tlj j|jk}ias—~ natjf`s%

Janl J|jk}ias+^RK) ,vJ ) cJ $) tsdj~ t` ~j}asatj |J  bs`k aii vdeo`v~ de  E J  c tlj

iasfj~t }`~~dcij kasfde de tlj L@F bjatzsj ~}anj% Ijasedef tlj vjdflt rjnt s vJ 

ak`zet~ t` `}tdkdwdef tlj b`ii`vdef n`erj| `cgjntdrj7

™J ,v) c$ ; yyvyy3 ( N 1l,v_ |J  ( c$ ( N 3|∌E J

l,∓v_ | ∓ c$ ,>%3$

Vj z~j tlj ldefj i`~~ bzentd`e l,|$ ; ka|,=) 1 ∓ |$) vldnl aii`v~ z~ t` z~j tlj

laso+ejfatdrj kdedef a}}s`anl t` n`}j vdtl kdiid`e~ `b ejfatdrj vdeo`v~ cjnaz~j tlj

~`iztd`e `ei oj}jeo~ `e a ~kaii ~jt `b ejfatdrj ~z}}`st rjnt`s~%

Bdfzsj >%> `bbjs~ a rd~zai n`k}asd~`e `b tlj }s`}`~jo J|jk}ias+^RK kjtl`o afade~t

tv` aitjseatdrj~ b`s tlj ta~h `b ojtjntdef tj~t+~jt katnlj~ b`s a ~defij j|jk}ias) a ~e`v+

:?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 55/130

n`rjsjo cdnnij% _lj fis~t s`v ~l`v~ a ~dk}ij ejasj~t+ejdflc`s a}}s`anl% _lj ~jn`eo

s`v ~l`v~ tlj `zt}zt `b `zs }s`}`~jo J|jk}ias+^RK% E`tj tlj ~zctij nlaefj~ de tlj

ijasejo L@F rjnt`s v) kahdef dt b`nz~ k`sj `e tlj cdnnij% _lj tldso s`v ~l`v~ tlj

`zt}zt `b ijasedef a od~taenj bzentd`e) satljs tlae a idejas nia~~dfijs% B`s tld~) vj a}}idjo

tlj ~defij+}`~dtdrj J|jk}ias+^RK bsakjv`sh de tlj ’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj)

 vdtl tlj j|jk}ias cjdef }ianjo at tlj `sdfde ,tld~ d~ n`enj}tzaii ~dkdias t` XKaid~djvdnw

aeo Jbs`~) 3==<) Bs`kj aeo Kaidh) 3==2[$% Vj `c~jsrjo tlat tlj njetjsjo+at+j|jk}ias

n`e~tsadet kaoj tlj od~taenj bzentd`e ij~~ }`vjsbzi tlae tlj idejas nia~~dfijs ,~jj Sj~zit~

~jntd`e$% Bdfzsj >%: ~l`v~ a bjv J|jk}ias+^RK~ bs`k tlj ’tsade‛ natjf`s ai`ef vdtl

tljds t`} ojtjntd`e~ `e tlj tj~t+~jt% E`tj l`v ~}jndfin janl ojtjnt`s d~ „ e`t gz~t t` tlj

tsade—~ `sdjetatd`e) czt jrje tlj t}j `b tsade%

>%1%1 Sjiatd`e~ld} t` I`nai Od~taenj Bzentd`e~

^denj J|jk}ias+^RK~ asj tsadejo de a }js+j|jk}ias ba~ld`e) tlj cjas a ~ts`ef sj+

~jkciaenj t` tlj I`nai Od~taenj Bzentd`e~ ojfiejo de Nla}tjs 3% L`vjrjs) tljsj asj

~`kj hj odbbjsjenj~ vldnl asj v`stlvldij t` j|}iade de ojtadi% Zeidhj od~taenj bzen+

td`e~ XBs`kj aeo Kaidh) 3==2) Kaid~djvdnw aeo Jbs`~) 3==<[) vldnl sjqzdsj a ~jt `b 

}`~dtdrj j|ak}ij~ ozsdef ijasedef) J|jk}ias+ RK~ `ei sjqzdsj a ~defij }`~dtdrj de+

~taenj% Vj aoodtd`eaii z~j kdiid`e~ `b dkafj vdeo`v~ a~ tlj ejfatdrj~ z~jo t` tsade

vj% _lj ~ldbt t`vaso~ z~def ij~~ }`~dtdrj oata aeo k`sj ejfatdrj oata d~ a e`tacij

nlaefj de }ldi`~`}l b`s ijasedef }js+j|jk}ias ~dkdiasdt kja~zsj~% C rdstzj `b cjdef

a l}js}iaej) tlj J|jk}ias+^RK ~dkdiasdt bzentd`e d~ e`t sjqzdsjo t` cj njetjsjo at

tlj j|jk}ias aeo aii`v~ z~ t` cjttjs aoa}t t` tlj tsadedef oata% Z~def a ~defij }`~dtdrj

de~taenj) vj acaeo`e tlj ejjo b`s ae iatjet rasdacij~ ,vldnl vjsj z~jo t` jen`oj

a rasdacij ezkcjs `b bsdjeo~ ozsdef od~taenj bzentd`e ijasedef de ^jntd`e 3$) aeo tlj

sj~zitdef }s`cijk d~ n`erj|% _ld~ kjae~ tlat tlj `}tdkdwatd`e }s`cijk der`irjo d~ kznl

::

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 56/130

ba~tjs t` ~`irj ,c z~def ldfli `}tdkdwjo ^RK }anhafj~ tzejo b`s iasfj+~naij }s`cijk~$

aeo vj o` e`t larj t` v`ss ac`zt i`nai kdedka ,a laiikash `b n`erj|dt$% K`sj

dk}`staeti) sjqzdsdef a ~defij }`~dtdrj de~taenj kjae~ tlat `zs oj}jeojenj `e natjf`s 

iacji~ d~ kznl i`vjs tlae tlj od~taenj bzentd`e~ de ^jntd`e 3 ,vldnl sjqzdsjo tlj z~j

`b natjf`sdj~ t` ojfiej tlj }`~dtdrj nia~~$% _ld~ d~ ae dk}`staet ~tj} de tlj odsjntd`e `b 

n`k}ijtji k`rdef ava bs`k natjf`sdj~ ozsdef ijasedef%

Cjnaz~j J|jk}ias+^RK~ nae cj detjs}sjtjo a~ j|jk}ias+~}jndfin ~dkdiasdt kja~zsj~)

`zs bsakjv`sh ~lasj~ ~`kj ~dkdiasdtdj~ vdtl od~taenj+ijasedef a}}s`anlj~) de }astdnzias

tl`~j tlat ijase }js+j|jk}ias od~taenj bzentd`e~ ,j%f%) XBs`kj aeo Kaidh) 3==2) Kai+

d~djvdnw aeo Jbs`~) 3==<[$% L`vjrjs) tlj nszndai odbbjsjenj cjtvjje a }js+j|jk}ias

nia~~dfijs aeo a }js+j|jk}ias od~taenj bzentd`e d~ tlat tlj iattjs b`snj~ tlj j|jk}ias

dt~jib t` larj tlj ka|dkaii attadeacij ~dkdiasdt% Ae J|jk}ias+^RK la~ kznl k`sj

bsjjo`k de ojfiedef tlj ojnd~d`e c`zeoas) aeo d~ cjttjs acij t` den`s}`satj de}zt bs`k

tlj ejfatdrj ~ak}ij~ ,~jj Bdfzsj >%> b`s a n`k}asd~`e) t` cj od~nz~~jo iatjs$%

>%1%3 Detjs}sjtdef ^RK~ tsadejo vdtl a ~defij }`~dtdrj de~taenj

_lj Sj}sj~jetjs _lj`sjk XValca) 146>[ tjii~ z~ tlat tlj ~`iztd`e t` ae ^RK }s`cijk

,a t}j `b I3+sjfziasdwjo ijasedef }s`cijk$ vdii tahj tlj b`ii`vdef b`sk7

vj ; αj|j (d∌E j

αd|d ,>%>$

Ljsj) αj d~ tlj ozai rasdacij a~~`ndatjo vdtl tlj ~defij }`~dtdrj de~taenj) aeo αd d~ a ozai

 rasdacij a~~`ndatjo vdtl tlj ejfatdrj~% Vj larj `c~jsrjo tlat aii `b `zs J|jk}ias+^RK~

asj acij t` ~j}asatj tlj ~defij }`~dtdrj bs`k tlj ejfatdrj~) aeo tlat tlj ~z}}`st rjnt`s~

s`zfli baii `e tljds sj~}jntdrj kasfde% |j d~ `ej ~z}}`st rjnt`s vdtl αj ≌ 1) aeo vj

fjt a ~kaii ~jt `b ejfatdrj ~z}}`st rjnt`s~ vdtl αd ≌ ∓1% Aii e`e+~z}}`stdef rjnt`s~

:2

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 57/130

larj αd ; =% Deodnatdef tlj ~jt `b ejfatdrj ~z}}`st rjnt`s~ ,ai~` naiijo laso ejfatdrj~$

a~ E^R) vj fjt7

vj ≌ |j ∓ d∌E^R

|d ,>%?$

Dt e`v cjn`kj~ nijas tlat tlj J|jk}ias+^RK ijase~ a tjk}iatj vl`~j }`~dtdrj }ast

 vdii sj~jkcij tlj j|jk}ias aeo tlj ejfatdrj }ast d~ a ~zk `rjs ejfatdrj ~z}}`st rjnt`s~%

>%3 Naidcsatd`e

Z~def tlj }s`njozsj ac`rj) vj tsade ae je~jkcij `b J|jk}ias+^RK) `ej b`s janl }`~dtdrj

de~taenj de tlj tsadedef ~jt% L`vjrjs) ozj t` tlj deoj}jeojet tsadedef }s`njozsj) tljds

`zt}zt~ asj e`t ejnj~~asdi n`k}atdcij% A n`kk`e ~tsatjf t` sjn`endij tlj `zt}zt~

`b kzitd}ij nia~~dfijs~ d~ t` }jsb`sk naidcsatd`e c fittdef a }s`cacdidt od~tsdcztd`e t` a

ljio+`zt ~jt `b ejfatdrj aeo }`~dtdrj ~ak}ij~ X]iatt) 1444) Bs`kj aeo Kaidh) 3==2[% _ld~

}s`njozsj d~ `btje naiijo ]iatt—~ ^naidef Aif`sdtlk) aeo fjejsaii a}}idjo t` a ljio+`zt

~jt vldnl la~ tlj ~akj od~tsdcztd`e `b ~ak}ij~ a~ tlj tsadedef+oata% L`vjrjs) de `zs

na~j) ~denj janl j|jk}ias+^RK d~ ~z}}`~jo t` fisj `ei `e rd~zaii ~dkdias j|ak}ij~) vj

naee`t ~a b`s ~zsj vldnl `b tlj ljio+`zt ~ak}ij~ ~l`zio cj n`e~dojsjo a~ }`~dtdrj~ a

 }sd`sd% B`s j|ak}ij) b`s a bs`etai rdjv `b ae tsade) `ei `tljs bs`etai rdjv~ `b ~dkdias

tsade~ ~l`zio cj n`e~dojsjo a~ }`~dtdrj~% B`stzeatji) gz~t idhj ozsdef tsadedef) vlat vj

nae cj ~zsj ac`zt d~ tlat tlj nia~~dfijs ~l`zio e`t fisj `e ejfatdrj vdeo`v~% _ljsjb`sj)

 vj ijt janl j|jk}ias ~jijnt dt~ `ve }`~dtdrj~ aeo tlje z~j tlj ^RK `zt}zt ~n`sj~ `e

tlj~j }`~dtdrj~) de aoodtd`e t` i`t~ `b ljio+`zt ejfatdrj~) t` naidcsatj tlj J|jk}ias+^RK%

_ `ctade janl j|jk}ias—~ naidcsatd`e }`~dtdrj~) vj sze tlj J|jk}ias+^RK `e tlj

 raidoatd`e ~jt) nsjatj a ~jt `b e`e+sjozeoaet ojtjntd`e~ z~def e`e+ka|dkzk ~z}}sj~~d`e)

:6

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 58/130

aeo n`k}ztj tlj `rjsia} ~n`sj cjtvjje sj~zitdef ojtjntd`e~ aeo fs`zeo+tsztl c`zeodef+

c`|j~% Vj tsjat aii ojtjntd`e~ vldnl `rjsia} c k`sj tlae =%: vdtl fs`zeo+tsztl c`|j~

a~ }`~dtdrj~ ,tld~ d~ tlj ~taeoaso ]A^NAI R@N nsdtjsd`e b`s a ~znnj~~bzi ojtjntd`e$%

 Aii ojtjntd`e~ vdtl ae `rjsia} i`vjs tlae =%3 asj tsjatjo a~ ejfatdrj~) aeo vj fit a

i`fd~tdn bzentd`e t` tlj~j ~n`sj~% E`tj tlat) aitl`zfl vj naee`t fzasaetjj tlat ldfli 

`rjsia}}def n`ssjnt ojtjntd`e~ vdii deojjo cj rd~zaii ~dkdias t` tlj j|jk}ias) vdtl rjs 

ldfl }s`cacdidt tlj vdii cj) ~denj tlj vjsj ldfli saehjo c tlj j|jk}ias+^RK de tlj

fis~t }ianj%

@zs naidcsatd`e ~tj} nae cj detjs}sjtjo a~ a ~dk}ij sj+~naidef aeo ~ldbtdef `b tlj

ojnd~d`e c`zeoas ,~jj Bdfzsj >%2$ „ }``si }jsb`skdef j|jk}ias~ vdii cj ~z}}sj~~jo

c lardef tljds ojnd~d`e c`zeoas k`rj t`vaso~ tlj j|jk}ias aeo vjii+}jsb`skdef

j|jk}ias~ vdii cj c``~tjo c lardef tljds ojnd~d`e c`zeoas k`rj ava bs`k tlj

j|jk}ias% Vldij tlj sj~zitdef ojnd~d`e c`zeoas d~ e` i`efjs ae `}tdkai ~`iztd`e b`s

tlj i`nai+^RK }s`cijk) jk}dsdnaii vj b`zeo tld~ }s`njozsj fsjati dk}s`rj~ tlj detjs+

j|jk}ias `sojsdef% Fdrje a ojtjntd`e | aeo tlj ijasejo ~dfk`do }asakjtjs~ ,αJ ) γJ $)

tlj naidcsatjo ojtjntd`e ~n`sj b`s j|jk}ias J  d~ a~ b`ii`v~7

b ,|yvJ ) αJ ) γJ $ ;1

1 ( j∓αJ,v_ J|∓γJ$

Vldij tlj i`fd~tdn fittdef d~ }jsb`skjo deoj}jeojeti b`s janl j|jk}ias) vj b`zeo

tlat dt fdrj~ z~ a n`e~dojsacij c``~t de ojtjntd`e }jsb`skaenj `rjs z~def sav ^RK

`zt}zt ~n`sj~% At tj~t+tdkj) vj nsjatj ojtjntd`e~ bs`k janl nia~~dfijs c tlsj~l`iodef tlj

sav ^RK `zt}zt ~n`sj at ∓1 ,tlj ejfatdrj kasfde$ aeo tlje sj~naij tljk z~def janl

j|jk}ias—~ ijasejo ~dfk`do }asakjtjs~%

:<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 59/130

Bdfzsj >%27 J|jk}ias+^RK naidcsatd`e% _lj naidcsatd`e ~tj} sj~naij~ tlj ^RK ~n`sj~ czt o`j~ e`t abbjnttlj `sojsdef `b tlj katnlj~) aii`vdef z~ t` n`k}asj tlj `zt}zt~ `b kzitd}ij deoj}jeojeti+tsadejoJ|jk}ias+^RK~%

>%> J|jk}ias N`+`nnzssjenj Katsdnj~

 Abtjs naidcsatd`e) vj nae nsjatj ojtjntd`e vdeo`v~ b`s janl j|jk}ias de a ~idodef+

 vdeo`v ba~ld`e% A n`kk`e aeo ~dk}ij kjnlaed~k b`s ~z}}sj~~def sjozeoaet sj~}`e~j~

d~ e`e+ka|dkzk ~z}}sj~~d`e ,EK^$8 l`vjrjs) z~def EK^ odsjnti `e j|jk}ias~ kjae~

tlat kzitd}ij j|jk}ias~ vdii cj n`k}jtdef b`s ojtjntd`e~ vdeo`v~% L`vjrjs) ja~ t`

sjn`fedwj `cgjnt~ vdii `btje larj kae rd~zai a~~`ndatd`e~ nsjatjo as`zeo tlj `cgjnt

`b detjsj~t ‟ deb`skatd`e vldnl n`k}jtdtdrj EK^ n`k}ijtji od~sjfaso~% De~tjao `b 

 gz~t z~def tlj sav a~~`ndatd`e ~n`sj) vj }s`}`~j t` azfkjet janl ojtjntd`e vdtl ae

j|jk}ias n`etj|t ~n`sj vldnl z~j~ tlj dojetdtdj~ aeo ~n`sj~ `b ejasc ojtjntd`e~ t`

c``~t tlj sav ojtjntd`e ~n`sj% B`s janl ojtjntd`e vj fjejsatj a n`etj|t bjatzsj ~dkdias

t` XC`zsojr jt ai%) 3=1=) Bjiwje~wvaic jt ai%) 3=1=[ vldnl }``i~ de a~~`ndatd`e ~n`sj~ `b 

ejasc ,`rjsia}}def$ ojtjntd`e~ aeo fjejsatj~ tlj fieai ojtjntd`e ~n`sj c a vjdfltjo

~zk `b tlj i`nai a~~`ndatd e ~n`sj aeo tlj n`etj|t ~n`sj% A ~jt `b  H  ojtjntd`e~ O d~

}s`oznjo c a}}idef a ~jt `b  E  j|jk}ias~ t` a ~defij dkafj aeo }jsb`skdef EK^

ans`~~ ojtjntd`e~ vdtl tlj ~akj j|jk}ias do% Vldij EK^ kahj~ ~zsj tlat vj tls`v

ava sjozeoaet ojtjntd`e~ bs`k tlj ~akj j|jk}ias) vj vdii ~tdii cj ijbt vdtl kae 

:4

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 60/130

`rjsia}}def ojtjntd`e~ cjnaz~j kae j|jk}ias asj ~dkdias t` janl `tljs%

Vj nae tldeh `b janl ojtjntd`e a~ a tsd}ijt n`e~d~t~ `b a sjntaefzias ojtjntd`e sjfd`e

Sd) tlj j|jk}ias do jd) aeo tlj j|jk}ias—~ rd~zai a~~`ndatd`e ~n`sj ~d%

O ; x,Sd) jd) ~d${H d;1 ,>%:$

Sd ∌ ? ,>%2$

jd ∌ x1) % % % ) E  { ,>%6$

~d ; v_ jd|Sd

( cjd ( 1%= ,>%<$

Cjnaz~j vj tlsj~l`iojo ojtjntd`e~ at ∓1%=) aoodef 1%= t` janl J|jk}ias+^RK—~

sav `zt}zt ~n`sj fzasaetjj~ tlat ~d ≪ =% Vj e`v ojfiej tlj j|jk}ias n`etj|t bjatzsj

b ,Sd$ ∌ E () a~~`ndatjo vdtl ojtjntd e sjfd`e Sd) a~ b`ii`v~7

b ,Sd$ ; Xb 1 b 3 % % % b  E [ ,>%4$

b  g ; ka|,jh; g$∤,Sh∌B d$

~h ,>%1=$

B d ; xS 7 @^ ,S)Sd$ 5 %:{ ,>%11$

^dk}i }zt) b`s a ojtjntd`e sjfd`e Sd) vj `ei n`e~dojs `tljs j|jk}ias ojtjntd`e~

de tlj ~jt B d ,tlj ~jt `b sjfd`e~ vldnl `rjsia} vdtl Sd c k`sj tlae =%:$% _lj g+tl

n`k}`ejet `b tlj j|jk}ias n`etj|t bjatzsj d~ tlje tlj ka|dkzk ~n`sj `b a ojtjntd`e

 vdtl j|jk}ias do j g % _lj fieai ~n`sj b`s sjfd`e Sd d~ tlje7

 ~d ; k_ jdb ,Sd$ ,>%13$

K  ; Xkj1 kj3 % % % kjE [ ,>%1>$

2=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 61/130

 vljsj K  d~ tlj E  Ö E  j|jk}ias n`+`nnzssjenj katsd| vldnl jen`oj~ l`v `btje

j|jk}ias~ jd aeo j g ~dkzitaej`z~i nsjatjo a n`ssjnt ojtjntd`e `e tlj tsaderai ~jt% Vj

j~~jetdaii i``} `rjs aii ojtjntd`e~ de tlj tsaderai ~jt) aeo n`zet l`v `btje j|jk}ias~

jd aeo j g }s`oznjo a n`ssjnt ojtjntd`e% @enj vj `ctade tlj fieai a~~`ndatd`e ~n`sj ~d

z~def tlj n`+`nnzssjenj katsd|) vj z~j ~taeoaso e`e+ka|dkzk ~z}}sj~~d`e t` nsjatj a

fieai) ~}as~j ~jt `b ojtjntd`e~ }js dkafj%

>%? Jraizatd`e `b J|jk}ias+^RK~

Vj jraizatj tlj J|jk}ias+^RK bsakjv`sh tv` odbbjsjet va~% Bds~t) vj n`k}asj tlj

ijasejo }js+j|jk}ias tjk}iatj~ afade~t tjk}iatj~ n`k}ztjo odsjnti bs`k tlj }`~dtdrj

de~taenj) vdtl`zt tlj z~j `b ae ejfatdrj~% ^jn`eo) vj n`k}asj tlj bzentd`eai b`sk

`b J|jk}ias+^RK~) afade~t a od~taenj bzentd`e njetjsjo as`zeo tlj j|jk}ias ,~jj

^jntd`e 3$% Vj z~j tlj ~akj j|ant }js+j|jk}ias bjatzsj sj}sj~jetatd`e de aii tlsjj na~j~)

aeo }jsb`sk tsadedef vdtl tlj ~akj }asakjtjs~ b`s tlj tv` ijasedef+ca~jo a}}s`anlj~%

E`skaidwjo L@F7 _` ~tzo tlj jbbjnt `b ijasedef) vj ejjo a va `b nsjatdef rd~zai

~dkdiasdtdj~ vdtl`zt ae ijasedef% Vj b`zeo tlat a ~dk}ij o`t }s`oznt vdtl a e`skaidwjo

tjk}iatj v`sh~ vjii a~ a ejasj~t+ejdflc`s ca~jidej% _lj ijasedef+bsjj vj d~ `ctadejo c 

nsjatdef tlj =+kjae) e`skaidwjo L@F oj~nsd}t`s odsjnti bs`k tlj j|jk}ias—~ bjatzsj~

,|j ∌ B ($ a~ b`ii`v~7

 vj ; |j ∓ º|1B  ,>%1?$

º| ;|_ j 1B 

B ,>%1:$

De tld~ jqzatd`e 1B  d~ tlj B  odkje~d`eai rjnt`s `b `ej~) aeo de `sojs t` cj =+kjae

21

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 62/130

tlj fieai tjk}iatj la~ c`tl }`~dtdrj aeo ejfatdrj }ast~% _lj doja cjldeo z~def tlj

e`skaidwjo L@F tjk}iatj n`kj~ bs`k `zs `c~jsratd`e tlat tlj tjk}iatj~ `ctadejo bs`k

tlj J|jk}ias+^RK aif`sdtlk asj kjae = aeo tlj }`~dtdrj }ast i``h~ rjs kznl idhj

tlj `sdfdeai sav L@F bjatzsj~% @ej detjs}sjtatd`e `b tlj e`skaidwjo L@F tjk}iatj

d~ tlj ~`iztd`e `b a ’rdstzai‛ ^RK ijasedef }s`nj~~ vljsj ejfatdrj ~z}}`st rjnt`s~ asj

zedb`ski od~tsdcztjo fsaodjet tjk}iatj~% De `tljs v`so~) tlj e`skaidwjo L@F tjk}iatj

dk}idndti a~~zkj~ tlat aii fsaodjet~ de tlj ejfatdrj v`sio asj jqzaii idhji) vldij tlj

J|jk}ias+^RK a}}s`anl o`j~ e`t kahj ae a~~zk}td`e~ ac`zt tlj ejfatdrj v`sio ,aeo

j|}idndti kdej~ tlj ejfatdrj~ bs`k a iasfj ~jt$%

L@F Od~taenj Bzentd`e~7 _` n`k}asj tlj tjk}iatj o`t+}s`oznt sj}sj~jetatd`e `b 

~dkdiasdt bzentd`e~ afade~t `zs jasidjs ’njetjsjo+at+j|jk}ias‛ }ldi`~`}l) vj }jsb`skjo

j|}jsdkjet~ `e ijasedef z~def tlj odaf`eai Kalaiae`cd~ bzentd`eai b`sk `b Jqzatd`e 3%1%

_` kahj ~zsj tlat tlj `ei odbbjsjenj de ijasedef d~ tlj nlaefj `b tlj bzentd`eai b`sk)

 vj }jsb`skjo ijasedef z~def tlj ~akj ~defij+}`~dtdrj+de~taenj `cgjntdrj bzentd`e a~ tlj

J|jk}ias+^RK) a~ vjii a~ tlj ~akj ~tsatjf b`s kdedef ejfatdrj vdeo`v~% _lj `ei 

odbbjsjenj d~ tlat de~tjao `b a}}idef tlj ijasejo vjdflt rjnt`s de sav bjatzsj ~}anj

v_ j |d) vj a}}i dt de tlj od~taenj+t`+j|jk}ias ~}anj) v_ 

j ojd%

_lj bzii ]A^NAI R@N ojtjntd`e nlaiijefj sj~zit~ vdii cj od~nz~~jo iatjs) czt vj

~zkkasdwj tlj sj~zit~ ljsj b`s n`k}ijtjej~~% Z~def tlj ~akj naidcsatd`e ~tj} ,vldnl

 vdii cj oj~nsdcjo de ^jntd`e >%3$ b`s aii > kjtl`o~) vj ~jj tlat E`skaidwjo L@F `ctade~

a kA] `b  %11=) I`nai Od~taenj Bzentd`e~ `ctade a kA] `b  %1:6) aeo J|jk}ias+^RK~

`ctade a kA] `b  %14<% _lj bzii ~jt `b sj~zit~ nae cj ~jje de _acij >%1%

>%?%1 ]A^NAI R@N @cgjnt Ojtjntd e _a~h 

Vj jraizatj `zs J|jk}ias+^RK bsakjv`sh `e tlj vjii+j~tacid~ljo ]A^NAI R@N cjenl+

kash ta~h `b `cgjnt ojtjntd`e% B`s `zs j|}jsdkjet~) vj z~j a ~defij ~`zsnj `b j|jk}ias~7

23

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 63/130

Bdfzsj >%67 @cgjnt Ojtjntd`e aeo A}}jasaenj _sae~bjs% Janl j|ak}ij ~l`v~ a ojtjntd`e bs`k `zsje~jkcij `b J|jk}ias+^RK~ ai`ef vdtl tlj a}}jasaenj tsae~bjssjo odsjnti bs`k tlj ~`zsnj j|jk}ias) t`ojk`e~tsatj tlj ldfl qzaidt `b rd~zai aidfekjet% C`tt`k s`v ~l`v~ `cgjnt natjf`s ojtjntd`e badizsj~%

tlj ]A^NAI R@N 3==6 oata~jt XJrjsdeflak jt ai%) 3=1=[ „ a }`}zias oata~jt z~jo t`

cjenlkash `cgjnt ojtjntd`e aif`sdtlk~% Ozsdef tsadedef) vj ijase a ~j}asatj nia~~dfijs v

b`s janl `b tlj 13) 2=< j|jk}ias~ bs`k tlj 3= natjf`sdj~ de :) =11 tsaderai dkafj~% Vj

kdej laso ejfatdrj~ bs`k `zt+`b+nia~~ dkafj~ de tlj tsade ~jt aeo }jsb`sk naidcsatd`e

z~def aii }`~dtdrj aeo ejfatdrj dkafj~ de tsaderai ,^jj ^jntd`e >%3$%

 At tj~t tdkj) janl J|jk}ias+^RK nsjatj~ ojtjntd`e vdeo`v~ de a ~idodef+vdeo`v

ba~ld`e) czt de~tjao `b z~def a ~taeoaso e`e+ka|dka+~z}}sj~~d`e vj z~j ae j|jk}ias

n`+`nnzsjenj ca~jo kjnlaed~k b`s ~z}}sj~~def sjozeoaet sj~}`e~j~% B`s janl ojtjntd`e

 vj fjejsatj a n`etj|t bjatzsj ~dkdias t` XC`zsojr jt ai%) 3=1=)Bjiwje~wvaic jt ai%) 3=1=[

 vldnl }``i~ de tlj ^RK ~n`sj~ `b ejasc ,`rjsia}}def$ ojtjntd`e~ aeo fjejsatj~ tlj

fieai ojtjntd`e ~n`sj c a vjdfltjo ~zk `b tlj i`nai ^RK ~n`sj aeo tlj n`etj|t ~n`sj%

@enj vj `ctade tlj fieai ojtjntd`e ~n`sj) vj z~j ~taeoaso e`e+ka|dkzk ~z}}sj~~d`e t`

nsjatj a fieai) ~}as~j ~jt `b ojtjntd`e~ }js dkafj%

Vj sj}`st sj~zit~ `e tlj 3=+natjf`s ]A^NAI R@N 3==6 n`k}> `cgjnt ojtjntd`e

2>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 64/130

J|jk}ias Kjtl`o~ Fi`cai

_}j EE EE(Nai OBZE OBZE(Nai J^RK J^RK(Nai J^RK(N` NW O_ IO]K

ajs`}iaej =%==2 =%=:2 =%=?? =%123 =%=23 =%3=? =%3=< =%323 =%136 =%3<6cdnnij =%=4? =%34> =%344 =%>2? =%>>> =%?=6 =%?<= =%?=4 =%3:> =%:1=cdso =%=== =%=13 =%==2 =%==< =%=43 =%=4> =%=66 | =%==: =%==2c`at =%==: =%=>? =%=4> =%=42 =%=44 =%1== =%1?> | =%=1: =%1?:c`ttij =%=== =%==4 =%==: =%=46 =%=3: =%1=> =%1>1 | =%1=6 =%32:cz~ =%==2 =%3=6 =%1:> =%>12 =%3>2 =%>1= =%>46 =%>4> =%3=: =%>46nas =%=1= =%321 =%3=3 =%>22 =%>34 =%?=1 =%?11 =%?>3 =%3>= =%:=3nat =%=43 =%=16 =%=43 =%=43 =%=4: =%=42 =%=:3 | =%==: =%12>nlads =%==1 =%=4? =%=4: =%=4< =%=4: =%1=? =%112 | =%=31 =%12:n`v =%=43 =%111 =%=46 =%1=6 =%11< =%1?6 =%1<2 | =%13< =%122odedeftacij =%==1 =%==? =%==> =%==3 =%=12 =%=3> =%111 | =%=1? =%3?:o`f =%==? =%=>> =%=43 =%=4> =%=4? =%=46 =%=>1 | =%==? =%=:=l`s~j =%=42 =%3?> =%3:> =%3>? =%>?= =%><? =%??6 | =%133 =%?:3k`t`scdhj =%=4? =%1<< =%142 =%33> =%3<6 =%>3= =%>4? =%>6: =%1=> =%><>}js~`e =%==: =%11? =%=46 =%13= =%13? =%143 =%124 | =%1=1 =%>23}`ttjo}iaet =%=1< =%=3= =%=13 =%=>6 =%=3= =%=42 =%113 | =%=33 =%=4=~ljj} =%==4 =%134 =%=?= =%116 =%116 =%126 =%332 | =%=:2 =%16?~`ba =%==< =%==> =%=?6 =%=12 =%=4< =%11= =%16= | =%=:= =%33<tsade =%=42 =%1<> =%1<= =%361 =%3=: =%341 =%>24 =%>>? =%13= =%>?1trk`edt`s =%1?? =%14: =%334 =%34> =%331 =%>1: =%>== | =%3?< =%><?

kA] =%=>4 =%11= =%113 =%1:: =%1:= =%14< =%336 | =%=46 =%322

_acij >%17 J|jk}ias+^RK @cgjnt Ojtjntd`e Sj~zit~ `e ]A^NAI R@N 3==6% _lj > odbbjsjet i`nai~dkdiasdt bzentd`e~ vj n`k}asj afade~t asj7 EE ,E`skaidwjo L`f Ejasj~t Ejdflc`s$) OBZE ,ijasedef aodaf`eai Kalaiae`cd~ od~taenj bzentd`e$) aeo J^RK ,J|jk}ias+^RK$% B`s janl kjtl`o vj ai~` }jsb`sk

naidcsatd`e) a~ deodnatjo vdtl a (Nai) aeo a}}i tlj n`etj|tzai n`+`nnzssjenj katsd| ,(N`$ t` tlj cj~t}jsb`skdef J^RK kjtl`o% _lj tv` fi`cai kjtl`o~ asj `zs dk}ijkjetatd`e `b Oaiai+_sdff~ ,ijasedef a~defij fi`cai tjk}iatj$) aeo IO]K ,Iatjet ojb`skacij }ast k`oji$% @zs kjtl`o cjat~ `zt tlj Oaiai+_sdff~ca~jidej ans`~~ aii natjf`sdj~% @zs a}}s`anl `ctade~ a kA] `b  33%6 sdraidef tlj kA] `b 32%6 `ctadejo bs`kBjiwje~wvaic jt ai XBjiwje~wvaic jt ai%) 3=1=[ od~nsdkdeatdrji tsadejo }ast+ca~jo kd|tzsj k`oji% NW~taeo~ b`s XNlzk aeo Wd~~jskae) 3==6[ aeo IO]K ~taeo~ b`s tlj Iatjet Ojb`skacij ]ast+ca~jo K`oji`b XBjiwje~wvaic jt ai%) 3=1=[% Ozj t` n`k}ztatd eai sja~`e~) vj n`k}ztjo tlj EE ca~jidej~ `e tlj}js~`e natjf`s vdtl 13:= j|jk}ias~%

2?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 65/130

nlaiijefj% Bdfzsj >%6 ~l`v~ ~jrjsai ojtjntd`e~ ,fsjje c`|j~$ }s`oznjo c `zs J|jk}ias+

^RK bsakjv`sh% Vj ai~` ~l`v tlj ~z}js+dk}`~jo j|jk}ias ,jii`v c`|j~$ a~~`ndatjo

 vdtl janl ojtjntd`e% B`ii`vdef tlj }s`t`n`i `b tlj R@N Nlaiijefj) vj jraizatj `zs

~~tjk `e a }js+natjf`s ca~d~ `e tlj tj~t ~jt) n`e~d~tdef `b ?) 4:3 dkafj~% Vj n`k}asj

tlj }jsb`skaenj `b `zs a}}s`anl ,J^RK(N`+`n$ t` ~jrjsai j|jk}ias ca~jidej~ a}ast

bs`k tlj R@N sj~zit~ sj}`stjo de XBjiwje~wvaic jt ai%) 3=1=) Nlzk aeo Wd~~jskae)

3==6[% _lj~j sj~zit~ larj cjje ~zkkasdwjo de _acij >%1 a~ Arjsafj ]sjnd~d`e }js nia~~%

@zs sj~zit~ ~l`v tlat ~taeoaso Ejasj~t Ejdflc`s 1 ,EE$ o`j~ e`t v`sh at aii% Vldij

tlj }jsb`skaenj dk}s`rj~ abtjs naidcsatd`e ,EE(Nai$) dt d~ ~tdii e`t n`k}asacij t` `tljs

a}}s`anlj~ ozj t` dt~ ianh `b k`ojidef ejfatdrj oata% Vj ai~` n`k}asjo afade~t a

od~taenj bzentd`e b`skziatd`e ~dkdias t` tlj `ej }s`}`~jo de XKaid~djvdnw aeo Jbs`~)

3==<[ czt ijasejo z~def a ~defij }`~dtdrj de~taenj% _lj sj~zit~ nijasi deodnatj tlat

tlj j|tsa n`e~tsadet ozj t` a od~taenj bzentd`e }asakjtjsdwatd`e d~ v`s~j tlae z~def a

l}js}iaej% _` ldflidflt tlj dk}`staenj `b z~def tlj n`+`nnzsjenj kjnlaed~k ac`rj)

 vj ai~` sj}`st `zs sj~zit~ z~def naidcsatd`e ,J^RK(Nai$%

@e tlj ]A^NAI tj~t ~jt) `zs bzii ~~tjk `ctade~ a kjae Arjsafj ]sjnd~d`e ,kA]$

`b  %336) vldnl d~ n`k}jtdtdrj vdtl vdtl Bjiwje~wvaic—~ ~tatj+`b+tlj+ast ojb`skacij }ast+

ca~jo kd|tzsj k`oji% E`tj l`vjrjs) tlat `zs ~~tjk o`j~ e`t z~j }ast~ ,tl`zfl tlj 

n`zio cj ja~di aoojo$ ~` tlj n`k}asd~`e d~ e`t jetdsji bads% _ljsjb`sj) vj ai~` n`k}asj

`zs }jsb`skaenj t` Oaiai/_sdff~ ca~jidej) vldnl z~j~ a ~defij natjf`s+vd~j idejas

^RK vdtl e` }ast~) aeo attade~ a kA] `b %=46) vldnl d~ ij~~ tlae laib `b `zs~% Vj ai~`

n`k}asjo afade~t tlj ]A^NAI R@N 3==6 vdeedef jets) tlj j|jk}ias+ca~jo kjtl`o

`b Nlzk jt ai% XNlzk aeo Wd~~jskae) 3==6[) aeo b`zeo tlat `zs ~~tjk cjat~ dt `e ?

`zt `b 2 natjf`sdj~ b`s vldnl tlj ~zckdttjo sj~zit~% De XNlzk aeo Wd~~jskae) 3==6[)

1Vj j|}jsdkjetjo vdtl kzitd}ij ~dkdiasdt kjtsdn~ aeo b`zeo tlat a o`t }s`oznt vdtl a e`skaidwjo

L@F tjk}iatj v`shjo tlj cj~t% _lj e`skaidwjo L@F tjk}iatj d~ nsjatjo c ~zctsantdef a n`e~taet bs`ktlj }`~dtdrj L@F bjatzsj~ t` kahj tljk =+kjae%

2:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 66/130

tlj vdeedef jets b`s kae `b tlj natjf`sdj~ de ]A^NAI 3==6) `cgjnt natjf`sdj~ asj

sj}sj~jetjo c ~t`sdef j|jk}ias~) czt tlj zeojsidef a~~zk}td`e d~ tlat aii j|jk}ias~

bs`k tlj ~akj nia~~ ~lasj a n`kk`e od~tsdcztd e `b bjatzsj~% _lz~) Nlzk jt ai%

sjqzdsj nia~~+vd~j aeo a~}jnt+vd~j iacjidef `b tsadedef oata t` csjah z} cs`ao ca~dn+ijrji

`cgjnt natjf`sdj~ ~znl a~ nas~ det` fiejs rd~zai ~zc+natjf`sdj~ ,j%f%) nas+ijbt) nas+bs`et$%

Zeb`stzeatji tld~ t}j `b iacjidef d~ tjod`z~ aeo odbfinzit t` `ctade b`s oata~jt~ `b 

~dfedfinaet ~dwj% K`sj`rjs) b`s ~`kj `cgjnt~ ,j%f%) c`ttij~) ~`nnjs caii~) fsa~~$ dt d~ e`t

nijas db a~}jnt+vd~j iacjidef d~ jrje vjii+ojfiejo%

>%: J|jk}ias+^RK Aeai~d~

De tld~ ~jntd`e) vj aeaiwj de ojtadi ~`kj }s`}jstdj~ `b `zs J|jk}ias+^RK `cgjnt

ojtjntd e ~~tjk% De }astdnzias) vj ~tzo `cgjnt ojtjntd`e }jsb`skaenj vlje z~def

odbbjsjet ejfatdrj ~jt~ ,c rasdef tljds n`etjet a~ vjii a~ tljds ~dwj$ a~ vjii a~ odbbjsjet

}`~dtdrj ~jt~ ,c rasdef tlj ezkcjs `b j|jk}ias~ t` cj z~jo at tj~t+tdkj$%

>%:%1 Denizodef ^akj+Natjf`s De~taenj~ de tlj Ejfatdrj ^jt

De tlj ia~t ~jntd`e) vj ~l`vjo tlat tlj J|jk}ias+^RK tsadedef }s`njozsj v`sh~ vdtl a

~defij }`~dtdrj de~taenj aeo kdiid`e~ `b ejfatdrj~% Vj ai~` od~nz~~jo tlat tld~ dk}idj~ tlat

tlj j|jk}ias—~ ojnd~d`e c`zeoas d~ ojfiejo) de iasfj }ast) c vlat dt d~ e`t% Cjnaz~j tlj

ejfatdrj ~jt va~ nsjatjo c jidkdeatdef dkafj~ n`etadedef ~akj+natjf`s ,`s) de+nia~~$

de~taenj~) dt oj}jeo~ `e tlj }`~dtdrj—~ natjf`s% _` ojtjskdej l`v tlj }jsb`skaenj `b 

`zs ~~tjk oj}jeo~ `e jidkdeatdef ~akj+natjf`s de~taenj~ bs`k tlj ejfatdrj ~jt) vj

sj}jat tlj ]A^NAI R@N 3==6 j|}jsdkjet~ bs`k ~jntd`e >%?%1) czt tld~ tdkj vj aii`v

tlj ejfatdrj~ t` n`kj bs`k ae  dkafj) denizodef de+nia~~ dkafj~% B`s janl j|jk}ias de

tsaderai) vj z~j aii `b tlj dkafj~ bs`k tlj tsade ~jt ,j|nizodef tlj j|jk}ias—~ `ve

22

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 67/130

_}j J^RK' J^RK'(Nai J^RK'(N` J^RK(N`

ajs`}iaej =%=?> =%131 =%11? =%3=<cdnnij =%312 =%>1= =%>43 =%?<=

cdso =%=43 =%=>3 =%=4: =%=66c`at =%=4? =%=4< =%1?> =%1?>

c`ttij =%=1= =%=>2 =%13? =%1>1cz~ =%1:: =%3=3 =%>3> =%>46nas =%1<: =%32? =%>?> =%?11

nat =%=4> =%=4> =%=>: =%=:3nlads =%=?4 =%=66 =%11? =%112

n`v =%116 =%13< =%14> =%1<2odedeftacij =%=11 =%=13 =%=42 =%111

o`f =%=4? =%=4: =%=:> =%=>1l`s~j =%32: =%3:< =%><1 =%??6

k`t`scdhj =%33? =%3>: =%>2= =%>4?

}js~`e =%1=3 =%13< =%123 =%124}`ttjo}iaet =%=12 =%=4> =%=2: =%113~ljj} =%=2: =%1:4 =%31= =%332~`ba =%=4? =%=3> =%131 =%16=

tsade =%1:= =%313 =%>=3 =%>24trk`edt`s =%1<: =%326 =%3<1 =%>==

kA] =%11> =%1?3 =%14: =%336

_acij >%37 J|jk}ias+^RK sj~zit~ vlje kdedef ejfatdrj~ bs`k c`tl `zt+nia~~ aeo de+nia~~ dkafj~ %

Vj azfkjet tlj J|jk}ias+^RK a}}s`anl c aii`vdef tlj ejfatdrj~ t` n`kj bs`k ae dkafj) denizodefde+nia~~ dkafj~ ,deodnatjo c J^RK'$% B`s n`k}asd~`e) tlj sdfltk`~t n`izke ,J^RK(N`$ d~ tlj `sdfdeaikjtl`o ,tahje bs`k _acij >%1$ vljsj de+nia~~ dkafj~ asj jidkdeatjo bs`k tlj ejfatdrj ~jt%

dkafj$ t` ojfiej tlj ejfatdrj ~jt% Abtjs ijasedef tlj v—~) vj }jsb`sk naidcsatd`e ,~jj

^jntd`e >%3$ aeo n`+`nnzssjenj katsd| j~tdkatd`e ,~jj ^jntd`e >%>$ tlj ~akj va a~

cjb`sj ,d%j%) ztdidwdef tlj j|jk}ias—~ natjf`s$%

 A~ nae cj ~jje bs`k tlj sj~zit~ de _acij >%3) tlj fieai kA] d~ =%14:) vldnl d~ `ei 

~idflti cji`v tlj =%336 kA] bs`k tlj `sdfdeai kjtl`o% _ld~ ~zffj~t~ tlat `enj tlj

ejfatdrj ~jt d~ iasfj je`zfl) tlj bjv }`tjetdaii n`ebz~def ’}`~dtdrj~‛ de tlj ejfatdrj

~jt asj e`t batai t` `zs ~~tjk% _lj }jsnjetafj os`} ~jjk~ t` cj badsi n`e~d~tjet b`s

tlj vjii+}jsb`skdef natjf`sdj~% E`tdnj tlat b`s ? natjf`sdj~ ,cdso) c`at) n`v) o`f$) tlj

J^RK'(N` sj~zit~ vjsj jqzai t` `s cjttjs tlae tlj `sdfdeai J^RK(N` kjtl`o% _lj

sj~zit~ b`s n`v) a sja~`eaci+}jsb`skdef natjf`s) jrje dk}s`rjo bs`k =%1<2 t` =%14?%

26

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 68/130

>%:%3 ^dwj `b Ejfatdrj ^jt

Cjnaz~j J|jk}ias+^RK~ kz~t cj tsadejo vdtl i`t~ `b ejfatdrj~) janl j|jk}ias—~ oata+

kdedef ~tj} sjqzdsj~ a}}s`|dkatji >= kdeztj~ `b v`sh `e a ~defij N]Z% Vldij dt d~ ja~ 

t` }asaiijidwj tld~ }s`njozsj z~def a niz~tjs `b n`k}ztjs) dt d~ cjejfindai t` bzstljs sjoznj

tlj tdkj `b kdedef% Sjozndef tlj ezkcjs `b ejfatdrj~ d~ `ej ja~ va t` ~}jjo+z} tlj

tsadedef }s`njozsj8 tljsjb`sj) de tld~ ~jntd`e vj a~h tlj eatzsai qzj~td`e7 l`v o`j~

`cgjnt ojtjntd`e }jsb`skaenj oj}jeo `e tlj ~dwj `b tlj ejfatdrj ~jt0 Vj }jsb`sk tld~

jraizatd`e `e 2 R@N natjf`sdj~7 cz~) n`v) odedeftacij) k`t`scdhj) ~ljj}) aeo tsade vldij

n`e~dojsdef odbbjsjet ejfatdrj ~jt ~dwj~7 3:) 3:=) 1===) aeo tlj `sdfdeai 3:==% Bdfzsj >%<~l`v~ tlj ~zkkasdwjo sj~zit~ b`s tlj tlsjj kjtl`o~7 J^RK) J^RK(Nai) aeo J^RK(N`%

Ojtadijo ezkcjs~ nae cj ~jje de _acij >%>) _acij >%?) _acij >%:) aeo _acij >%2% Vj e`tj

tlat densja~def tlj ejfatdrj ~jt ~dwj bs`k 3: t` 3:= ejfatdrj~ dkafj~ fdrj~ z~ a kznl

ldfljs c``~t tlae densja~def dt bs`k 1=== t` tlj bzii 3:== dkafj~% _ld~ ~zffj~t~ tlat

`enj janl j|jk}ias la~ ~jje a ~zbfindjet ezkcjs `b ejfatdrj~) dt~ }jsb`skaenj vdii e`t

~dfedfinaeti dk}s`rj vdtl k`sj ejfatdrj~%

2<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 69/130

= :== 1=== 1:== 3=== 3:===%=:

=%1

=%1:

=%3

=%3:

=%>

.Ejfatdrj~

      A      ]

]A^NAI R@N kA]' rjs~z~ Ejfatdrj ^jt ^dwj

 

J^RK

J^RK(Nai

J^RK(N`

Bdfzsj >%<7 Rasdef ejfatdrj ~jt ~dwj~% Vj n`k}ztj tlj }jsb`skaenj `b tlj J|jk}ias+^RK aif`sdtlk a~a kA] `rjs 2 ]A^NAI natjf`sdj~7 cz~) n`v) odedeftacij) k`t`scdhj) ~ljj}) aeo tsade% _lj |+a|d~ deodnatj~

tlj ezkcjs `b ejfatdrj dkafj~ z~jo aeo tlj +a|d~ d~ tlj ]A^NAI R@N 3==6 sj~zitdef kA] ~n`sj b`s janl`b tlj > kjtl`o~%

24

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 70/130

_}j J^RK J^RK(C J^RK(K

cz~ =%1=> =%324 =%3?<n`v =%1== =%133 =%1?4odedeftacij =%==1 =%=?< =%=42

k`t`scdhj =%12? =%3>3 =%3:>~ljj} =%=4< =%1:? =%12?tsade =%1=1 =%142 =%3:2

kA] =%=4? =%16= =%14?

_acij >%>7 J|jk}ias+^RK sj~zit~ vdtl 3: ejfatdrj~%

_}j J^RK J^RK(C J^RK(K

cz~ =%1<: =%342 =%>?<

n`v =%131 =%1:2 =%12?odedeftacij =%==: =%=1< =%113

k`t`scdhj =%3?< =%3<? =%>><~ljj} =%=:> =%12> =%14<tsade =%13? =%3:< =%>1:

kA] =%13> =%142 =%3?2

_acij >%?7 J|jk}ias+^RK sj~zit~ vdtl 3:= ejfatdrj~%

_}j J^RK J^RK(C J^RK(K

cz~ =%312 =%>=? =%><?

n`v =%1>= =%1?< =%16<odedeftacij =%=13 =%=14 =%11<k`t`scdhj =%324 =%>=3 =%>66~ljj} =%=6? =%163 =%3>4tsade =%1<? =%>=: =%>21

kA] =%1?6 =%3=< =%362

_acij >%:7 J|jk}ias+^RK sj~zit~ vdtl 1=== ejfatdrj~

_}j J^RK J^RK(C J^RK(K

cz~ =%3>2 =%>1= =%>46

n`v =%11< =%1?6 =%1<2

odedeftacij =%=12 =%=3> =%111k`t`scdhj =%3<6 =%>3= =%>4?~ljj} =%116 =%126 =%332tsade =%3=: =%341 =%>24

kA] =%12> =%31= =%3<1

_acij >%27 J|jk}ias+^RK sj~zit~ vdtl 3:== ejfatdrj~% _ld~ d~ tlj ~akj a~ tlj kade sj~zit de tlj kade_acij >%1

6=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 71/130

>%:%> Oj}jeojenj @e tlj Ezkcjs `b J|jk}ias~

^denj tlj n`k}ij|dt `b `zs Je~jkcij `b J|jk}ias+^RK~ ~naij~ idejasi vdtl tlj ezkcjs

`b j|jk}ias~) dt d~ raizacij t` ~tzo tlj `cgjnt ojtjntd`e }jsb`skaenj `b je~jkcij~ `b 

 rasdef ~dwj~% Vj a~h7 dt }`~~dcij t` `ctade sjaii f``o `cgjnt ojtjntd`e sj~zit~ vdtl

a ~kaii ezkcjs `b j|jk}ias~0 B`s janl `cgjnt natjf`s de R@N) vj tsade aii `b tlj

j|jk}ias~ z~def `zs bzii a}}s`anl) aeo tlje ~`st tljk z~def tlj `zt}zt `b naidcsatd`e

,~jj ^jntd`e sjb~jn7naidcsatd`e$% Naidcsatd`e ~n`sj~ janl j|jk}ias deoj}jeojeti aeo

tlz~ tlj j|jk}ias~ vldnl }s`oznj tlj iasfj~t ezkcjs `b f``o katnlj~ ,`e ljio+`zt

 raidoatd`e oata$ vdii n`kj fis~t de tlj ~`stdef% Fdrje tld~ `sojsdef `b j|jk}ias~) vjn`e~dojs je~jkcij~ `b densja~def ~dwj) ~tastdef vdtl tlj ~defij cj~t j|jk}ias) aeo

densjkjetaii aoodef tlj ej|t+cj~t `ej) zetdi vj larj aii j|jk}ias~ de tlj je~jkcij%

Vj sj}`st tlj ]A^NAI R@N }jsb`skaenj b`s rasdef ezkcjs~ `b j|jk}ias~ z~def tlj

}js+j|jk}ias naidcsatd`e ~tsatjf) aeo e`t tlj bzii n`+`nnzssjenj katsd|% _lj ]A^NAI

 R@N A] rjs~z~ ezkcjs `b j|jk}ias~ }i`t~ nae cj ~jje de Bdfzsj >%4% Vj ~l`v tlj sj~zit~

de tv` odbbjsjet va~7 A] r~% i`f ezkcjs `b j|jk}ias~) aeo A] r~% }jsnjet `b j|jk}ias~

de tlj natjf`s% Vj e`tj tlat tld~ nzsrj d~ b`s tlj k`~t }ast k`e`t`ednaii densja~def% A~

nae cj ~jje bs`k tld~ nzsrj) tlj }jsb`skaenj ~atzsatj~ b`s tlj~j natjf`sdj~ vlje ac`zt

laib tlj j|jk}ias~ asj nl`~je% _lj ffiat tadi `b janl nzsrj ~zffj~t~ tlat aoodef tlj ia~t

laib `b j|jk}ias~ ,kae `b vldnl asj cao$ o`j~e—t kahj `zs }jsb`skaenj os`}% _ld~

~zffj~t~ tlat `zs naidcsatd`e ~tj} v`sh~ vjii at ~z}}sj~~def cao j|jk}ias~% ^denj tlj

sze+tdkj n`k}ij|dt d~ odsjnti }s`}`std`eai t` tlj ezkcjs `b j|jk}ias~ de tlj je~jkcij)

tld~ sj~zit ~zffj~t~ tlat vj nae fjt kznl ba~tjs `cgjnt natjf`s ojtjnt`s~ c n`e~tszntdef

ae je~jkcij vdtl e` k`sj tlae laib tlj ezkcjs `b j|jk}ias~ de janl natjf`s%

61

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 72/130

1==

1=1

1=3

1=>

1=?

=

=%=:

=%1

=%1:

=%3

=%3:

=%>

=%>:

=%?

=%?:

Ezkcjs `b j|jk}ias~ ,i`f ~naij$

      A      ]

]A^NAI R@N A] r~% Ezkcjs `b j|jk}ias~ ,i`f ~naij$

 

cdnnij

cz~

nas

l`s~j

k`t`scdhj

~ljj}

tsade

= 1= 3= >= ?= := 2= 6= <= 4= 1===

=%=:

=%1

=%1:

=%3

=%3:

=%>

=%>:

=%?

=%?:

]jsnjetafj `b j|jk}ias~

      A      ]

]A^NAI R@N A] r~% ]jsnjetafj `b J|jk}ias~

 

cdnnij

cz~

nas

l`s~j

k`t`scdhj

~ljj}

tsade

Bdfzsj >%47 ]A^NAI R@N Sj~zit~ r~% Ezkcjs `b J|jk}ias~% Vj sj}`st tlj R@N A] b`s je~jkcij~ `b densja~def ~dwj% B`s janl natjf`s) a ~`stjo id~t d~ nsjatjo bs`k tlj `zt}zt `b naidcsatd`e) aeo je~jkcij~`b ~dwj 1 tls`zfl E  ,E  d~ tlj ezkcjs `b j|jk}ias~ de a natjf`s$ asj jraizatjo `e tlj ojtjntd`e ta~h%_lj t`} }i`t ~l`v~ }jsb`skaenj rjs~z~ tlj sav ezkcjs `b j|jk}ias~% _lj c`tt`k }i`t ~l`v~ tlj ~akjdeb`skatd`e) czt z~def a idejas ~naij) aeo j|}sj~~jo a~ a }jsnjet `b tlj t`tai ezkcjs `b j|jk}ias~ vdtldetlj natjf`s%

63

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 73/130

>%:%? _lj S`ij `b Bjatzsj Odkje~d`eaidt 

 Ae`tljs dk}`staet oj~dfe ojnd~d`e de `zs J|jk}ias+^RK bsakjv`sh d~ tlj odkje~d`eai+

dt `b bjatzsj~ sj}sj~jetdef janl j|jk}ias% De tld~ ~jntd`e) vj ras tlj bjatzsj odkje~d`e

c rasdef janl j|jk}ias—~ L@F tjk}iatj ~dwj% Vj sj}jat tlj ]A^NAI R@N j|}jsdkjet~

 vdtl iasfjs+~dwjo tjk}iatj~ a~ vjii a~ vdtl ~kaiijs+~dwjo tjk}iatj~% @zs ojbazit tjk}iatj

~dwj d~ s`zfli 1== njii~) vdtl tlj iasfj tjk}iatj~ cjdef s`zfli 3== njii~) ~kaii tjk}iatj~

cjdef s`zfli  := njii~) aeo tde `ej~ vdtl 3: njii~% _lj L@F dedtdaidwatd`e/bsakdef

}s`njozsj d~ bzii oj~nsdcjo de A}}jeod| C%3% A ~zkkas `b tlj sj~zit~ arjsafjo ans`~~

2 natjf`sdj~ nae cj ~jje de Bdfzsj >%1=% _lj bzii sj~zit tacij~ asj de _acij >%6) _acij >%<)_acij >%4) aeo _acij >%1=% Vj e`tj tlat iasfjs tjk}iatj~ ~idflti dk}s`rj sj~zit~ b`s tlj

sav J^RK kjtl`o aeo J^RK(Nai8 l`vjrjs) a tjk}iatj ~dwj `b  1== d~ tlj cj~t vlje

z~def tlj n`+`nnzssjenj katsd|% L`vjrjs) tlj aoraetafj `b z~def ~kaii tjk}iatj~ `rjs

iasfjs `ej~ d~ tlat tsadedef ~kaii tjk}iatj~ a~ vjii a~ jraizatdef tljk `e tj~t+dkafj~ d~

de fjejsai kznl ba~tjs%

6>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 74/130

3= ?= 2= <= 1== 13= 1?= 12= 1<= 3===%1

=%1:

=%3

=%3:

=%>

=%>:

L@F _jk}iatj ^dwj

    k     A     ]

kA] rjs~z~ L@F _jk}iatj ^dwj

 

J^RK

J^RK(Nai

J^RK(N`

Bdfzsj >%1=7 ]A^NAI R@N Sj~zit~ b`s Rasdef L@F _jk}iatj ^dwj~% Vj n`k}ztj tlj }jsb`skaenj `b 

tlj J|jk}ias+^RK aif`sdtlk a~ a kA] `rjs 2 ]A^NAI natjf`sdj~7 cz~) n`v) odedeftacij) k`t`scdhj)~ljj}) aeo tsade% _lj |+a|d~ deodnatj~ tlj ~dwj `b tlj L@F tjk}iatj~ z~jo aeo tlj +a|d~ d~ tlj ]A^NAI

 R@N 3==6 sj~zitdef kA] ~n`sj b`s janl `b tlj > kjtl`o~%

6?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 75/130

_}j J^RK J^RK(Nai J^RK(N`

cz~ =%1== =%13: =%1<?n`v =%1>6 =%134 =%31<odedeftacij =%==1 =%==1 =%==2

k`t`scdhj =%1<= =%162 =%3:2~ljj} =%1=> =%13= =%3=?tsade =%=4> =%=42 =%11?

kA] =%1=3 =%1=< =%12?

_acij >%67 J|jk}ias+^RK sj~zit~ vdtl tde L@F tjk}iatj~ ,f`ai~dwj7 3:) ka|odk7 :$%

_}j J^RK J^RK(Nai J^RK(N`

cz~ =%163 =%31: =%>12n`v =%1?4 =%122 =%33>odedeftacij =%==6 =%=4> =%=:>k`t`scdhj =%3:4 =%324 =%>62~ljj} =%11> =%1:6 =%3>3tsade =%111 =%116 =%1>>

kA] =%1>: =%16= =%333

_acij >%<7 J|jk}ias+^RK sj~zit~ vdtl ~kaiijs L@F tjk}iatj~ ,f`ai~dwj7 :=) ka|odk7 <$%

_}j J^RK J^RK(Nai J^RK(N`

cz~ =%3>2 =%>1= =%>46n`v =%11< =%1?6 =%1<2

odedeftacij =%=12 =%=3> =%111k`t`scdhj =%3<6 =%>3= =%>4?~ljj} =%116 =%126 =%332tsade =%3=: =%341 =%>24

kA] =%12> =%31= =%3<1

_acij >%47 J|jk}ias+^RK sj~zit~ vdtl ojbazit+~dwjo L@F tjk}iatj~ ,f`ai~dwj7 1==) ka|odk7 13$%

_}j J^RK J^RK(Nai J^RK(N`

cz~ =%3>3 =%>:3 =%>4?

n`v =%1>3 =%1:: =%12?odedeftacij =%=4> =%=42 =%1=3k`t`scdhj =%>=< =%>1= =%>6>~ljj} =%=61 =%1<= =%316tsade =%3=2 =%>=< =%>>6

kA] =%16? =%3>> =%32?

_acij >%1=7 J|jk}ias+^RK sj~zit~ vdtl iasfj L@F tjk}iatj~ ,f`ai~dwj7 3==) ka|odk7 1:$%

6:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 76/130

]ast DD7 Cj`eo @cgjnt Ojtjntd`e

Dt abb`so~ ae dkkjodatj ~tj}) l`vjrjs) t` a~~`ndatdrj deoj|def) tlj ca~dn doja `b 

 vldnl d~ a }s`rd~d`e vljsjc ae dtjk ka cj naz~jo at vdii t` ~jijnt dkkjodatji 

aeo azt`katdnaii ae`tljs% _ld~ d~ tlj j~~jetdai bjatzsj `b tlj kjkj|% _lj }s`nj~~`b tdef tv` dtjk~ t`fjtljs d~ tlj dk}`staet tldef%

 Raeejras Cz~l) A~ Vj Ka _ldeh ,14?:$

62

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 77/130

Vj larj) z} zetdi e`v) ojr`tjo `zs attjetd`e t` nsjatdef rd~zai a~~`ndatd`e~) `s de

tlj v`so~ `b Raeejras Cz~l) ’tdef tv` dtjk~ t`fjtljs%‛ Vj ojk`e~tsatjo tlat tlj

 rd~zai a~~`ndatd`e~ }s`oznjo c tlj J|jk}ias+^RK aif`sdtlk nae cj z~jo b`s `cgjnt

natjf`s ojtjntd`e) kznl idhj `tljs natjf`s+ca~jo ojtjntd`e ~~tjk~ XOaiai aeo _sdff~)

3==:) Bjiwje~wvaic jt ai%) 3=1=) Nlzk aeo Wd~~jskae) 3==6[% L`vjrjs) tlj njetsai

cjejfit `b `zs a}}s`anl d~ tlat vj asj acij t` j~tacid~l a odsjnt ideh cjtvjje a ojtjntd`e

aeo a ~defij tsadedef j|jk}ias~% De Nla}tjs ?) vj ~l`vna~j tlj qzaidt `b `zs rd~zai

a~~`ndatd`e~ aeo ojk`e~tsatj tlat tlj aidfekjet cjtvjje k`~t ojtjntd`e~ aeo tljds

a~~`ndatjo j|jk}ias d~ f``o je`zfl t` ijt z~ tsae~bjs ae aradiacij j|jk}ias kjta+oata

,j%f%) ~jfkjetatd`e) fj`kjts) >O k`oji$ odsjnti `et` tlj ojtjntd`e% Vj }jsb`sk ae

jraizatd`e `e tlj ]A^NAI R@N 3==6 oata~jt) n`e~dojs tlj tlsjj tsae~bjs+ta~h~ ,d%j%)

~jfkjetatd`e tsae~bjs) fj`kjts tsae~bjs) aeo sjiatjo `cgjnt }sdkdef$) aeo rd~zaidwj tlj

j|jk}ias+j|jk}ias ~dkdiasdt ~tszntzsj a~ a fsa}l% @zs kjta+oata tsae~bjs f`j~ cj`eo

 vlat d~ sjqzdsjo de k`~t `cgjnt ojtjntd`e ta~h~ ,d%j%) a natjf`s+iacjijo c`zeodef c`|$)

aeo tlj sj~zitdef detjs}sjtatd`e~ ,j%f%) `cgjnt }`~j) `cgjnt fj`kjts) `cgjnt attsdcztj~$

nae tlje cj z~jo a~ }ast `b `rjsaii ~njej zeojs~taeodef%

De Nla}tjs : vj n`e~dojs a n`etj|tzai `cgjnt }sjodntd`e ta~h XKaid~djvdnw aeo Jbs`~)

3==4[ vldnl vj tanhij c b`skdef a fsa}l `rjs j|jk}ias~ aeo tljds ~}atdai sjiatd`e~ld}~)

 vldnl vj naii tlj Rd~zai Kjkj| K`oji% Vj a}}i `zs ’natjf`s+bsjj‛ Rd~zai Kjkj|

K`oji t` a rasdaet `b Aet`ed` _`ssaica—~ N`etj|t Nlaiijefj) vljsj tlj f`ai d~ t` }sjodnt

tlj a}}jasaenj `b a ldooje `cgjnt ~`iji ca~jo `e tlj ldooje sjfd`e—~ ~}atdai sjiatd`e~ld}

t` a ~jt `b rd~dcij `cgjnt~ de tlj ~njej% Vj jraizatj `zs Rd~zai Kjkj| K`oji `e tlj

N`etj|t Nlaiijefj ta~h aeo ~l`v ae dk}s`rjkjet `rjs natjf`s+ca~jo ca~jidej~%

66

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 78/130

Nla}tjs ?

 A~~`ndatd`e aeo Kjta+oata _sae~bjs

Vj larj aisjao ~l`ve tlat J|jk}ias+^RK~ asj ldfli+tzejo t` tlj a}}jasaenj `b tlj

~`zsnj j|jk}ias aeo tlat vj asj acij t` `ctade ldfl qzaidt n`ssj~}`eojenj~ cjtvjje

j|jk}ias~ aeo ojtjntd`e vdeo`v~% ^znl ldfl qzaidt aidfekjet kjae~ tlat J|jk}ias+

^RK~ nae cj z~jo b`s k`sj tlae c`zeodef c`| }sjodntd`e ‟ ae kjta+oata a~~`ndatjo

 vdtl j|jk}ias~ nae cj tsae~bjssjo `et` tlj ojtjntd`e vdeo`v% _lj laiikash `b `zs

a}}s`anl d~ tlat vj o` e`t sjqzdsj a n`k}ij| dkafj aidfekjet }s`nj~~ ,zeidhj XCjsf

jt ai%) 3==:) Idz jt ai%) 3==4[$ ‟ tlj j|jk}ias+~}jndfin ojtjnt`s aisjao o`j~ aii tlj

 v`sh! Vj n`e~dojs kjta+oata de tlj b`sk `b a }d|ji+vd~j iacjidef ~znl a~ a ~jfkjetatd`e

`s a fj`kjtsdn iacjidef aeo vj ~dk}i tsae~bjs tlj j|jk}ias+aidfejo kjta+oata `et`

tlj ojtjntd`e z~def tlj tsae~b skatd e j~tdkatjo cjtvjje tlj ~`zsnj j|jk}ias aeo tlj

ojtjntd`e vdeo`v ,a ~naias ~naij aeo a 3O tsae~iatd`e$%

De tld~ nla}tjs vj }sj~jet sj~zit~ `e a}}idef ae je~jkcij `b J|jk}ias+^RK~

,a~ ojfiejo de Nla}tjs >$ t` a ~jt `b kjta+oata tsae~bjs ta~h~7 ~jfkjetatd`e tsae~bjs)

fj`kjts tsae~bjs) >O k`oji tsae~bjs) a~ vjii a~ sjiatjo `cgjnt }sdkdef% B`s tlj tsae~bjs

a}}idnatd`e~ vj z~j tlj J|jk}ias+^RK kjtl`o vdtl naidcsatd`e cjnaz~j jrje tl`zfl

z~def tlj j|jk}ias n`+`nnzsjenj katsd| c``~t~ `cgjnt ojtjntd`e }jsb`skaenj) dt z~j~

6<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 79/130

J|jk}ias

Kjta+oata

A}}jasaenj

^jfkjetatd`e

J|jk}iasOjtjnt`sv

A}}jasaenj

J|jk}ias

A}}jasaenj

Kjta+oata Kjta+oata

^jfkjetatd`e ^jfkjetatd`e

Ojtjnt`sv Ojtjnt`sv

Bdfzsj ?%17 ^jfkjetatd`e _sae~bjs% @cgjnt ~jfkjetatd`e~ asj tsae~bjssjo bs`k tlj j|jk}ias odsjnti `et` tlj ojtjntd`e vdeo`v%

kzitd}ij `rjsia}}def j|jk}ias~ t` ~n`sj vdeo`v~ ,at tlj n`~t `b b`sfjttdef vldnl ~defij

a~~`ndatd`e d~ tlj cj~t$% Naidcsatd`e }s`oznj~ kznl ldfljs qzaidt aidfekjet~ cjnaz~j

ojtjntd`e~ asj ~n`sjo deoj}jeojeti%

?%1 ^jfkjetatd`e aeo Fj`kjts _sae~bjs

B`s tlj ta~h `b ~jfkjetatd`e) tlj f`ai d~ t` j~tdkatj vldnl }d|ji~ cji`ef t` a fdrje

`cgjnt aeo vldnl o` e`t% Bdfzsj ?%1 ~l`v~ ~`kj qzaidtatdrj ~jfkjetatd`e tsae~bjs

j|ak}ij~ `e a vdoj rasdjt `b `cgjnt nia~~j~% B`s qzaetdtatdrj jraizatd e) vj a~hjo

iacjijs~ t` ~jfkjet aeo fj`kjtsdnaii aee`tatj aii `b tlj de~taenj~ de tlj ’cz~‛ natjf`s 

de tlj ]A^NAI R@N 3==6 oata~jt% B`s tlj ~jfkjetatd`e ta~h) `zs kjtl`o }jsb`sk~ at

a }d|jivd~j annzsan `b 4=%2*% B`s fj`kjts j~tdkatd`e) tlj f`ai d~ t` a~~dfe iacji~ t`

}d|ji~ deodnatdef kjkcjs~ld} t` `ej `b > ’ijbt)‛ ’bs`et)‛ aeo ’sdflt‛ o`kdeaet `sdjetatd`e

nia~~j~ XL`djk jt ai%) 3==:[% Vj n`k}asj `zs J|jk}ias+^RK ~~tjk afade~t tv`

ca~jidej~7 ,a$ L`djk—~ }sj+tsadejo fjejsdn fj`kjtsdn nia~~ j~tdkatd`e aif`sdtlk XL`djk

jt ai%) 3==:[8 ,c$ Z~def XBjiwje~wvaic jt ai%) 3=1=[ t` ojtjnt `cgjnt~ b`ii`vjo c ~dk}ij

EE t` nsjatj a~~`ndatd`e~% Vj `ctade a 23%>* }d|jivd~j iacjidef annzsan z~def `zs

J|jk}ias+^RK a}}s`anl a~ n`k}asjo t` tlj ?>%=* `ctadejo z~def XL`djk jt ai%) 3==:[

64

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 80/130

Kjta+oataKjta+oata Kjta+oata

J|jk}ias J|jk}ias J|jk}iasOjtjnt`s v Ojtjnt`s v Ojtjnt`s v

Fj`kjts Fj`kjts Fj`kjts

A}}jasaenj A}}jasaenj A}}jasaenj

Bdfzsj ?%37 Qzaidtatdrj Fj`kjts _sae~bjs% Vj tsae~bjs fj`kjtsdn iacjidef bs`k cz~ j|jk}ias~ `et`n`ssj~}`eodef ojtjntd`e~%

aeo :1%=* z~def XBjiwje~wvaic jt ai%) 3=1=[(EE% _ld~ nijasi ~l`v~ tlat vldij `zs

tsae~bjs d~ ~dk}ij) dt d~ ojfiedtji e`t tsdrdai a~ dt sjidj~ `e `ctadedef ~ts`ef aidfekjet

cjtvjje tlj j|jk}ias aeo tlj ojtjntd`e ,~jj qzaidtatdrj sj~zit~ de Bdfzsj ?%3$% Fi`cai

kjtl`o~ badi t` fjejsatj ~znl aidfekjet~) ijaodef t` kznl i`vjs }jsb`skaenj%

?%3 >O K`oji _sae~bjs

Vj aee`tatjo a ~zc~jt `b nlads j|jk}ias~ vdtl >O k`oji~ bs`k F``fij—~ >O Vasjl`z~j

,aeo aidfejo vdtl F``fij ^hjtnl+Z} >O k`oji+t`+dkafj aidfekjet t``i$% Fdrje a

~defij j|jk}ias) iacjijs~ vjsj a~hjo t` fieo tlj k`~t rd~zaii ~dkdias k`oji de tlj >O

Vasjl`z~j b`s tlat de~taenj aeo }jsb`sk tlj aidfekjet% Ozj t` tlj ldfl qzaidt `b `zs

azt`katdnaii+fjejsatjo a~~`ndatd`e~) vj vjsj acij t` ~dk}i tsae~bjs tlj j|jk}ias+

aidfejo >O k`oji odsjnti `et` tlj ojtjntd`e vdeo`v vdtl`zt ae aoodtd`eai aidfekjet)

~jj Bdfzsj ?%>%

<=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 81/130

Bdfzsj ?%>7 >O K`oji tsae~bjs% De janl `b tlj~j > j|ak}ij~) tlj fsjje c`| de tlj t`} dkafj ~l`v~ tljojtjntd`e vdeo`v) aeo tlj c`tt`k ~l`v~ tlj azt`katdnaii tsae~bjssjo >O k`oji%

<1

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 82/130

Bdfzsj ?%?7 Sjiatjo @cgjnt ]sdkdef% A cdnnij/k`t`scdhj/l`s~j j|jk}ias d~ z~jo t` }sjodnt c`zeodefc`| b`s ’}js~`e‛%

?%> Sjiatjo @cgjnt ]sdkdef

J|jk}ias~ `btje ~l`v ae detjs}ia `b kzitd}ij `cgjnt~) tlz~ ae `tljs `cgjnt~ vldnl

~zbfindjeti `rjsia} vdtl tlj j|jk}ias nae cj rdjvjo a~ aoodtd`eai kjta+oata cji`efdef

t` tlj j|jk}ias% _ld~ ~zffj~t~ z~def ojtjnt`s~ `b `ej natjf`s t` lji} ’}sdkj‛ `cgjnt~

`b ae`tljs natjf`s% Vj i``h at tlj b`ii`vdef ta~h7 }sjodntdef a c`zeodef c`| b`s

’}js~`e‛ fdrje a ojtjntd`e `b natjf`s  \ ) vljsj \  d~ jdtljs a l`s~j) k`t`scdhj) `s

cdnnij ,~jj Bdfzsj ?%? b`s qzaidtatdrj sj~zit~$% Vj ~a tlat a }js~`e d~ ’sdodef‛ ae \  db dt~ `rjsia} vdtl tlj \  c`zeodef c`| ~zbfindjet ,@^ 5 %1$% Vj qzaetdtatdrji jraizatjo

tlj }js~`e }sjodntd`e }jsb`skaenj aeo n`k}asjo afade~t a ca~jidej vldnl }sjodnt~ a

}js~`e }sj~jenj ca~jo `e kag`sdt r`tdef% @zs kjtl`o n`e~dojsaci `zt}jsb`sk~ tlj

ca~jidej ,63%?2* a~ n`k}asjo t` :<%26* b`s tlj ca~jidej$) ~zffj~tdef tlat `zs j|jk}ias

a~~`ndatd`e~ }s`rdoj f``o aidfekjet `b j|jk}ias~ a~ vjii a~ tljds sjiatjo `cgjnt~% _lj

sj~zit~ nae cj ~jje de _acij ?%1%

<3

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 83/130

Natjf`s Kag`sdt R`tdef XKaid~djvdnw jt ai%) 3=11[

cdnnij 2>%?* 63%<*k`t`scdhj :=%=* 26%?*

l`s~j 23%2* 66%3*

_acij ?%17 D~ tljsj a }js~`e sdodef tld~ l`s~j0 Vj }sjodnt bs`k `zs cdnnij) k`t`scdhj) aeo l`s~jojtjnt`s~ vljtljs tljsj d~ a }js~`e sdodef tlj `cgjnt% @zs a}}s`anl d~ cjttjs tlae tlj kag`sdt r`tjca~jidej) ~zffj~tdef tlat j|jk}ias~ asj z~jbzi at }sjodntdef ejasc) sjiatjo `cgjnt~%

?%? J|jk}ias De+}adetdef

Vlje tlj oata~jt o`j~ e`t n`kj jqzd}}jo vdtl ae kjta+oata cj`eo `cgjnt iacji~

aeo c`zeodef c`|j~) dt d~ e`t }`~~dcij t` }jsb`sk ae hdeo `b tsae~bjs% De ~znl a na~j)

 vj nsjatj ae ’detjs}sjtatd`e‛ rda a ~dk}ij `cgjnt de+}adetdef z~def tlj j|jk}ias—~ sav

a}}jasaenj) a~ ~l`ve Bdfzsj ?%: aeo Bdfzsj ?%2% De janl j|ak}ij vj ~l`v tlj ~`zsnj

j|jk}ias ai`ef vdtl dt~ ijasejo tjk}iatj vj `e tlj ijbt) aeo tlj ojtjntd`e vdeo`v aeo

j|jk}ias de+}adetdef `e tlj sdflt% E`tj l`v `zs j|jk}ias de+}adetdef sj~zit~ ~l`vna~j

tlj qzaidt `b `zs azt`katdn a~~`ndatd`e~) jt afade% ^`kj badizsj~ `b `zs de+}adetdef nae

cj ~jje de Bdfzsj ?%6 aeo Bdfzsj ?%<% E`tj tlat qzdtj `btje vlje vj }s`oznj a badizsj

,ojfiejo c natjf`s kd~katnl$) asj kd~tahj~ asj `btje cjtvjje ~jkaetdnaii sjijraet

natjf`sdj~ ,j%f%) a n`v kd~tahje a~ a ~ljj}) `s a l`s~j kd~tahje a~ a o`f$% Jrje vlje

 vj j~tdkatj tlj vs`ef `cgjnt natjf`s) vj nae ~tdii `btje }sjodnt tlj `cgjnt—~ }`~j qzdtj

 vjii%

<>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 84/130

v11<

Ojtjntd`e De+}adetdef

v33

Ojtjntd`e De+}adetdef

v1:= Ojtjntd`e De+}adetdef

v>4: Ojtjntd`e De+}adetdef

Bdfzsj ?%:7 ^znnj~~bzi J|jk}ias _sae~bjs~ 1% Vj ~`kj `b tlj t`} j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI

  R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%

<?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 85/130

v3=?

Ojtjntd`e De+}adetdef

v3>=

Ojtjntd`e De+}adetdef

v<3<

Ojtjntd`e De+}adetdef

v12:

Ojtjntd`e De+}adetdef

Bdfzsj ?%27 ^znnj~~bzi J|jk}ias _sae~bjs~ 3% Vj ~`kj `b tlj t`} j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI

  R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%

<:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 86/130

v3<3

Ojtjntd`e De+}adetdef

v:63

Ojtjntd`e De+}adetdef

v3:<

Ojtjntd`e De+}adetdef

v1=6 Ojtjntd`e De+}adetdef

Bdfzsj ?%67 J|jk}ias _sae~bjs Badizsj~ 1% Vj ~`kj `b tlj t`} badijo j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI

  R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%

<2

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 87/130

v3:= Ojtjntd`e De+}adetdef

v?:>

Ojtjntd`e De+}adetdef

v?21

Ojtjntd`e De+}adetdef

v3=6

Ojtjntd`e De+}adetdef

Bdfzsj ?%<7 J|jk}ias _sae~bjs Badizsj~ 3% Vj ~`kj `b tlj t`} badijo j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI  R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%

<6

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 88/130

?%: Rd~zaidwdef J|jk}ias+J|jk}ias A~~`ndatd`e~

’%%%vj ~jj a n`k}idnatjo ejtv`sh `b ~dkdiasdtdj~ `rjsia}}def aeo nsd~~+ns`~~def‛

Izovdf Vdttfje~tjde

De tld~ ~jntd`e) vj nsjatj a rd~zaidwatd`e `b tlj j|jk}ias+j|jk}ias rd~zai a~~`ndatd`e

~tszntzsj deoznjo c tlj J|jk}ias+^RK aif`sdtlk `e tlj ]A^NAI R@N 3==6 oata~jt%

@zs rd~zaidwatd`e d~ a ~`st `b ’Rd~zai Kjkj|‛ fsa}l) vldnl vj rd~zaidwj z~def tlj }`}zias

`}je ~`zsnj fsa}l rd~zaidwatd`e idcsas Fsa}lrdw XJii~`e jt ai%) 3==1[% Vj ai~` w``k

det` }`std`e~ `b tld~ fsa}l t` aeaiwj tlj ’rd~zai tsadi‛ ~dkdiasdt ~tszntzsj as`zeo a

}astdnzias j|jk}ias%

Vldij tlj n`enj}t `b a ejtv`sh `b `rjsia}}def ~dkdiasdtdj~ n`kj~ bs`k Vdttfje~tjde—~

bakdi sj~jkciaenj~ XVdttfje~tjde) 14:>[) tlj doja `b a rd~zai tsadi d~ odsjnti tahje bs`k

Cz~l—~ ’A~ Vj Ka _ldeh%‛ _lj aoraetafj `b tldehdef ac`zt j|jk}ias~ de tjsk~ `b rd~zai

tsadi~ d~ tlat a tsadi aii`v~ b`s a ~defij `cgjnt de~taenj t` }astdnd}atj de kzitd}ij rd~zai

tsadi~% _` nsjatj a fsa}l) vj n`eejnt tv` j|jk}ias~ vdtl ae jofj db c`tl j|jk}ias~ fisj

`e janl `tljs vdtl a naidcsatjo ~n`sj fsjatjs tlae =%6 aeo db tljds c`zeodef c`| `rjsia}

d~ fsjatjs tlae =%:% Vldij tld~ d~ a satljs ~tsdnt sjqzdsjkjet b`s nsjatdef j|jk}ias+

j|jk}ias jofj~) vjahjs n`e~tsadet~ nsjatj fsa}l~ vldnl asj laso t` rd~zaidwj de tv`

odkje~d`e~% B`s tlj ~ahj `b niasdt) vj nl`~j tlj~j }asakjtjs~ jrje tl`zfl tlj o` e`t

aiva~ sj~zit de a n`eejntjo fsa}l% Vj od~}ia fsa}l~ b`s ~jrjsai ]A^NAI R@N `cgjnt

natjf`sdj~ z~def tlj J|jk}ias+^RK aif`sdtlk% _lj~j ’Rd~zai Kjkj|j~‛ nae cj ~jje de

tlj b`ii`vdef }afj~7 tsade ,~jj Bdfzsj ?%4$) k`t`scdhj ,~jj Bdfzsj ?%1=$) aeo ajs`}iaej

,~jj Bdfzsj ?%11$%

Dt d~ ai~` dk}`staet t` e`tj tlat kae `b tlj doja~ cjldeo Cz~l—~ kjkj| asj ~i`vi 

~tastdef t` a}}jas de n`k}ztjs rd~d`e% @ej ldfli sjijraet a}}s`anl d~ tlj ]l`t`cd`~

aif`sdtlk XHjkjikanljs+^lidwjskae jt ai%) 3=11[ b`s eardfatdef a iasfj n`iijntd`e `b 

<<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 89/130

dkafj~ `b tlj ~akj }js~`e% C nsjatdef ideh~ cjtvjje rd~zaii ~dkdias }l`t`~ `b tlj ~akj

}js~`e—~ banj) dt d~ }`~~dcij t` sjeojs a k`rdj vldnl ~zkkasdwj~ tlj rd~zai jr`iztd`e

`b a ~defij }js~`e) jrje nsjatdef n`k}jiidef aedkatd`e~ `b tlj banj a~ dt tsae~dtd`e~

bs`k odbbjsjet ~tatj~ ,j%f%) a k`rdj vldnl tsae~dtd`e~ bs`k a }js~`e ~kdidef t` tlj ~akj

}js~`e bs`vedef$% Vldij tld~ a}}s`anl la~ kaoj dt~ va det` tlj }`}zias Banj K`rdj~

bjatzsj `b F``fij—~ ]dna~a }l`t`+~lasdef ~`btvasj) dt d~ dk}`staet t` sjaidwj tlat dt osav~

dt~ ~znnj~~ bs`k tlj katzsdt `b banj ojtjntd`e aeo banj sjn`fedtd`e tjnle`i`f% _lj hj 

nlaiijefj d~ de j|tjeodef tlj~j doja~ t` e`e+banj o`kade~%

<4

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 90/130

 _sade Rd~zai Kjkj| Fsa}l

1=| w``k

Bdfzsj ?%47 _sade N`enj}t Rd~zai Kjkj|% _lj ~dkdiasdt fsa}l d~ nsjatjo c a}}idef janl J|jk}ias+^RK t` de+nia~~ dkafj~%

4=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 91/130

K`t`scdhj Rd~zai Kjkj| Fsa}l

1=| w``k1=| w``k

Bdfzsj ?%1=7 K`t`scdhj N`enj}t Rd~zai Kjkj|% _lj ~dkdiasdt fsa}l d~ nsjatjo c a}}idef janlJ|jk}ias+^RK t` de+nia~~ dkafj~%

41

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 92/130

1=| w``k1=| w``k

Ajs`}iaej Rd~zai Kjkj| Fsa}l

Bdfzsj ?%117 Ajs`}iaej N`enj}t Rd~zai Kjkj|% _lj ~dkdiasdt fsa}l d~ nsjatjo c a}}idef janlJ|jk}ias+^RK t` de+nia~~ dkafj~%

43

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 93/130

Nla}tjs :

N`etj|tzai @cgjnt ]sjodntd`e

L`v bas nae `z f` cjb`sj szeedef ae `cgjnt ojtjnt`s0

  Aet`ed` _`ssaica

De sjai ~njej~ n`k}`~jo `b kae odbbjsjet `cgjnt~) tlj ~}atdai n`efifzsatd`e `b `ej

`cgjnt nae bandidtatj sjn`fedtd`e `b sjiatjo `cgjnt~ XCas aeo Ziikae) 1442[) aeo qzdtj

`btje akcdfzdtdj~ de sjn`fedtd`e naee`t cj sj~`irjo vdtl`zt i``hdef cj`eo tlj ~}atdai

j|tjet `b tlj `cgjnt de qzj~td e% _lz~) aif`sdtlk~ vldnl g`deti sjn`fedwj kae `cgjnt~

at `enj c tahdef ann`zet `b n`etj|tzai sjiatd`e~ld}~ larj cjje qzdtj }`}zias% Vldij

jasi ~~tjk~ sjidjo `e laeo+n`ojo szij~ b`s detjs+`cgjnt n`etj|t ,j%f%) XLae~`e aeo

Sd~jkae) 146<) ^tsat aeo Bd~nlijs) 1441[$) k`sj k`ojse a}}s`anlj~ t}dnaii }jsb`sk

debjsjenj de a }s`cacdid~tdn fsa}ldnai k`oji vdtl sj~}jnt t` natjf`sdj~ vljsj `cgjnt

detjsantd`e~ asj k`ojijo a~ ldfljs `sojs }`tjetdai~ XLj jt ai%) 3==?) Hzkas aeo Ljcjst)

3==:) ^l`tt`e jt ai%) 3==2) Sacde`rdnl jt ai%) 3==6) Faiijfzdii`~ jt ai%) 3==<) ]asdhl jt ai%)

3==<) Sz~~jii jt ai%) 3==6[% @ej dk}`staet dk}idndt a~~zk}td`e kaoj c aii ~znl k`oji~

d~ tlat detjsantd`e~ cjtvjje `cgjnt de~taenj~ nae cj aojqzatji k`ojijo a~ sjiatd`e~ld}~

cjtvjje lzkae+ojfiejo `cgjnt natjf`sdj~%

De tld~ nla}tjs vj nlaiijefj tld~ ’natjf`s a~~zk}td`e‛ b`s `cgjnt+`cgjnt detjsantd`e~

4>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 94/130

aeo }s`}`~j a e`rji natjf`s+bsjj a}}s`anl b`s k`ojidef `cgjnt sjiatd`e~ld}~% Vj

}s`}`~j a ejv bsakjv`sh) tlj Rd~zai Kjkj| K`oji) b`s sj}sj~jetdef aeo sja~`edef ac`zt

`cgjnt dojetdtdj~ aeo tljds n`etj|tzai sjiatd`e~ld}~ de ae j|jk}ias+ca~jo) e`e+}asakjtsdn

  va% Vj jraizatj `zs k`oji `e Aet`ed _ ssaica—~ }s`}`~jo N`etj|t Nlaiijefj X_`ssaica)

3==>a[ afade~t a ca~jidej natjf`s+ca~jo ~~tjk%

@zs ~tastdef }`det d~ Raeejras Cz~l—~ `c~jsratd`e tlat ~tsdnt natjf`sdnai deoj|def `b 

n`enj}t~ la~ ~jrjsj idkdtatd`e~ XCz~l) 14?:[ ,~jj od~nz~~d`e de Nla}tjs 1$% Ae`tljs

k`tdratd`e d~ K`~lj Cas vl` cjidjrj~ tlat }sjodntd`e }ia~ a hj s`ij de tlj lzkae

csade XCas) 3==4[% Acaeo`edef sdfdo `cgjnt natjf`sdj~) vj jkcsanj Cz~l—~ aeo Cas—~

cjidjb de tlj }sdkas s`ij `b a~~`ndatd`e~) czt zeidhj Cz~l) vj adk t` od~n`rjs tlj~j

a~~`ndatd`e~ azt`katdnaii bs`k tlj oata% At tlj n`sj `b `zs k`oji d~ ae j|jk}ias+

ca~jo sj}sj~jetatd`e `b `cgjnt~ XE`~`b~h) 14<2) Kaid~djvdnw aeo Jbs`~) 3==<[% _lj

 Rd~zai Kjkj| nae tlje cj tl`zflt `b a~ a ra~t fsa}l) vdtl e`oj~ sj}sj~jetdef aii tlj

`cgjnt de~taenj~ de tlj oata~jt) aeo asn~ sj}sj~jetdef tlj odbbjsjet t}j~ `b a~~`ndatd`e~

cjtvjje tljk ,Bdfzsj :%1$% _ljsj asj tv` t}j~ `b asn~ de `zs k`oji) jen`odef tv`

odbbjsjet sjiatd`e~ld}~ cjtvjje `cgjnt~7 1$ rd~zai ~dkdiasdt ,j%f% tld~ nas i``h~ idhj tlat

nas$) aeo 3$ n`etj|tzai a~~`ndatd`e~ ,j%f% tld~ nas d~ ej|t t` tld~ czdiodef$%

@enj tlj fsa}l d~ czdit) dt nae cj z~jo t` detjs}sjt a e`rji dkafj ,Bdfzsj :%1) ijbt$ c 

fis~t n`eejntdef ~jfkjet~ vdtlde tlj dkafj vdtl ~dkdias j|jk}ias~) aeo tlje }s`}afatdef

n`etj|tzai deb`skatd`e cjtvjje tlj~j j|jk}ias~ tls`zfl tlj fsa}l% Vlje ae j|jk}ias

fjt~ antdratjo) rd~zaii ~dkdias j|jk}ias~ a~ vjii a~ `tljs n`etj|tzaii sjijraet `cgjnt~

fjt antdratjo a~ vjii% _ld~ va) j|jk}ias+t`+j|jk}ias ~dkdiasdt de tlj Kjkj| fsa}l nae

~jsrj a~ Cz~l—~ ’tsadi~‛ t` ideh n`enj}t~ t`fjtljs de a e`e+}asakjtsdn) qzjs+oj}jeojet

 va) vdtl`zt tlj z~j `b }sjojfiejo natjf`sdj~% B`s j|ak}ij) de Bdfzsj :%1) vj ~l`zio cj

acij t` debjs tlat a nas ~jje bs`k tlj sjas `btje n`+`nnzs~ vdtl ae `cidqzj czdiodef vaii

,czt e`t a bs`etai vaii$ „ ~`kjtldef vldnl natjf`s+ca~jo k`oji~ v`zio cj laso+}sj~~jo

4?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 95/130

Bdfzsj :%17 _lj Rd~zai Kjkj| fsa}l jen`oj~ `cgjnt ~dkdiasdt ,~`ido cianh jofj$ aeo ~}atdai n`etj|t

,o`ttjo sjo jofj$ cjtvjje }ads~ `b `cgjnt j|jk}ias~% A ~}atdai n`etj|t bjatzsj d~ ~t`sjo b`s janl n`etj|tjofj% _lj Kjkj| fsa}l nae cj z~jo t` detjs}sjt a ejv dkafj c a~~`ndatdef dkafj ~jfkjet~ vdtl

j|jk}ias~ de tlj fsa}l ,`saefj jofj~$ aeo }s`}afatdef tlj deb`skatd`e% Bdfzsj cj~t rdjvjo de n`i`s%

t` anldjrj%

B`skaii) vj ojfiej tlj Rd~zai Kjkj| K`oji a~ a fsa}l F ; ,R) J   ) J N ) xO{) xb {$

n`e~d~tdef `b  E  `cgjnt j|jk}ias e`oj~ R ) ~dkdiasdt jofj~ J   ) n`etj|t jofj~ J N ) E 

}js+j|jk}ias ~dkdiasdt bzentd`e~ xO{) aeo tlj ~}atdai bjatzsj~ xb { a~~`ndatjo vdtl janl

n`etj|t jofj%

:%1 Czdiodef a IacjiKj Rd~zai Kjkj| Fsa}l

Vj j|tsant a iasfj oataca~j `b j|jk}ias `cgjnt~ aeo tljds fs`zeo+tsztl ~jfkjetatd`e

ka~h~ bs`k tlj IacjiKj XSz~~jii jt ai%) 3==<[ oata~jt aeo ijase tlj ~tszntzsj `b tlj

 Rd~zai Kjkj| de ae `bffidej ~jttdef% Vj z~j `cgjnt~ bs`k tlj >= k`~t bsjqzjeti `nnzssdef

natjf`sdj~ de IacjiKj% ^dkdiasdt jofj~ asj nsjatjo z~def tlj }js+j|jk}ias od~taenj

bzentd`e ijasedef bsakjv`sh `b XKaid~djvdnw aeo Jbs`~) 3==<[) aeo n`etj|t jofj~ asj

nsjatjo janl tdkj tv` j|jk}ias~ asj `c~jsrjo de tlj ~akj dkafj% Vj larj a t`tai

`b  E  ; <6) <=3 j|jk}ias~ de tlj Rd~zai Kjkj|) yJ   y ; 362) 6<3 ~dkdiasdt jofj~) aeo

yJ N y ; 4<4) 1=2 n`etj|t jofj~%

Vj z~j tlj }js+j|jk}ias od~taenj+bzentd`e ijasedef aif`sdtlk `b XKaid~djvdnw aeo

4:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 96/130

a~~`ndatd`e~

0

nas

o``s

s`ao

vdeo`v

vdeo`v

tsjj

nas

tsjj

vdeo`v

vljjivljji0

Natjf`s

J~tdkatd`e

nas}js~`evljjis`aovdeo`vbjenjczdiodef

~dojvaiho``stsjj

Bdfzsj :%37 _ ssaica—~ N`etj|t Nlaiijefj7 ’L`v bas nae `z f` vdtl`zt szeedef a i`nai `cgjntojtjnt`s0‛ _lj ta~h d~ t` sja~`e ac`zt tlj dojetdt `b tlj ldooje `cgjnt ,oje`tjo c a ’0‛$ vdtl`zt i`nai

deb`skatd`e% De `zs natjf`s+bsjj Rd~zai Kjkj| k`oji) `cgjnt }sjodntd`e~ asj fjejsatjo de tlj b`sk `b j|jk}ias a~~`ndatd`e~ b`s tlj ldooje `cgjnt% De a natjf`s+ca~jo k`oji) tlj natjf`s `b tlj ldooje `cgjnt

d~ odsjnti j~tdkatjo%

Jbs`~) 3==<[) a~ oj~nsdcjo de Nla}tjs 3) t` ijase tlj `cgjnt ~dkdiasdt jofj~% Vj nsjatj

a ~dkdiasdt jofj cjtvjje tv` j|jk}ias~ db tlj asj ojjkjo ~dkdias c janl `tljs~—

od~taenj bzentd`e~% Vj z~j a fi|jo θ ; %====1 aeo ώ ; 1== b`s aii j|jk}ias~ ,~jj

Jqzatd`e 3%2$%

Vlje tv` `cgjnt~ `nnzs de~doj a ~defij dkafj) vj jen`oj tljds 3+O ~}atdai sjiatd`e+

~ld} det` a n`etj|t bjatzsj rjnt`s b  ∌ 1= ,rd~zaidwjo a~ sjo o`ttjo jofj~ de Bdfzsj :%1$%

_lj n`etj|t bjatzsj rjnt`s jen`oj~ sjiatdrj `rjsia}) sjiatdrj od~}ianjkjet) sjiatdrj ~naij)

aeo sjiatdrj ljdflt `b tlj c`tt`k+k`~t }d|ji cjtvjje tv` j|jk}ias sjfd`e~ de a ~defij

dkafj% _ld~ bjatzsj na}tzsj~ tlj ~}atdai sjiatd`e~ld} cjtvjje tv` sjfd`e~ aeo o`j~

e`t tahj det` ann`zet ae a}}jasaenj deb`skatd`e ‟ dt d~ a fjejsaidwatd`e `b tlj ~}atdai

bjatzsj~ z~jo de XFaiijfzdii`~ jt ai%) 3==<[% Vj kja~zsj tlj ~dkdiasdt cjtvjje tv`

n`etj|t bjatzsj~ z~def a Faz~~dae hjseji7 H ,b ) b $ ; j∓α1yy b ∓ b  yy3  vdtl α1 ; 1%=%

:%3 _ ssaica—~ N`etj|t Nlaiijefj

_lj detzdtd`e tlat vj v`zio idhj t` jraizatj d~ tlat kae z~jbzi sjfziasdtdj~ `b tlj rd~zai

 v`sio asj i`~t vlje ojaidef ~`iji vdtl natjf`sdj~ ,j%f%) tlj ~doj rdjv `b a czdiodef

~l`zio a~~`ndatj k`sj vdtl a ~doj rdjv `b a nas tlae a bs`etai rdjv `b a nas$% _lj hj 

42

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 97/130

k`tdratd`e cjldeo tlj Rd~zai Kjkj| d~ tlat n`etj|t ~l`zio oj}jeo `e tlj a}}jasaenj `b 

ae `cgjnt aeo e`t gz~t tlj natjf`s dt cji`ef~ t`% De `sojs t` tj~t tld~ l}`tlj~d~ afade~t

tlj n`kk`ei ljio }santdnj `b ac~tsantdef ava a}}jasaenj det` natjf`sdj~) vj ejjo a

sdnl jraizatd`e oata~jt a~ vjii a~ a kjaedefbzi jraizatd`e ta~h%

Vj b`zeo tlat tlj N`etj|t Nlaiijefj X_ ssaica) 3==>a[ sjnjeti }s`}`~jo c Aet`ed`

_ ssaica fit~ `zs ejjo~ }jsbjnti% _lj jraizatd`e ta~h d~ de~}dsjo c tlj qzj~td e7

’L`v bas nae `z f` vdtl`zt szeedef ae `cgjnt ojtjnt`s0‛ _lj f`ai d~ t` sjn`fedwj a

~defij `cgjnt de tlj dkafj vdtl`zt }jjhdef at }d|ji~ cji`efdef t` tlat `cgjnt% _`ssaica

}sj~jetjo ae aif`sdtlk b`s }sjodntdef tlj natjf`s aeo ~naij `b ae `cgjnt z~def `ei 

n`etj|tzai deb`skatd`e X_`ssaica) 3==>c[) czt ld~ e`td`e `b n`etj|t d~ ~njej+njetjsjo

,d%j) tlj a}}jasaenj `b tlj jetdsj dkafj d~ z~jo b`s }sjodntd`e$% ^denj tlj n`etj|t vj

 vd~l ~tzo d~ `cgjnt+njetjsjo) vj z~j ae `cgjnt+njetjsjo b`skziatd`e `b tlj N`etj|t

Nlaiijefj% Vldij dt d~ e`t nijas db tlj ac~`iztj }jsb`skaenj ezkcjs~ `e tlj N`etj|t

Nlaiijefj asj rjs kjaedefbzi de tljk~jirj~) vj bjji tlat dt d~ ae dojai ta~h b`s ~tzodef

`cgjnt+njetjsjo n`etj|t aeo tlj s`ij `b natjf`sdwatd`e a~~zk}td`e~ de ~znl k`oji~%

De `zs rasdaet `b tlj N`etj|t Nlaiijefj) tlj f`ai d~ t` }sjodnt tlj natjf`s `b aldooje `cgjnt td ~`iji ca~jo `e dt~ ~}atdai sjiatd`e~ld}~ t` ~`kj }s`rdojo `cgjnt~ ‟

 vdtl`zt z~def tlj }d|ji~ cji`efdef t` tlj ldooje `cgjnt at aii% B`s `zs ~tzo) vj z~j

kaezaii }s`rdojo sjfd`e~ aeo natjf`s iacji~ `b H  ~z}}`stdef `cgjnt~ de~doj a ~defij

dkafj% Vj sjbjs t` tlj dojetdtdj~ `b tlj H  ~z}}`stdef `cgjnt~ de tlj dkafj a~ xt1) % % % ) tH {

,vljsj t ∌ x1) % % % ) yN y{$ aeo tlj ~jt `b H  3O ~}atdai sjiatd`e~ld} bjatzsj~ cjtvjje janl

~z}}`stdef `cgjnt aeo tlj ldooje `cgjnt a~ xb d1) % % % ) b dH {%

46

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 98/130

:%> Debjsjenj de tlj Rd~zai Kjkj| K`oji

De tld~ ~jntd`e) vj j|}iade l`v t` z~j tlj Rd~zai Kjkj| fsa}l ,azt`katdnaii n`e+

~tszntjo bs`k oata$ t` }jsb`sk debjsjenj b`s tlj N`etj|t Nlaiijefj ldooje+`cgjnt

}sjodntd`e ta~h% E`t kahdef tlj ’natjf`s a~~zk}td e)‛ tlj k`oji d~ ojfiejo vdtl

sj~}jnt t` j|jk}ias a~~`ndatd`e~ b`s tlj ldooje `cgjnt% Debjsjenj de tlj k`oji sjtzse~ a

n`k}atdcdidt ~n`sj cjtvjje jrjs j|jk}ias aeo tlj ldooje `cgjnt) aeo nae cj tl`zfl

`b a~ sjtzsedef ae `sojsjo id~t `b j|jk}ias a~~`ndatd`e~% Ozj t` tlj eatzsj `b j|jk}ias

a~~`ndatd`e~ a~ `}}`~jo t` natjf`s a~~dfekjet~) a ~z}}`stdef `cgjnt nae cj a~~`ndatjo

 vdtl kzitd}ij j|jk}ias~ a~ `}}`~jo t` a ~defij natjf`s% Vj nsjatj ~`bt j|jk}ias a~~`nd+

atd`e~ cjtvjje janl `b tlj ~z}}`stdef `cgjnt~ aeo tlj j|jk}ias~ de tlj Rd~zai Kjkj|

z~def tlj ~dkdiasdt bzentd`e~ xO{ ,~jj ^jntd`e :%1$%

x^ 1) % % % ) ^  H { asj tlj a}}jasaenj bjatzsj~ b`s tlj H  ~z}}`stdef `cgjnt~% Aa g d~ tlj

abfiedt cjtvjje j|jk}ias a de tlj Rd~zai Kjkj| aeo tlj g+tl ~z}}`stdef `cgjnt aeo d~

nsjatjo c jraizatdef ^  g zeojs a—~ od~taenj bzentd e Aa g ; j∓Oa,^ g$% ά,jd) j g) b dg$ d~ tlj

}adsvd~j n`k}atdcdidt cjtvjje j|jk}ias jd aeo j g zeojs tlj ~}atdai bjatzsj b dg % Ijt V ac

cj tlj aoganjen katsd| sj}sj~jetatd`e `b tlj ~dkdiasdt jofj~ ,V zr ; X,z) r$ ∌ J   [$%

Debjsjenj de tlj Rd~zai Kjkj| K`oji d~ o`ej c `}tdkdwdef tlj b`ii`vdef n`eodtd`eai

od~tsdcztd`e vldnl ~n`sj~ tlj a~~dfekjet `b ae ascdtsas j|jk}ias jd t` tlj ldooje `cgjnt

ca~jo `e n`etj|tzai sjiatd`e~7

 },jdyA1) % % % ) AH ) b d1) % % % ) b dH $ ∛H 

 g;1

E a;1

Aa g ά,jd) ja) b dg$ ,:%1$

i`f ά,jd) j g) b dg$ ;

,z)r$∌J N

V dzV  grH ,b dg) b zr$,z)r$∌J N

V dzV  gr,:%3$

4<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 99/130

_lj sja~`e b`s tlj ~zkkatd`e de~doj Jqzatd`e :%3 d~ tlat dt affsjfatj~ n`etj|tzai de+

tjsantd`e~ bs`k ~dkdias j|jk}ias~% C o`def tld~) vj jbbjntdrji ’oje~db‛ tlj n`etj|tzai

detjsantd`e~ de tlj Rd~zai Kjkj|% Ae detjs}sjtatd`e `b tld~ oje~dfinatd`e }s`njozsj d~ tlat

 vj asj nsjatdef a hjseji oje~dt j~tdkat`s b`s ae ascdtsas }ads `b j|jk}ias~ ,jd) j g$ rda

a vjdfltjo ~zk `b hjseji~ }ianjo at n`etj|t bjatzsj~ de tlj oata ~jt xb zr{ 7 ,z) r$ ∌ J N 

 vljsj tlj vjdflt~ V dzV  gr kja~zsj rd~zai ~dkdiasdt cjtvjje }ads~ ,jd) j g$ aeo ,jz) jr$ %

Vj j|}jsdkjetjo vdtl z~def a ~defij hjseji) ά,jd) j gy b dg$ ; H ,b dg ) b jd)jg $) aeo b`zeo

tlat tlj detjfsatd`e `b kzitd}ij bjatzsj~ rda tlj oje~dfinatd`e oj~nsdcjo ac`rj d~ a hj 

defsjodjet b`s ~znnj~~bzi Rd~zai Kjkj| debjsjenj%

Bdeaii) abtjs }jsb`skdef debjsjenj de tlj Rd~zai Kjkj| K`oji) vj asj ijbt vdtl a

~n`sj b`s janl j|jk}ias% At tld~ ~tafj) a~ bas a~ `zs k`oji d~ n`enjsejo) tlj sjn`fedtd e

la~ aisjao cjje }jsb`skjo% L`vjrjs) ~denj tlj ta~h vj asj jraizatjo `e d~ natjf`s+

ca~jo) vj n`kcdej tlj sjtzsejo j|jk}ias~ det` a r`tj b`s natjf`sdj~ z~def Iznj—~

 A|d`k `b Nl`dnj XKjode aeo ^nlabbjs) 146<[ vldnl arjsafj~ tlj j|jk}ias sj~}`e~j~

}js+natjf`s%

N`IA+ca~jo ]asakjtsdn K`oji

Vj v`zio idhj t` jraizatj tlj Rd~zai Kjkj| k`oji afade~t a k`sj tsaodtd`eai) natjf`s+

ca~jo bsakjv`sh vdtl }asakjtsdn detjs+natjf`s sjiatd`e~ld}~% @ej `b tlj k`~t sjnjet

aeo ~znnj~~bzi a}}s`anlj~ d~ tlj N`IA k`oji XFaiijfzdii`~ jt ai%) 3==<[% N`IA ijase~ a

~jt `b }asakjtjs~ b`s janl }ads `b natjf`sdj~ vldnl n`ssj~}`eo t` sjiatdrj ~tsjeftl~ `b 

tlj b`zs odbbjsjet t`})ac`rj)cji`v)de~doj ~}atdai sjiatd`e~ld}~% De tlj na~j `b ojaidef

 vdtl natjf`sdj~ odsjnti vj n`e~dojs a n`eodtd`eai od~tsdcztd`e `rjs tlj natjf`s `b tlj

ldooje `cgjnt td tlat bant`s~ a~ a ~tas fsa}l vdtl H  ijarj~ ,vdtl tlj ldooje `cgjnt

cjdef n`eejntjo t` aii tlj ~z}}`stdef `cgjnt~$% μ asj k`oji }asakjtjs~) ά d~ a }adsvd~j

}`tjetdai tlat kja~zsj~ tlj n`k}atdcdidt `b tv` natjf`sdj~ vdtl a ~}jndfijo ~}atdai

44

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 100/130

sjiatd`e~ld}) aeo W  d~ a e`skaidwatd`e n`e~taet ~znl tlat tlj n`eodtd`eai od~tsdcztd`e

~zk~ t` 1%

 },tdyt1) % % % ) tH ) b d1) % % % ) b dH )μ$ ; 1W 

H  g;1

ά,td) t g) b dg )μ$ ,:%>$

B`ii`vdef XFaiijfzdii`~ jt ai%) 3==<[) vj z~j a bjatzsj bzentd`e l,b $ tlat n`k}ztj~

tlj abfiedt cjtvjje bjatzsj b  aeo a ~jt `b }s`t`t}dnai ~}atdai sjiatd`e~ld}~% Vj azt`kat+

dnaii fieo ]  }s`t`t}dnai ~}atdai sjiatd`e~ld}~ c niz~tjsdef aii ~}atdai bjatzsj rjnt`s~ xb {

de tlj tsadedef ~jt rda tlj }`}zias H+kjae~ aif`sdtlk% Ijt l,b $ ∌ ]  cj tlj e`skaidwjo

 rjnt`s `b abfiedtdj~ t` niz~tjs njetjs~ xn1) % % % )n] {%μ

d~ tlj ~jt `b aii }asakjtjs~ de tld~k`oji) vdtl μ,td) t g$ ∌ ]  cjdef tlj }asakjtjs~ a~~`ndatjo vdtl tlj }ads `b natjf`sdj~

,td) t g$%

i`f ά,td) t g) b dg )μ$ ; Xl,b dg$_ [ μ,td) t g$ ,:%?$

ld,b $ ∛ j∓αyy b ∓ndyy3 ,:%:$

Vj tsdjo z~def tlj b`zs }s`t`t}dnai sjiatd`e~ld}~ n`ssj~}`eodef t` ac`rj) cji`v)

de~doj) aeo `zt~doj a~ de XFaiijfzdii`~ jt ai%) 3==<[) czt b`zeo tlat z~def H+kjae~ vdtl

~dfedfinaeti iasfjs ezkcjs `b }s`t`t}j~ ]  ; >= }s`oznjo ~z}jsd`s sj~zit~% B`s ijasedef

μ) vj b`zeo tlj ka|dkzk idhjidl``o μ z~def fsaodjet oj~njet% _lj tsadedef `cgjntdrj

bzentd`e va~ `}tdkdwjo t` kdkdn vlat la}}je~ ozsdef tj~tdef `e tlj N`etj|t Nlaiijefj

ta~h% ^denj tlj od~tsdcztd`e~ b`s tlj N`etj|t Nlaiijefj ta~h asj ojfiejo vdtl sj~}jnt t` a

~defij natjf`s rasdacij ,~jj Jqzatd`e :%>$) vj n`zio n`k}ztj tlj }astdtd`e bzentd`e

odsjnti aeo odo e`t sjqzdsj ae a}}s`|dkatd`e~ a~ de XFaiijfzdii`~ jt ai%) 3==<[ ,vldnl

sjqzdsjo tsadedef de a i``} fsa}l$%

1==

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 101/130

Sjoznjo HOJ Kjkj| K`oji

^denj tlj Rd~zai Kjkj| K`oji aeo tlj N`IA+de~}dsjo k`oji kahj odbbjsjet a~~zk}td`e~

 vdtl sj~}jnt t` `cgjnt~ ,natjf`s+ca~jo r~% j|jk}ias+ca~jo$ aeo n`etj|t ,}asakjtsdn

 r~% e`e}asakjtsdn$) vj bjji dt v`zio ai~` cj z~jbzi t` j|akdej a lcsdo k`oji) ozccjo

tlj Sjoznjo HOJ Kjkj| K`oji) vldnl z~j~ a e`e}asakjtsdn k`oji `b n`etj|t ca~jo

`e Hjseji Oje~dt J~tdkatd e,HOJ$ czt `}jsatj~ `e `cgjnt natjf`sdj~% _lj Sjoznjo

HOJ Kjkj| K`oji d~ nsjatjo c n`iia}~def aii j|jk}ias~ cji`efdef t` a ~defij natjf`s 

det` bzii+n`eejntjo n`k}`ejet~ vldnl nae cj tl`zflt `b a~ aoodef natjf`sdj~ det` tlj

 Rd~zai Kjkj| fsa}l% Dojetdtdj~ cjtvjje deodrdozai j|jk}ias~ asj i`~t) aeo tlz~ vji`~j tlj fiej ojtadi~ `b a ~}atdai n`etj|t% C b`skdef natjf`sdj~) vj nae e` i`efjs ~a a

}astdnzias ~}atdai sjiatd`e~ld} d~ cjtvjje a cizj ~doj rdjv `b a nas aeo ae `cidqzj csdnh 

czdiodef) vj nae `ei ~a dt d~ a sjiatd`e~ld} cjtvjje a nas aeo a czdiodef% E`v tlat

 vj asj ijbt vdtl ae ze`sojsjo caf `b ~}atdai sjiatd`e~ld}~ xb { cjtvjje tv` natjf`sdj~)

 vj ejjo a va t` kja~zsj n`k}atdcdidt cjtvjje a ejvi `c~jsrjo b  aeo tlj ~t`sjo

sjiatd`e~ld}~%

Vj z~j tlj ~akj b`sk `b tlj N`etj|t Nlaiijefj n`eodtd`eai od~tsdcztd`e a~ de

Jqzatd`e :%>% Vj z~j a Hjseji Oje~dt J~tdkat`s,HOJ$ b`s jrjs }ads `b natjf`sdj~)

aeo tlj }`tjetdai ά nae cj tl`zflt `b a~ a katsd| `b ~znl j~tdkat`s~% _lj z~j `b 

e`e}asakjtsdn }`tjetdai~ de fsa}ldnai k`oji~ la~ cjje aisjao j|}i`sjo de tlj o`kade

`b tj|tzsj aeai~d~ X]afjt aeo I`ef~tabb) 144<[% οdg d~ tlj Hs`ejnhjs ojita bzentd`e%

i`f ά,td) t g) b dg$ ; ,z)r$∌J N οdzοgrH ,b dg) b zr$,z)r$∌J N

οdzοgr,:%2$

_lj Sjoznjo Kjkj| k`oji) cjdef natjf`s+ca~jo aeo e`e}asakjtsdn) affsjfatj~ tlj

~}atdai sjiatd`e~ld}~ ans`~~ kae odbbjsjet }ads~ `b j|jk}ias~ bs`k tv` natjf`sdj~% Vldij

 vj z~jo a fi|jo hjseji H  vldnl kja~zsj~ od~taenj d~`ts`}dnaii ans`~~ tlj odkje~d`e~

1=1

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 102/130

Rd~zai Kjkj|

 }  j  s  ~    e  

n  a  s  t  s  j  j  

v  d   e  o     v  

l  j  a  o   c  z  d   i   o   d   e   f  

~  h     v  a  i   i   

s    a  o   ~  d   o   j  v  a  i   h  

~  d    f  e  n  l  a  d   s  

o       s  k  `  z  e  t  a  d   e  

t  a  c  i   j  b   i       s  

~  t  s  j  j  t  i   d    f  l  t  

i   a  k   }   }  i   a  e  t  

 }    i   j  c  a  i   n    e     

v  l  j  j  i   

t  j  |  t   f  s  a  ~  ~  

n    i   z  k  e  

 }  a  e  j  t  s  a  ~  l  

c  i   d   e  o    f  s    z  e  o   

a  s  k  

}js~`e

nastsjj

vdeo`vljao

czdiodef~hvaiis`ao

~dojvaih~dfe

nladso``s

k`zetadetacij

bi``s~tsjjtidflt

iak}}iaet}`ij

cain`evljji

tj|tfsa~~

n`izke}aejtsa~lcideo

fs`zeoask

HOJ

 }  j  s  ~    e  

n  a  s  t  s  j  j  

v  d   e  o   `  v  

l  j  a  o   c  z  d   i   o   d   e   f  

~  h     v  a  i   i   

s     a  o   ~  d   o   j  v  a  i   h  

~  d    f  e  n  l  a  d   s  

o   `    s  k  `  z  e  t  a  d   e  

t  a  c  i   j  b   i       s  

~  t  s  j  j  t  i   d    f  l  t  

i   a  k   }   }  i   a  e  t  

 }    i   j  c  a  i   n    e     

v  l  j  j  i   

t  j  |  t   f  s  a  ~  ~  

n    i   z  k  e  

 }  a  e  j  t  s  a  ~  l  

c  i   d   e  o    f  s    z  e  o   

a  s  k  

}js~`enas

tsjjvdeo`v

ljaoczdiodef

~hvaiis`ao

~dojvaih~dfenladso``s

k`zetadetacijbi``s

~tsjjtidfltiak}}iaet}`ij

cain`evljjitj|t

fsa~~n`izke

}aejtsa~lcideo

fs`zeo

ask

N`IA

 }  j  s  ~    e  

n  a  s  t  s  j  j  

v  d   e  o     v  

l  j  a  o   c  z  d   i   o   d   e   f  

~  h     v  a  i   i   

s    a  o   ~  d   o   j  v  a  i   h  

~  d    f  e  n  l  a  d   s  

o       s  k  `  z  e  t  a  d   e  

t  a  c  i   j  b   i       s  

~  t  s  j  j  t  i   d    f  l  t  

i   a  k   }   }  i   a  e  t  

 }    i   j  c  a  i   n    e     

v  l  j  j  i   

t  j  |  t   f  s  a  ~  ~  

n    i   z  k  e  

 }  a  e  j  t  s  a  ~  l  

c  i   d   e  o    f  s     z  e  o   

a  s  k  

}js~`enastsjj

vdeo`v

ljaoczdiodef

~hvaiis`ao

~dojvaih~dfenladso``s

k`zetadetacijbi``s

~tsjjtidfltiak}

}iaet}`ij

cain`evljjitj|t

fsa~~n`izke

}aejtsa~lcideo

fs`zeoask

= =%1 =%3 =%> =%? =%: =%2 =%6 =%< =%4 1=

=%1

=%3

=%>

=%?

=%:

=%2

=%6

=%<

=%4

1

Sjnaii

      ]     s     j     n      d     ~      d     `     e

N`etj|t Nlaiijefj ]sjodntd`e N`ebdojenj

 

Rd~zaiKjkj|

HOJ

N`IA

=

=%3

=%?

=%2

=%<

1

N`etj|t Nlaiijefj Sjn`fedtd`e Annzsan b`s >= Natjf`sdj~

 } j s  ~   e 

n a s  t  s  j j 

v  d  e o    v  

l j a o  c z  d  i  o  d  e f  

~ h     v  a i  i  

s    a o  ~ d  o  j v  a i  h  

~ d  f  e n l a d  s  

o      s  k   z  e t  a d  e 

t  a c i  j b  i      s  

~ t  s  j j t  i  d  f  l t  

i  a k  }  } i  a e t  

 }   i  j c a i  n   e    

v  l j j i  

t  j |  t  f  s  a ~ ~ 

n   i  z  k e 

 } a e j t  s  a ~ l 

c i  d  e o  f  s    z  e o  

a s  k  

Rd~zai Kjkj|

HOJ

N`IA

Bdfzsj :%>7 N`etj|t Nlaiijefj Jraizatd`e a%$ N`etj|t Nlaiijefj n`ebz~d`e katsdnj~ b`s tlj > kjtl`o~7 Rd~zai Kjkj|) HOJ) aeo N`IA% c%$ Sjn`fedtd`e ]sjnd~d`e rjs~z~ Sjnaii vlje tlsj~l`iodef `zt}zt ca~jo

`e n`efiojenj% n$ ^doj c ~doj n`k}asd~`e `b tlj > kjtl`o~— annzsandj~ b`s >= natjf`sdj~%

`b  b ) tlj aoraetafj `b ~znl a e`e}asakjtsdn a}}s`anl d~ tlat vdtl je`zfl oata tlj

}astdnziasdtdj~ `b H  o` e`t kattjs% Vj ai~` j|}jsdkjetjo vdtl a Ejasj~t Ejdflc`s ca~jo

k`oji) czt b`zeo tlj Hjseji Oje~dt J~tdkatd`e a}}s`anl t` cj ~z}jsd`s%

:%? N`etj|t Nlaiijefj Jraizatd`e7 Rd~zai Kjkj| rjs~z~

Natjf`sdj~

B`s tlj N`etj|t Nlaiijefj jraizatd`e) vj z~j 3== saeo`ki ~jijntjo oje~ji iacjijo

dkafj~ bs`k IacjiKj XSz~~jii jt ai%) 3==<[% @zs tj~t~jt n`etade~ >=?< t`tai `cgjnt~ bs`k

>= odbbjsjet natjf`sdj~% B`s ae dkafj vdtl H  `cgjnt~) vj ~`irj H  N`etj|t Nlaiijefj

}s`cijk~ vdtl `ej ldooje `cgjnt aeo H +1 ~z}}`stdef `cgjnt~% Qzaidtatdrj sj~zit~ `e tld~

}sjodntd`e ta~h nae cj ~jje de Bdfzsj :%?%

Vj jraizatj tlj }jsb`skaenj `b `zs Rd~zai Kjkj| k`oji) tlj Sjoznjo Kjkj|

HOJ k`oji) aeo tlj N`IA+de~}dsjo k`oji vdtl sj~}jnt t` natjf`sdwatd`e }jsb`skaenj

,n`ebz~d`e katsdnj~ nae cj ~jje de t`} ijbt `b Bdfzsj :%>$% _lj `rjsaii sjn`fedtd`e

1=3

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 103/130

tacij

ffi``s

vaii

o``s

ffi``s

tacij

o``s

vaii

~dojvaih

nas

s`ao

}js~`e

Rd~zai Kjkj| J|jk}ias ]sjodntd`e~De}zt Dkafj ( Ldooje Sjfd`e Natjf`sdwatd`e Sj~zit~

   R  d  ~  z  a   i

  K  j  k

  j  |

  H  O  J

  N    I  A

   R  d  ~  z  a   i

  K  j  k

  j  |

  H  O  J

  N    I  A

   R  d  ~  z  a   i

  K  j  k

  j  |

  H  O  J

  N    I  A

   R  d  ~  z  a   i

  K  j  k  j  |

  H  O  J

  N    I  A

   R  d  ~  z  a   i

  K  j  k

  j  |

  H  O  J

  N    I  A

   R  d  ~  z  a   i

  K  j  k

  j  |

  H  O  J

  N    I  A

   R  d  ~  z  a   i

  K  j  k  j  |

  H  O  J

  N    I  A

Bdfzsj :%?7 Qzaidtatdrj Sj~zit~ `e tlj N`etj|t Nlaiijefj% J|jk}ias }sjodntd e~ asj bs`k tlj Rd~zaiKjkj| k`oji aeo natjf`sdwatd`e sj~zit~ asj bs`k tlj Rd~zai Kjkj| k`oji) tlj HOJ K`oji) aeoN`IA XFaiijfzdii`~ jt ai%) 3==<[%

1=>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 104/130

annzsan `b tlj Rd~zai Kjkj| K`oji) Sjoznjo Kjkj| K`oji) aeo N`IA asj %:36)

%?>=) aeo %?:6 sj~}jntdrji% E`tj tlat tlj Rd~zai Kjkj| K`oji }jsb`sk~ ~dfedfinaeti 

cjttjs tlae tlj ca~jidej~% _ahdef a ni`~js i``h at tlj }js+natjf`s annzsandj~ `b tlj

tlsjj kjtl`o~ ,~jj c`tt`k `b Bdfzsj :%>$) vj ~jj tlat tlj N`IA+ca~jo kjtl`o badi~ `e

kae natjf`sdj~% _lj arjsafj }js+natjf`s sjn`fedtd`e annzsandj~ `b tlj tlsjj kjtl`o~

asj7 %:>?) %?:?) aeo %31>% _lj Rd~zai Kjkj| K`oji ~tdii }jsb sk~ tlj cj~t) czt vj

~jj a ~dfedfinaet os`} de }jsb`skaenj b`s tlj natjf`s+ca~jo N`IA kjtl`o% N`IA d~

cda~jo t`vaso~ tlj }`}zias natjf`sdj~) sjtzsedef tlj k`~t bsjqzjeti `nnzssdef natjf`s 

,vdeo`v$ qzdtj `btje% @rjsaii) tlj Rd~zai Kjkj| K`oji anldjrj~ tlj cj~t }jsb`skaenj

b`s 31 `zt `b tlj >= natjf`sdj~%

De aoodtd`e) vj }i`t }sjnd~d`e sjnaii nzsrj~ b`s janl `b tlj tlsjj kjtl`o~ t` ojtjskdej

db ldfl n`efiojenj sjtzsejo c janl k`oji d~ n`ssjiatjo vdtl ldfl sjn`fedtd`e satj~ ,t`}

sdflt `b Bdfzsj :%>$% _lj Rd~zai Kjkj| k`oji la~ tlj k`~t ~dfedfinaet ldfl+}sjnd~d`e

i`v+sjnaii sjfdkj) ~zffj~tdef tlat dt~ n`efiojenj d~ a f``o kja~zsj `b ~znnj~~% _lj

sjiatdrji ffiat nzsrj b`s tlj N`IA kjtl`o d~ sjiatjo t` tlj }s`cijk `b `rjsn`k}je~atd`e

b`s }`}zias nia~~j~ a~ kjetd`ejo ac`rj% _lj od~tsdcztd`e~ sjtzsejo c N`IA tjeo t`

ojfjejsatj t` a ~defij e`e+wjs` raizj ,k`~t `btje `e `ej `b tlj }`}zias natjf`sdj~

~znl a~ vdeo`v$% _ld~ d~ vl tlj ka|dkzk }s`cacdidt sjtzsejo c N`IA d~ e`t a f``o

kja~zsj `b n`efiojenj%

Vj ai~` ojk`e~tsatj tlj }`vjs `b tlj Rd~zai Kjkj| t` }sjodnt a}}jasaenj ~`iji 

ca~jo `e n`etj|tzai detjsantd`e~ vdtl `tljs `cgjnt~ aeo tljds rd~zai a}}jasaenj% _lj

kdooij s`v `b Bdfzsj :%? ojk`e~tsatj~ ~`kj `b tlj~j a~~`ndatd`e~% E`tj l`v de s`v 1)

a }iaz~dcij rdjv}`det d~ ~jijntjo satljs tlae gz~t a saeo`k nas% De s`v > vj ~jj tlat tlj

a}}jasaenj `b ~e`v `e `ej k`zetade ~zffj~t~ tlat tlj `tljs }`std`e `b tlj dkafj ai~`

n`etade~ a ~e`v k`zetade% De ~zkkas) vj }sj~jetjo a natjf`s+bsjj Rd~zai Kjkj|

K`oji aeo a}}idjo dt t` tlj ta~h `b n`etj|tzai `cgjnt sjn`fedtd`e vdtlde tlj j|}jsdkjetai

1=?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 105/130

bsakjv`sh `b tlj N`etj|t Nlaiijefj% @zs j|}jsdkjet~ n`efisk `zs detzdtd`e tlat k`rdef

cj`eo natjf`sdj~ d~ cjejfindai b`s dk}s`rjo k`ojidef `b sjiatd`e~ld}~ cjtvjje `cgjnt~%

:%?%1 B`ii`v+z} V`sh 

^denj dt~ dedtdai }zcidnatd`e de 3==4) ~jrjsai detjsj~tdef a}}s`anlj~ larj cjje }zcid~ljo

 vldnl asj jdtljs deffizjenjo c `zs Rd~zai Kjkj| v`sh) `s asj rjs sjijraet% ]djts`

]js`ea—~ jbb`st t` `sfaedwj rd~zai he`vijofj de tlj b`sk `b a Rd~d}joda X]js`ea) 3=1=[

d~ k`sj ahde t` a rd~zai jenni`}joda tlae a rd~zai vjc `b n`enj}t~8 l`vjrjs) dt d~

ejrjstljij~~ k`tdratjo c Raeejras Cz~l—~ doja~ sjfasodef tlj kjkj| aeo tlz~ ldfli 

sjijraet t` `zs a}}s`anl%

Sjfasodef tlj z~j `b `cgjnt+fsa}l~ de rd~d`e) tlj v`sh `b XBd~ljs aeo Laesalae)

3=1=[ z~j~ a ~dkdias natjf`s+bsjj k`oji czt a}}idj~ dt t` tlj }s`cijk `b >O k`oji

~jasnl vlje fdrje tlj a}}jasaenj~ `b a ~kaii ~jt `b ~z}}`stdef >O `cgjnt~% XIjj aeo

Fsazkae) 3=1=[ larj ~znnj~~bzii z~jo `cgjnt fsa}l~ aeo fsa}l+ca~jo oj~nsd}t`s~ b`s

ze~z}jsrd~jo `cgjnt natjf`s od~n`rjs%

1=:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 106/130

Nla}tjs 2

N`eniz~d`e7 _`vaso tlj Rd~zai Kjkj|

De ld~ od~~jstatd`e) vj ojk`e~tsatjo tlj kae aoraetafj~ `b }`~def tlj sjn`fedtd`e

}s`cijk a~ `cgjnt a~~`ndatd`e a~ `}}`~jo t` tlj t}dnai natjf`s+ca~jo `cgjnt eakdef

}asaodfk% Vj }sj~jetjo ae j|jk}ias+ca~jo `cgjnt sjn`fedtd e bsakjv sh) ~l`vna~jo

tlj }`vjs `b }js+j|jk}ias ~dkdiasdt kja~zsj~ b`s nsjatdef ldfl+qzaidt rd~zai a~~`nd+

atd`e~) a~ vjii a~ jraizatjo `zs je~jkcij `b J|jk}ias+^RK a}}s`anl `e tlj }`}zias

]A^NAI R@N `cgjnt ojtjntd`e cjenlkash ta~h% E`t `ei o`j~ `zs kjtl`o }jsb`sk

n`k}jtdtdrji vdtl ~tatj+`b+tlj+ast kjtl`o~) czt dt nae cj a}}idjo t` kjta+oata tsae~bjs

ta~h~ vldnl asj laso t` tanhij vdtl natjf`s+ca~jo k`oji~% Cji`v vj `ztidej ~jrjsai

hj aoraetafj~ `b `zs bsakjv`sh `rjs natjf`s+ca~jo a}}s`anlj~7

1% N`enj}tzai ~dk}idndt7 De~tjao `b ijasedef `ej n`k}ij| natjf`s+~}jndfin k`oji)

  vj ijase a ~j}asatj ~dk}ij tjk}iatj+ca~jo ojtjnt`s b`s janl j|jk}ias de `zs

oata~jt% Janl `b `zs ijasedef }s`cijk~ der`irj~ a ~defij }`~dtdrj de~taenj aeo d~

n`erj|) tlz~ aii`vdef z~ t` z~j jbfindjet `}tdkdwatd`e idcsasdj~ ozsdef ijasedef%

3% Kjta+oata tsae~bjs7 @zs ojtjntd`e~ }s`oznj ldfl qzaidt aidfekjet~ vldnl aii`v

z~ t` tsae~bjs ae kjta+oata a~~`ndatjo vdtl tlj j|jk}ias odsjnti `et` tlj

ojtjntd`e vdeo`v) vdtl`zt ae aoodtd`eai dtjsatdrj aidfekjet ~tj}% _ld~ ~l`v~

1=2

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 107/130

tlat `enj vj na~t tlj sjn`fedtd`e }s`cijk de tjsk~ `b a~~`ndatd`e) vj asj acij t`

}s`oznj kznl k`sj tlae gz~t a natjf s+iacjijo c`zeodef c`|% ]s`cijk~ ~znl a~

~jfkjetatd`e aeo fj`kjts tsae~bjs nae cj ja~di laeoijo c `zs j|jk}ias+ca~jo

a}}s`anl%

>% ]asaiijidwacdidt/Ja~di j|tjeoacij7 Cjnaz~j `zs j|jk}ias~ asj tsadejo deoj+

}jeojeti) dt d~ }`~~dcij t` od~tsdcztj ijasedef ans`~~ a niz~tjs aeo tlje ~dk}i 

n`enatjeatj tlj sj~zitdef J|jk}ias+^RK~ t` nsjatj `ej iasfj je~jkcij% Cjnaz~j

`b tlj kdedkai detjsantd`e ans`~~ j|jk}ias~) vj nae ’degjnt he`vijofj‛ det` `zs

je~jkcij `b J|jk}ias+^RK~ c azfkjetdef dt vdtl aoodtd`eai j|jk}ias~) vdtl`zt

lardef t` sj+tsade jrjs j|jk}ias%

?% Dkafj/`cgjnt katnldef7 Vj larj ~l`ve `zs bsakjv`sh nae cj z~jo b`s katnldef

`cgjnt~ a~ vjii a~ ns`~~+o`kade dkafj katnldef) ~zffj~tdef tlat `tljs a}}idnatd`e~)

sjqzdsdef tljds `ve e`td`e `b rd~zai ~dkdiasdt) n`zio cjejfit bs`k `zs j|jk}ias+

ca~jo a}}s`anl%

Fdrje tlj }`vjs `b j|jk}ias+ca~jo rd~zai a~~`ndatd`e~) n`z}ijo vdtl c`tl densja~jo

n`k}ztdef ~}jjo~ aeo dk}s`rjo annj~~ t` iasfj+~naij n`k}ztatd`eai sj~`zsnj~) dt d~

}jsla}~ tlj sdflt tdkj b`s rd~d`e sj~jasnljs~ t` ~jsd`z~i n`e~dojs Raeejras Cz~l—~ doja~

sjfasodef a kjk`s+ca~jo k`oji `b tl`zflt7 ’Dt `}jsatj~ c a~~`ndatd`e% Vdtl `ej dtjk

de dt~ fsa~}) dt ~ea}~ de~taeti t` tlj ej|t tlat d~ ~zffj~tjo c tlj a~~`ndatd`e `b tl`zflt~)

de ann`soaenj vdtl ~`kj detsdnatj vjc `b tsadi~ nassdjo c tlj njii~ `b tlj csade%‛ XCz~l)

14?:[ @zs v`sh `e tlj j|jk}ias+ca~jo Rd~zai Kjkj| K`oji ~zffj~t~ tlat tljsj d~

kjsdt t` tlj doja `b ae detsdnatj vjc `b rd~zai tsadi~% A~ vj larj ~l`ve de Nla}tjs :)

dt d~ }`~~dcij t` nsjatj ~znl a rd~zai ejtv`sh) vdtl j|jk}ias~ a~ e`oj~ aeo j|jk}ias

sjiatd`e~ld}~ a~ jofj~% C a}}idef `zs Rd~zai Kjkj| K`oji t` a n`etj|tzai `cgjnt

}sjodntd`e ta~h) vj vjsj acij t` }sjodnt tlj a}}jasaenj `b a ldooje `cgjnt ~`iji ca~jo

1=6

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 108/130

`e dt~ ~}atdai sjiatd`e~ld}~ t` a ~jt `b `c~jsrjo `cgjnt~ de tlj dkafj% _lz~ jkcsandef

a kjkj|+ca~jo va `b tldehdef ac`zt tlj rd~zai v`sio jetadi~ tldehdef ac`zt rd~zai

he`vijofj a~ l}js+idehjo rd~zai he`vijofj satljs tlae a~ natjf`s+ca~jo he`vijofj%

^znl a natjf`s+bsjj rdjv `b tlj v`sio la~ tlj nlaenj t` nlaefj tlj va vj aoosj~~

tlj }s`cijk `b dkafj zeojs~taeodef) kznl idhj tlj va F``fij aeo Vdhd}joda nlaefjo

tlj va vj tldeh ac`zt he`vijofj%

Bdeaii) tljsj d~ fs`vdef jrdojenj tlat tlj doja~ sjfasodef rd~zai a~~`ndatd`e~ asj ljsj

t` ~ta% Fdrje tlj s`ij vldnl aeai`fdj~) a~~`ndatd`e~ aeo }sjodntd`e~ }ia de n`fedtdrj

ejzs`~ndjenj XCas) 3==4[) tlj sjnjet ~ndjetdfin fieodef~ vldnl ~zffj~t tlat lzkae~—

 rd~zai kjk`s d~ iasfjs tlae }sjrd`z~i tl`zflt XCsao jt ai%) 3==4[) a~ vjii a~ tlj

~znnj~~ `b he`vijofj f`def odfdtai ,j%f%) Vdhd}joda) Banjc``h) F``fij$) vj bjji tlat

tlj tdkj d~ sdflt t` ~tast jkcsandef tlj }`vjs `b rd~zai a~~`ndatd`e~% Vj l`}j tlat

`zs tlj~d~ kahj~ a ~ts`ef na~j b`s j|jk}ias+ca~jo sj}sj~jetatd`e~ aeo tlj ~tsjeftl `b 

}js+j|jk}ias ~dkdiasdt bzentd`e~ ‟ vj l`}j tlat tlj ~znnj~~ `b `zs J|jk}ias+^RK

bsakjv`sh aeo Rd~zai Kjkj| K`oji vdii k`tdratj tlj ej|t fjejsatd`e `b sj~jasnljs~ t`

qzj~td`e tlj ejnj~~dt `b sjidef `e `cgjnt natjf`sdj~ b`s `cgjnt/~njej detjs}sjtatd e%

1=<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 109/130

 A}}jeod| A 

Ns`~~+O`kade Dkafj Katnldef

Bdfzsj A%17 Ns`~~+O`kade Dkafj Katnldef7 Vj asj detjsj~tjo de ojfiedef rd~zai ~dkdiasdt cjtvjjedkafj~ ans`~~ odbbjsjet o`kade~) ~znl a~ }l`t`~ tahje `rjs odbbjsjet ~ja~`e~ aeo idfltdef) }adetdef~)

~hjtnlj~) jtn% Vlat kahj~ tld~ nlaiijefdef d~ tlat tlj rd~zai n`etjet d~ `ei ~dkdias `e tlj ldfljs ~njej

ijrji) czt qzdtj od~~dkdias `e tlj }d|ji ijrji%

De tld~ tlj~d~) vj ~zkkasdwj tlj sj~zit~ `b ae a}}idnatd`e `b tlj J|jk}ias+^RK

bsakjv`sh t` tlj }s`cijk `b ns`~~+o`kade katnldef X^lsdra~tara jt ai%) 3=11[% Vj

csdjffi od~nz~~ tlj }s`cijk aeo `ei ~l`v a ~zc~jt `b `zs sj~zit~ b`s ~jrjsai odbbjsjet

ns`~~+o`kade a}}idnatd`e~%

_lj njetsai jijkjet n`kk`e t` kae dkafj+katnldef a}}idnatd`e~ de n`k}ztjs

fsa}ldn~ d~ ~jasnldef a iasfj oata~jt t` fieo rd~zaii+~dkdias katnlj~ t` a fdrje qzjs ‟

cj dt ae dkafj }atnl) a bzii dkafj) `s a ~}atd`+tjk}`sai ci`nh% L`vjrjs) ojfiedef a f``o

 rd~zai ~dkdiasdt kjtsdn t` z~j b`s katnldef nae `btje cj ~zs}sd~defi odbfinzit% Fsaetjo)

de kae ~dtzatd`e~ vljsj tlj oata d~ sja~`eaci l`k`fjej`z~ ,j%f%) odbbjsjet }atnlj~

 vdtlde tlj ~akj tj|tzsj dkafj XJbs`~ aeo Bsjjkae) 3==1[) `s odbbjsjet bsakj~ vdtlde

1=4

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 110/130

tlj ~akj rdoj` X^nl`oi jt ai%) 3===[$) a ~dk}ij }d|ji+vd~j) ~zk+`b+~qzasjo+odbbjsjenj~

katnldef v`sh~ qzdtj vjii% Czt vlat ac`zt tlj na~j~ vlje tlj rd~zai n`etjet d~ `ei 

~dkdias `e tlj ldfljs ~njej ijrji) czt qzdtj od~~dkdias `e tlj }d|ji ijrji) ~znl a~ de

Bdfzsj A%1% B`s de~taenj) kjtl`o~ tlat z~j ~njej katnldef ,j%f%) XLa~ aeo Jbs`~)

3==6) Oaij jt ai%) 3==4[$ `btje ejjo t` katnl dkafj~ ans`~~ odbbjsjet diizkdeatd`e~)

odbbjsjet ~ja~`e~) odbbjsjet nakjsa~) jtn% Idhjvd~j) sj+tj|tzsdef ae dkafj de tlj ~tij `b 

a }adetdef XLjstwkaee jt ai%) 3==1) Jbs`~ aeo Bsjjkae) 3==1[ sjqzdsj~ kahdef rd~zai

n`ssj~}`eojenj cjtvjje tv` rjs odbbjsjet o`kade~ ‟ }l`t`~ aeo }adetdef~% _lj ns`~~+

o`kade katnldef d~ jrje k`sj nsdtdnai b`s a}}idnatd`e~ ~znl a~ ^hjtnl3]l`t` XNlje

jt ai%) 3==4[ aeo NF3Sjai XG`le~`e jt ai%) 3=1=[) vldnl adk t` csdef o`kade~ a~

odbbjsjet a~ ~hjtnlj~ aeo NF sjeojsdef~ det` n`ssj~}`eojenj vdtl eatzsai }l`t`fsa}l~%

De aii `b tlj~j na~j~) }d|ji+vd~j katnldef basj~ j|tsjkji }``si) cjnaz~j }jsnj}tzaii 

~kaii odbbjsjenj~ nae sj~zit de ascdtsas iasfj }d|ji+vd~j odbbjsjenj~% Vlat d~ ejjojo d~ a

 rd~zai kjtsdn tlat nae ~l`v ~`kj s`cz~tej~~ t` ~kaii) zedk}`staet rd~zai odbbjsjenj~)

 jt ~tdii na}tzsj tlj dk}`staet rd~zai ~tszntzsj~ tlat kahj tv` dkafj~ a}}jas ~dkdias%

_ld~ d~ }sjnd~ji vlat kahj~ tld~ }s`cijk ~` odbfinzit ‟ tlj rd~zai ~dkdiasdt aif`sdtlk

~`kjl`v ejjo~ t` he`v vldnl rd~zai ~tszntzsj~ asj dk}`staet b`s a lzkae `c~jsrjs

aeo vldnl asj e`t%

Vj ~l`v tlat a e`rji rjs~d`e `b tlj J|jk}ias+^RK aif`sdtlk }s`rdoj~ a rjs ~dk}ij)

 jt ~zs}sd~defi jbbjntdrj a}}s`anl t` rd~zai katnldef vldnl d~ }astdnziasi vjii+~zdtjo

b`s katnldef dkafj~ ans`~~ odbbjsjet o`kade~% Fdrje ae dkafj sj}sj~jetjo c ~`kj

bjatzsj~ tlj adk d~ t` b`nz~ tlj katnldef `e tlj bjatzsj~ tlat asj tlj k`~t rd~zaii 

dk}`staet b`s tld~ }astdnzias dkafj% _lj njetsai doja d~ z~j tlj e`td`e `b ’oata+osdrje

zedqzjej~~‛% Vj l}`tlj~dwj tlat tlj dk}`staet }ast~ `b tlj dkafj~ asj tl`~j tlat

asj k`sj zedqzj `s sasj vdtlde tlj rd~zai v`sio ,sj}sj~jetjo ljsj c a iasfj oata~jt$%

E`tj tlat ~denj tlj ~akj i`nai bjatzsj~ n`zio sj}sj~jet rjs odbbjsjet rd~zai n`etjet

11=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 111/130

Bdfzsj A%37 ^hjtnl aeo ]adetdef Katnldef @zs a}}s`anl v`sh~ vjii ans`~~ odbbjsjet rd~zai o`kade~denizodef ~hjtnlj~ aeo }adetdef~

oj}jeodef `b n`etj|t) `zs e`td`e `b zedqzjej~~ la~ t` cj ~njej+oj}jeojet) d%j%) janl

qzjs ~njej ojndoj~ vlat—~ tlj cj~t va t` vjdflt dt~ n`e~tdtzjet }ast~% C b`nz~def

`e tlj fi`caii ~aidjet }ast~ `b tlj dkafj) tlj a}}s`anl nae cj ~znnj~~bzii z~jo b`s

fjejsdn ns`~~+o`kade katnldef) vdtl`zt kahdef ae o`kade+~}jndfin nlaefj~) a~ ~l`ve

`e Bdfzsj A%3%

Vj `}jsatd`eaidwj tld~ oata+osdrje zedqzjej~~ c z~def tlj J|jk}ias+^RK bsakj+

 v`sh) czt de a }zsji natjf`s+bsjj kaeejs% B`s a qzjs dkafj) vj tsade ae J|jk}ias+

^RK t` od~n`rjs vldnl }ast~ `b ae dkafj asj k`~t od~nsdkdeatdrj de sjiatd`e~ld} t` tlj

sj~t `b tlj oata~jt% _lz~ de tld~ ~njeasd`) tlj J|jk}ias+^RK—~ ejfatdrj ~jt o`j~ e`t z~j

ae nia~~+deb`skatd`e ,vldnl va~ sjqzdsjo de XKaid~djvdnw jt ai%) 3=11[$%

Bdfzsj A%> ~l`v ~`kj j|ak}ij ~hjtnl qzjsdj~ aeo tlj n`ssj~}`eodef t`} 2 sjtsdjrai

sj~zit~ b`s `zs a}}s`anl aeo tlj ca~jidej~ ‟ dt nae cj ~jje tlat `zs a}}s`anl e`t `ei 

`zt}jsb`sk~ aii `b tlj ca~jidej~) czt sjtzse~ dkafj~ ~l`vdef tlj tasfjt `cgjnt de a rjs 

~dkdias }`~j aeo rdjv}`det a~ tlj qzjs ~hjtnl% Qzaidtatdrj j|ak}ij~ `b }adetdef+t`+

dkafj katnldef nae cj ~jje de Bdfzsj A%?% Qzaidtatdrj }adetdef3F]^ j|ak}ij~ nae cj

~jje de Bdfzsj A%:%

111

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 112/130

Bdfzsj A%>7 ^hjtnl3Dkafj Qzaidtatdrj J|ak}ij% @zs a}}s`anl z~def tlj J|jk}ias+^RK aif`sdtlk d~n`k}asjo t` ~jrjsai }`}zias tjnledqzj~ b`s katnldef dkafj~%

Bdfzsj A%?7 ]adetdef3Dkafj Qzaidtatdrj J|ak}ij% @zs a}}s`anl z~def tlj J|jk}ias+^RK aif`sdtlk d~n`k}asjo t` ~jrjsai }`}zias tjnledqzj~ b`s katnldef dkafj~%

Bdfzsj A%:7 ]adetdef3F]^ Qzaidtatdrj J|ak}ij~% De tlj~j tv` j|ak}ij~) vj ~l`v tlj de}zt }adetdef a~

 vjii a~ tlj t`} > katnlj~ z~def FD^_ aeo `zs a}}s`anl% Vj od~}ia F]^ i`naidwatd`e sj~zit~ a~ a oje~dt ka} `rjsiado `e tlj fi`cj%

113

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 113/130

 A}}jeod| C

J|jk}ias+^RK~7 Ae @}je ^`zsnj

Dk}ijkjetatd`e

Vj csdjffi oj~nsdcj tlj dk}ijkjetatd`e `b tlj J|jk}ias+^RK aif`sdtlk z~jo t` tsade

tl`z~aeo~ `b j|jk}ias~ ans`~~ a niz~tjs `b 3==( kanldej~) de a Ka}Sjoznj+idhj ba~ld`e%

Vldij vj ijase v—~ b`s a iasfj ezkcjs `b j|jk}ias~) janl j|jk}ias—~ ijasedef }s`cijk

aeo naidcsatd`e nae cj ~`irjo deoj}jeojeti aii`vdef b`s ja~ }asaiijidwatd`e% _lj Katiac

~`zsnj n`oj `b tlj J|jk}ias+^RK bsakjv`sh ,Nla}tjs > aeo Nla}tjs ?$ d~ aradiacij

a~ a }zcidn) `}je+~`zsnj }s`gjnt `e FdtLzc% FdtLzc d~ a n`oj ~lasdef vjc~dtj vdtl

kae k`ojse ~`ndai+ejtv`shdef bjatzsj~ czdit+de% _lj }ldi`~`}l cjldeo l`~tdef `}je+

~`zsnj }s`gjnt~ `e FdtLzc d~ tlat dt kahj~ n`iiac`satd`e aeo detjsantd`e kznl ja~djs

tlae z}i`aodef a tascaii `e a vjc~dtj% FdtLzc kahj~ dt ja~ t` z~j a }s`gjnt) tsanh dt~

}s`fsj~~) aeo ai~` n`etsdcztj ae czf+fi|j~ aeo jelaenjkjet~ b`s tlj n`k}ztjs rd~d`e

n`kkzedt t` jeg`%

1% J|jk}ias+^RK ]s`gjnt7 ltt}7//vvv%n~%nkz%joz/utkaid~dj/}s`gjnt~/dnnr11/

3% J|jk}ias+^RK N`oj7 ltt}~7//fdtlzc%n`k/qzaet`kc`ej/j|jk}ias~rk

11>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 114/130

_lj J|jk}ias+^RK ~~tjk sjqzdsj~ annj~~ t` a niz~tjs vdtl a ~lasjo ejtv`sh osdrj% _`

cj acij t` sze `e a rasdjt `b niz~tjs~) tlj ca~dn }sdend}ij vldnl vj jk}i` d~ tlat `b a

~defij }s`nj~~ vldnl n`zio cj sj}idnatjo E  tdkj~% _ljsj d~ `ej kade osdrjs }s`fsak)

 vldnl vlje i``}def `rjs j|jk}ias~) z~j~ i`nhfiij~ t` ~jj db ae j|jk}ias la~ aisjao cjje

tsadejo% Db ae j|jk}ias la~ e`t cjje tsadejo) tlj nzssjet }s`nj~~ fsac~ tlj j|jk}ias~

aeo vsdtj~ a i`nhfiij t` od~h ‟ ae `tljs }s`nj~~ vdii ~dk}i ~hd} `rjs ae aisjao fied~ljo

j|jk}ias `s a i`nhjo j|jk}ias%

C%1 J|jk}ias+^RK _sadedef ]asakjtjs~

Vj z~j idc~rk XNlaef aeo Ide) 3==1[ t` tsade janl j|jk}ias—~ v% Vj aitjseatj cjtvjje

ijasedef tlj vjdflt~ fdrje ae antdrj ~jt `b ejfatdrj vdeo`v~) aeo kdedef aoodtd`eai

ejfatdrj vdeo`v~ z~def tlj nzssjet v a~ de XBjiwje~wvaic jt ai%) 3=1=[% Vj z~j tlj

~akj sjfziasdwatd`e }asakjtjs N 1 ; =%: aeo N 3 ; %=1 b`s aii j|jk}ias~) czt b`zeo `zs

 vjdflt rjnt`s~ t` cj s`cz~t t` a vdoj saefj `b N ~) j~}jndaii ~denj tlj asj sj+~naijo

ozsdef naidcsatd`e% Vj z~j 3:== a~ tlj ka|dkzk ezkcjs `b ejfatdrj~ t` kdej) vldnl

n`ssj~}`eo~ t` a ~defij }a~~ `rjs tlj ejfatdrj~% Vj n`e~dojs dkafj~ a~ vjii a~ tljds

ijbt+sdflt ffid}}jo n`zetjs}ast~ b`s c`tl tsadedef aeo tj~tdef%

C%3 ^idodef Vdeo`v~ aeo Bjatzsj ]sakdo~

@ej `b tlj cdffj~t osavcanh~ `b ~jfkjetatd`e+ca~jo ’~jfkjet+tlje+sjn`fedwj‛ tjnl+

edqzj~ d~ tlat tlj sjqzdsj ~dfedfinaet n`k}ztatd`eai sj~`zsnj~ t` fjejsatj naeodoatj

~jfkjet~ ‟ `enj a ~jfkjetatd`e d~ n`k}ztjo b`s ae dkafj aeo bjatzsj n`k}ztatd`e

la~ cjje a}}idjo t` tlj sj~zitdef ~jfkjet~) dt d~ t}dnaii vd~j t` ~t`sj tlj sj~zit~

`e laso+od~h t` ar`do zeejnj~~asdi szeedef tlj ~jfkjetatd`e aif`sdtlk a ~jn`eo

11?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 115/130

tdkj% ^jfkjetatd`e+ca~jo a}}s`anl }sjnizoj njstade a}}idnatd`e~ ~znl a~ sjai+tdkj /

detjsantdrj+tdkj a}}idnatd`e~ ,ozj t` ~}jjo `b n`k}ztdef a ~jfkjetatd`e$) a~ vjii a~

a}}idef tljk t` oata~jt~ `b kdiid`e~ `b dkafj~ ,ozj t` ~t`safj ~}anj sjqzdsjkjet~$%

_lj ~jn`eo ~l`stn`kdef `b z~def dkafj ~jfkjetatd`e b`s `cgjnt sjn`fedtd`e d~ tlat dt

dets`oznj~ a ~jn`eo }ianj vljsj `cgjnt sjn`fedtd`e n`zio badi% C badidef t` fs`z} }d|ji~

det` sja~`eacij nlzeh~ vdtl ~zbfindjet ~}atdai ~z}}`st b`s sjn`fedtd`e) ~jfkjetatd`e+

ca~jo a}}s`anlj~ nae ~dk}i badi at }s`ozndef ae  sja~`eacij ~jfkjet~ b`s ~`kj

odbfinzit+t +~jfkjet `cgjnt~% Vlat tld~ kjae~ d~ tlat jrje db vj vjsj acij t` `ctade

dojai }js+j|jk}ias ~dkdiasdt kja~zsj~) vj kdflt ~tdii e`t cj acij t` sjn`fedwj jrjs 

~defij `cgjnt oj}dntjo de a ~njej% _ld~ d~ a ~jrjsj idkdtatd`e) j~}jndaii cjnaz~j a}}idef

}js+j|jk}ias ~dkdiasdt kja~zsj~ ,vldnl vj larj aisjao ~}jet a n`e~dojsacij tdkj

tsadedef$ d~ ~dfedfinaeti ba~tjs tlae fjejsatdef tlj antzai ~jfkjet~% Vlat tld~ ~zffj~t~

d~ tlat vj ~l`zio fjejsatj k`sj ~jfkjet~ de ij~~ tdkj ‟ db tlj }js+j|jk}ias ~dkdiasdt 

bzentd`e~ asj f``o je`zfl) tlje tlj n`zio ojndoj vldnl ~jfkjet~ t` hjj}%

 A~ vdii cj ~jje de tld~ A}}jeod|) L@F+ca~jo ~idodef vdeo`v a}}s`anlj~ `bbjs a

~dfedfinaet ~}jjoz} `rjs tljds ~jfkjetatd`e+ca~jo n`zetjs}ast~% _lj ~}jjoz} d~ ~`~dfedfinaet tlat vj nae }s`nj~~ kae k`sj sjfd`e~ }js dkafj) tlz~ aiijrdatdef `zs

  v`ss tlat ~jfkjetatd`e kdflt n`k}ijtji kd~~ ~`kj `cgjnt~% Vj fis~t `ztidej tlj

kzitd~naij L@F bjatzsj+}sakdo) tlje }sj~jet tlj tv` odbbjsjet va~ de vldnl vj nae

}jsb`sk i`naidwatd`e vdtlde tlj }sakdo% _lj ciawdef+ba~t ojtjntd`e ~tsatjf ca~jo `e

n`er`iztd`e sjqzdsj~ rjs idttij kjk`s `rjsljao aeo d~ ~zdtacij b`s ojaidef vdtl a ~kaii

ezkcjs `b j|jk}ias~ ,j%f%) tlj kdedef ~tafj de J|jk}ias+^RK ijasedef$% Zeb`stzeatji)

z~def ~idodef+vdeo`v n`er`iztd`e b`s a iasfj ezkcjs `b j|jk}ias~ nae ~tdii cj ~i`v% @zs

n`etsdcztd`e de tld~ nla}tjs d~ tlj ci`nh+katnldef kjtl`o `b ojtjntd`e vldnl sjqzdsj~ a

~defij katsd|+kzitd}idnatd`e aeo d~ kznl ba~tjs vlje i`naidwdef tl`z~aeo~ `b j|jk}ias~

de~doj a ~defij dkafj ,j%f%) a}}idef ae je~jkcij `b J|jk}ias+^RK~ ozsdef tj~tdef$%

11:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 116/130

Kzitd~naij L@F ]sakdo

Ld~t`fsak~ `b @sdjetjo Fsaodjet~ ,L@F$ asj ~`kj `b tlj k`~t vdoji z~jo oj~nsd}+

t`s~ ozj t` tljds ~}jjo `b n`k}ztatd`e~) deljsjet s`cz~tej~~ t` ~idflt `cgjnt rasda+

td`e~/ojb`skatd`e~) aeo acdidt t` na}tzsj a n`as~j ,jt sdfdo$ ~}atdai ia`zt `b bjatzsj~%

Vldij dedtdaii dets`oznjo b`s }joj~tsdae ojtjntd`e XOaiai aeo _sdff~) 3==:[ z~def a idejas

^RK) L@F d~ ai~` z~jo c tlj ~tatj+`b+tlj+ast ojb`skacij }ast k`oji `b XBjiwje~wvaic

jt ai%) 3=1=[%

Satljs tlae deoj}jeojeti n`k}ztdef L@F oj~nsd}t`s~ b`s odbbjsjet sjfd`e~ de ae

dkafj) L@F d~ k`~t `btje n`k}ztjo de a }sakdo+idhj ba~ld`e% Dkafj }sakdo~ X Aoji~`e

jt ai%) 1441[ asj a nia~~dnai tjnledqzj~ de rd~d`e) aeo asj nsdtdnai de kahdef L@F

n`k}ztatd`e~ ba~t% @ej `b tlj laiikash~ `b z~def bjatzsj }sakdo~ b`s i`naidwatd`e d~ tlat

 vj o` e`t sjqzdsj j|}idndti i``}def `rjs sjfd`e~ de tlj dkafj aeo deoj}jeojeti naiidef

a bjatzsj~,sjfd`e$ bzentd`e% ^denj L@F bjatzsj~ asj a n`enatjeatd`e `b ~kaiijs L@F

njii~) tv` ldfli `rjsia}}def naeodoatj sjfd`e~ bs`k tlj ~akj ijrji `b tlj }sakdo

 vdii ~lasj kae `b tlj ~akj ci`nh~% De `tljs v`so~) dt ~zbfinj~ t` n`k}ztj tlj L@F

oj~nsd}t`s b`s janl njii de ae dkafj `ei `enj) tlje n`enatjeatdef odbbjsjet njii~ vlje

 vj vaet oj~nsd}t`s~ n`k}ztjo b`s odbbjsjet sjfd`e~% Vj z~j 1= ijrji~ }js `ntarj vlje

nsjatdef tlj }sakdo aeo z~j 1%= a~ tlj fiej~t dkafj ~naij%

Sj}sj~jetdef @cgjnt~ a~ L@F _jk}iatj~

Vj z~j tlj L@F oj~nsd}t`s bs`k XBjiwje~wvaic jt ai%) 3=1=[ vldnl z~j~ B  ; >1 ezkcjs~

t` sj}sj~jet janl njii) aeo a ci`nh ~dwj `b  < }d|ji~ ,~cde ; <$% Vlje dedtdaidwdef

j|jk}ias~) de~tjao `b vas}def janl j|jk}ias t` a nae`ednai bsakj ,a~ d~ ejnj~~as de

k`e`idtldn kjtl`o~ XOaiai aeo _sdff~) 3==:[$) vj ijt janl j|jk}ias ojfiej dt~ `ve

L@F odkje~d`e~ bs`k tlj fs`zeo+tsztl c`zeodef c`|—~ a~}jnt satd`% Z~def a ~defij

112

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 117/130

}`~dtdrj de~taenj kjae~ tlat vj o` e`t ejjo t` v`ss ac`zt tlj aidfekjet `b kzitd}ij

de~taenj~ ,vldnl kz~t cj aoosj~~jo de k`e`idtldn nia~~dfijs~ c ae aidfekjet ~tsatjf 

ahde t` tlj iatjet s``t+fiitjs z}oatj ~tj} `b Iatjet ^RK~ XBjiwje~wvaic jt ai%) 3==<[$%

Vlje dt d~ ejnj~~as t` j|tsant bjatzsj~ `rjs a tasfjt sjfd`e) }s`rdojo a~ a c`zeodef

c`|) vj ~dk}i n`k}ztj tlj dkafj+vdoj kzitd~naij L@F }sakdo aeo ~jasnl b`s a ~idnj

`b tlj }sakdo vl`~j ~}atdai j|tjet cj~t katnlj~ tlj tasfjt c`zeodef c`| ,ann`sodef t`

`rjsia} ~n`sj$% ^denj vj n`zio j|tsant kzitd}ij sjntaefzias sjfd`e~ ,bs`k odbbjsjet ijrji~

`b tlj }sakdo$ t` sj}sj~jet tlj j|jk}ias) dt d~ ai~` ejnj~~as t` }s`rdoj aoodtd`eai n`e+

~tsadet~ sjfasodef tlj ~dwj `b tlj oj~dsjo L@F odkje~d`e~% Vj z~j tv` ~dk}ij ljzsd~tdn~

b`s nl``~def l`v t` bsakj ae j|jk}ias7 fieodef a tjk}iatj vldnl n`k}sd~j~ s`zfli 

f`ai~dwj njii~) aeo a ka|dkzk njii odkje~d`e) ka|njiiodkje~d`e% De tlj L@F j|}jsd+

kjet~ z~jo tls`zfl`zt tld~ tlj~d~) vj z~j f`ai~dwj;1== aeo ka|njiiodkje~d`e;13%

_ld~ kjae~ tlat vj i``h b`s a }sakdo ~idnj vldnl la~ ldfl `rjsia} vdtl tlj fs`zeo+

tsztl c`zeodef c`|) n`k}sd~j~ s`zfli 1== njii~ aeo la~ a ka|dkzk njii odkje~d`e `b 

13% B`s j|ak}ij) db ae j|jk}ias d~ sj}sj~jetjo vdtl a 1= Ö 1= L@F oj~nsd}t`s) tlje tlj

t`tai ezkcjs `b odkje~d`e~ de tlj oj~nsd}t`s vdii cj >1== ,a~~zkdef B  ; >1$%

I`naidwdef a ~defij j|jk}ias rda n`er`iztd`e

B`s tlj }zs}`~j `b i`naidwdef a ~defij j|jk}ias) sj}sj~jetjo c a tjk}iatj v) vdtlde tlj

L@F }sakdo) vj a}}i a n`er`iztd`e ans`~~ janl ijrji `b tlj }sakdo deoj}jeojeti%

Fdrje a ijrji `b tlj }sakdo) sj}sj~jetjo a~ a K  Ö E  Ö B  katsd| `b ezkcjs~ ,B  cjdef

tlj odkje~d`eaidt `b tlj oj~nsd}t`s$) aeo v cjdef `b ~dwj K  Ö E  Ö B ) vj }jsb`sk a

n`er`iztd`e vldnl sj~zit~ de a ,K  ∓ K $ Ö ,E  ∓ E $ katsd| `b ~n`sj~% _ld~ kjtl`o `b 

i`naidwatd`e d~ rjs ba~t ~denj dt o`j~ e`t sjqzdsj tlj nsjatd`e `b ae j|}idndt bjatzsj katsd|

‟ tlj `rjsia}}def L@F bjatzsj~ `ei ejjo t` cj ~t`sjo de }sakdo b`sk% Vj z~j tld~

n`er`iztd`e+ca~jo ~tsatjf ozsdef tlj laso ejfatdrj kdedef ~tj} `b tlj J|jk}ias+^RK

116

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 118/130

aif`sdtlk cjnaz~j tsadedef d~ }jsb`skjo deoj}jeojeti b`s janl j|jk}ias%

 A}}idef Od~taenj Bzentd`e~ rda _v` N`er`iztd`e~

N`er`iztd`e nae ai~` cj z~jo b`s i`naidwdef j|jk}ias~ sj}sj~jetjo c od~taenj bzentd`e~)

aitl`zfl tlj n`k}ij|dt d~ k`sj tlae tvdnj tlat `b idejas tjk}iatj~ ,cjnaz~j tlj

n`er`iztd`e kz~t cj a}}idjo tvdnj aeo a ~qzasjo L@F }sakdo kz~t cj n`k}ztjo$% A 

~qzasjo L@F }sakdo d~ `ctadejo c ~qzasdef aii `b tlj jijkjet~ de tlj `sdfdeai L@F

}sakdo% _` ~jj vldnl tv` n`er`iztd`e~ kz~t cj }jsb`skjo) n`e~dojs tlj odaf`eai

Kalaiae`cd~ od~taenj bzentd`e b`sk ,~akj a~ a~ ojfiejo de Jqzatd`e 3%1$7

Oj,|d$ ; ,|j ∓ |d$_ V j,|j ∓ |d$ ,C%1$

vj ; odaf,V j$ ,C%3$

Vj nae }jsb`sk ~`kj kaed}ziatd`e~ ,vldnl j|}i`dt tlj bant tlat V j d~ odaf`eai$ t`

 vsdtj tlj od~taenj bzentd`e a~ b`ii`v~7

Oj,|d$ ; |_ j V j|j ( |_ 

d V j|d ∓ 3|_ j V j|d ,C%>$

Oj,|d$ ; H ( v_ j |d ( v_ 

j |d ,C%?$

H  ; |_ j V j|j ,C%:$

|_ d V j|d ; v_ 

j |d ,C%2$

 |dXh[ ; ,|dXh[$

3

,C%6$

 vj ; ∓3V j|j ,C%<$

 A~ nae cj ~jje de Jqzatd`e C%?) tlj fis~t }ast `b tlj od~taenj bzentd`e jraizatd`e) H )

d~ deoj}jeojet `b tlj tj~t vdeo`v |d aeo ejjo~ t` cj n`k}ztjo `ei `enj) aeo nae cj

11<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 119/130

nanljo b`s bztzsj z~j% _lj }ast `b tlj od~taenj bzentd`e jraizatd`e vldnl oj}jeo~ `e

tlj tj~t+vdeo`v—~ bjatzsj~ nae cj cs`hje o`ve det` tv` n`er`iztd`e~7 tlj fis~t z~def

tlj vjdflt rjnt`s vj) czt a}}idjo t` tlj ~qzasjo L@F }sakdo

 |d) aeo tlj ~jn`eo z~def

 vj aeo a}}idjo t` tlj sav L@F }sakdo%

I`naidwdef kae j|jk}ias~ rda a ~defij katsd| kzitd}idnatd`e

De tlj na~j `b a}}idef tl`z~aeo~ `b j|jk}ias~ t` tlj L@F }sakdo j|tsantjo bs`k a

~defij dkafj) dt d~ cjejfindai t` j|}idndti n`e~tsznt tlj O Ö E  bjatzsj+katsd|) vljsj O

d~ tlj t`tai odkje~d`eaidt `b tlj L@F tjk}iatj) aeo E  d~ tlj t`tai ezkcjs `b naeodoatj

c`zeodef c`|j~ j|tsantjo bs`k a ~defij dkafj ,ans`~~ aii }sakdo ijrji~$% Vj naii tld~

a}}s`anl tlj ci`nh+katnldef kjtl`o b`s ~idodef+vdeo`v i`naidwatd`e% _lj hj de~dflt

 vldnl aii`v~ z~ t` o` tld~ d~ tlj sjaidwatd`e tlat tlj sj~zit `b a}}idef ae K  Ö E  L@F

tjk}iatj d~ jqzdraijet t` a}}idef ae jeiasfjo) =+}aoojo) ,K (∂K $Ö,E (∂E $ tjk}iatj

 vljsj ∂K  aeo ∂E  deodnatj l`v kznl t` }ao% Cjnaz~j a}}idnatd`e `b a tjk}iatj d~ a

~dk}ij o`t+}s`oznt) aoodef wjs`~ vdii e`t nlaefj tlj sj~zit ,a~ i`ef a~ vj ai~` }ao tlj

L@F }sakdo vdtl =~ t` ann`zet b`s c`zeoas jbbjnt~$% Vj j|}i`dt tlj bant tlat vj z~jo

 ka|njiiodkje~d`e; 13 ozsdef j|jk}ias dedtdaidwatd`e ,~jj A}}jeod| C%3$% Aii `b `zs

j|jk}ias~ nae tlje cj =+}aoojo det` a n`kk`e 13 Ö 13 bsakj ,d%j%) O ; ??2?$%

Fdrje a ejv dkafj) vj fis~t j|tsant aii 13 Ö 13 vdeo`v~ bs`k janl ijrji `b dt~

=+}aoojo L@F }sakdo% De }santdnj vj fjt E  ≌ 3=) === od~tdent vdeo`v~ vlje z~def

~cde; < aeo z~def dkafj~ vldnl n`k}sd~j a}}s`|dkatji  >== Ö ?== }d|ji~% Aii `b `zs

j|jk}ias~ nae e`v cj a}}idjo t` tld~ dkafj cjnaz~j tlj larj ai~` cjje =+}aoojo t`

tlj nae`ednai 13 Ö 13 bsakj% De tlj na~j `b z~def idejas tjk}iatj~) a~ d~ tlj na~j vdtl

tlj J|jk}ias+^RK aif`sdtlk) aii `b `zs j|jk}ias~ nae cj a}}idjo rda a ~defij katsd|

kzitd}idnatd`e ,jrje db tlj vjsj dedtdaii `b odbbjsjet odkje~d`e~$% Vlje a}}idef

od~taenj bzentd`e~ ,d%j%) odaf`eai Kalaiae`cd~ kjtsdn~$ de a ~dkdias ci`nh+ba~ld`e ,a~

114

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 120/130

 va~ ejnj~~as b`s tlj j|}jsdkjet~ de ^jntd`e >%?$) vj ejjo t` c`tl =+}ao tlj j|jk}ias—~

bjatzsj~ aeo ijasejo od~taenj bzentd`e vjdflt~ t` fit tlj n`kk`e 13 Ö 13 bsakj%

Vldij tlj ci`nh+katnldef kjtl`o~ tahj~ k`sj n`k}ztjs kjk`s t` ~t`sj tlj

ldfli+sjozeoaet bjatzsj+katsd| aeo tahj~ a}}s`|dkatji  ? ~jn`eo~ t` n`k}ztj }js

dkafj) tld~ n`k}ztatd`e d~ deoj}jeojet `b tlj ezkcjs `b j|jk}ias~% _lj ~defij katsd|+

kzitd}idnatd e d~ qzdtj ba~t aeo }s`rdoj~ a ~dfedfinaet ~}jjoz} `rjs n`er`iztd`e vlje

tlj ezkcjs `b j|jk}ias~ d~ k`ojsatji iasfj% Vj b`zeo tlat b`s z} t` := j|jk}ias~)

tlj n`er`iztd`e+ca~jo kjtl`o d~ ba~tjs) vldij b`s k`sj tlae := j|jk}ias~) tlj ~defij+

katsd|+kzitd}idnatd`e kjtl`o d~ ba~tjs%

C%> J|jk}ias+^RK sze+tdkj n`k}ij|dt 

_lj sze+tdkj n`k}ij|dt `b `zs a}}s`anl at tj~t tdkj ~naij~ idejasi vdtl tlj ezkcjs

`b }`~dtdrj de~taenj~ ,czt zeidhj hjseji+^RK kjtl`o~) e`t tlj ejfatdrj~$% L`vjrjs) de

}santdnj) tlj c`ttijejnh a}}jas~ t` cj }js+dkafj ta~h~ ,i`aodef) n`k}ztdef L@F }sakdo

jtn%$ „ tlj antzai }js+de~taenj n`k}ztatd`e d~ gz~t a ~defij o`t+}s`oznt) vldnl nae cj

o`ej j|tsjkji ba~t% B`s ae arjsafj ]A^NAI nia~~ ,∹ >== tsadedef j|ak}ij~ djiodef

∹ >== ~j}asatj nia~~dfijs~$ `zs kjtl`o d~ `ei  2 tdkj~ ~i`vjs tlae a natjf`s+ca~jo

kjtl`o ~znl a~ XBjiwje~wvaic jt ai%) 3=1=[% K`sj fjejsaii) cjnaz~j `b tlj i`ef+tadijo

od~tsdcztd`e `b `cgjnt~ de tlj v`sio ,1=* `b `cgjnt~ `ve 4=* `b j|jk}ias~ X_`ssaica

jt ai%) 3==<[$) tlj j|tsa n`~t `b z~def j|jk}ias~ r~% natjf`sdj~ vdii fsjati odkded~l a~

tlj ezkcjs `b natjf`sdj~ densja~j~%

13=

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 121/130

Cdcid`fsa}l 

XAoji~`e) 3==1[ Aoji~`e) J% ,3==1$% @e ~jjdef ~tzbb7 tlj }jsnj}td`e `b katjsdai~ c 

lzkae~ aeo kanldej~% De ]s`n% ^]DJ %

XAoji~`e jt ai%) 1441[ Aoji~`e) J% L%) ^dk`enjiid) J% ]%) aeo Bsjjkae) V% _% ,1441$%

]sakdo~ aeo kzitd~naij sj}sj~jetatd`e~% ]s`n% 1>tl Jzs`}jae N`ebjsjenj `e Rd~zai

 ]jsnj}td`e%

XAla jt ai%) 1441[ Ala) O% V%) Hdcijs) O%) aeo Aicjst) K% H% ,1441$% De~taenj+ca~jo

ijasedef aif`sdtlk~% Kanldej Ijasedef) 2%

XAsd~t tij) [ Asd~t tij% Natjf`sdj~%

XAthj~`e jt ai%) 1446[ Athj~`e) N%) K``sj) A%) aeo ^nlaai) ^% ,1446$% I`naii vjdfltjo

ijasedef% Astdfindai Detjiidfjenj Sjrdjv%

XCas) 3==6[ Cas) K% ,3==6$% _lj }s`antdrj csade7 Z~def aeai`fdj~ aeo a~~`ndatd`e~ t`

fjejsatj }sjodntd`e~% _sjeo~ de N`fedtdrj ^ndjenj%

XCas) 3==4[ Cas) K% ,3==4$% _lj }s`antdrj csade7 kjk`s b`s }sjodntd`e~% ]ldi`~`}ldnai

_sae~antd`e~ `b tlj S`ai ^`ndjt C) >2?713>:„13?>%

XCas aeo Ziikae) 1442[ Cas) K% aeo Ziikae) ^% ,1442$% ^}atdai n`etj|t de sjn`fedtd e%

 ]jsnj}td`e) 3:7>?>„>:3%

131

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 122/130

XCa~sd) 1443[ Ca~sd) S% ,1443$% Sjn`fedtd`e c }s`t`t}j~% Detjseatd`eai G`zseai `b 

N`k}ztjs Rd~d`e) 14714„1?6%

XCji`efdj jt ai%) 3==3[ Cji`efdj) ^%) Kaidh) G%) aeo ]zwdnla) G% ,3==3$% ^la}j katnldef

aeo `cgjnt sjn`fedtd`e z~def ~la}j n`etj|t~% ]AKD %

XCjsf jt ai%) 3==:[ Cjsf) A% N%) Cjsf) _% I%) aeo Kaidh) G% ,3==:$% ^la}j katnldef aeo

`cgjnt sjn`fedtd`e z~def i`v od~t`std`e n`ssj~}`eojenj% NR]S%

XCjshjij) 161=[ Cjshjij) F% ,161=$% _sjatd~j N`enjsedef tlj ]sdend}ij~ `b Lzkae

 He`vijofj%

XC`zsojr jt ai%) 3=1=[ C`zsojr) I%) Kagd) ^%) Cs`|) _%) aeo Kaidh) G% ,3=1=$% Ojtjntdef

}j`}ij z~def kztzaii n`e~d~tjet }`~jijt antdratd`e~% JNNR %

XCsao jt ai%) 3==4[ Csao) _%) H`ehij) _%) Airasjw) F%) aeo @idra) A% ,3==4$% Rd~zai

i`ef+tjsk kjk`s la~ a ka~~drj ~t`safj na}andt b`s `cgjnt ojtadi~% ]s`njjodef~ `b 

tlj Eatd`eai Anaojk `b ^ndjenj~) Z^A%

XCz~l) 14?:[ Cz~l) R% ,14?:$% A~ vj ka tldeh% _lj Atiaetdn K`etli %

XNlaef aeo Ide) 3==1[ Nlaef) N%+N% aeo Ide) N%+G% ,3==1$% IDC^RK7 a idcsas b`s ~z}+

 }`st rjnt`s kanldej~% ^`btvasj aradiacij at ltt}7//vvv%n~dj%etz%joz%tv/ungide/

idc~rk%

XNla}jiij) 3==6[ Nla}jiij) @% ,3==6$% _sadedef a ~z}}`st rjnt`s kanldej de tlj }sdkai%

 Ejzsai N`k}ztatd`e) 14711::„116<%

XNlje jt ai%) 3==4[ Nlje) _%) Nljef) K%+K%) _ae) ]%) ^lakds) A%) aeo Lz) ^%+K% ,3==4$%

^hjtnl3}l`t`7 detjsejt dkafj k`etafj% ANK _sae~% Fsa}l%) 3<%

XNlje jt ai%) 3==1[ Nlje) T%) Wl`z) \%) ) aeo Lzaef) _% ^% ,3==1$% @ej+nia~~ ~rk b`s

ijasedef de dkafj sjtsdjrai% DND]%

133

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 123/130

XNlzk aeo Wd~~jskae) 3==6[ Nlzk) @% aeo Wd~~jskae) A% ,3==6$% Ae j|jk}ias k`oji

b`s ijasedef `cgjnt nia~~j~% NR]S%

XN`kaedndz aeo Kjjs) 3==3[ N`kaedndz) O% aeo Kjjs) ]% ,3==3$% Kjae ~ldbt7 A s`cz~ta}}s`anl t`vaso bjatzsj ~}anj aeai~d~% ]AKD ) 3?,:$72=>„214%

XOaiai aeo _sdff~) 3==:[ Oaiai) E% aeo _sdff~) C% ,3==:$% Ld~t`fsak~ `b `sdjetjo fsaod+

jet~ b`s lzkae ojtjntd`e% NR]S%

XOaij jt ai%) 3==4[ Oaij) H%) G`le~`e) K% H%) ^zeharaiid) H%) Katz~dh) V%) aeo ]fi~tjs) L%

,3==4$% Dkafj sj~t`satd`e z~def `eidej }l`t n`iijntd`e~% De Detjseatd`eai N`ebjsjenj

`e N`k}ztjs Rd~d`e%

XJojikae) 144:[ Jojikae) ^% ,144:$% Sj}sj~jetatd`e) ~dkdiasdt aeo tlj nl`sz~ `b 

}s`t`t}j~% Kdeo~ aeo Kanldej~%

XJbs`~ aeo Bsjjkae) 3==1[ Jbs`~) A% A% aeo Bsjjkae) V% _% ,3==1$% Dkafj qzditdef b`s

tj|tzsj ~etlj~d~ aeo tsae~bjs% ]s`njjodef~ `b ^DFFSA]L 3==1%

XJii~`e jt ai%) 3==1[ Jii~`e) G%) Fae~ejs) J% S%) H`zt~`fi`~) J%) E`stl) ^% N%) aeo V``olzii)

F% ,3==1$% Fsa}lrdw + `}je ~`zsnj fsa}l osavdef t``i~% De Fsa}l Osavdef) }afj~

?<>„?<?%

XJrjsdeflak jt ai%) 3=1=[ Jrjsdeflak) K%) F``i) I% R%) Vdiidak~) N% H% D%) Vdee) G%) aeo

Wd~~jskae) A% ,3=1=$% _lj }a~nai rd~zai `cgjnt nia~~j~ ,r`n$ nlaiijefj% DGNR %

XBjiwje~wvaic jt ai%) 3==<[ Bjiwje~wvaic) ]%) KnAiij~tjs) O%) aeo Sakaeae) O% ,3==<$%

 A od~nsdkdeatdrji tsadejo) kzitd~naij) ojb`skacij }ast k`oji% NR]S%

XBjiwje~wvaic jt ai%) 3=1=[ Bjiwje~wvaic) ]% B%) Fds~nldnh) S% C%) KnNaiij~tjs) O%) aeo

Sakaeae) O% ,3=1=$% @cgjnt ojtjntd`e vdtl od~nsdkdeatdrji tsadejo }ast ca~jo

k`oji~% ]AKD %

13>

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 124/130

XBd~ljs aeo Laesalae) 3=1=[ Bd~ljs) K% aeo Laesalae) ]% ,3=1=$% N`etj|t+ca~jo

~jasnl b`s >o k`oji~% De ANK ^DFFSA]L A~da 3=1= }a}js~) ^DFFSA]L A^DA —1=)

}afj~ 1<371„1<371=% ANK%

XBs`kj aeo Kaidh) 3==2[ Bs`kj) A% aeo Kaidh) G% ,3==2$% Dkafj sjtsdjrai aeo sjn`fed+

td`e z~def i`nai od~taenj bzentd`e~% ED]^%

XBs`kj jt ai%) 3==6[ Bs`kj) A%) ^defjs) T%) ^la) B%) aeo Kaidh) G% ,3==6$% Ijasedef

fi`caii+n`e~d~tjet i`nai od~taenj bzentd`e~ b`s ~la}j+ca~jo dkafj sjtsdjrai aeo

nia~~dfinatd`e% DNNR %

XBs`kj) 3==6[ Bs`kj) A% I% ,3==6$% Ijasedef I`nai Od~taenj Bzentd`e~ b`s J|jk}ias+

 Ca~jo @cgjnt Sjn`fedtd`e% ]lO tlj~d~) JJN^ Oj}astkjet) Zedrjs~dt `b Naidb`seda)

Cjshjij%

XFaiijfzdii`~ jt ai%) 3==<[ Faiijfzdii`~) N%) Sacde`rdnl) A%) aeo Cji`efdj) ^% ,3==<$%

@cgjnt natjf`sdwatd`e z~def n`+`nnzssjenj) i`natd`e aeo a}}jasaenj% JNNR %

XFz aeo Sje) 3=1=[ Fz) N% aeo Sje) \% ,3=1=$% Od~nsdkdeatdrj kd|tzsj+`b+tjk}iatj~

b`s rdjv}`det nia~~dfinatd`e% JNNR %

XLaijr jt ai%) 3==4[ Laijr) A%) E`srdf) ]%) aeo ]jsjdsa) B% ,3==4$% _lj zesja~`eacij

jbbjntdrjej~~ `b oata% Detjiidfjet ^~tjk~%

XLae~`e aeo Sd~jkae) 146<[ Lae~`e) A% aeo Sd~jkae) J% ,146<$% Rd~d`e~7 A n`k}ztjs

~~tjk b`s detjs}sjtdef ~njej~% N`k}ztjs Rd~d`e ^~tjk~) }afj~ >=>„>>>%

XLa~ aeo Jbs`~) 3==6[ La~) G% aeo Jbs`~) A% A% ,3==6$% ^njej n`k}ijtd`e z~def

kdiid`e~ `b }l`t`fsa}l~% ANK _sae~antd`e~ `e Fsa}ldn~ ,^DFFSA]L 3==6$) 32%

XLj jt ai%) 3==?[ Lj) \%) Wjkji) S% ^%) aeo Nassjdsa+]js}de¿ae) K% ¿ A% ,3==?$% Kzitd~naij

n`eodtd`eai saeo`k fijio~ b`s dkafj iacjidef% NR]S) }afj~ 24:„6=3%

13?

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 125/130

XLjstwkaee jt ai%) 3==1[ Ljstwkaee) A%) Gan`c~) N%) @idrjs) E%) Nzsij~~) C%) aeo ^aij~de)

O% ,3==1$% Dkafj aeai`fdj~% De ]s`njjodef~ `b ^DFFSA]L 3==1%

XL`djk jt ai%) 3==:[ L`djk) O%) Jbs`~) A% A%) aeo Ljcjst) K% ,3==:$% Azt`katdn }l`t`}`}+z}% ANK _sae~antd`e~ `e Fsa}ldn~ ,^DFFSA]L 3==:$) 3?,>$%

XLzkj) 16>4[ Lzkj) O% ,16>4$% A _sjatd~j `b Lzkae Eatzsj%

XG`le~`e jt ai%) 3=1=[ G`le~`e) K% H%) Oaij) H%) Ardoae) ^%) ]fi~tjs) L%) Bsjjkae) V% _%)

aeo Katz~dh) V% ,3=1=$% NF3sjai7 Dk}s`rdef tlj sjaid~k `b n`k}ztjs fjejsatjo

dkafj~ z~def a iasfj n`iijntd`e `b }l`t`fsa}l~% DJJJ _sae~antd`e~ `e Rd~zaidwatd`e

aeo N`k}ztjs Fsa}ldn~%

XHjkjikanljs+^lidwjskae jt ai%) 3=11[ Hjkjikanljs+^lidwjskae) D%) ^ljnltkae) J%)

Fasf) S%) aeo ^jdtw) ^% K% ,3=11$% J|}i`sdef }l`t`cd`~% ^DFFSA]L %

XHzkas aeo Ljcjst) 3==:[ Hzkas) ^% aeo Ljcjst) K% ,3==:$% A ldjsasnldnai fijio bsakj+

 v`sh b`s zedfijo n`etj|t+ca~jo nia~~dfinatd`e% DNNR %

XIad jt ai%) 3=11[ Iad) H%) C`) I%) Sje) \%) aeo B`|) O% ,3=11$% ^}as~j od~taenj ijasedef

b`s `cgjnt sjn`fedtd`e n`kcdedef sfc aeo oj}tl deb`skatd`e%

XIah`bb) 14<6[ Iah`bb) F% ,14<6$% V`kje) Bdsj) aeo Oaefjs`z~ _ldef~7 Vlat Natjf`sdj~

 Sjrjai Ac`zt tlj Kdeo%

XIjj aeo Fsazkae) 3=1=[ Ijj) T% G% aeo Fsazkae) H% ,3=1=$% @cgjnt+fsa}l~ b`s n`etj|t+

avasj natjf`s od~n`rjs% N`k}ztjs Rd~d`e aeo ]attjse Sjn`fedtd`e) DJJJ N`k}ztjs

^`ndjt N`ebjsjenj `e) =71„<%

XIjdcj jt ai%) 3==?[ Ijdcj) C%) Ij`easod~) A%) aeo ^nldjij) C% ,3==?$% N`kcdejo `c+

 gjnt natjf`sdwatd`e aeo ~jfkjetatd`e vdtl ae dk}idndt ~la}j k`oji% V`sh~l`} `e

^tatd~tdnai Ijasedef de N`k}ztjs Rd~d`e) JNNR %

13:

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 126/130

XId jt ai%) 3=1=[ Id) B%) Nassjdsa) G%) aeo ^kdenld~j~nz) N% ,3=1=$% @cgjnt sjn`fedtd`e a~

saehdef l`id~tdn fifzsj+fs`zeo l}`tlj~j~% NR]S%

XId jt ai%) 3=11[ Id) T%) Fz) I%) aeo Haeaoj) _% ,3=11$% S`cz~ti aidfedef a ~la}j k`ojiaeo dt~ a}}idnatd`e t` nas aidfekjet `b zehe`ve }`~j% ]attjse Aeai~d~ aeo Kanldej

 Detjiidfjenj) DJJJ _sae~antd`e~ `e) ]],44$71%

XIdz jt ai%) 3==4[ Idz) N%) Tzje) G%) aeo _`ssaica) A% ,3==4$% E`e}asakjtsdn ~njej

}as~def7 Iacji tsae~bjs rda oje~j ~njej aidfekjet% NR]S%

XI`nhj) 12<4[ I`nhj) G% ,12<4$% Ae J~~a N`enjsedef Lzkae Zeojs~taeodef%

XKaid~djvdnw aeo Jbs`~) 3==6[ Kaid~djvdnw) _% aeo Jbs ~) A% A% ,3==6$% Dk}s`rdef

~}atdai ~z}}`st b`s `cgjnt~ rda kzitd}ij ~jfkjetatd`e~% CKRN %

XKaid~djvdnw aeo Jbs`~) 3==<[ Kaid~djvdnw) _% aeo Jbs`~) A% A% ,3==<$% Sjn`fedtd`e c 

a~~`ndatd`e rda ijasedef }js+j|jk}ias od~taenj~% NR]S%

XKaid~djvdnw aeo Jbs`~) 3==4[ Kaid~djvdnw) _% aeo Jbs`~) A% A% ,3==4$% Cj`eo natj+

f`sdj~7 _lj rd~zai kjkj| k`oji b`s sja~`edef ac`zt `cgjnt sjiatd e~ld}~% ED]^%

XKaid~djvdnw jt ai%) 3=11[ Kaid~djvdnw) _%) Fz}ta) A%) aeo Jbs`~) A% A% ,3=11$% Je~jkcij

`b j|jk}ias+~rk~ b`s `cgjnt ojtjntd`e aeo cj`eo% DNNR %

XKjode aeo ^nlabbjs) 146<[ Kjode) O% I% aeo ^nlabbjs) K% ,146<$% N`etj|t tlj`s `b 

nia~~dfinatd`e ijasedef% ]~nl`i`fdnai Sjrdjv) <:73=6„3><%

XKzs}l) 3==3[ Kzs}l) F% I% ,3==3$% _lj Cdf C``h `b N`enj}t~% _lj KD_ ]sj~~%

XE`~`b~h) 14<2[ E`~`b~h) S% K% ,14<2$% Attjetd`e) ~dkdiasdt) aeo tlj dojetdfinatd`e+

natjf`sdwatd`e sjiatd`e~ld}% G`zseai `b J|}jsdkjetai ]~nl`i`f7 Fjejsai) 11:,1$7>4„

:6%

132

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 127/130

X]afjt aeo I`ef~tabb) 144<[ ]afjt) S% aeo I`ef~tabb) D% O% ,144<$% _j|tzsj ~etlj~d~ rda

a e`enaz~ai e`e}asakjtsdn kzitd~naij kash`r saeo`k fijio% DJJJ _sae~antd`e~ `e

 Dkafj ]s`nj~~def%

X]asdhl jt ai%) 3==<[ ]asdhl) O%) Wdtednh) N% I%) aeo Nlje) _% ,3==<$% Bs`k a}}jasaenj

t` n`etj|t+ca~jo sjn`fedtd`e7 Oje~j iacjidef de ~kaii dkafj~% NR]S%

X]ash jt ai%) 3=1=[ ]ash) O%) Sakaeae) O%) aeo B`vihj~) N% ,3=1=$% Kzitdsj~`iztd e

k`oji~ b`s `cgjnt ojtjntd`e% JNNR %

X]js`ea) 3=1=[ ]js`ea) ]% ,3=1=$% Rd~d`e `b a rd~d}joda% ]s`njjodef~ `b tlj DJJJ %

X]iatt) 1444[ ]iatt) G% N% ,1444$% ]s`cacdid~tdn `zt}zt~ b`s ~z}}`st rjnt`s kanldej~ aeo

n`k}asd~`e~ t` sjfziasdwjo idhjidl``o kjtl`o~% Aoraenj~ de Iasfj Kasfde Nia~~dfijs~%

XSacde`rdnl jt ai%) 3==6[ Sacde`rdnl) A%) Rjoaiod) A%) Faiijfzdii`~) N%) Vdjv`sa) J%) aeo

Cji`efdj) ^% ,3==6$% @cgjnt~ de n`etj|t% DNNR %

XSakaeae aeo Cahjs) 3=11[ Sakaeae) O% aeo Cahjs) ^% ,3=11$% I`nai od~taenj bzen+

td`e~7 A ta|`e`k) ejv afi`sdtlk~) aeo ae jraizatd`e%

XSje jt ai%) 3==:[ Sje) I%) ]atsdnh) A%) Jbs`~) A% A%) L`ofde~) G% H%) aeo Sjlf) G% K%

,3==:$% A oata+osdrje a}}s`anl t` qzaetdbdef eatzsai lzkae k`td`e% ANK _sae~an+

td`e~ `e Fsa}ldn~ ,^DFFSA]L 3==:$) 3?,>$71=4=„1=46%

XS`~nl) 146<[ S`~nl) J% ,146<$% ]sdend}ij~ `b natjf`sdwatd`e% N`fedtd`e aeo Natjf`sdwa+

td`e) }afj~ 36„?<%

XS`~nl aeo Kjsrd~) 146:[ S`~nl) J% aeo Kjsrd~) N% C% ,146:$% Bakdi sj~jkciaenj~7

^tzodj~ de tlj detjseai ~tszntzsj `b natjf`sdj~% N`fedtdrj ]~nl`i`f ) 67:6>„2=:%

136

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 128/130

XSz~~jii jt ai%) 3==2[ Sz~~jii) C%) Jbs`~) A% A%) ^drdn) G%) Bsjjkae) V%) aeo Wd~~jskae) A%

,3==2$% Z~def kzitd}ij ~jfkjetatd`e~ t` od~n`rjs `cgjnt~ aeo tljds j|tjet de dkafj

n`iijntd`e~% NR]S%

XSz~~jii jt ai%) 3==4[ Sz~~jii) C%) Jbs`~) A% A%) ^drdn) G%) Bsjjkae) V% _%) aeo Wd~~jskae)

 A% ,3==4$% ^jfkjetdef ~njej~ c katnldef dkafj n`k}`~dtj~% ED]^%

XSz~~jii jt ai%) 3==<[ Sz~~jii) C%) _`ssaica) A%) Kzs}l) H%) ) aeo Bsjjkae) V% _% ,3==<$%

Iacjikj7 a oataca~j aeo vjc+ca~jo t``i b`s dkafj aee`tatd`e% DGNR %

XSz~~jii jt ai%) 3==6[ Sz~~jii) C% N%) _`ssaica) A%) Idz) N%) Bjsfz~) S%) aeo Bsjjkae) V% _%

,3==6$% @cgjnt sjn`fedtd`e c ~njej aidfekjet% ED]^%

X^arasj~j aeo Bjd+Bjd) 3==6[ ^arasj~j) ^% aeo Bjd+Bjd) I% ,3==6$% >o fjejsdn `cgjnt

natjf`sdwatd`e) i`naidwatd`e aeo }`~j j~tdkatd`e% DNNR %

X^nlaeh) 14<>[ ^nlaeh) S% N% ,14<>$% Oeakdn Kjk`s7 A _lj`s `b Sjkdeodef aeo

 Ijasedef de N`k}ztjs~ aeo ]j`}ij% Nakcsdofj Zedrjs~dt ]sj~~%

X^nl`oi jt ai%) 3===[ ^nl`oi) A%) ^wjid~hd) S%) ^aij~de) O% L%) aeo J~~a) D% ,3===$% Rdoj`

tj|tzsj~% De ]s`njjodef~ `b ^DFFSA]L —==%

X^nlih`}b jt ai%) 3==1[ ^nlih`}b) C%) ]iatt) G% N%) ^lavj+_ai`s) G%) ^k`ia) A% G%) aeo

Vdiidak~`e) S% N% ,3==1$% J~tdkatdef tlj ~z}}`st `b a ldfl+odkje~d`eai od~tsdcztd`e%

 Ejzsai N`k}ztatd`e%

X^ld aeo Kaidh) 3===[ ^ld) G% aeo Kaidh) G% ,3===$% E`skaidwjo nzt~ aeo dkafj ~jf+

kjetatd`e% ]AKD ) 33,<$%

X^ldsh) 3==:[ ^ldsh) N% ,3==:$% @et`i`f d~ `rjssatjo7 Natjf`sdj~) ideh~) aeo taf~%

13<

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 129/130

X^l`tt`e jt ai%) 3==2[ ^l`tt`e) G%) Vdee) G% K%) S`tljs) N%) aeo Nsdkded~d) A% ,3==2$%

_j|t`ec``~t7 G`det a}}jasaenj) ~la}j aeo n`etj|t k`ojidef b`s kzitd+nia~~ `cgjnt

sjn`fedtd`e aeo ~jfkjetatd`e% JNNR %

X^lsdra~tara jt ai%) 3=11[ ^lsdra~tara) A%) Kaid~djvdnw) _%) Fz}ta) A%) aeo Jbs`~) A% A%

,3=11$% Oata+osdrje rd~zai ~dkdiasdt b`s ns`~~+o`kade dkafj katnldef% ^zckdttjo t`

^DFFSA]L A~da%

X^drdn aeo Wd~~jskae) 3==>[ ^drdn) G% aeo Wd~~jskae) A% ,3==>$% Rdoj` f``fij7 A tj|t

sjtsdjrai a}}s`anl t` `cgjnt katnldef de rdoj`~% De DNNR ) }afj~ 1?6=„) Va~ldeft`e)

ON) Z^A% DJJJ N`k}ztjs ^`ndjt%

X^hafj~tao) 1442[ ^hafj~tao) ]% ,1442$% _lj kdeo—~ kanldej~7 _lj tzsdef kanldej)

tlj kjkj|) aeo tlj }js~`eai n`k}ztjs% ^JKD@_DNA%

X^tsat aeo Bd~nlijs) 1441[ ^tsat) _% aeo Bd~nlijs) K% ,1441$% N`etj|t+ca~jo rd~d e7

Sjn`fedwdef `cgjnt~ z~def deb skatd`e bs`k c`tl 3+o aeo >+o dkafjs% ]AKD ) 1>71=:=„

1=2:%

X_l`ka~ jt ai%) 3==4[ _l`ka~) A%) Bjssasd) R%) Ijdcj) C%) _ztjiaas~) _%) aeo F``i) I% G% R%

,3==4$% ^la}j+bs`k+sjn`fedtd`e7 Sjn`fedtd`e jeacij~ kjta+oata tsae~bjs% NRDZ %

X_`ssaica) 3==>a[ _ ssaica) A% ,3==>a$% _lj n`etj|t nlaiijefj%

ltt}7//vjc%kdt%joz/t`ssaica/vvv/nas~AeoBanj~DeN`etj|t%ltki %

X_`ssaica) 3==>c[ _`ssaica) A% ,3==>c$% N`etj|tzai }sdkdef b`s `cgjnt ojtjntd`e% Detjs+

eatd`eai G`zseai `b N`k}ztjs Rd~d`e) :>7124„141%

X_`ssaica jt ai%) 3==<[ _`ssaica) A%) Bjsfz~) S%) aeo Bsjjkae) V% _% ,3==<$% <= kdiid`e

tde dkafj~7 a iasfj oata~jt b`s e`e+}asakjtsdn `cgjnt aeo ~njej sjn`fedtd`e% ]AKD )

>=%

134

8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…

http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 130/130

XRjoaiod jt ai%) 3==4[ Rjoaiod) A%) Fzi~lae) R%) Raska) K%) aeo Wd~~jskae) A% ,3==4$%

Kzitd}ij hjseji~ b`s `cgjnt ojtjntd e% DNNR %

XValca) 146>[ Valca) F% ,146>$% A nia~~ `b a}}s`|dkatj ~`iztd`e~ t` idejas `}jsat`sjqzatd`e~% G`zseai `b A}}s`|dkatd`e _lj`s ) 4,1$721 „ 66%

XVjdecjsfjs) 3==6[ Vjdecjsfjs) O% ,3==6$% Jrjstldef d~ kd~njiiaej`z~ 7 tlj }`vjs `b 

tlj ejv odfdtai od~`sojs% Ejv T`sh 7 _dkj~ C``h~%

XVdttfje~tjde) 14:>[ Vdttfje~tjde) I% ,14:>$% ]ldi`~`}ldnai Derj~tdfatd`e~% Cianhvjii

]zcid~ldef%

X\da` jt ai%) 3=1=[ \da`) G%) La~) G%) Jldefjs) H%) @idra) A%) aeo _`ssaica) A% ,3=1=$%

^ze oataca~j7 Iasfj+~naij ~njej sjn`fedtd`e bs`k accj t` w``% NR]S%

X\z jt ai%) 3==2[ \z) I%) Nsakkjs) H%) aeo ^nlzzskae~) O% ,3==2$% S`cz~t ~z}}`st

 rjnt`s kanldej tsadedef rda n`erj| `ztidjs aciatd`e% AAAD %

XWlaef jt ai%) 3==2[ Wlaef) L%) Cjsf) A%) Kadsj) K%) aeo Kaidh) G% ,3==2$% ^RK+HEE7

Od~nsdkdeatdrj ejasj~t ejdflc`s nia~~dfinatd`e b`s rd~zai natjf`s sjn`fedtd`e% NR]S)