CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond -...
Transcript of CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond -...
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 1/130
J|jk}ias+ca~jo Sj}sj~jetatd`e~ b`s@cgjnt Ojtjntd`e) A~~`ndatd`e aeo
Cj`eo
_`ka~w Kaid~djvdnw
NKZ+SD+_S+11+>3
^zckdttjo de }astdai bzifiiikjet `b tljsjqzdsjkjet~ b`s tlj ojfsjj `b
O`nt`s `b ]ldi`~`}l de S`c`tdn~
_lj S`c`tdn~ De~tdtztjNasejfdj Kjii`e Zedrjs~dt
]dtt~czsfl) ]jee~iraeda 1:31>
Azfz~t 3=11
_lj~d~ N`kkdttjj7]s`bj~~`s Aij|jd A% Jbs`~) Nlads
]s`bj~~`s Kastdai Ljcjst
]s`bj~~`s _ahj` Haeaoj]s`bj~~`s ]djts` ]js`ea) Naitjnl
n 3=11 c _`ka~w Kaid~djvdnw% Aii sdflt~ sj~jsrjo%
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 2/130
3
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 3/130
B`s tlj rjnt`s~)
b`s ~z}}`stdef k sj~jasnl aii tlj~j jas~%
>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 4/130
Ac~tsant
Sjn`fedwdef aeo sja~`edef ac`zt tlj `cgjnt~ b`zeo de ae dkafj d~ `ej `b tlj hj
}s`cijk~ de n`k}ztjs rd~d`e% _ld~ tlj~d~ d~ ca~jo `e tlj doja tlat de `sojs t` zeojs~taeo
a e`rji `cgjnt) dt d~ `btje e`t je`zfl t` sjn`fedwj tlj `cgjnt natjf`s dt cji`ef~ t` ,d%j%)
ae~vjsdef ’Vlat d~ tld~0‛$% Vj asfzj tlat a k`sj kjaedefbzi detjs}sjtatd`e nae cj
`ctadejo c idehdef tlj de}zt `cgjnt vdtl a ~dkdias sj}sj~jetatd`e de kjk`s ,d%j%) a~hdef
’Vlat d~ tld~ idhj0‛$% De tld~ tlj~d~) vj }sj~jet a kjk`s+ca~jo ~~tjk b`s sjn`fedwdef
aeo detjs}sjtdef `cgjnt~ de dkafj~ c j~tacid~ldef rd~zai a~~`ndatd`e~ cjtvjje ae de}zt
dkafj aeo a iasfj oataca~j `b `cgjnt j|jk}ias~% _lj~j rd~zai a~~`ndatd`e~ nae tlje
cj z~jo t` }sjodnt }s`}jstdj~ `b tlj e`rji `cgjnt vldnl naee`t cj ojoznjo ~`iji bs`k
natjf`s kjkcjs~ld} ,j%f%) vldnl va d~ dt bandef0 vlat d~ dt~ ~jfkjetatd`e0 d~ tljsj a
}js~`e ~dttdef `e dt0$%
]ast D `b tld~ tlj~d~ d~ ojodnatjo t` j|jk}ias sj}sj~jetatd`e~ aeo aif`sdtlk~ b`s
nsjatdef rd~zai a~~`ndatd`e~% Vj }s`}`~j I`nai Od~taenj Bzentd`e~ aeo J|jk}ias+^RK~)
vldnl asj tsadejo ~j}asatji b`s janl j|jk}ias aeo aii`v ae de~taenj+~}jndfin e`td`e `b
rd~zai ~dkdiasdt% Vj ~l`v tlat ae je~jkcij `b J|jk}ias+^RK~ }jsb`sk~ n`k}jtdtdrji
t` ~tatj+`b+tlj+ast `e tlj ]A^NAI R@N `cgjnt ojtjntd`e ta~h% De ]ast DD) vj b`nz~ `e
tlj aoraetafj~ `b z~def j|jk}ias~ `rjs a }zsji natjf`s+ca~jo a}}s`anl% Cjnaz~j
J|jk}ias+^RK~ ~l`v f``o aidfekjet cjtvjje ojtjntd`e vdeo`v~ aeo tljds a~~`ndatjo
j|jk}ias~) vj ~l`v tlat dt d~ }`~~dcij t` tsae~bjs ae aradiacij j|jk}ias kjta+oata
,~jfkjetatd`e) fj`kjtsdn ~tszntzsj) >O k`oji) jtn%$ odsjnti `et` tlj ojtjntd`e~) vldnl
nae tlje cj z~jo a~ }ast `b `rjsaii ~njej zeojs~taeodef% Bdeaii) vj n`e~tsznt a Rd~zai
Kjkj|) a ra~t fsa}l `rjs j|jk}ias~ jen`odef c`tl rd~zai a~ vjii a~ ~}atdai sjiatd`e~ld}~)
aeo a}}i dt t` ae `cgjnt }sjodntd`e ta~h% @zs sj~zit~ ~l`v tlat j|jk}ias~ }s`rdoj a
cjttjs e`td`e `b `cgjnt n`etj|t tlae natjf`s+ca~jo a}}s`anlj~%
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 5/130
Id~t `b Bdfzsj~
1%1 Rd~zai ^e`ek~ aeo ] i~jk % % % % % % % % % % % % % % % % % % % % % % % 3=
3%1 Ijasedef a od~taenj bzentd`e de od~taenj+t`+j|jk}ias rjnt`s ~}anj % % % % 34
3%3 J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 1/3 % % % % % % % % % % >6
3%> J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 3/3 % % % % % % % % % % ><
3%? IacjiKj ^jfkjet Iacjidef7 Jraizatdef Ijasejo Od~taenj Bzentd`e~ % % ?1
3%: J|jk}ias A~~ ndatd`e de tlj IacjiKj tj~t ~jt % % % % % % % % % % % % % % % % ??
3%2 Ojtjntdef aeo ^jfkjetdef `cgjnt~ de IacjiKj% % % % % % % % % % % % % % % % ?:
3%6 Dkafj ]as~def rda Kzitd}ij ^jfkjetatd`e~ aeo I`nai Od~taenj Bzentd`e~ ?6
>%1 @cgjnt Natjf`s Ojtjnt`s r~% Je~jkcij b J|jk}ias Ojtjnt`s~% % % % % % % ?4
>%3 Natjf`s ^RK r~% J|jk}ias+^RK~ % % % % % % % % % % % % % % % % % % % % % :=
>%> L@F katnldef n`k}asd~`e % % % % % % % % % % % % % % % % % % % % % % % % % :1
>%? Ijasedef ae J|jk}ias+~}jndfin L@F _jk}iatj % % % % % % % % % % % % % % % :3
>%: A bjv ’tsade‛ J|jk}ias+^RK~ % % % % % % % % % % % % % % % % % % % % % % % % :?
>%2 J|jk}ias+^RK naidcsatd`e % % % % % % % % % % % % % % % % % % % % % % % % % % :4
>%6 @cgjnt Ojtjntd e aeo A}}jasaenj _sae~bjs % % % % % % % % % % % % % % % % % 2>
>%< Rasdef ejfatdrj ~jt ~dwj~ % % % % % % % % % % % % % % % % % % % % % % % % % % 24
:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 6/130
>%4 ]A^NAI R@N Sj~zit~ r~% Ezkcjs `b J|jk}ias~ % % % % % % % % % % % % % % 63
>%1= ]A^NAI R@N Sj~zit~ b`s Rasdef L@F _jk}iatj ^dwj~ % % % % % % % % % % % 6?
?%1 ^jfkjetatd`e _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % % % % % 64
?%3 Qzaidtatdrj Fj`kjts _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % <=
?%> >O K`oji _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % <1
?%? Sjiatjo @cgjnt ]sdkdef % % % % % % % % % % % % % % % % % % % % % % % % % % % % <3
?%: ^znnj~~bzi J|jk}ias _sae~bjs~ 1 % % % % % % % % % % % % % % % % % % % % % % % <?
?%2 ^znnj~~bzi J|jk}ias _sae~bjs~ 3 % % % % % % % % % % % % % % % % % % % % % % % <:
?%6 J|jk}ias _sae~bjs Badizsj~ 1 % % % % % % % % % % % % % % % % % % % % % % % % % <2
?%< J|jk}ias _sae~bjs Badizsj~ 3 % % % % % % % % % % % % % % % % % % % % % % % % % <6
?%4 _sade N`enj}t Rd~zai Kjkj| % % % % % % % % % % % % % % % % % % % % % % % % % 4=
?%1= K`t`scdhj N`enj}t Rd~zai Kjkj| % % % % % % % % % % % % % % % % % % % % % % 41
?%11 Ajs`}iaej N`enj}t Rd~zai Kjkj| % % % % % % % % % % % % % % % % % % % % % % 43
:%1 _lj Rd~zai Kjkj| fsa}l % % % % % % % % % % % % % % % % % % % % % % % % % % % 4:
:%3 _ ssaica—~ N`etj|t Nlaiijefj % % % % % % % % % % % % % % % % % % % % % % % % % 42
:%> N`etj|t Nlaiijefj Jraizatd`e % % % % % % % % % % % % % % % % % % % % % % % % 1=3
:%? Qzaidtatdrj Sj~zit~ `e tlj N`etj|t Nlaiijefj % % % % % % % % % % % % % % % % 1=>
A%1 Ns`~~+O`kade Dkafj Katnldef % % % % % % % % % % % % % % % % % % % % % % % 1=4
A%3 ^hjtnl aeo ]adetdef Katnldef % % % % % % % % % % % % % % % % % % % % % % % % 111
A%> ^hjtnl3Dkafj % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 113
A%? ]adetdef3Dkafj % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 113
A%: ]adetdef3F]^ Qzaidtatdrj J|ak}ij~ % % % % % % % % % % % % % % % % % % % % % 113
2
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 7/130
Id~t `b _acij~
3%1 ^jfkjet Oj~nsd}t`s7 1? Jijkjetas Bjatzsj~ % % % % % % % % % % % % % % % % >2
3%3 Iacji n`ebz~d`e abtjs od~taenj bzentd e ijasedef % % % % % % % % % % % % % % >4
>%1 J|jk}ias+^RK @cgjnt Ojtjntd`e Sj~zit~ `e ]A^NAI R@N 3==6% % % % % % 2?
>%3 J|jk}ias+^RK sj~zit~ vlje denizodef de+nia~~ dkafj~ ozsdef ejfatdrj
kdedef % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 26
>%> J|jk}ias+^RK sj~zit~ vdtl 3: ejfatdrj~ % % % % % % % % % % % % % % % % % % 6=
>%? J|jk}ias+^RK sj~zit~ vdtl 3:= ejfatdrj~ % % % % % % % % % % % % % % % % % 6=
>%: J|jk}ias+^RK sj~zit~ vdtl 1=== ejfatdrj~ % % % % % % % % % % % % % % % % % 6=
>%2 J|jk}ias+^RK sj~zit~ vdtl 3:== ejfatdrj~ % % % % % % % % % % % % % % % % % 6=
>%6 J|jk}ias+^RK sj~zit~ vdtl tde L@F tjk}iatj~ % % % % % % % % % % % % % % 6:
>%< J|jk}ias+^RK sj~zit~ vdtl ~kaii L@F tjk}iatj~ % % % % % % % % % % % % % 6:
>%4 J|jk}ias+^RK sj~zit~ vdtl ojbazit+~dwjo L@F tjk}iatj~ % % % % % % % % % 6:
>%1= J|jk}ias+^RK sj~zit~ vdtl iasfj L@F tjk}iatj~ % % % % % % % % % % % % % 6:
?%1 D~ tljsj a }js~`e sdodef tld~ l`s~j0 % % % % % % % % % % % % % % % % % % % % % <>
6
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 8/130
N`etjet~
1 Dets`ozntd`e 1>
1%1 Canhfs`zeo `e Natjf sdwatd e _lj`sdj~ % % % % % % % % % % % % % % % % % % 12
1%1%1 ]s`cijk~ Vdtl Rd~zai Natjf`sdj~ % % % % % % % % % % % % % % % % % % 14
1%1%3 Vl Natjf`sdwj0 % % % % % % % % % % % % % % % % % % % % % % % % % % % 3=
1%3 Sjiatjo V`sh % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 31
1%> _lj~d~ @rjsrdjv % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 3?
]ast D7 Ijasedef J|jk}ias Sj}sj~jetatd`e~ 3:
3 I`nai Od~taenj Bzentd`e~ 36
3%1 Ijasedef Od~taenj Bzentd`e~ Vdtl Bsdjeo~ % % % % % % % % % % % % % % % % % 34
3%3 Sjiatjo V`sh % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % >>
3%> Jraizatdef I`nai Od~taenj Bzentd`e~ `e IacjiKj % % % % % % % % % % % % % % >?
3%>%1 ^jfkjet+ca~jo Bjatzsj~ % % % % % % % % % % % % % % % % % % % % % % % >:
3%>%3 Od~taenj Bzentd`e Ijasedef Ojtadi~ % % % % % % % % % % % % % % % % % >:
3%>%> ^jfkjet Iacjidef % % % % % % % % % % % % % % % % % % % % % % % % % % ?=
3%>%? ]as~def Dkafj~ rda Kzitd}ij ^jfkjetatd`e~ % % % % % % % % % % % % ?1
3%>%: B`ii`v+z} V`sh % % % % % % % % % % % % % % % % % % % % % % % % % % % % ?:
<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 9/130
> J|jk}ias+^RK~ ?<
>%1 Vl` Ejjo~ bsdjeo~0 Ijasedef Vdtl a ^defij ] ~dtdrj De~taenj % % % % % % :3
>%1%1 Sjiatd`e~ld} t` I`nai Od~taenj Bzentd`e~ % % % % % % % % % % % % % % ::
>%1%3 Detjs}sjtdef ^RK~ tsadejo vdtl a ~defij }`~dtdrj de~taenj % % % % % :2
>%3 Naidcsatd e % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % :6
>%> J|jk}ias N`+`nnzssjenj Katsdnj~ % % % % % % % % % % % % % % % % % % % % % % :4
>%? Jraizatd`e `b J|jk}ias+^RK~ % % % % % % % % % % % % % % % % % % % % % % % % 21
>%?%1 ]A^NAI R@N @cgjnt Ojtjntd e _a~h % % % % % % % % % % % % % % % % % 23
>%: J|jk}ias+^RK Aeai~d~ % % % % % % % % % % % % % % % % % % % % % % % % % % % 22
>%:%1 Denizodef ^akj+Natjf`s De~taenj~ de tlj Ejfatdrj ^jt % % % % % % 22
>%:%3 ^dwj `b Ejfatdrj ^jt % % % % % % % % % % % % % % % % % % % % % % % % % 2<
>%:%> Oj}jeojenj @e tlj Ezkcjs `b J|jk}ias~ % % % % % % % % % % % % % 61
>%:%? _lj S ij b Bjatzsj Odkje~d`eaidt % % % % % % % % % % % % % % % % % 6>
]ast DD7 Cj`eo @cgjnt Ojtjntd e 62
? A~~`ndatd`e aeo Kjta+oata _sae~bjs 6<
?%1 ^jfkjetatd`e aeo Fj`kjts _sae~bjs % % % % % % % % % % % % % % % % % % % % 64
?%3 >O K`oji _sae~bjs % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % <=
?%> Sjiatjo @cgjnt ]sdkdef % % % % % % % % % % % % % % % % % % % % % % % % % % % % <3
?%? J|jk}ias De+}adetdef % % % % % % % % % % % % % % % % % % % % % % % % % % % % % <>
?%: Rd~zaidwdef J|jk}ias+J|jk}ias A~~`ndatd`e~ % % % % % % % % % % % % % % % % <<
: N`etj|tzai @cgjnt ]sjodntd`e 4>
:%1 Czdiodef a IacjiKj Rd~zai Kjkj| Fsa}l % % % % % % % % % % % % % % % % % % 4:
4
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 10/130
:%3 _ ssaica—~ N`etj|t Nlaiijefj % % % % % % % % % % % % % % % % % % % % % % % % % 42
:%> Debjsjenj de tlj Rd~zai Kjkj| K`oji % % % % % % % % % % % % % % % % % % % % 4<
:%? N`etj|t Nlaiijefj Jraizatd`e7 Rd~zai Kjkj| rjs~z~ Natjf`sdj~ % % % % % % 1 = 3
:%?%1 B`ii`v+z} V`sh % % % % % % % % % % % % % % % % % % % % % % % % % % % % 1=:
2 N`eniz~d`e7 _ vaso tlj Rd~zai Kjkj| 1=2
A Ns`~~+O`kade Dkafj Katnldef 1=4
C J|jk}ias+^RK~7 Ae @}je ^`zsnj Dk}ijkjetatd e 11>
C%1 J|jk}ias+^RK _sadedef ]asakjtjs~ % % % % % % % % % % % % % % % % % % % % % 11?
C%3 ^idodef Vdeo`v~ aeo Bjatzsj ]sakdo~ % % % % % % % % % % % % % % % % % % % 11?
Kzitd~naij L@F ]sakdo % % % % % % % % % % % % % % % % % % % % % % % 112
I`naidwdef kae j|jk}ias~ rda a ~defij katsd| kzitd}idnatd`e % % 114
C%> J|jk}ias+^RK sze+tdkj n`k}ij|dt % % % % % % % % % % % % % % % % % % % % % 13=
1=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 11/130
11
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 12/130
Anhe`vijofjkjet~
D v`zio fis~t idhj t` tlaeh k aord~`s) Aij|jd ,Ai`~la$ Jbs`~) b`s }s`rdodef kj vdtl tlj
jen`zsafjkjet t` }zs~zj tlj cdf }s`cijk~ de n`k}ztjs rd~d`e% E`t `ei la~ lj cjje a
kjet`s) czt lj ai~` }iajo ~jrjsai `tljs hj s`ij~ de k ~ndjetdfin ojrji`}kjet7 a bsdjeo)
a nsdtdn) a rd~d`eas) a }ldi`~`}ljs) aeo k`~t dk}`staeti) a n`iiac`sat`s%
Dt v`zio cj ~dii t` tldeh tlat k osjak `b czdiodef detjiidfjet kanldej~ ~tastjo
vlje D jetjsjo fsaozatj ~nl``i% Cjb`sj n`kdef t` NKZ) k bakdi jen`zsafjo kj
t` }zs~zj k osjak~) aeo tlz~ e`ej `b tld~ v`sh v`zio larj cjje }`~~dcij vdtl`zt
tlj ~z}}`st `b k bakdi) j~}jndaii k k`tljs W`fia Kaid~djvdnw aeo k cs`tljs Katt
Kaid~djvdnw%
D v`zio ai~` idhj t` tlaeh k bjii`v S`c`fsao~ b`s kahdef fsaozatj ~nl``i tszi
akawdef% De }astdnzias) D v`zio idhj t` tlaeh Fj`bb L`iidefjs) C`sd~ ^`bkae) G`eatlae
Lzaef) Kash ]aiatznnd) aeo Laee~ _a}}jdejs b`s lji}def kj sjaidwj tlat tlj 3==:
S`c`fsao~ vjsj ae akawdef czenl% _lj `tljs rd~d`easdj~ ,^aet`~l Odrraia) ^azsacl
^defl) Acldear ^lsdra~tara$ lji}jo kahj VAS] lanhdef jeg`acij aeo vdtl`zt tljds
lji} aeo vdiidefej~~ t` id~tje t` k }s`cijk~) tl`~j detje~j iatj+edflt ojczffdef
~j~~d`e~ v`zio e`t larj cjje }`~~dcij% Aeo fieaii) D v`zio idhj t` tlaeh Gjeedbjs Cs`ve
vdtl`zt vl`k k tdkj at NKZ v`zio larj cjje ~dfedfinaeti ij~~ jeg`acij%
Sjfasodef bzeodef) D v`zio idhj t` tlaeh tlj Eatd`eai ^ndjenj B`zeoatd`e) b`s
avasodef kj tlj E^B Fsaozatj Sj~jasnl Bjii`v~ld}) vldnl lji}jo bzeo k sj~jasnl
bs`k 3==2 zetdi 3==4% _lj fieai jas `b k sj~jasnl at NKZ va~ kaoj }`~~dcij c a
sj~jasnl avaso bs`k tlj K^S+NKZ Njetjs b`s N`k}ztatd`eai _ldehdef%
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 13/130
Nla}tjs 1
Dets`ozntd`e
’@zs dej}tdtzoj de fjttdef at tlj sjn`so d~ iasfji naz~jo c tlj astdfindaidt `b
~~tjk~ `b deoj|def% Vlje oata `b ae ~`st asj }ianjo de ~t`safj) tlj asj fiijo
ai}lacjtdnaii `s ezkjsdnaii) aeo deb`skatd`e d~ b`zeo ,vlje dt d~$ c tsandef dt
o`ve bs`k ~zcnia~~ t` ~zcnia~~%%% _lj lzkae kdeo o`j~ e`t v`sh tlat va% Dt
`}jsatj~ c a~~`ndatd`e% Vdtl `ej dtjk de dt~ fsa~}) dt ~ea}~ de~taeti t` tlj ej|t
tlat d~ ~zffj~tjo c tlj a~~`ndatd`e `b tl`zflt~) de ann`soaenj vdtl ~`kj detsdnatj
vjc `b tsadi~ nassdjo c tlj njii~ `b tlj csade%%% ^jijntd`e c a~~`ndatd`e) satljs tlae
c deoj|def) ka jt cj kjnlaedwjo% @ej naee`t l`}j tlz~ t` jqzai tlj ~}jjo aeo
ffij|dcdidt vdtl vldnl tlj kdeo b`ii`v~ ae a~~`ndatdrj tsadi) czt dt ~l`zio cj }`~~dcij
t` cjat tlj kdeo ojnd~drji de sjfaso t` tlj }jskaejenj aeo niasdt `b tlj dtjk~
sj~zssjntjo bs`k ~t`safj% N`e~dojs a bztzsj ojrdnj b`s deodrdozai z~j) vldnl d~ a
~`st `b kjnlaedwjo }sdratj fiij aeo idcsas% Dt ejjo~ a eakj) aeo t` n`de `ej at
saeo`k) ’kjkj|‛ vdii o`% A kjkj| d~ a ojrdnj de vldnl ae deodrdozai ~t`sj~ aii
ld~ c``h~) sjn`so~) aeo n`kkzednatd`e~) aeo vldnl d~ kjnlaedwjo ~` tlat dt ka
cj n`e~zitjo vdtl j|njjodef ~}jjo aeo ffij|dcdidt% Dt d~ ae jeiasfjo detdkatj~z}}ijkjet t` ld~ kjk`s%
Raeejras Cz~l) A~ Vj Ka _ldeh ,14?:$
1>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 14/130
Raeejras Cz~l—~ rd~d`e) a~ j|}`zeojo de tlj ac`rj j|njs}t bs`k ld~ ~jkdeai 14?:
j~~a) ’A~ vj ka tldeh‛ XCz~l) 14?:[) ~tjk~ bs`k tlj sjaidwatd`e tlat tljsj d~ a fsjat
kd~katnl cjtvjje tlj va de vldnl deb`skatd`e d~ t}dnaii `sfaedwjo aeo tlj va de
vldnl tlj lzkae kdeo v`sh~ ,d%j%) natjf`sdnai deoj|def rjs~z~ a~~`ndatd`e$% N`enjsejo
vdtl tlj tsae~kd~~d`e aeo annj~~dcdidt `b ~ndjetdfin doja~) Cz~l bazitjo tlj ’astdfindaidt `b
~~tjk~ `b deoj|def‛ aeo }s`}`~jo tlj Kjkj|1) a }l~dnai ojrdnj vldnl v`zio cj z~jo
t` sjtsdjrj deb`skatd`e ca~jo `e a~~`ndatdrj ideh~ de~tjao `b ~tsdnt natjf`sdnai deoj|def%
_lj a~~`ndatdrj ideh~ vjsj t` cj jetjsjo kaezaii c tlj z~js aeo n`zio cj `b ~jrjsai
odbbjsjet t}j~% Nlade~ `b ideh~ v`zio b`sk det` i`efjs a~~`ndatdrj tsadi~) nsjatdef ejv
eassatdrj~ de tlj n`enj}t ~}anj% Cz~l cjidjrjo tlat tlj Kjkj| v`zio cj deraizacij t` a
~ndjetd~t) antdef a~ ae j|tjeojo ~kast kjk`s vldnl v`zio e`t `ei ~t`sj tlj a~~`ndatdrj
tsadi~ j~tacid~ljo ozsdef s`ztdej sj~jasnl) czt n`zio ai~` azt`katdnaii ~zffj~t e`rji
a~~`ndatd`e~ vldnl kdflt e`t cj `crd`z~ t` a lzkae `}jsat`s%
K`tdratjo c tlj doja tlat ’tlj }s`nj~~ `b tdef tv` dtjk~ t`fjtljs d~ tlj dk}`staet
tldef)‛ Cz~l cjidjrjo tlj Kjkj| v`zio sjr`iztd`edwj tlj va vj `sfaedwj aeo sjtsdjrj
deb`skatd`e% Cz~l—~ Kjkj|) kznl idhj Aiae _zsdef—~ _zsdef Kanldej) va~ ae attjk}t t`
k`oji vlat dt kjae~ t` cj a n`fedtdrj afjet8 l`vjrjs) vldij _zsdef }s`}`~jo tlj _zsdef
Kanldej de ae attjk}t t` kjnlaed~j ascdtsas sja~`edef ,aif`sdtlk j|jnztd`e$) Cz~l
tsdjo t` kjnlaed~j tlj a~~`ndatdrj }s`nj~~j~ vldnl asj aeai`f`z~ t` z~def kjk`s X^h+
afj~tao) 1442[% Vldij tlj _zsdef Kanldej cjnakj tlj ca~d~ `b k`ojse n`k}ztatd`e)
tlj Kjkj| va~ ~jje ojnaoj~ iatjs a~ }d`ejjsdef l}jstj|t aeo tlj V`sio Vdoj Vjc%
N`e~dojsdef tlj hj s`ij `b tlj detjsejt de k`ojse idbj) dt d~ ~abj t` ~a tlat kae `b
Cz~l—~ doja~ asj qzdtj aidrj t`oa% _lj va de vldnl vj fieo deb`skatd`e `e tlj detjsejt
t`oa d~ k`sj ~dkdias t` Cz~l—~ Kjkj| tlae tlj va de vldnl a ~ndjetd~t z~jo t` fieo
sjijraet deb`skatd`e at tlj idcsas ,canh de 14?:$%
1 A }`stkaetjaz `b kjk`s aeo deoj|) `s kjk`s j|tjeojs%
1?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 15/130
_lj ~znnj~~ `b tlj detjsejt la~ osakatdnaii nlaefjo tlj va vj tldeh ac`zt
he`vijofj% De ld~ c``h) ’Jrjstldef d~ Kd~njiiaej`z~7 _lj ]`vjs `b tlj Ejv Odfdtai
Od~`sojs‛ XVjdecjsfjs) 3==6[) Oardo Vjdecjsfjs j|}`zeo~ tlj rdjv tlat vj) a~ a ~`ndjt)
~l`zio jkcsanj odfdtai he`vijofj ,j%f%) Vdhd}joda) F``fij$ cjnaz~j `enj deb`skatd`e
d~ }ianjo `eidej) dt cjn`kj~ ~dfedfinaeti k`sj z~jbzi% Zeidhj c``h~ aeo `tljs }l~dnai
jetdtdj~ vldnl kz~t cj }l~dnaii `sfaedwjo de >O ~}anj ,j%f%) `e c``h~ljirj~$) odfdtai
aeo l}js+idehjo he`vijofj d~ kznl k`sj rjs~atdij cjnaz~j tljsj d~ e` ejjo t` j|}idndti
a~~dfe dtjk~ t` a ~defij ~zcgjnt/natjf`s% Idhj Vjdecjsfjs) Nia ^ldsh) a }s`kdejet
tldehjs vl` vsdtj~ ac`zt tlj detjsejt aeo `tljs ojnjetsaidwjo tjnle`i`fdj~) asfzj~
jijfaeti afade~t natjf`sdj~ de ld~ astdnij) ’@et`i`f d~ @rjssatjo7 Natjf`sdj~) Ideh~)
aeo _af~‛ X^ldsh) 3==:[% ^ldsh cjidjrj~ de tlj }`vjs `b tv` }`vjsbzi czdiodef ci`nh~
vldnl nae cj z~jo t` `sfaedwj deb`skatd`e7 1$ ideh~) vldnl nae }`det t` aetldef) aeo
3$ taf~) vldnl asj a va `b attanldef iacji~ t` ideh~% ]s`rdodef kzitd}ij asfzkjet~
afade~t tsaodtd`eai natjf`sdwatd`e ~nljkj~) ^ldsh asfzj~ tlat ’vlat vj—sj ~jjdef vlje
vj ~jj tlj Vjc d~ antzaii a saodnai csjah vdtl }sjrd`z~ natjf`sdwatd`e ~tsatjfdj~) satljs
tlae ae j|tje~d`e `b tljk%‛ Ideh~ dk}i tlat tljsj vdii fjejsaii j|d~t kzitd}ij }atl~
vldnl vj n`zio tsarjs~j t` fieo tlat `ej nszndai cdt `b deb`skatd`e vj asj i``hdef b`s%
E`t aii csaenlj~ `b ~ndjenj larj bzii bjit tlj vsatl `b tld~ ejv odfdtai od~`sojs%
@ej kdflt asfzj tlat tlj }s`cijk `b `cgjnt sjn`fedtd`e) a~ t}dnaii b`zeo de tlj fijio
`b n`k}ztjs rd~d`e aeo s`c`tdn~) banj~ kae }s`cijk~ vldnl) z~def Cz~l—~ v`so~) asj
’iasfji naz~jo c tlj astdfindaidt `b ~~tjk~ `b deoj|def%‛ _lj z~j `b natjf`sdj~ ,nia~~j~$
t` sj}sj~jet rd~zai n`enj}t~ d~ ~` }sjraijet de n`k}ztjs rd~d`e aeo kanldej ijasedef
tlat k`~t sj~jasnljs~ o` e`t fdrj dt a ~jn`eo tl`zflt% Banjo vdtl a ejv ta~h) `ej ~dk}i
nasrj~ z} tlj ~`iztd`e ~}anj det` nia~~j~ ,j%f%) nas~) }j`}ij) czdiodef~$) a~~dfe~ nia~~
iacji~ t` tsadedef j|ak}ij~ aeo a}}idj~ `ej `b tlj kae }`}zias kanldej ijasedef t``i~
t` `ctade a nia~~dfijs%
1:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 16/130
De tld~ v`sh) vj asj aor`natdef a odbbjsjet va `b tldehdef ac`zt sjn`fedtd`e ‟ e`t
a~ `cgjnt eakdef) czt satljs a~ `cgjnt a~~`ndatd`e% _lj doja) ~zffj~tjo c jrdojenj bs`k
n`fedtdrj ~ndjenj) d~ tlat tlj njetsai qzj~td`e `b sjn`fedtd`e kdflt e`t cj ’Vlat d~ dt0‛
czt satljs ’Vlat d~ dt idhj0‛ XCas) 3==6[% _lj jtk`i`f `b tlj rjs v`so ’sj+n`fedwj‛
,t` he`v afade$ ~z}}`st~ tlj rdjv tlat a~~`ndatd`e }ia~ a hj s`ij de sjn`fedtd`e%
Zeojs tld~ k`oji) vlje banjo vdtl a e`rji `cgjnt) tlj ta~h d~ t` a~~`ndatj dt vdtl tlj
k`~t ~dkdias `cgjnt~ de `ej—~ kjk`s% _lj~j sjkjkcjsjo `cgjnt~) de tzse) }s`rdoj tlj
kjta+oata ,j%f%) `cgjnt eakj) fj`kjts) a~~`ndatjo antd`e~$ ejjojo t` detjs}sjt tlj
e`rji `cgjnt% _ld~ aii`v~ b`s a oeakdn ojfiedtd`e `b natjf`sdj~ ca~jo `e oata aradiacdidt
aeo ta~h ,j%f%) ae `cgjnt nae cj a rjldnij) a nas) a R`ir`) `s C`c—~ R`ir`$%
Ae dk}`staet cjejfit `b `cgjnt a~~`ndatd`e `rjs `cgjnt eakdef d~ tlat tljsj d~ e` ejjo
t` odrdoj tlj v`sio z} det` sdfdo) }sj+ojfiejo natjf`sdj~ a }sd`sd% De~tjao) janl `cgjnt
de~taenj z~j~ dt~ ejasj~t ejdflc`s~ ,de ~`kj bjatzsj ~}anj$ t` debjs dt~ `ve dojetdt) a~
fjejsai `s a~ ~}jndfin a~ tlj aradiacij oata aii`v~% B`s j|ak}ij) db `zs oata~jt o`j~ e`t
n`etade kae nas~) tlje tlj cj~t tlat vj nae ~a ac`zt a ejv nas de~taenj d~ tlat dt nae
cj katnljo t` ae`tljs ’nas‛% Czt a~ tlj ezkcjs `b odbbjsjet nas~ de tlj oata~jt fs`v~)
vj ~l`zio cj acij t` fieo rjs ~}jndfin nas katnlj~ vldnl vdii aii`v z~ t` sjn`fedwj tlj
~akj `cgjnt de~taenj a~ ’sjo L`eoa Ann`so‛%
1%1 Canhfs`zeo `e Natjf`sdwatd`e _lj`sdj~
’Sj~tsdntdef tlj sj}sj~jetatd`e~ ojsdrjo bs`k ~njej~ t` cjdef n`enj}tzai ak`zet~
t` dk}`~def a ~jrjsj laeodna} `e tlj rd~zai ~~tjk‛
^ldk`e Jojikae
_` zeojs~taeo ae dkafj d~ t` sjn`fedwj dt~ n`e~tdtzjet~ `cgjnt~ vljsj `cgjnt ’sjn`f+
edtd`e‛ d~ z~zaii a~~zkjo t` kjae `cgjnt eakdef ‟ fdrje ae dkafj) tlj f`ai d~ t`
12
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 17/130
eakj tlj oj}dntjo `cgjnt~ ,aeo }`~~dci ~l`v tlj `cgjnt~— ~}atdai j|tjet$% Czt ~denj `zs
iaefzafj o`j~ e`t larj a eakj b`s jrjs }`~~dcij `cgjnt de~taenj) tld~ sjqzdsj~ tlat
`cgjnt natjf`sdj~ cj z~jo b`s eakdef }zs}`~j~% L`vjrjs) f`def bs`k `cgjnt~ t` `cgjnt
natjf`sdj~ d~ ae j|tsjkji e`d~ aeo i`~~ }s`nj~~7 ’a }dntzsj d~ v`stl a tl`z~aeo v`so~‛
‟ e`t `ej `s tv` t}dnaii z~jo b`s natjf`sdwatd`e% Cjb`sj od~nz~~def tlj n`etsdcztd`e~ `b
tld~ tlj~d~) dt d~ v`stlvldij f`def canh de tdkj t` cjttjs zeojs~taeo tlj s`ij natjf`sdj~
larj }iajo de rd~d`e aeo sjiatjo od~nd}idej~%
_lj`sdj~ `b natjf sdwatd e oatj canh t` tlj aendjet Fsjjh~% Asd~t tij ojfiejo nat+
jf`sdj~ a~ od~nsjtj jetdtdj~ nlasantjsdwjo c a ~jt `b }s`}jstdj~ ~lasjo c aii tljds
kjkcjs~ X Asd~t`tij) [% Ld~ natjf`sdj~ asj kztzaii j|niz~drj) aeo jrjs kjkcjs `b
a natjf`s d~ jqzai% _ld~ nia~~dnai rdjv d~ ~tdii tlj k`~t vdoji annj}tjo va `b sja+
~`edef ac`zt natjf`sdj~ aeo ta|`e`kdj~ de laso ~ndjenj~%3 L`vjrjs) a~ }`detjo `zt
c Izovdf Vdttfje~tjde) tld~ d~ aik`~t njstadei e`t tlj va k`~t `b `zs jrjsoa
n`enj}t~ v`sh ,j%f%) vlat d~ tlj ~jt `b }s`}jstdj~ tlat ojfiej tlj n`enj}t ’fakj‛ aeo
e`tldef ji~j0 XVdttfje~tjde) 14:>[$% Jk}dsdnai jrdojenj b`s t}dnaidt ,j%f%) a s`cde
d~ a k`sj n`kk`ei ndtjo j|ak}ij `b ’cdso‛ tlae a nldnhje$ aeo kzitd}ij natjf`s
kjkcjs~ld}~ ,j%f%) nldnhje d~ c`tl ’cdso‛ aeo ’b``o‛$ bzstljs n`k}idnatj tlj Asd~t`tjidae
rdjv% Vdttfje~tjde asfzjo tlat tldef~ sjbjssjo t` c tlj ~akj eakj asj e`t ejnj~~asdi
n`eejntjo c `ej j~~jetdai bjatzsj ‟ de~taenj~ cji`efdef t` a ~defij n`enj}t kdflt cj
n`eejntjo c `rjsia}}def ~dkdiasdtdj~) zeojs tlj doja he`ve a~ bakdi sj~jkciaenj~%
K`tdratjo c Vdttfje~tjde—~ e`td`e `b bakdi sj~jkciaenj~) tlj fs`zeo+csjahdef
v`sh `b n`fedtdrj }~nl`i`fd~t Jijae`s S`~nl XS`~nl aeo Kjsrd~) 146:) S`~nl) 146<[
ojk`e~tsatjo tlat lzkae~ o` e`t nzt z} tlj v`sio det` ejat natjf`sdj~ ojfiejo c
~lasjo }s`}jstdj~) czt de~tjao z~j ~dkdiasdt a~ tlj ca~d~ `b natjf`sdwatd`e% S`~nl—~
}s`t`t}j tlj`s }`~tziatj~ tlat ae `cgjnt—~ natjf`s d~ ojtjskdejo c dt~ ~dkdiasdt t`
3_ld~ d~ tlj sja~`e vl tlj ojcatj sjfasodef ]izt`—~ ~tatz~ a~ a }iaejt la~ cjje ~` rdnd`z~%
16
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 18/130
,a ~jt `b$ }s`t`t}j~ vldnl ojfiej janl natjf`s) aii`vdef b`s rasdef ,fsaojo$ ojfsjj
`b kjkcjs~ld}% ^znl }s`t`t}j k`oji~ larj cjje ~znnj~~bzii z~jo b`s `cgjnt sjn`fed+
td`e XCa~sd) 1443) Jojikae) 144:[% F`def jrje bzstljs) j|jk}ias tlj`s XKjode aeo
^nlabbjs) 146<) E`~`b~h) 14<2[ sjgjnt~ tlj ejjo b`s ae j|}idndt natjf`s sj}sj~jetatd`e)
asfzdef de~tjao tlat a n`enj}t nae cj dk}idndti b`skjo rda aii dt~ `c~jsrjo de~taenj~% De
j|jk}ias tlj`s) tljsj d~ e` ejjo t` b`sk ae j|}idndt ac~tsantd`e ~znl a~ a }s`t`t}j ‟
natjf`sdwatd`e d~ }jsb`skjo c katnldef a e`rji de}zt t` a iasfj ~jt `b tsadedef de~taenj~%
B`s a fsjat dets`ozntd`e t` tlj`sdj~ natjf`sdwatd`e ,denizodef }s`t`t}j aeo j|jk}ias
tlj`sdj~$) vj sjbjs tlj sjaojs t` Fsjf`s Kzs}l—~ Cdf C``h `b N`enj}t~ XKzs}l)
3==3[%
_ljsj d~ ai~` aczeoaet n`etjk}`sas jrdojenj de tlj fijio `b n`fedtdrj ~ndjenj vldnl
~zffj~t~ tlat tljsj d~ k`sj t` rd~zai zeojs~taeodef tlae nae cj na}tzsjo c sdfdo
natjf`sdj~% C ~tzodef lzkae ~zcgjnt~) Azoj @idra aeo n`iijafzj~ XCsao jt ai%) 3==4[
larj ~l`ve tlat rd~zai kjk`s la~ a ka~~drj ~t`safj na}andt b`s `cgjnt ojtadi~% _ljds
fieodef~ ~zffj~t tlat rd~zai i`ef+tjsk sj}sj~jetatd`e~ asj k`sj ojtadijo tlae }sjrd`z~i
tl`zflt) aeo ~l`v tlat ’rd~zai i`ef+tjsk kjk`s sj}sj~jetatd e~ nae n`etade e`t `ei
fd~t deb`skatd`e c ai~` ojtadi~ ~zbfindjet t` od~nsdkdeatj cjtvjje j|jk}ias~%‛ XCsao
jt ai%) 3==4[ _ld~ ~zffj~t~ tlat lzkae~ o` e`t ~dk}i ac~tsant+ava `cgjnt ojtadi~ c
}iandef tljk det` ejat aeo od~tdent natjf`sdj~%
Fzdojo c detzdtd`e bs`k tlj`sdj~ `b lzkae rd~zai }jsnj}td`e aeo jrdojenj bs`k
n`fedtdrj ejzs`~ndjenj) K`~lj Cas `ztidej~ tlj dk}`staenj `b aeai`fdj~) a~~`ndatd`e~)
aeo }sjodntd`e~ de tlj lzkae csade XCas) 3==4[% Lj asfzj~ tlat tlj f`ai `b rd~zai
}jsnj}td`e d~ e`t t` sjn`fedwj ae `cgjnt de tlj tsaodtd`eai ~je~j `b natjf`sdwdef dt ,d%j%)
a~hdef ’Vlat d~ tld~0‛$) czt de~tjao idehdef tlj de}zt vdtl ae aeai`f`z~ sj}sj~jetatd`e
de kjk`s ,d%j)% a~hdef ’Vlat d~ tld~ idhj0‛$%> @enj a e`rji de}zt d~ idehjo vdtl
>Cas z~j~ tlj tjsk ’aeai`f‛ b`s tlj }s`nj~~ `b j~tacid~ldef a ideh cjtvjje a }sjrd`z~i ~jje `cgjntaeo a e`rji `ej) vldij vj ~dk}i sjbjs t` tlj~j ideh~ a~ rd~zai a~~`ndatd`e~%
1<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 19/130
aeai`f`z~ sj}sj~jetatd`e~) a~~`ndatjo sj}sj~jetatd`e~ asj antdratjo sa}doi aeo }sjodnt
tlj sj}sj~jetatd`e~ `b vlat d~ k`~t idhji t` `nnzs ej|t% Cas ~zffj~t~ tlat tlj }sdend}ij
`b }sjodntd`e d~ a ~ts`ef naeodoatj b`s a zedrjs~ai }sdend}ij tlat nae j|}iade a kag`sdt `b
tlj csade—~ `}jsatd`e XCas) 3==4[% Czt }sjodntd`e) a~ asfzjo c Cas) sjidj~ `e tlj acdidt
t` j~tacid~l kjaedefbzi aeai`fdj~ ,`s z~def `zs tjskde`i`f) rd~zai a~~`ndatd`e~$%
1%1%1 ]s`cijk~ Vdtl Rd~zai Natjf`sdj~
Vj sjbjs t` `cgjnt~ c tlj natjf`sdj~ tlj cji`ef t` ~` jbb`stij~~i de `zs oadi idrj~ tlat
dt d~ e`t nijas vl kanldej~ ~l`zio larj ~znl a laso tdkj }jsb`skdef tlj ~akj ta~h%
L`vjrjs) dt d~ zenijas vljtljs natjf`sdwatd`e d~ z~jbzi b`s n`k}ztjs rd~d`e% B`s j|ak}ij)
kae `cgjnt natjf`sdj~ asj bzentd`eai% _lj~j natjf`sdj~ `btje j|ldcdt rd~zai }`i~jk
‟ `cgjnt de~taenj~ tlat larj rd~zaii e`tldef t` o` vdtl janl `tljs ,j%f%) ’nlads‛$%
K`sj`rjs) natjf`sdj~ asj iaefzafj oj}jeojet ‟ ae `cgjnt natjf`s de `ej iaefzafj
kdflt e`t j|d~t de ae`tljs XIah`bb) 14<6[% Tjt ae`tljs ~`zsnj `b rd~zai }`i~jk
}astdnzias t` 3O dkafj ~jt~ d~ rdjv+oj}jeojenj% _ahje `e dt~ `ve) a ~doj+rdjv `b a nas
la~ rd~zaii e`tldef de n`kk`e vdtl a bs`etai rdjv `b a nas ,Bdfzsj 1%1 sdflt$% _ljsjb`sj)
tsdef laso t` kahj a ~defij nas n`enj}t ojtjnt`s fisj `e c`tl ~jjk~ n`zetjs}s`ozntdrj
‟ c ijasedef natjf`s+~}jndfin k`oji~) vj kdflt antzaii cj tsdef t` ~`irj a k`sj
odbfinzit }s`cijk tlae d~ ejnj~~as%
De aoodtd`e) rd~zai `cgjnt natjf`sdwatd`e d~ e`t jrje n`e~d~tjet ans`~~ deodrdozai~%
N`e~dojs) b`s de~taenj) tlj IacjiKj oata~jt XSz~~jii jt ai%) 3==<[ vljsj lzkae iacjijs~
nae nl``~j ae Jefid~l v`so/}lsa~j tlj idhj b`s `cgjnt aee`tatd`e% Bdfzsj 1%1 ,ijbt$~l`v~ a t}dnai j|ak}ij `b rd~zai ~e`ek~ ‟ tv` rd~zaii ~dkdias `cgjnt~ tlat larj
cjje ascdtsasdi a~~dfejo odbbjsjet iacji~ ,’czdiodef‛ r~% ’l`z~j‛$%
14
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 20/130
L`z~j Nas NasCzdiodef
Bdfzsj 1%17 _}dnai j|ak}ij~ `b rd~zai ~e`ek~ aeo rd~zai }`i~jk tlat asj n`kk`e de IacjiKj X Sz~~jii
jt ai%) 3==<[ aee`tatd`e~% Rd~zai ~e`ek~7 tv` `cgjnt~ tlat asj rd~zaii qzdtj ~dkdias czt larj odbbjsjetnia~~ iacji~ ,ijbt$% Rd~zai }`i~jk7 tv` `cgjnt~ tlat larj e`tldef de n`kk`e rd~zaii czt asj iacjijo t`cj tlj ~akj nia~~ ,sdflt$%
1%1%3 Vl Natjf`sdwj0
Czt dt kdflt e`t cj t`` }s`ozntdrj t` n`enjetsatj `e tlj rasd`z~ natjf`sdwatd`e tlj`sdj~
vdtl`zt n`e~dojsdef tlj fieai adk ‟ vlat o` vj ejjo natjf`sdj~ b`s0 @ej asfzkjet d~
tlat natjf`sdwatd e d~ a t``i t` bandidtatj he`vijofj tsae~bjs% J%f%) lardef cjje attanhjo
`enj c a tdfjs) dt d~ nsdtdnaii dk}`staet t` ojtjskdej db a ejvi `c~jsrjo `cgjnt cji`ef~
t` tlj tdfjs natjf`s ~` a~ t` ztdidwj tlj deb`skatd`e bs`k tlj }sjrd`z~ jen`zetjs% E`tj
tlat ljsj sjn`fedwdef tlj j|}idndt natjf`s d~ zedk}`staet) a~ i`ef a~ tlj tv` tdfjs~n`zio cj a~~`ndatjo vdtl janl `tljs% Ae`tljs asfzkjet b`s z~def natjf`sdj~ d~ tlj
jeacij n`kkzednatd`e% Db tv` afjet~ ~lasj a n`kk`e idefzd~tdn r`naczias) tlje tlj
nae jbbjntdrji j|nlaefj deb`skatd`e ac`zt tlj v`sio%
Czt vlat db tljsj d~ e` ~defij ldjsasnldnai `sfaedwatd`eai ~nljkj tlat d~ ~zdtacij
b`s `sfaedwdef tlj rd~zai v`sio0 A~ asfzjo c Ai`e Laijr de ’_lj Zesja~`eacij
Jbbjntdrjej~~ `b Oata)‛ XLaijr jt ai%) 3==4[) a }ast `b tlj v`sio d~ j|}iadejo vjii c
}asakjtsdn/katljkatdnai k`oji~) aeo a }ast `b tlj v`sio d~ cjttjs j|}iadejo c oata%
Vj asfzj tlat tlj rd~zai v`sio nae cj ~zbfindjeti j|}iadejo c oata) aeo tlj ~jasnl b`s
a ~defij `sfaedwatd`eai ~nljkj b`s tlj rd~zai v`sio kdflt cj bztdij%
3=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 21/130
1%3 Sjiatjo V`sh
@b n`zs~j) tlj doja `b a~~`ndatdef a ejv de~taenj vdtl ~`kjtldef ~jje de tlj }a~t la~
a i`ef aeo sdnl ld~t`s) ~tastdef vdtl tlj Csdtd~l Jk}dsdnd~t~ XI`nhj) 12<4) Cjshjij)
161=) Lzkj) 16>4[) aeo n`etdezdef a~ j|jk}ias tlj`s de n`fedtdrj }~nl`i`f XKjode
aeo ^nlabbjs) 146<)E`~`b~h) 14<2[) na~j+ca~jo sja~`edef de AD X^nlaeh) 14<>[) de~taenj+
ca~jo kjtl`o~ de kanldej ijasedef X Ala jt ai%) 1441[) oata+osdrje tsae~bjs de fsa}l+
dn~ XSje jt ai%) 3==:[) jtn% De n`k}ztjs rd~d`e) tld~ t}j `b e`e+}asakjtsdn tjnledqzj
la~ cjje qzdtj ~znnj~~bzi at a rasdjt `b ta~h~ denizodef7 `cgjnt aidfekjet XCji`efdj
jt ai%) 3==3) Cjsf jt ai%) 3==:[) ~njej sjn`fedtd`e X_`ssaica jt ai%) 3==<) \da` jt ai%)
3=1=[) dkafj }as~def XIdz jt ai%) 3==4[) ak`ef `tljs~% L`vjrjs) b`s `cgjnt ojtjntd`e)
oata+osdrje kjtl`o~ ~znl a~ XSz~~jii jt ai%) 3==6) Kaid~djvdnw aeo Jbs`~) 3==<[) larj
e`t cjje n`k}jtdtdrj afade~t od~nsdkdeatdrj a}}s`anlj~ ,tl`zfl tlj lcsdo kjtl`o
`b XNlzk aeo Wd~~jskae) 3==6[ n`kj~ ni`~j$% Vl d~ tld~0 De `zs rdjv) tlj }sdkas
odbfinzit ~tjk~ bs`k tlj ka~~drj ak`zet~ `b ejfatdrj oata tlat kz~t cj n`e~dojsjo de
tlj ojtjntd`e }s`cijk% De dkafj nia~~dfinatd`e) vljsj oata~jt ~dwj~ t}dnaii saefj bs`k a
bjv tl`z~aeo~ t` a kdiid`e) z~def tlj h Ejasj~t Ejdflc`s~ ,h+EE$ aif`sdtlk t` n`k}ztj
od~taenj~ t` aii tsadedef dkafj~ d~ ~tdii qzdtj bja~dcij% De `cgjnt ojtjntd`e) l`vjrjs) tlj
ezkcjs `b ejfatdrj vdeo`v~ nae f` a~ ldfl a~ lzeosjo~ `b kdiid`e~) }jsb`skdef h+EE
z~def c`tl }`~dtdrj~ aeo ejfatdrj~ d~ }s`ldcdtdrji j|}je~drj% Z~def ljzsd~tdn~) ~znl a~
~zc~ak}idef `s dfe`sdef tlj ejfatdrj ~jt) sj~zit~ de a ~zc~taetdai os`} de }jsb`skaenj%
De n`etsa~t) nzssjet ~tatj+`b+tlj+ast kjtl`o~ de `cgjnt ojtjntd`e , XOaiai aeo _sdff~)
3==:[) XBjiwje~wvaic jt ai%) 3=1=[) aeo tljds ojsdratdrj~$ asj }astdnziasi vjii+~zdtjo b`s
laeoidef iasfj ak`zet~ `b ejfatdrj oata% _lj jk}i` ’oata+kdedef‛ t` dtjsatdrji ~dbt
tls`zfl kdiid`e~ `b ejfatdrj~ aeo fieo tlj ’laso‛ `ej~ vldnl asj tlje z~jo t` tsade a
od~nsdkdeatdrj nia~~dfijs% Cjnaz~j tlj nia~~dfijs d~ a idejas ^RK) jrje tlj laso ejfatdrj~
31
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 22/130
o` e`t ejjo t` cj j|}idndti ~t`sjo czt asj sj}sj~jetjo }asakjtsdnaii) de tjsk~ `b a
ojnd~d`e c`zeoas%
L`vjrjs) tlj }asakjtsdn eatzsj `b tlj~j nia~~dfijs~) vldij a cij~~def b`s laeoidef
ejfatdrj oata) cjn`kj~ k`sj }s`cijkatdn vlje sj}sj~jetdef tlj }`~dtdrj~% _}dnaii)
aii }`~dtdrj j|ak}ij~ `b a fdrje `cgjnt natjf`s asj sj}sj~jetjo a~ a vl`ij) dk}idn+
dti a~~zkdef tlat tlj asj aii sjiatjo t` janl `tljs rd~zaii % Zeb`stzeatji) k`~t
~taeoaso ~jkaetdn natjf`sdj~ ,j%f%) ’nas‛) ’nlads‛) ’tsade‛$ o` e`t b`sk n`ljsjet rd~zai
natjf`sdj~ XKaid~djvdnw aeo Jbs`~) 3==<[) tlz~ tsjatdef tljk }asakjtsdnaii sj~zit~ de
vjah aeo `rjsi+fjejsdn ojtjnt`s~% _` aoosj~~ tld~ }s`cijk) a ezkcjs `b a}}s`anlj~
larj z~jo ~jkd+}asakjtsdn kd|tzsj k`oji~) fs`z}def tlj }`~dtdrj~ det` niz~tjs~ ca~jo
`e kjta+oata ~znl a~ c`zeodef c`| a~}jnt satd` XBjiwje~wvaic jt ai%) 3=1=[) `cgjnt
~naij X]ash jt ai%) 3=1=[) `cgjnt rdjv}`det XFz aeo Sje) 3=1=[) }ast iacji~ XC`zsojr
jt ai%) 3=1=[) jtn% Czt tlj i`v ezkcjs `b kd|tzsj n`k}`ejet~ z~jo de }santdnj kjae~
tlat tljsj d~ ~tdii n`e~dojsacij rasdatd`e vdtlde janl niz~tjs% A~ a sj~zit) tlj aidfekjet) `s
rd~zai n`ssj~}`eojenj) cjtvjje tlj ijasejo k`oji aeo a ojtjntjo de~taenj d~ t`` n`as~j
t` cj z~acij b`s `cgjnt a~~`ndatd`e aeo iacji tsae~bjs% Vldij }ast+ca~jo k`oji~ XBjiwje+
~wvaic jt ai%) 3=1=[ aii`v odbbjsjet i`naidwatd`e~ `b }ast~ vdtlde od~tdent ojtjntd`e~) tlj
sjqzdsjkjet tlat tlj kz~t cj ~lasjo ans`~~ aii kjkcjs~ `b a natjf`s kjae~ tlat tlj~j
’}ast~‛ asj ai~` j|tsjkji rafzj aeo tlj sj~zitdef n`ssj~}`eojenj~ asj zedetzdtdrj% De
fjejsai) dt kdflt cj cjttjs t` tldeh `b tlj~j }ast~ a~ ~`bt) ojb`skacij ~zc+tjk}iatj~% _lj
]`~jijt~ a}}s`anl XC`zsojr jt ai%) 3=1=[ attjk}t~ t` aoosj~~ tld~ }s`cijk c kaezaii
iacjidef }ast~ aeo z~def tljk t` tsade a ~jt `b }`~j+~}jndfin }ast ojtjnt`s~% Vldij rjs
jen`zsafdef) tlj ljar kaezai iacjidef czsoje d~ a cdf idkdtatd`e `b tld~ kjtl`o%
Kjta+oata tsae~bjs la~ ai~` cjje z~jo de a ’sjn`fedwj+tlje+tsae~bjs‛ ~jttdef ,j%f%)
X_l`ka~ jt ai%) 3==4) Id jt ai%) 3=11[$% _lj a}}s`anl `b X_l`ka~ jt ai%) 3==4[ aoo~ a
kjta+oata sja~`edef ~tafj t` tlj sjn`fedtd`e ~~tjk `b XIjdcj jt ai%) 3==?[) czt ~denj
33
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 23/130
`cgjnt natjf`sdj~ asj sj}sj~jetjo }asakjtsdnaii) a e`e+tsdrdai kjta+oata }s`}afatd`e
~tj} kz~t cj denizojo% Dt d~ ai~` v`stlvldij kjetd`edef tlj a}}s`anl `b X^arasj~j
aeo Bjd+Bjd) 3==6[) vldnl `zt}jsb`sk~ X_l`ka~ jt ai%) 3==4[ aeo }s`rdoj~ a k`sj
n`k}ant sj}sj~jetatd e `b tlj `cgjnt natjf`s% L`vjrjs c`tl X_l`ka~ jt ai%) 3==4[
aeo X^arasj~j aeo Bjd+Bjd) 3==6[ asj natjf`s+ca~jo aeo kz~t n`}j vdtl tlj }s`cijk~
deljsjet de ojaidef vdtl rd~zai natjf`sdj~ ,~jj od~nz~~d`e de ^jntd`e 1%1%1$% _lj a}}s`anl
`b XId jt ai%) 3=11[ fis~t z~j~ a n`iijntd`e `b }`~j+~je~dtdrj `cgjnt ojtjnt`s~ t` }s`oznj
ojtjntd`e~ de tlj dkafj aeo tlje }jsb sk~ a iaeokash+ca~jo aidfekjet z~def ae dtjsatdrj
aif`sdtlk% A~ vdii cj ~jje de Nla}tjs ?) `zs a}}s`anl o`j~ e`t sjqzdsj ae aoodtd`eai
aidfekjet ~tj}~ abtjs ojtjntd`e) e`s o`j~ dt sjqzdsj tsadedef ~j}asatj natjf`s+ca~jo
`cgjnt ojtjnt`s~ t` nsjatj dedtdai ojtjntd`e~%
3>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 24/130
1%> _lj~d~ @rjsrdjv
]ast D `b tld~ tlj~d~ d~ ojodnatjo t` ijasedef j|jk}ias sj}sj~jetatd`e~ b`s `cgjnt natjf`s
ojtjntd e% Nla}tjs 3 k`tdratj~ tlj }s`cijk `b ijasedef j|jk}ias+~}jndfin ~dkdiasdt
kja~zsj~ aeo }sj~jet~ I`nai Od~taenj Bzentd`e~ aeo l`v tlj nae cj z~jo vdtl Kzitd}ij
^jfkjetatd`e aif`sdtlk~ b`s i`naidwdef j|jk}ias~ de dkafj~ XKaid~djvdnw aeo Jbs`~)
3==<[% Nla}tjs > d~ ac`zt J|jk}ias+^RK~ XKaid~djvdnw jt ai%) 3=11[) vldnl asj a
kassdafj `b tlj j|jk}ias+ca~jo kjtl`o`i`f vdtl od~nsdkdeatdrj tsadedef b`zeo de
a}}s`anlj~ ~znl a~ XBjiwje~wvaic jt ai%) 3=1=[% ]ast D ~l`v~ tlat c j|}i`dtdef iasfj
ak`zet~ `b rd~zai oata) vj nae ijase j|jk}ias+~}jndfin ~dkdiasdt kja~zsj~) vldnl asj
~z}jsd`s t` ijasedef+bsjj ejasj~t+ejdflc`s a}}s`anlj~% Vj n`enizoj ]ast D c ~l`vdef
tlat vdtl ae je~jkcij `b J|jk}ias+^RK~) vj nae anldjrj }jsb`skaenj `e tlj ]A^NAI
R@N ojtjntd`e ta~h tlat d~ `e }as vdtl tlj kznl k`sj n`k}ij| iatjet }ast+ca~jo k`oji
`b XBjiwje~wvaic jt ai%) 3=1=[) at `ei a k`oj~t n`k}ztatd`eai n`~t densja~j%
]ast DD d~ ojodnatjo t` tlj }`vjs `b j|jk}ias a~~`ndatd`e~ ‟ vj ~l`v sj~zit~ `e ta~h~
vldnl f` cj`eo `cgjnt natjf`s ojtjntd`e% Vj jraizatj tlj J|jk}ias+^RK a}}s`anl
`e a odrjs~j ~jt `b kjta+oata tsae~bjs ta~h~ XKaid~djvdnw jt ai%) 3=11[ ,~jfkjetatd`e
tsae~bjs) qzaidtatdrj fj`kjts tsae~bjs) aeo sjiatjo `cgjnt }sdkdef$ a~ vjii a~ rd~zaidwj
tlj J|jk}ias+^RK—~ deoznjo j|jk}ias+j|jk}ias ~dkdiasdt ~tszntzsj `e ]A^NAI% Vj
n`enizoj vdtl a n`etj|tzai `cgjnt }sjodntd`e ta~h XKaid~djvdnw aeo Jbs`~) 3==4[) vldnl
sjqzdsj~ tlj b`skatd`e `b a fsa}l `rjs j|jk}ias~) vldnl vj naii tlj Rd~zai Kjkj| ) aeo
~l`v ae dk}s`rjkjet `rjs natjf`s+ca~jo ca~jidej~% De tlj A}}jeod|) vj `rjsrdjv
ae a}}idnatd`e `b tlj J|jk}ias+^RK aif`sdtlk t` tlj }s`cijk `b ns`~~+o`kade dkafj
katnldef X^lsdra~tara jt ai%) 3=11[ ,j%f%) katnldef }adetdef~ t` }l`t`~ aeo katnldef
~hjtnlj~ t` }l`t`~$ a~ vjii a~ od~nz~~ `zs n`etsdcztd`e~ de tlj b`sk `b ae `}je ~`zsnj
J|jk}ias+^RK `cgjnt ojtjntd`e idcsas%
3?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 25/130
]ast D7 Ijasedef J|jk}ias
Sj}sj~jetatd`e~
’Sj}sj~jet aii tlj oata vdtl a e`e}asakjtsdn k`oji satljs tlae tsdef t`
~zkkasdwj dt vdtl a }asakjtsdn k`oji) cjnaz~j vdtl rjs iasfj oata ~`zsnj~) tlj
oata l`io~ a i`t `b ojtadi%%% E`v f` `zt aeo fatljs ~`kj oata) aeo ~jj vlat dt nae
o`%‛
Ai`e Laijr) ]jtjs E`srdf) aeo Bjseaeo` ]jsjdsa)
’_lj Zesja~`eacij Jbbjntdrjej~~ `b Oata‛ ,F``fij) 3==4$
3:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 26/130
Oj~}dtj tlj cjejfit~) }`~def sjn`fedtd`e a~ j|jk}ias a~~`ndatd`e d~ e`t ae ja~
ta~h% @ej sjqzdsjkjet d~ a rjs iasfj oata~jt) sdnl je`zfl t` n`etade kae odbbjsjet
`cgjnt~ aeo kae de~taenj~% Sjnjeti) vdtl tlj a}}jasaenj `b iasfj dkafj n`iijntd`e~)
~jrjsai ~~tjk~ larj ~l`ve tlat ~dk}ij h+ejasj~t+ejdflc`s ,h+EE$ a}}s`anlj~ nae `btje
}jsb`sk ~zs}sd~defi vjii X_`ssaica jt ai%) 3==<) La~ aeo Jbs`~) 3==6) Sz~~jii jt ai%)
3==6[% L`vjrjs) aii tlj~j kjtl`o~ katnl tlj dkafj a~ a vl`ij) vldnl jbbjntdrji idkdt~
tljk t` `}jsatdef `e tlj n`as~j ~njej ijrji ,tljsj d~ ~dk}i e`t je`zfl oata de tlj
v`sio t` `c~jsrj aii }`~~dcij `cgjnt~ de aii }`~~dcij n`efifzsatd`e~$% _` katnl deodrdozai
`cgjnt~ vdtlde ~njej~) vj kz~t }astdtd`e tlj dkafj det` nlzeh~ vldnl asj ~kaii je`zfl
t` cj katnlacij de a sja~`eaci+~dwjo oataca~j) czt iasfj je`zfl t` jen`oj ~}jndfin
`cgjnt~) e`t fjejsdn ’rd~zai v`so~‛ X^drdn aeo Wd~~jskae) 3==>[% @ej a}}s`anl d~ t`
aii`v }astdai ~njej katnlj~ de~tjao `b katnldef jetdsj ~njej~ XSz~~jii jt ai%) 3==4[8
l`vjrjs) ~denj vj asj detjsj~tjo de ojtjntdef jetdsj `cgjnt~) vj b`nz~ `e katnldef
`cgjnt+~dwjo dkafj nlzeh~ ,b`s vldnl vj z~j jdtljs kzitd}ij ~jfkjetatd e~ aeo ~idodef
vdeo`v~$%
De aoodtd`e t` a tsadedef oata~jt) j|jk}ias+ca~jo a}}s`anlj~ sjqzdsj a ~zdtacij
e`td`e `b rd~zai ~dkdiasdt% De `sojs t` kahj `zs a}}s`anl v`sh `e a k`ojsatji+~dwjo
tsadedef ~jt) vj ~l`v tlat ijasedef rd~zai ~dkdiasdt bzentd`e~ d~ ~z}jsd`s t` z~def a
~defij laeo+nsabtjo kjtsdn) a~ d~ n`kk`e de ejasj~t+ejdflc`s a}}s`anlj~% Vj }s`}`~j
tv` t}j~ `b ~dkdiasdt bzentd`e~) I`nai Od~taenj Bzentd`e~ aeo J|jk}ias+^RK~) a~ vjii
a~ aif`sdtlk~ b`s ijasedef tljk bs`k oata% Vldij tlj ezkcjs `b ijasedef }s`cijk~ vj
~`irj ~naij~ idejasi vdtl tlj ezkcjs `b j|jk}ias~) }js+j|jk}ias ijasedef }s`cijk~ asj
kznl ja~djs t` ojai vdtl cjnaz~j janl j|jk}ias—~ ijasedef }s`cijk d~ ~dk}ijs tlae a
fi`cai natjf`s+vd~j }s`cijk% K`sj dk}`staeti) }js+j|jk}ias ~dkdiasdt bzentd`e~ fdrj
z~ a rasdacij aeo `cgjnt+oj}jeojet e`td`e `b ~dkdiasdt) aii`vdef z~ t` sja~`e ac`zt a
odrjs~j ~jt `b `cgjnt t}j~%
32
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 27/130
Nla}tjs 3
I`nai Od~taenj Bzentd`e~
Vlje vj i``h at tlj v`sio as`zeo z~) vj e`tdnj tlat ~dkdiasdt d~ ojfiejo odbbjsjeti
b`s odbbjsjet t}j~ `b `cgjnt~ de tlj v`sio% V`s~j jt) `cgjnt~ nae j|ldcdt ~dkdiasdt
`e kae odbbjsjet) `btje n`etsaodnt`s) ijrji~7 ~la}j) ~dwj) n`i`s) tj|tzsj) jtn% B`s
j|ak}ij) Aoji~`e odrdoj~ tlj v`sio det` ’tldef~‛ ,~znl a~ nas~) }j`}ij$ aeo ’~tzbb‛
,fsa~~) }arjkjet) dnj nsjak) jtn%$ X Aoji~`e) 3==1[% B`s ’tldef~‛) idhj nas~) `cgjnt ~la}j
d~ ae dk}`staet nzj vljsja~ `cgjnt n`i`s d~ z~zaii e`t% Czt b`s ’~tzbb‛) idhj fsa~~) vldnl
o`j~ e`t `ve dt~ c`zeoasdj~) ~la}j d~ z~jij~~ ,de bant) ojtsdkjetai$ czt n`i`s aeo tj|tzsj
asj j|tsjkji dk}`staet% _ljsjb sj) t` fieo vlat a fdrje `cgjnt de~taenj d~ ~dkdias
t`) dt d~ dk}jsatdrj tlat tlj sdflt od~taenj kjtsdn b`s tlat de~taenj cj z~jo% Czt) `b
n`zs~j) t` he`v tlj sdflt od~taenj kjtsdn sjqzdsj~ he`vdef vlat tlat `cgjnt d~! A~ d~
`btje tlj na~j de rd~d`e) vj asj banjo vdtl a odbfinzit nldnhje aeo jff }s`cijk% De tlj
na~j `b od~taenj bzentd`e~) vj tanhij tld~ }s`cijk c ijasedef a ~j}asatj n`kcdeatd`e
`b jijkjetas od~taenj~ ,~znl a~ n`i`s) tj|tzsj aeo ~la}j$ b`s janl j|jk}ias de `zs
oataca~j% _` kahj tldef~ odbfinzit) t}dnai lzkae `cgjnt iacji~ ,vldnl asj fjejsaii tlj
ca~dn+ijrji natjf`sdj~ t` vldnl ae `cgjnt cji`ef~ t`$ asj e`t f``o je`zfl t` kahj tlj
a~~zk}td`e tlat ae j|jk}ias ~l`zio cj ~dkdias t` aii `tljs j|jk}ias~ vdtl tlj ~akj
36
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 28/130
iacji ,~jj Bdfzsj 1%1 aeo od~nz~~d`e de ~jntd`e 1%1%1$% Vj de~tjao }s`}`~j a iasfji
oata+osdrje a}}s`anl vldnl vjahi z~j~ tlj `cgjnt iacji~ t` azt`katdnaii ijase b`s janl
j|jk}ias a od~taenj bzentd`e aeo a ~kaii ~jt `b j|jk}ias~ vldnl asj rd~zaii ~dkdias t`
tlj j|jk}ias%
_lj od~taenj bzentd`e~ ojfiejo de tld~ ~jntd`e asj }`~dtdrj idejas n`kcdeatd`e~ `b
jijkjetas od~taenj~% Vj z~j `ej n`kk`e od~taenj bzentd`e }asakjtjsdwatd e b`s
aii j|jk}ias~) czt ijase de~taenj+~}jndfin }asakjtjs~ c ~`irdef a odbbjsjet ~z}jsrd~jo
ijasedef }s`cijk b`s janl j|jk}ias% Janl j|jk}ias la~ dt~ `ve od~taenj bzentd`e ‟ vj
oje`tj j|jk}ias j—~ od~taenj bzentd`e a~ Oj,|d$ ; O,|dy|j)vj$) vldnl tahj~ a~ de}zt
~`kj qzjs bjatzsj~ |d ∌ B aeo sjtzse~ tlj ~dkdiasdt cjtvjje tlj j|jk}ias) |j ∌ B
aeo tlj de}zt |d ca~jo `e tlj e`td`e `b ~dkdiasdt jen`ojo c vj ∌ B (%
Vj z~j a }`}zias nia~~ `b od~taenj bzentd`e~ vldnl asj ai~` naiijo odaf`eai Kala+
iae`cd~ od~taenj kjtsdn~% _lj ojnd~d`e c`zeoas `b janl od~taenj bzentd`e d~ ae jiid}~`do
njetjsjo at |j) vljsj janl n`k}`ejet `b vj oje`tj~ tlj ji`efatd`e `b tlj jiid}~j
ai`ef tlat }astdnzias odkje~d`e% ^denj vj hjj} janl od~taenj bzentd`e aenl`sjo at tlj
j|jk}ias) vj nae tldeh `b ae ejv }`det |d) e`t de dt~ sav bjatzsj ~}anj) czt de tljod~taenj+t`+j|jk}ias+j ~}anj ,~jj Bdfzsj 3%1$% Ijt z~ ojfiej ojd t` cj tlj B odkje~d`eai
e`e+ejfatdrj ’od~taenj ~qzasjo rjnt`s‛ cjtvjje |j aeo tlj de}zt |d) aeo ojdXh[ tlj
h+tl n`k}`ejet `b tld~ rjnt`s% Vj vj nae tlz~ vsdtj j|jk}ias j—~ od~taenj bzentd`e a~
b`ii`v~7
Oj,|d$ ; ,|j ∓ |d$_ V j,|j ∓ |d$ ,3%1$
Oj,|d$ ; v_ j ojd ,3%3$
vj ; odaf,V j$ ,3%>$
ojdXh[ ; ,|jXh[ ∓ |dXh[$3 ,3%?$
3<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 29/130
n
a
c
+
+
+
j
a
c
n
+
+
Sav Bjatzsj ^}anj ’Od~taenj+t`+j|jk}ias‛
rjnt`s ~}anj
r~%
+
j
+
+
Bdfzsj 3%17 Ijasedef a od~taenj+bzentd`e de ’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj% De~tjao `b }jsb`sk+def ijasedef de tlj sav bjatzsj ~}anj ,vljsj tlj ojnd~d`e c`zeoas d~ e`e+idejas$) vj ka} tlj }s`cijkt` a od~taenj+t`+j|jk}ias ~}anj ,vljsj tlj ojnd~d`e c`zeoas d~ idejas$% _lj j|jk}ias) deodnatjo c j)d~ e`v }ianjo at tlj `sdfde) cjnaz~j dt~ `ve od~taenj+t +j|jk}ias vdii cj =% Ae oata }`det |d nae cjka}}jo t` tlj }`~dtdrj }`det ojd de tlj od~taenj+t`+j|jk}ias ~}anj a~ deodnatjo c Jqzatd`e 3%?%
B`skziatdef tlj }s`cijk de a ’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj) vj nae ~jj
tlat ae a}}idnatd`e `b tlj od~taenj bzentd`e d~ ~dk}i a o`t }s`oznt de tlj ejv ~}anj
,Jqzatd`e 3%3$% Ae dkkjodatj aoraetafj d~ tlat dt aii`v~ z~ t` z~j `ej `b tlj kae
vjii+zeojs~t``o t``i~ fjasjo t`vaso~ ijasedef idejas ojnd~d`e c`zeoasdj~%
3%1 Ijasedef Od~taenj Bzentd`e~ Vdtl Bsdjeo~
B`s janl j|jk}ias vj ijase vj c b`skdef a ~z}jsrd~jo+ijasedef }s`cijk bs`k tlj
iacji~ a~~`ndatjo vdtl tsadedef j|ak}ij~% L`vjrjs) ~denj tljsj d~ a fsjat ojai `b rd~zai
odrjs~dt vdtlde a ~defij `cgjnt natjf`s ,a~ asfzjo de ^jntd`e 1%1%1$) vj vaet t` ar`do
b`sndef ae j|jk}ias t` cj ~dkdias t` aii de+nia~~ de~taenj~% De~tjao `b z~def aii `tljs
34
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 30/130
`cgjnt de~taenj~ vdtl tlj ~kaii nia~~ a~ j—~ }`~dtdrj~) vj ijt tlj ijasedef }s`nj~~ nl``~j
vldnl j|ak}ij~ j d~ antzaii ~dkdias t` ,sjbjssjo t` a~ j—~ bsdjeo~$% B`s tld~ }zs}`~j)
vj z~j ae j|jk}ias+~}jndfin cdeas rjnt`s αj ,vl`~j ijeftl d~ jqzai t` tlj ezkcjs `b
j|jk}ias~ vdtl tlj ~akj iacji a~ j$) vljsj tlj e`e+wjs` jijkjet~ `b αj deodnatj vldnl
j|ak}ij~ asj j—~ bsdjeo~% ^denj vj o` e`t he`v vldnl j|jk}ias~ asj rd~zaii ~dkdias
t` janl `tljs) vj asj banjo vdtl a nldnhje+aeo+jff }s`cijk aeo kz~t ijase vj aeo αj
g`deti% Ai~`) a~ d~ n`kk`e de dkcaiaenjo nia~~dfinatd`e }s`cijk~) vj ijase a cda~ tjsk
cj de aoodtd`e t` tlj `sdjetatd`e `b tlj l}js+}iaej vj%
B`s ijasedef) vj ~tast `zt vdtl a iasfj aee`tatjo ~jt `b `cgjnt j|jk}ias~ x,|d) td${E d;1)
vljsj td ∌ x1) % % % ) K { deodnatj~ tlj nia~~ tlat janl j|ak}ij cji`ef~ t`% Vj deoj}je+
ojeti ijase a ~j}asatj od~taenj bzentd`e b`s janl j|jk}ias) tlz~ `zs a}}s`anl ~naij~
idejasi vdtl tlj ezkcjs `b j|jk}ias~ E % Cjnaz~j b`s janl j|jk}ias—~ ijasedef }s`cijk)
vj ~`irj a tv`+nia~~ }s`cijk) `zs a}}s`anl d~ e`t ~je~dtdrj t` tlj `rjsaii ezkcjs `b
`cgjnt nia~~j~) K % Fdrje a ~defij j|ak}ij) vj }astdtd`e aii `tljs `cgjnt~ det` tlj de+nia~~
j|ak}ij~ ~jt) N j ; x g 7 t g ; tj{) aeo dt~ n`k}ijkjet) tlj `zt+nia~~ ~jt%
_lj `zt}zt `b }js+j|jk}ias ijasedef d~ a ~defij vjdflt rjnt`s) a ~naias cda~) aeo tljj|jk}ias—~ bsdjeo deodnat`s rjnt`s7
Xv∖j) c∖j)α∖
j[ ; asfkdev)c)α
™,v) c)α$ ,3%:$
Ijasedef tlj od~taenj bzentd`e d~ }jsb`skjo c `}tdkdwdef tlj b`ii`vdef `cgjntdrj
bzentd`e b`s janl j|jk}ias ,~denj vj deoj}jeojeti ijase janl j|jk}ias—~ od~taenj
bzentd`e) vj os`} tlj j ~zc~nsd}t b`s niasdt$7
™,v) c)α$ ;θ
3yyvyy3 (
d∌N
αd I,∓,v_ od ( c$$ (d/∌N
I,v_ od ( c$ ∓ ώyyα yy3 ,3%2$
>=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 31/130
v ≪ =
α ∌ x=) 1{yN y
De `zs b`skziatd`e) ijt I,|$ cj a i`~~ bzentd`e) ~znl a~ tlj ldefj+i`~~ Ildefj,|$ ;
ka|,1 ∓ |) =$ `s tlj ~qzasjo ldefj+i`~~ Ildefj∓~q,|$ ; ka|,1 ∓ |) =$3) aeo ijt N cj tlj
~jt `b j|jk}ias~ tlat larj tlj ~akj iacji a~ tlj b`nai j|jk}ias ,d%j%) tlj j|jk}ias vl`~j
od~taenj bzentd`e vj asj ijasedef$% Vj dedtdaii z~jo tlj n`e~tsadet tlat
d αd ; H )
vljsj H d~ tlj tasfjt ezkcjs `b j|jk}ias~ vj b`snj t` cj ~dkdias t` j XKaid~djvdnw aeo
Jbs`~) 3==<[) czt iatjs b`zeo tlat z~def a tjsk vldnl aii`v~ janl j|jk}ias t` ~jijnt
a rasdacij ezkcjs `b bsdjeo~ kahj~ k`sj ~je~j ,tld~ d~ tlj ia~t tjsk) k`oziatjo c
ώ$ XKaid~djvdnw aeo Jbs`~) 3==4[% @e `ej laeo) c `}tdkdwdef tlj od~taenj bzentd`e
ijasedef `cgjntdrj bzentd`e vj ~tsdrj t` kadetade a ka|+kasfde ~j}asatd`e cjtvjje tlj
j|jk}ias }iz~ dt~ ~jt `b bsdjeo~ aeo tlj ~jt `b ejfatdrj j|ak}ij~% @ej tlj `tljs laeo)
vj ts t` ~jijnt tlj bsdjeo~ ~znl tlat tlj aii`v b`s a iasfj ~j}asatd`e) }sjbjssdef t`
nl``~j a iasfj ezkcjs `b bsdjeo~%
Vdtl`zt tlj α }asakjtjs aeo vdtl e` n`e~tsadet `e v) Jqzatd`e 3%2 d~ gz~t tlj
}sdkai b`sk `b kae n`erj| ~tatd~tdnai ijasedef tjnledqzj~ ,~znl a~ I`fd~tdn Sjfsj~~d`e
aeo ^z}}`st Rjnt`s Kanldej~$% De `zs na~j vj z~j tlj n`erjetd`e tlat ~kaiijs raizj~ `b
v_ od dk}i a fsjatjs ojfsjj `b ~dkdiasdt t` j|jk}ias j ,tlj ~jkaetdn~ `b a od~taenj$)
~` tlj }`~dtdrdt n`e~tsadet `e v d~ kjaet t` je~zsj tlat a iasfj jijkjetas od~taenj
nae ejrjs dk}i a rjs ~kaii ,}`tjetdaii ejfatdrj$ od~taenj% _ld~ ai~` je~zsj~ tlat
tlj j|jk}ias la~ ka|dkai ~dkdiasdt vdtl dt~jib% ^denj tlj }sj~jenj `b tlj cdeas α—~
sjeojs~ tlj }s`cijk e`e+n`erj| ,a~ d~ n`kk`e de iatjet rasdacij k`oji~$) vj }s`njjo
dtjsatdrji j~tdkatdef α fdrje v aeo j~tdkatdef v fdrje α% Ozsdef janl dtjsatd`e) vj
>1
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 32/130
asj fzasaetjjo t` ejrjs densja~j tlj raizj `b `zs `cgjntdrj bzentd`e ,Jqzatd`e 3%2$ aeo
tlz~ jbfindjeti fieo a i`nai kdedkzk% Vj ~tast vdtl ae dedtdai od~taenj bzentd`e v= aeo
csjah o`ve Jqzatd`e 3%2 c dtjsatdrji ~`irdef tlj b`ii`vdef tv` n`erj| ~zc+}s`cijk~7
αh ; asfkde
α
d∌N
αdI,∓vh ¶ od$ ∓ ώyyα yy3 ,3%6$
Xvh(1) ch(1[ ; asfkde,v)c$
θ
3yyvyy3 (
d7αhd;1
I,∓,v ¶ od$ ( c$ (d/∌N
I,v ¶ od ( c$ ,3%<$
Fdrje vh) dt d~ tsdrdai t` ~`irj b`s αh ,Jqzatd`e 3%6$ cjnaz~j janl αhd nae cj ~`irjo
b`s deoj}jeojeti7
αh ; asfkde
α
d∌N
αdI,∓vh ¶ od$ ∓ ώyyα yy3 ,3%4$
αh ; asfkde
α
d∌N
αd,I,∓vh ¶ od$ ∓ ώ$ ,3%1=$
αhd ; asfkde
αd
αd,I,∓vh ¶ od$ ∓ ώ$ ,3%11$
αhd ;
=) db I,∓vh ¶ od$ ≪ ώ
1) db I,∓vh ¶ od$ 9 ώ
,3%13$
_lj ~zc+}s`cijk `b ijasedef vh(1 fdrje αh ,Jqzatd`e 3%<$ tahj~ tlj b`sk `b a
nia~~dnai n`erj| ~z}jsrd~jo ijasedef }s`cijk% Cjnaz~j vj z~j a ldefj+i`~~) vj z~j tlj
}zcidni aradiacij }sdkai ^RK ~`irjs bs`k XNla}jiij) 3==6[ ,vdtl a ~idflt k`odfinatd`e
t` laeoij tlj }`~dtdrdt n`e~tsadet$% _lj jetdsj }s`njozsj n`erjsfj~ vljeα
h(1
;α
h
)~denj tld~ dk}idj~ tlat vh(1 ; vh% Abtjs ~`irdef janl j|jk}ias—~ ijasedef }s`cijk)
vj ~naij tlj sj~zitdef od~taenj bzentd`e ~znl tlat a od~taenj raizj `b 1 ,de~tjao `b =$
n`ssj~}`eo~ t` tlj ojnd~d`e c`zeoas aeo a raizj `b = ,de~tjao `b ∓vE B $ n`ssj~}`eo~
t` }jsbjnt ~dkdiasdt%
>3
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 33/130
3%3 Sjiatjo V`sh
I`nai ijasedef la~ a i`ef ld~t`s ‟ tjnledqzj~ ca~jo `e i`naii vjdfltjo idejas sjfsj~+
~d`e X Athj~`e jt ai%) 1446[ larj cjje ~znnj~~bzii a}}idjo t` `tljs o`kade~ ~znl a~
s`c`t n`ets`i% De rd~d`e) tlj v`sh `b Bs`kj jt ai% XBs`kj aeo Kaidh) 3==2) Bs`kj jt ai%)
3==6[ ojai~ vdtl a iasfj ezkcjs `b `cgjnt natjf`sdj~ aeo }jsb`sk~ j|jk}ias+katnldef
b`s tlj ta~h `b dkafj nia~~dfinatd`e ,`e Naitjnl 1=1$% Bs`kj—~ v`sh) vldnl va~ tlj fis~t
t` }s`}`~j ijasedef }js+j|jk}ias ~dkdiasdt kja~zsj~) la~ cjje rjs deffizjetdai de tlj
ojrji`}kjet `b `zs v`sh% Dt d~ }jsla}~ e`t ~zs}sd~def tlat Bs`kj—~ v`sh `e ijasedef
i`nai+od~taenj bzentd`e~ ,idhj `zs~$ d~ ai~` ljardi deffizjenjo c Vdttfje~tjde—~ doja `b
bakdi sj~jkciaenj~ XBs`kj) 3==6[%
L`vjrjs) dt d~ v`stl ldflidfltdef tlj cdffj~t odbbjsjenj cjtvjje `zs v`sh aeo tlat
`b XBs`kj jt ai%) 3==6[7 vldij vj aoosj~~ tlj }s`cijk `b `cgjnt i`naidwatd`e) XBs`kj
jt ai%) 3==6[ `ei n`e~dojsjo dkafj nia~~dfinatd`e% At tlj sj}sj~jetatd`eai ijrji) XBs`kj
jt ai%) 3==6[ z~jo i`nai bjatzsj katnldef) aii`vdef odbbjsjet rdjv~ `b tlj ~akj `cgjnt
t` }`tjetdaii cj katnljo) vldij vj z~jo kznl k`sj sdfdo a~~jkcidj~ `b bjatzsj~
je~zsdef tlat osa~tdnaii odbbjsjet rdjv~ `b ae `cgjnt vdii e`t cj n`ebz~jo ,jrje db ~`kj
tsae~b`skatd`e `b i`nai bjatzsj~ n`zio }ianj tlje de n`ssj~}`eojenj$% Aoodtd`eaii) vj
z~j tlj cj~t katnldef j|jk}ias~ b`s detjs}sjtatd`e vldnl kjae~ tlat dt d~ kznl k`sj
dk}`staet de `zs bsakjv`sh t` `ctade rd~zaii ~dkdias katnlj~) vldij XBs`kj jt ai%)
3==6[ `ei jraizatjo tljds a}}s`anl `e natjf`s }sjodntd`e%
N`eejntd`e t` e`e+n`erj| ijasedef% _ljsj d~ ai~` ae dk}`staet n`eejntd`e cj+
tvjje `zs z~j `b bsdjeo deodnat`s rasdacij~ de Jqzatd`e 3%2 aeo tlj S`cz~t ^z}}`st
Rjnt`s Kanldej tsadedef aif`sdtlk `b X \z jt ai%) 3==2[% _lj ~l`v tlat tlj z~j `b
deodnat`s rasdacij~ d~ jqzdraijet t` z~def tlj Sak}+I`~~) a tszenatjo rasdaet `b tlj ldefj
i`~~ vldnl sjtzse~ a n`e~taet i`~~ b`s oata }`det~ ~zbfindjeti bas bs`k tlj kasfde%
>>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 34/130
Isak},|$ ; kde,1) Ildefj,|$$
Ildefj,|$ ; ka|,=) 1 ∓ |$
Vlat tld~ kjae~ d~ `zs od~taenj bzentd`e b`skziatd`e n`zio cj sj+vsdttje vdtl`zt
ae cdeas rasdacij~ ‟ tlj ~akj jbbjnt n`zio cj `ctadejo c sj}iandef tlj n`erj| i`~~
bzentd`e `e tlj }`~dtdrj~ vdtl tlj e`e+n`erj| Isak}% X \z jt ai%) 3==2[ ai~` ~l`vjo tlat
vlje z~def ae aitjseatdef `}tdkdwatd`e aif`sdtlk) dt o`j~ e`t kattjs vljtljs vj b`snj
αd t` cj cdeas `s de tlj saefj X=) 1[ ~denj tlj ~`iztd`e vdii aiva~ cj jdtljs = `s 1%
3%> Jraizatdef I`nai Od~taenj Bzentd`e~ `e IacjiKj
Vj asj zitdkatji detjsj~tjo de }as~def a ~njej det` dt~ n`e~tdtzjet `cgjnt~ ‟ zeojs~taeo+
def a~ kznl a~ vj nae ac`zt ae de}zt dkafj% O`def ~` b`s a sja~`eaci fjejsai nia~~
`b dkafj~ sjqzdsj~ laeoidef a iasfj ezkcjs `b odbbjsjet `cgjnt~ tlat `nnzs de jrjsoa
idbj% _ljsjb`sj) tlj nl`dnj `b tlj sdflt tsadedef oata d~ `b tlj ztk`~t dk}`staenj% @b aii
tlj nzssjeti aradiacij oata~jt~) tlj `ei `ej n`etadedef a iasfj ezkcjs `b sjai+v`sio
~njej~) vdtl a vdoj rasdjt `b jrjsoa `cgjnt~ tlat asj e`t `ei iacjijo czt ai~` ~jf+
kjetjo) d~ tlj IacjiKj oata~jt XSz~~jii jt ai%) 3==<[% IacjiKj d~ ae `ef`def `eidej
dkafj+aee`tatd`e jbb`st der`irdef kae iacjijs~% A~ a sj~zit) e`t `ei asj tlj dkafj~
z~js+n`etsdcztjo) ~}aeedef a vdoj saefj `b ~njej~) czt z~js~ asj bsjj t` iacji janl `cgjnt
vdtl ae Jefid~l tj|t ~tsdef tlj idhj) }s`rdodef a f``o ~ak}idef `b tlj od~tsdcztd`e `b
`cgjnt eakj~ ’de tlj vdio‛%
A~ a ~`zsnj `b j|jk}ias~) vj z~j a ~zc~jt `b IacjiKj vldnl n`e~d~t~ `b `rjs :===
dkafj~% Abtjs dfe`sdef tde `cgjnt~) vj nijae z} tlj `cgjnt aee`tatd`e~ c od~nasodef
>?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 35/130
az|didas v`so~ bs`k tlj iacji~ ,z~def tlj bzentd`e }s`rdojo de IacjiKj t``ihdt$) aeo
hjj} aii `cgjnt~ vl`~j zedqzj iacji `nnzs~ at ija~t : tdkj~% _ld~ fdrj~ z~ a t`tai `b
13) 4=: `cgjnt~ ~}aeedef 161 zedqzj iacji~%
3%>%1 ^jfkjet+ca~jo Bjatzsj~
Vj sj}sj~jet janl `cgjnt c E B ; 1? odbbjsjet bjatzsj~% _lj bjatzsj~ na}tzsj odbbjsjet
a~}jnt~ `b ~la}j) tj|tzsj) n`i`s) aeo dkafj i`natd`e b`s ae dkafj ~jfkjet ,~jj _acij
3%1$% _ na}tzsj deb`skatd`e ac`zt ~la}j vj n`k}ztj7 tlj njetjsjo `cgjnt ka~h de
a nae`ednai >3 Ö >3 bsakj) tlj ~dwj `b tlj sjfd`e) aeo tlj ~dwj `b sjfd`e—~ c`zeodef
c`|% _` na}tzsj tj|tzsj vj n`k}ztj e`skaidwjo tj|t`e ld~t`fsak~ de tlj detjsd`s `b
tlj `cgjnt) aeo) ~j}asatji) ai`ef tlj c`zeoasdj~ `b tlj `cgjnt% B`s n`i`s vj n`k}ztj
tlj kjae SFC+raizj) dt~ ~taeoaso ojrdatd`e) a~ vjii a~ a n`i`s ld~t`fsak% Bdeaii) t`
na}tzsj he`vijofj ac`zt tlj }`~dtd`e `b tlj ~jfkjet de ae dkafj) vj n`k}ztj a n`as~j
,cizssjo$ < Ö < ac~`iztj ~jfkjetatd`e ka~h a~ vjii a~ tlj e`skaidwjo ljdflt `b tlj
t`}+k`~t aeo c`tt`k+k`~t }d|ji de tlj sjfd`e%
3%>%3 Od~taenj Bzentd`e Ijasedef Ojtadi~
Vj ojfiej janl `b `zs od~taenj bzentd`e~ a~ a idejas n`kcdeatd`e `b jijkjetas od~taenj~%
Vj z~j 1? n`i`s) ~la}j) tj|tzsj) aeo i`natd`e bjatzsj~ t` sj}sj~jet j|jk}ias~% De~tjao `b
}jsb`skdef ijasedef de `ej g`det bjatzsj ~}anj nsjatjo c n`enatjeatdef tlj 1? odbbjsjet
bjatzsj~) vj }jsb`sk ijasedef de tlj sjoznjo 1?O jijkjetas+od~taenj ~}anj% _lz~
tlj od de Jqzatd`e 3%2 d~ tlj rjnt`s `b 1? Jznidojae od~taenj~ cjtvjje tlj j|jk}ias
vl`~j ~dkdiasdt vj asj ijasedef ,tlj b`nai j|jk}ias$ aeo tlj d+tl j|ak}ij% Vj z~j
I,|$ ; ka|,1 ∓ |) =$3) tlj ldefj+~qzasjo i`~~ bzentd`e) b`s ijasedef vldnl aii`v~ z~ t`
z~j a ~jn`eo `sojs ^RK ~`irjs XNla}jiij) 3==6[%
>:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 36/130
_}j Eakj Odkje~d`e
^la}jNjetjsjo Ka~h >3|>3;1=3?CC J|tjet 3
]d|ji Asja 1
_j|tzsj
Sdflt C`zeoas _j|+Ld~t 1==_ } C`zeoas _j|+Ld~t 1==
Ijbt C`zeoas _j|+Ld~t 1==C`tt`k C`zeoas _j|+Ld~t 1==
Detjsd`s _j|+Ld~t 1==
N`i`s
Kjae N`i`s >
N`i`s ~to >N`i`s Ld~t`fsak >>
I`natd`e Ac~`iztj Ka~h <|<;2?_ } Ljdflt 1C`t Ljdflt 1
_acij 3%17 ^jfkjet Oj~nsd}t`s7 1? Jijkjetas Bjatzsj~ z~jo t` sj}sj~jet `cgjnt~% Jijkjetas od~taenj~ asj ~dk}i tlj I3 od~taenj~ cjtvjje n`ssj~}`eodef bjatzsj rjnt`s~%
B`s jrjs j|jk}ias vj dedtdaidwj tlj od~taenj bzentd`e ijasedef }s`nj~~ vdtl v= ;
vtj|t`e) vljsj vtj|t`e la~ wjs`~ ai`ef aii jijkjetas od~taenj odkje~d`e~ j|nj}t tlj de+
tjsd`s tj|t`e ld~t`fsak bjatzsj ,a caf+`b+v`so~ bjatzsj$% Bs`k tlj odrjs~j n`kcdeatd`e~
`b ljzsd~tdnaii ojfiejo od~taenj~/vjdflt~ vj j|}jsdkjetjo vdtl) vtj|t`e }jsb`skjo
tlj cj~t b`s a vdoj assa `b `cgjnt t}j~) aeo vj z~j dt a~ tlj ca~jidej ’ijasedef+bsjj‛
ejasj~t+ejdflc`s kjtl`o%
>2
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 37/130
nas
nas nas ~zr nas nas
nas nas nas nas
nas
nas nas~ nas nas
nas nas nas nas
nas
nas nas nas nas
nas nas nas nas
tsjj b`sj~t tsjj rjfjtatd`e
}js~`e }js~`e }js~`e~taeodef v`kae }js~`e}js~`e
N j e t j s j o K a ~ h
C C
J | t j e t
_ ` }
_ j | t e ~
S d f l t _ j | t ` e ~
N ` i ` s ~ t o
K j a e N i ` s
D e t j
s d s _ j | t ` e ~
C ` t _ j | t e ~
I j b t _ j | t ` e ~
] d |
j i A s j a
_ ` }
L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N i ` s ~ t o
K j a e N ` i ` s
D e t j s d ` s _ j | t ` e ~
C ` t _ j | t ` e ~
I j b t _ j | t ` e ~
] d | j i A s j a
_ ` } L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N ` i s ~ t o
K j a e N ` i s
D e t j s d ` s _ j | t ` e ~
C ` t _ j | t ` e ~
I j b t _ j | t e ~
] d | j i A s j a
_ ` } L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N i ` s ~ t o
K j a e N i ` s
D e t j s d ` s _ j | t e ~
C ` t _ j | t ` e ~
I j b t _ j | t ` e ~
] d | j i A s j a
_ ` } L j d f l t
N ` i s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
Bdfzsj 3%37 J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 1/3% Fdrje ae j|jk}ias `e tlj t`}ijbt) tlj sjkadedef s`v ~l`v~ tlj t`} ? k`~t ~dkdias `cgjnt~ abtjs ijasedef a od~taenj bzentd`e% _lj
od~taenj bzentd`e d~ rd~zaidwjo a~ a od~tsdcztd`e `rjs jijkjetas od~taenj~ aeo ~l`ve de tlj c`tt`k ijbt%_lj ? j|jk}ias~ `e tlj c`tt`k sdflt asj tlj ? k`~t ~dkdias `cgjnt~ vdtl sj~}jnt t` tlj tj|t`e ld~t`fsakod~taenj%
>6
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 38/130
vdeo`v
czdiodef vdeo`v czdiodef vdeo`v
vdeo`v vdeo`v vdeo`v vdeo`v
vljji
k`t`scdhj tsjj kdss`s vljji
vljji vljji vljji vljji
czdiodef
~dojvaih nas nas nas
czdiodef czdiodef czdiodef czdiodef
czdiodef
vaii czdiodef }adetdef czdiodef
czdiodef czdiodef czdiodef czdiodef
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N i ` s ~ t o
K j a e N i ` s
D e t j s d ` s _ j | t ` e ~
C ` t _ j | t ` e ~
I j b t _ j | t ` e ~
] d | j i A s j a
_ ` } L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N i ` s ~ t o
K j a e N i ` s
D e t j s d ` s _ j | t ` e ~
C ` t _ j | t ` e ~
I j b t _ j | t ` e ~
] d | j i A s j a
_ ` } L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N i ` s ~ t o
K j a e N i ` s
D e t j s d ` s _ j | t ` e ~
C ` t _ j | t ` e ~
I j b t _ j | t ` e ~
] d | j i A s j a
_ ` } L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
N j e t j s j o K a ~ h
C C J | t j e t
_ ` } _ j | t ` e ~
S d f l t _ j | t ` e ~
N i ` s ~ t o
K j a e N i ` s
D e t j s d ` s _ j | t ` e ~
C ` t _ j | t ` e ~
I j b t _ j | t ` e ~
] d | j i A s j a
_ ` } L j d f l t
N i ` s L d ~ t
A c ~ i z t j K a ~ h
C ` t L j d f l t
Bdfzsj 3%>7 J|jk}ias A~~`ndatd`e de tlj IacjiKj _sadedef ^jt 3/3% Fdrje ae j|jk}ias `e tlj t`}ijbt) tlj sjkadedef s`v ~l`v~ tlj t`} ? k`~t ~dkdias `cgjnt~ abtjs ijasedef a od~taenj bzentd`e% _lj
od~taenj bzentd`e d~ rd~zaidwjo a~ a od~tsdcztd`e `rjs jijkjetas od~taenj~ aeo ~l`ve de tlj c`tt`k ijbt%_lj ? j|jk}ias~ `e tlj c`tt`k sdflt asj tlj ? k`~t ~dkdias `cgjnt~ vdtl sj~}jnt t` tlj tj|t`e ld~t`fsakod~taenj%
><
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 39/130
Detjs}sjtdef Ijasejo Od~taenj Bzentd`e~% Abtjs ijasedef tlj od~taenj bzentd`e~)
vj a}}i janl j|jk}ias—~ od~taenj bzentd`e t` aii `b tlj `tljs j|jk}ias~ ~jje ozsdef
tsadedef aeo ojfiej tlj ~z}}`st ~jt `b janl j|jk}ias a~ tlj j|jk}ias~ vldnl sjtzsejo a
~n`sj cji`v 1%=%
w ∌ ^z}},j$ ↟ Oj,|w$ 9 1%= ,3%1>$
De }santdnj tlj sj~zitdef ~z}}`st ~jt~ vdioi ras de ~dwj% B`s j|jk}ias~ bs`k fjejsdn
nia~~j~ ~znl a~ ’~h‛ vljsj vj j|}jnt kae ~hdj~ t` cj satljs ~dkdias) tlj ~z}}`st ~jt d~
iasfj% _lj ~z}}`st ~jt nae ai~` cj rjs ~kaii ‟ vldnl la}}je~ vlje dt~ n`ssj~}`eodef
j|jk}ias d~ jdtljs e`t rd~zaii od~tdentdrj `s akcdfz`z~i/den`ssjnti iacjijo% Vj }szej
ava tlj j|jk}ias~ vdtl ae jk}t ~z}}`st ~jt% ^jrjsai ijasejo od~taenj bzentd`e~
aeo tlj t`} jijkjet~ de tljds ~z}}`st ~jt~ asj ~l`ve aeo n`k}asjo t` tlj ejdflc`s~
fdrje a ~dk}ij tj|t`e+ld~t`fsak od~taenj de Bdfzsj 3%3 aeo Bdfzsj 3%>% _lj ijasejo
od~taenj bzentd`e~ asj o`def a f``o g`c at n`kcdedef jijkjetas od~taenj~ t` kja~zsj
~dkdiasdt% E`tdnj tlat ae j|jk}ias—~ ~z}}`st o`j~ e`t aiva~ n`etade j|jk}ias~ vdtl
tlj ~akj iacji% De }astdnzias ae j|jk}ias vdtl tlj iacji ’~taeodef }js~`e v`kae‛ va~
ojjkjo ~dkdias t` tlj tasfjt j|jk}ias vdtl iacji ’}js~`e‛ jrje tl`zfl tlj asj od~tdent
iacji~% Vj kja~zsj l`v `btje tld~ la}}je~ aeo ~l`v tlj t`} bjv jijkjet~ `b tlj iacji
n`ebz~d`e katsd| de _acij 3%3% E`tdnj tlat k`~t `b tlj~j n`ebz~d`e~ n`ssj~}`eo t` rd~zai
~t`} ~dfe ~dfe 6%<* s`ao ldflva s`ao >%?*}`ij ~tsjjtidflt 2%6* }adetdef }dntzsj >%?*
k`t`snnij k`t`scdhj 2%3* ~dojvaih s`ao >%3*
k`zetade~ k`zetade 2%3* ni`zo ~h >%1*fs`zeo fsa~~ ~dojvaih >%6* fsa~~ fs`zeo fsa~~ >%1*fsa~~ iave >%2* k`zetade k`zetade~ 3%6*
_acij 3%37 Iacji n`ebz~d`e abtjs od~taenj bzentd`e ijasedef% Vj ~l`v tlj t`} 13 iacji n`ebz~d`e~od~n`rjsjo abtjs od~taenj bzentd e ijasedef% B`s j|ak}ij) 6%<* `b tlj tdkj a ’~t`} ~dfe‛ j|jk}ias
a~~`ndatjo vdtl a ’~dfe‛ j|jk}ias~% _ld~ ~zffj~t~ tlat rd~zai ~dkdiasdt nae aiijrdatj ~`kj `b tlj }s`cijk~ vdtl akcdfz`z~ iacji~%
>4
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 40/130
~e`ek~%
3%>%> ^jfkjet Iacjidef
De `sojs t` ojtjskdej db tlj od~taenj bzentd`e~ asj `rjs+fittdef) vj n`e~dojs a ~jfkjet+
iacjidef ta~h `e a ljio+`zt ~zc~jt `b IacjiKj% @zs jraizatd e z~j~ a tj~t ~jt `b 1?6
`zto``s dkafj~ aii n`kdef bs`k `ej ~}jndfin ~zcb`iojs de IacjiKj ,t` kdedkdwj tlj
nlaenj~ `b ~dkdias oata cjdef z~jo b`s tsadedef aeo tj~tdef$% _ld~ tj~tdef ~zc~jt n`etade~
a t`tai `b 1) 1?2 `cgjnt~% B`s tlj ~jfkjet+iacjidef ta~h) vj sj}sj~jet c`tl j|jk}ias~
aeo tj~tdef sjfd`e~ c ~jfkjet+ca~jo bjatzsj~ ,a~ oj~nsdcjo de ^jntd`e 3%>%1$ j|tsantjo
bs`k tlj fs`zeo+tsztl ~jfkjetatd`e ka~h~% _lj k`sj odbfinzit }s`cijk `b azt`katdnaii
j|tsantdef `cgjnt~ bs`k dkafj~ vlje e` fs`zeo+tsztl ~jfkjetatd`e~ asj fdrje d~ ojiajo
zetdi ^jntd`e 3%>%?% Vj iacji tlj tj~tdef ~jfkjet c tlj iacji bs`k tlj ~defij k`~t
rd~zaii ~dkdias j|jk}ias z~def tlj ijasejo od~taenj~% Vj n`e~dojs a ~jt `b od~taenj
tlsj~l`io~ aeo n`k}ztj tlj }sjnd~d`e rjs~z~ sjnaii nzsrj% ]sjnd~d`e kja~zsj~ tlj
}s`cacdidt tlat a sjtzsejo iacji d~ dojetdnai t` tlj fs`zeo tsztl iacji aeo de `zs na~j
sjnaii kja~zsj~ tlj bsantd`e `b ~jfkjet~ tlat fjt iacjijo% A~ a ca~jidej) vj n`k}asj tlj
}jsb`skaenj `b `zs ijasejo od~taenj bzentd`e~ t` a ejasj~t+ejdflc`s nia~~dfijs z~def a
tj|t`e ld~t`fsak od~taenj% _lj }sjnd~d`e+sjnaii nzsrj nae cj ~jje de Bdfzsj 3%?% Db vj
`ei detjs}sjt tlj od~taenj bzentd`e~ tlat sjtzse a raizj cji`v 1%= ,vlat `zs ijasedef
b`skziatd`e ~zffj~t~ d~ tlj cj~t tldef t` o`$) vj `ctade iacji~ b`s 2=* `b `cgjnt~ vdtl a
}sjnd~d`e `b 41*% _ld~ ~zffj~t~ tlat tlj ijasejo od~taenj bzentd`e~ asj }s`rdodef a rjs
kjaedefbzi od~taenj b`s sjn`fedtd`e%
?=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 41/130
= =%3 =%? =%2 =%< 1=
=%3
=%?
=%2
=%<
1
Sjnaii'
] s j n d ~ d ` e
Ijasejo Od~taenj Bzentd`e~ b`s ^jfkjet Iacjidef
Ijasejo Od~taenj Bzentd`e~
O91 tlsj~l`io }`det
_j|t`e Ld~t`fsak
Bdfzsj 3%?7 IacjiKj ~jfkjet iacjidef7 ]js+J|jk}ias Od~taenj Bzentd e~ r~% _j|t`e Ld~t`fsak~%Vj ijase a odbbjsjet od~taenj bzentd`e b`s janl j|jk}ias aeo a}}i tljk t` a ljio+`zt ~jt `b `cgjnt~
bs`k IacjiKj% ^jfkjet bjatzsj~ asj `ctadejo bs`k fs`zeo+tsztl ~jfkjetatd`e~% E`tj tlat tlj Od~taenjBzentd`e~ asj ~dfedfinaeti cjttjs tlae _j|t`e ld~t`fsak~ de tlj ldfl+}sjnd~d`e) i`v+sjnaii sjfdkj%
3%>%? ]as~def Dkafj~ rda Kzitd}ij ^jfkjetatd`e~
Vj aisjao ~av de ^jntd`e 3%> tlat j|jk}ias+~}jndfin od~taenj bzentd`e~ nae cj z~jo t`
ojtjskdej tlj dojetdt `b a fs`zeo+tsztl ~jfkjet) czt l`v nae vj z~j tljk t` ~jfkjet
`zt `cgjnt~ de~doj e`rji) zeiacjijo dkafj~0 Vj tanhij tld~ }s`cijk de tv` ~tj}~% Vj z~j
tlj kzitd}ij ~jfkjetatd`e a}}s`anl XL`djk jt ai%) 3==:)Sz~~jii jt ai%) 3==2)Kaid~djvdnw
aeo Jbs`~) 3==6[) vldnl va~ ~l`ve t` }s`rdoj f``o ~}atdai ~z}}`st b`s kae odbbjsjet
`cgjnt t}j~) t` fjejsatj a iasfj n`iijntd`e `b naeodoatj ~jfkjet~% Vj tlje hjj} tlj
e`e+sjozeoaet ~zc~jt `b ~jfkjet~ vldnl fjejsatjo vjii+~n`sdef rd~zai a~~`ndatd`e~
vdtl tlj ~jt `b j|jk}ias~%
De XKaid~djvdnw aeo Jbs`~) 3==6[) vj asfzjo at fsjat ijeftl tlat z~def kzitd}ij
~jfkjetatd`e~ d~ cjttjs tlae `ej ‟ tlj hj de~dflt d~ tlat vldij e` ~defij ~jfkjetatd`e d~
idhji t` }s`oznj aii f``o ~jfkjet~) ~`kj `b tlj ~jfkjet~ de ~`kj `b tlj ~jfkjetatd`e~
asj rjs f``o% Vj z~j a rasdaet `b tlj kzitd}ij ~jfkjetatd`e a}}s`anl XSz~~jii jt ai%)
3==2)Kaid~djvdnw aeo Jbs`~) 3==6[) t` fjejsatj a ’~`z} `b ~jfkjet~‛ de a }zsji c`tt`k+
?1
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 42/130
z} ba~ld`e ,e` `cgjnt he`vijofj d~ }s`rdojo$% De }astdnzias) vj ras tlj }asakjtjs~ `b
tv` ~jfkjetatd`e jefdej~ ‟ Kjae+^ldbt ca~jo JOD^@E XN`kaedndz aeo Kjjs) 3==3[
aeo E`skaidwjo Nzt~ X^ld aeo Kaidh) 3===[ ‟ t` fjejsatj kzitd}ij dkafj ~jfkjetatd`e~
b`s jrjs de}zt dkafj% ^denj vj larj ~l`ve de XKaid~djvdnw aeo Jbs`~) 3==6[ tlat ~`kj
n`k}`~dtj `cgjnt~ asj rjs zeidhji t` n`kj `zt a~ a ~defij ~jfkjet de ae ~jfkjetatd`e)
czt nae cj vjii a}}s`|dkatjo c a kjsfj `b a bjv aoganjet ~jfkjet~% _ljsjb`sj) vj
azfkjet `zs dedtdai ~`z} `b ~jfkjet~ c n`e~dojsdef tlj kjsfj~ `b 3 `s > aoganjet
~jfkjet~ a~ od~nz~~jo de XKaid~djvdnw aeo Jbs`~) 3==6[% _lj sj~zitdef c`tt k+z}
~jfkjet sj}sj~jetatd`e nae }s`rdoj sjfd`e~ vdtl f``o ~}atdai ~z}}`st b`s c`tl ~la}j+
bsjj ’~tzbb‛ `cgjnt~ ~znl a~ fsa~~) s`ao) aeo ~h a~ vjii a~ fi|jo+j|tjet ’tldef~‛ ~znl a~
nas~) cdnnij~) aeo }j`}ij% Ae aoodtd`eai aoraetafj `b z~def a c`tt k+z} kjnlaed~k
t` fjejsatj naeodoatj sjfd`e~ d~ tlat dt d~ deoj}jeojet `b tlj ezkcjs `b j|jk}ias~ `s
`cgjnt natjf`sdj~ z~jo de bzstljs }s`nj~~def ~tafj~%
_lj od~taenj bzentd`e~ ijasejo ~` bas asj e`t rjs f``o at sjn`fedwdef cao ~jfkjet~
‟ tlj ejrjs ~av ae de tsadedef! Vj tlz~ azfkjet tlj oata z~jo de od~taenj bzentd`e
ijasedef t` n`etade a iasfj ezkcjs ,`rjs >=) ===$ `b cao ~jfkjet~ vldnl na}tzsj tlj
a}}jasaenj `b }attjse~ tlat o` e`t n`ssj~}`eo t` ae `cgjnt~%
Abtjs vj fjejsatj tlj ~`z} `b c`tt`k+z} ~jfkjet~) vj n`k}ztj tlj bzii katsd| `b
od~taenj~ cjtvjje aii j|jk}ias~ aeo ~jfkjet~% Vj `ei n`e~dojs tlj od~taenj~ cji`v
1%= aeo tlj sj~zitdef a~~`ndatd`e~ asj rjs ~}as~j% @e arjsafj) ij~~ tlae %3* `b tlj
}`tjetdai a~~`ndatd`e~ asj antdrj% Qzaidtatdrj j|ak}ij~ `b j|jk}ias a~~`ndatd`e de tlj
~`z} `b c`tt`k+z} ~jfkjet~ nae cj ~jje de Bdfzsj 3%:% Qzdtj `btje) a ~defij ~jfkjet
vdii a~~`ndatj vdtl kae j|jk}ias~ aeo vj n`e~tsznt a sjn`fedtd`e ~n`sj `zt `b tlj id~t
`b a~~`ndatdef od~taenj~% Ijttdef J cj tlj id~t `b j|jk}ias~ a~~`ndatdef vdtl ~jfkjet ^
?3
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 43/130
tlj sjn`fedtd`e n`efiojenj ~,^) J $ d~ n`e~tszntjo a~ b`ii`v~7
~,^) J $ ; 1/j∌J
1
Oj,^ $,3%1?$
B`s jraizatd`e }zs}`~j~) vj z~j tlj ljio+`zt ~zc~jt `b IacjiKj ojfiejo de ^jntd`e 3%>%
Vj iacji janl `cgjnt l}`tlj~d~ vdtl tlj k`~t bsjqzjeti `nnzssdef iacji ak`ef dt~
a~~`ndatd`e~% Vj ai~` sjtade aii ~jfkjet~ tlat a~~`ndatj vdtl at ija~t `ej j|jk}ias) aeo
tlz~ larj kzitd}ij ,}`tjetdaii aii n`ssjnt$ `rjsia}}def `cgjnt l}`tlj~j~% ^denj vj o`
e`t vaet t` }jeaidwj b`s tlj~j aitjseatdrj a~~`ndatd`e~ vj ojfiej ojtjntd`e }sjnd~d`e
a~ b`ii`v~7 vj n`e~dojs ae `cgjnt l}`tlj~d~ t` cj n`ssjnt db dt la~ a ~jfkjet `rjsia}~n`sj ,ojfiejo a~ de XKaid~djvdnw aeo Jbs`~) 3==6[$ `b at ija~t =%: vdtl a fs`zeo tsztl
sjfd`e tlat la~ tlj ~akj dojetdnai iacji a~ tlj l}`tlj~d~% Vj n`e~dojs aii `cgjnt~ de
taeojk aeo o` e`t }jeaidwj b`s kzitd}ij n`ssjnt `rjsia}}def a~~`ndatd`e~% Vj ras tlj
sjn`fedtd`e n`efiojenj t` nsjatj tlj }sjnd~d`e rjs~z~ sjnaii nzsrj de Bdfzsj 3%2%
De `sojs t` qzaetdb l`v vjii vj ~jfkjet `cgjnt~) b`s janl n`ssjnt ojtjntd`e vj
kja~zsj tlj `rjsia} ~n`sj cjtvjje tlj a~~`ndatjo fs`zeo tsztl sjfd`e~ aeo tlj `cgjnt
l}`tlj~j~% Vj ~l`v tlj arjsafj `rjsia} ~n`sj a~ vj ras tlj sjn`fedtd`e n`efiojenj
aeo n`k}asj tlat t` tlj arjsafj `rjsia} ~n`sj `b tlj cj~t ~jfkjet de `zs ~`z} `b
~jfkjet~% _lj n`ssj~}`eodef }i`t nae cj ~jje `e tlj sdflt ~doj `b Bdfzsj 3%2%
_lj acdidt t` sjtzse a ~kaii ezkcjs `b `cgjnt l}`tlj~j~ vdtl ldfl qzaidt ~jfkje+
tatd`e ka~h~ d~ nszndai b`s dkafj zeojs~taeodef% Jrje tl`zfl tlj detjs}ia cjtvjje
`cgjnt~ ,j%f%) XSacde`rdnl jt ai%) 3==6[$ d~ njstadei a nszndai n`k}`ejet b`s ojtjskdedef
tlj dojetdt `b aii tlj ~njej—~ rd~zai jijkjet~) vj nae ~tdii nsjatj kjaedefbzi ,}astdai$
}as~j~ z~def `zs i`nai od~taenj bzentd`e~ ai`ej% Vj nsjatj ae dkafj }as~j bs`k `zs
`rjsia}}def `cgjnt l} tlj~j~ a~ b`ii`v~7 fdrje a id~t `b `cgjnt l}`tlj~j~ de a ~defij
dkafj ~`stjo c tljds sjn`fedtd`e n`efiojenj aeo ae dedtdaii jk}t id~t `b `cgjnt~ de tlj
?>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 44/130
czdiodef czdiodef czdiodef czdiodef
czdiodef czdiodef czdiodef czdiodef
wjcsa ns`~~def wjcsa ns`~~def wjcsa ns`~~def wjcsa ns`~~def
nas
nas
nas
nas
nas
nas
nas
tsjj tsjjtsjjtsjj
~h ~h~h ~h
~dojvaih ~dojvaih~dojvaih ~dojvaih
^jfkjet IacjiKj J|jk}ias~
nas
Bdfzsj 3%:7 J|jk}ias A~~`ndatd`e de tlj IacjiKj tj~t ~jt% Janl j|ak}ij ~l`v~ a ~jfkjet fjejsatjobs`k kzitd}ij ~jfkjetatd`e~ ai`ef vdtl dt~ t`} ? j|jk}ias a~~`ndatd`e~%
??
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 45/130
1
@cgjnt Ojtjntd`e ]jsb`skaenj @cgjnt ^jfkjetatd`e Annzsan
Sjnaii Sjnaii
K j a e @ r j s i a } ^ n ` s j
]
s j n d ~ d ` e
Kjae @^
== ===%3 =%3
=%3=%3
=%?=%?
=%? =%?
=%2=%2
=%2 =%2=%< =%<
=%<=%<
1 1
11
Bdfzsj 3%27 Ojtjntdef aeo ^jfkjetdef `cgjnt~ de IacjiKj% C`tl nzsrj~ asj nsjatjo c rasdef tljsjn`fedtd`e n`efiojenj ,Jqzatd`e 3%1?$% _lj fis~t }i`t ~l`v~ tlj }sjnd~d`e+sjnaii nzsrj b`s tlj ta~h `b `cgjnt ojtjntd`e% A ojtjntd`e d~ ojjkjo n`ssjnt db dt sjtzse~ tlj ~akj iacji a~ vjii a~ la~ ae `rjsia} ~n`sj,@^$ fsjatjs tlae %: vdtl a fs`zeo+tsztl ~jfkjet% _lj ~jn`eo }i`t ~l`v~ tlj arjsafj ~jfkjetatd`eqzaidt `b n`ssjnt ojtjntd`e~ aeo n`k}asj~ tlat t` tlj kjae cj~t `rjsia} ~n`sj `b tlj de}zt kzitd}ij~jfkjetatd`e~%
}as~j) vj fsjjodi }ianj tlj nzssjet cj~t `cgjnt l}`tlj~j~ det` tlj id~t `b `cgjnt~ de tlj
}as~j vldij sjk`rdef aii l}`tlj~j~ tlat `rjsia} vdtl a ~n`sj `b =%: `s k`sj% _ld~ d~
jqzdraijet t` tlj t}dnai e`e+ka|dkzk ~z}}sj~~d`e aif`sdtlk ,a~ z~jo de XBjiwje~wvaic
jt ai%) 3==<[$) czt a}}idjo t` bsjj+b`sk ~jfkjet~ de~tjao `b c`zeodef c`|j~% _v`
sj~zitdef dkafj }as~def j|ak}ij~ nae cj ~jje de Bdfzsj 3%6%
_ld~ nla}tjs la~ ojk`e~tsatjo tlat kzitd}ij ~jfkjetatd`e~ aeo I`nai Od~taenj
Bzentd`e~ aii`v z~ t` }jsb`sk ojtjntd`e aeo ~jfkjetatd`e b`s a iasfj ezkcjs `b
odbbjsjet `cgjnt~% De aoodtd`e) vj larj ~l`ve tlat tlj detjfsai n`k}`ejet `b ~znl a
~jfkjet+tlje+sjn`fedwj a}}s`anl d~ tlj ijasedef `b j|jk}ias+~}jndfin od~taenj bzentd`e~%
3%>%: B`ii`v+z} V`sh
^denj dt~ }zcidnatd`e de 3==<) ~jrjsai sjiatjo a}}s`anlj~ larj cjje }zcid~ljo aeo dtd~ v`stlvldij kjetd`ejo tljk de tld~ tlj~d~% Sjfasodef tlj ’~jfkjet+tlje+sjn`fedwj‛
a}}s`anl) tlj a}}s`anl `b XId jt ai%) 3=1=[ la~ cjn`kj qzdtj }`}zias% XId jt ai%) 3=1=[
z~j fsa}l+nzt~ t` fjejsatj a ~`z} `b fifzsj+fs`zeo ~jfkjetatd`e~ aeo saeh tljk t`
}jsb`sk `cgjnt sjn`fedtd`e% @zs i`nai od~taenj bzentd`e bsakjv`sh la~ cjje j|tjeojo
?:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 46/130
t` tlj }s`cijk `b sjn`fedtd`e bs`k c`tl n`i`s aeo oj}tl ~je~`s~ XIad jt ai%) 3=11[% De
aoodtd`e) tljsj la~ cjje sd~def detjsj~t de i`nai od~taenj bzentd`e~ aeo kae odbbjsjet
b`skziatd`e~ larj cjje j|tje~drji aeaiwjo c Sakaeae aeo Cahjs XSakaeae aeo
Cahjs) 3=11[%
?2
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 47/130
czdiodef czdiodef czdiodef czdiodef
s`ao s`ao s`ao s`ao
nas nas nas nas
s`ao s`ao s`ao s`ao
czdiodef czdiodef czdiodef czdiodef
s`ao s`ao s`aos`ao
nas nasnasnas
nas nas nas
nas nas nas
~dojvaih
De}zt Dkafj ^jfkjet~ IacjiKj J|jk}ias~
Bdfzsj 3%67 Dkafj ]as~def rda Kzitd}ij ^jfkjetatd`e~ aeo I`nai Od~taenj Bzentd`e~% A }as~j d~nsjatjo c }jsb`skdef ~jfkjet+ca~jo e`e+ka|dkzk ~z}}sj~~d`e abtjs ~n`sdef ~jfkjet~ vdtl I`nai
Od~taenj Bzentd`e~%
?6
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 48/130
Nla}tjs >
J|jk}ias+^RK~
A kjsj ojnaoj af`) azt`katdnaii sjn`fedwdef jrjsoa `cgjnt~ de dkafj~ ,~znl a~ tlj
cz~ de Bdfzsj >%1$ va~ tl`zflt t` cj ae aik`~t ze~`iracij ta~h% Tjt t`oa) a ezkcjs `b
kjtl`o~ nae o` gz~t tlat vdtl sja~`eacij annzsan% Czt ijt z~ n`e~dojs tlj `zt}zt `b a
t}dnai `cgjnt ojtjnt`s ‟ a s`zfl c`zeodef c`| as`zeo tlj `cgjnt aeo a natjf`s iacji
,Bdfzsj >%1 ijbt$% Vldij tld~ kdflt cj ~zbfindjet b`s a sjtsdjrai ta~h ,’fieo aii cz~j~ de
tlj oataca~j‛$) dt ~jjk~ satljs ianhdef b`s ae ~`st `b ojj}js sja~`edef ac`zt tlj ~njej%
L`v d~ tlj cz~ `sdjetjo0 D~ dt a kded+cz~ `s a o`zcij+ojnhjs0 Vldnl }d|ji~ antzaii
cji`ef t` tlj cz~0 Vlat d~ dt~ s`zfl fj`kjts0 _lj~j asj aii rjs laso qzj~td`e~ b`s
a t}dnai `cgjnt ojtjnt`s% Czt vlat db) de aoodtd`e t` tlj c`zeodef c`|) vj asj acij t`
`ctade ae a~~`ndatd`e vdtl a rjs ~dkdias j|jk}ias bs`k tlj tsadedef ~jt ,Bdfzsj >%1
sdflt$) vldnl nae }s`rdoj a ldfl ojfsjj `b n`ssj~}`eojenj% ^zoojei) ae hdeo `b
kjta+oata }s`rdojo vdtl tlj tsadedef ~ak}ij ,a }d|ji+vd~j aee`tatd`e `s iacji ~znl a~
rdjv}`det) ~jfkjetatd`e) n`as~j fj`kjts) a >O k`oji) attsdcztj~) jtn%$ nae cj ~dk}i
tsae~bjssjo t` tlj ejv de~taenj%
?<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 49/130
Bdfzsj >%17 @cgjnt Natjf`s Ojtjnt`s r~% Je~jkcij `b J|jk}ias Ojtjnt`s~% @zt}zt `b a t}dnai `cgjntojtjnt`s d~ gz~t a c`zeodef c`| aeo a natjf`s iacji ,ijbt$% Czt `zs je~jkcij `b J|jk}ias+^RK~ d~ acij t`
a~~`ndatj janl ojtjntd`e vdtl a rd~zaii ~dkdias tsadedef j|jk}ias ,sdflt$) aii`vdef b`s odsjnt tsae~bjs `b kjta+oata ~znl a~ ~jfkjetatd`e) fj`kjts) jrje a >O k`oji ,c`tt`k$%
?4
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 50/130
Bdfzsj >%37 Natjf`s ^RK r~% J|jk}ias+^RK~% De~tjao `b tsadedef a ~defij }js+natjf`s nia~~dfijs) vj
tsade a ~j}asatj idejas ^RK nia~~dfijs b`s janl j|jk}ias de `zs oata~jt vdtl a ~defij }`~dtdrj j|ak}ij aeokdiid`e~ `b ejfatdrj vdeo`v~% Ejfatdrj~ n`kj bs`k dkafj~ e`t n`etadedef ae de~taenj~ `b tlj j|jk}ias—~natjf`s%
Vlat ~jjk~ oj~dsacij d~ ae a}}s`anl tlat la~ aii tlj ~tsjeftl~ `b a Oaiai+_sdff~ X Oaiai
aeo _sdff~) 3==:[ / Bjiwje~wvaic jt ai% XBjiwje~wvaic jt ai%) 3=1=[ ojtjnt`s „ }`vjsbzi
oj~nsd}t`s) jbfindjet od~nsdkdeatdrj bsakjv`sh) nijrjs kdedef `b laso+ejfatdrj~) jtn% „czt vdtl`zt tlj osavcanh~ dk}`~jo c a sdfdo) natjf`s+ca~jo sj}sj~jetatd`e `b tlj
}`~dtdrj~% _ }zt dt ae`tljs va) vlat vj vaet d~ a kjtl`o tlat d~ e`e+}asakjtsdn
vlje sj}sj~jetdef tlj }`~dtdrj~) czt }asakjtsdn ,`s at ija~t ~jkd+}asakjtsdn$ vlje
sj}sj~jetdef tlj ejfatdrj~% _ld~ d~ tlj hj k`tdratd`e cjldeo `zs J|jk}ias+ RK
a}}s`anl% Vlat vj }s`}`~j d~ a kassdafj `b tlj j|jk}ias+ca~jo kjtl`o`i`f) vldnl
aii`v~ z~ t` }s`}afatj sdnl aee`tatd`e~ bs`k j|jk}ias~ `et` ojtjntd`e vdeo`v~) vdtl
od~nsdkdeatdrj tsadedef) vldnl aii`v~ z~ t` ijase }`vjsbzi j|jk}ias+ca~jo nia~~dfijs~
bs`k ra~t ak`zet~ `b }`~dtdrj aeo ejfatdrj oata%
@zs `cgjnt ojtjnt`s d~ ca~jo `e a rjs ~dk}ij doja7 t` ijase a ~j}asatj nia~~dfijs b`s
janl j|jk}ias de tlj oata~jt ,~jj Bdfzsj >%3$% Vj sj}sj~jet janl j|jk}ias z~def a sdfdo
L@F tjk}iatj XOaiai aeo _sdff~) 3==:[% ^denj vj z~j a idejas ^RK) janl nia~~dfijs
nae cj detjs}sjtjo a~ a ijasejo j|jk}ias+~}jndfin L@F vjdflt rjnt`s% A~ a sj~zit)
de~tjao `b a ~defij n`k}ij| natjf`s ojtjnt`s) vj larj a iasfj n`iijntd`e `b ~dk}ijs
deodrdozai J|jk}ias+^RK ojtjnt`s~ `b rasd`z~ ~la}j~ aeo ~dwj~) janl ldfli tzejo t` tlj
j|jk}ias—~ a}}jasaenj% Czt) zeidhj a ~taeoaso ejasj~t+ejdflc`s ~nljkj) janl ojtjnt`s
d~ od~nsdkdeatdrji tsadejo% ^` vj asj acij t` fjejsaidwj kznl cjttjs vdtl`zt sjqzdsdef
:=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 51/130
Bdfzsj >%>7 L@F+katnldef n`k}asd~`e% Fdrje a cdnnij tsadedef ~ak}ij bs`k ]A^NAI ,sj}sj~jetjo vdtla L@F vjdflt rjnt`s v$) vj ~l`v tlj t`} 2 katnlj~ bs`k tlj ]A^NAI tj~t+~jt z~def tlsjj kjtl`o~% S`v 17 eadrj ejasj~t ejdflc`s ,z~def sav e`skaidwjo L@F$% S`v 37 _sadejo J|jk}ias+^RK ,e`tdnj l`v v
b`nz~j~ `e cdhj+~}jndfin jofj~$% S`v >7 Ijasejo od~taenj bzentd`e „ ae J|jk}ias+^RK czt tsadejo de tlj’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj) vdtl tlj j|jk}ias cjdef }ianjo at tlj `sdfde ,i``~ji n`ssj~}`eodef
t` XBs`kj aeo Kaidh) 3==2) Kaid~djvdnw aeo Jbs`~) 3==<[$%
ae je`sk`z~ oata~jt `b j|jk}ias~) aii`vdef z~ t` }jsb`sk ~zs}sd~defi vjii jrje `e a
k`ojsatji+~dwjo tsadedef oata~jt ~znl a~ tlj ]A^NAI R@N 3==6 XJrjsdeflak jt ai%)
3=1=[%
_lj ~dkdiasdt bzentd`e~ vj }s`}`~j de tld~ Nla}tjs tahj tlj b`sk `b a o`t }s`oznt
cjtvjje ae j|jk}ias—~ ijasejo vjdflt rjnt`s vj ,ai~` sjbjssjo t` a~ ae j|jk}ias+~}jndfin
tjk}iatj$ aeo |d) tlj sav bjatzsj~ sj}sj~jetdef ae de}zt `cgjnt ,}iz~ a cda~ cj$%
Oj,|d$ ; v_ j |d ( cj ,>%1$
_lj `zt}zt `b ~znl a ~dkdiasdt bzentd`e nae cj detjs}sjtjo a~ tlj ~dfejo od~taenj
bs`k tlj l}js}iaej sj}sj~jetjo c v_ j | ( cj ; =% _lj z~j `b a l}js}iaej }a+
sakjtjsdwatd`e d~ k`tdratjo c tlj fsjat ~znnj~~ `b tjk}iatj~ de tlj `cgjnt ojtjntd`e
n`kkzedt ,j%f%) kae `b tlj cj~t+}jsb`skdef a}}s`anlj~ asj tlj L@F+ca~jo Oaiai+
_sdff~/Bjiwje~wvaic jt ai% bakdi `b k`e`idtldn ojtjnt`s~$% L`vjrjs) zeidhj k`e`idtldn
a}}s`anlj~ vldnl ijase natjf`s+~}jndfin tjk}iatj~ ,j%f%) vo`f) vnas) vnlads) jtn$) `zs
tjk}iatj~ asj j|jk}ias+~}jndfin ,j%f%) vj1)vj3) % % % )vjE $ aeo tsadejo de a }js+j|jk}ias
ba~ld`e% De `sojs t` kadetade tlj ~}jndfindt `b tlj ijasejo tjk}iatj t` a ~defij j|jk}ias
:1
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 52/130
J|jk}ias Ijasejo v
Bdfzsj >%?7 Ijasedef ae J|jk}ias+~}jndfin L@F _jk}iatj% ^denj vj z~j idejas nia~~dfijs~) tlj sj~zitdef
ojnd~d`e c`zeoas nae cj detjs}sjtjo a~ ae j|jk}ias+~}jndfin L@F tjk}iatj% _lj rd~zaidwatd`e d~ tlj}`~dtdrj }ast `b tlj ijasejo l}js}iaej%
|j) vj ijase tlj tjk}iatj vj vdtl a ~defij }`~dtdrj de~taenj% _lj sj~zitdef vjdflt
rjnt`s nae tlje cj detjs}sjtjo a~ a tjk}iatj ldfli tzejo t` tlj de}zt j|jk}ias ,~jj
Bdfzsj >%?$% ^denj vj z~j tlj ~akj idejas ^z}}`st Rjnt`s Kanldej ,^RK$ a}}s`anl
a~ XOaiai aeo _sdff~) 3==:[ t` ijase vj) czt asj acij t` }s`oznj a tjk}iatj ldfli tzejo
t` tlj j|jk}ias |j) vj ozc tld~ a}}s`anl tlj J|jk}ias+^RK%
>%1 Vl` Ejjo~ bsdjeo~0 Ijasedef Vdtl a ^defij ]`~dtdrj
De~taenj
@ej v`zio dkafdej tlat tsadedef ae ^RK vdtl a ~defij }`~dtdrj j|ak}ij vdii caoi
`rjs+fit% Czt e`tj tlat vj sjqzdsj bas ij~~ bs`k a }js+j|jk}ias nia~~dfijs a~ n`k}asjo t`
a }js+natjf`s nia~~dfijs „ janl `b `zs ojtjnt`s~ `ei ejjo~ t` }jsb`sk vjii `e rd~zaii
~dkdias j|ak}ij~% ^denj janl nia~~dfijs d~ ~`irdef a kznl ~dk}ijs }s`cijk tlae de tlj
bzii+natjf`s na~j) vj nae z~j a ~dk}ij sjfziasdwjo idejas ^RK t` }sjrjet `rjs+fittdef%
Ae`tljs nszndai n`k}`ejet d~ tlat) vldij vj `ei larj a ~defij }`~dtdrj j|ak}ij) vj
:3
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 53/130
larj kdiid`e~ `b ejfatdrj j|ak}ij~ tlat vj kdej bs`k tlj tsadedef ~jt ,d%j%) bs`k dkafj~
tlat o` e`t n`etade ae de~taenj~ `b tlj j|jk}ias—~ natjf`s$% A~ a sj~zit) tlj j|jk}ias—~
ojnd~d`e c`zeoas d~ ojfiejo) de iasfj }ast) c vlat dt d~ e`t% @ej `b tlj hj n`etsdcztd`e~
`b `zs a}}s`anl d~ tlat vj ~l`v fjejsaidwatd`e d~ }`~~dcij bs`k a ~defij }`~dtdrj j|ak}ij
aeo a ra~t ~jt `b ejfatdrj~%
At tj~t+tdkj) vj deoj}jeojeti sze janl nia~~dfijs `e tlj de}zt dkafj ,~jj tlj ~idodef
vdeo`v od~nz~~d`e de A}}jeod| ~jntd`e C%3$ aeo z~j ~dk}ij e`e+ka|dkzk ~z}}sj~~d`e
t` nsjatj a fieai ~jt `b ojtjntd`e~) vljsj janl ojtjntd`e d~ a~~`ndatjo vdtl a ~defij
j|jk}ias% L`vjrjs) ~denj `zs deoj}jeojeti+tsadejo nia~~dfijs~ kdflt e`t `zt}zt odsjnti
n`k}asacij ~n`sj~) vj kz~t }jsb`sk naidcsatd e `e a raidoatd`e ~jt% _lj detzdtd`e
na}tzsjo c tld~ naidcsatd`e ~tj} d~ tlat odbbjsjet j|jk}ias~ vdii `bbjs osa~tdnaii odbbjsjet
fjejsaidwatd`e }`tjetdai% A ljardi `nnizojo `s tszenatjo `cgjnt de~taenj vdii larj
}``sjs fjejsaidwatd e tlae a nijaejs j|jk}ias) tlz~ s`cz~tej~~ afade~t jrje a ~defij
cao nia~~dfijs d~ dk}jsatdrj t` `ctadedef f``o `rjsaii }jsb`skaenj% ^denj `zs nia~~dfijs~
asj tsadejo vdtl`zt ~jjdef ae `tljs }`~dtdrj de~taenj~ czt dt~jib) vj nae z~j tljk b`s
naidcsatd`e de a ’ijarj+aii+czt+`ej+`zt‛ ba~ld`e%
Dt d~ v`stlvldij }`detdef `zt ~`kj `b tlj hj odbbjsjenj~ cjtvjje `zs a}}s`anl aeo
`tljs sjiatjo ^RK+ca~jo tjnledqzj~ ~znl a~ `ej+nia~~ ^RK~ X^nlih`}b jt ai%) 3==1) Nlje
jt ai%) 3==1[) kzitd+nia~~ hjseji ^RK~) hjseji+ijasedef a}}s`anlj~ X Rjoaiod jt ai%) 3==4[)
aeo tlj hEE+^RK aif`sdtlk XWlaef jt ai%) 3==2[% Aii `b tlj~j a}}s anlj~ sjqzdsj
ka}}def tlj j|jk}ias~ det` a n`kk`e bjatzsj ~}anj `rjs vldnl a ~dkdiasdt hjseji
nae cj n`k}ztjo ,vldnl vj ar`do$) czt k`sj dk}`staeti) hjseji kjtl`o~ i`~j tlj
~jkaetdn~ `b ~defij+j|jk}ias a~~`ndatd`e~ vldnl asj ejnj~~as b`s ldfl qzaidt kjta+oata
tsae~bjs%
Fdrje a ~jt `b tsadedef j|jk}ias~) vj sj}sj~jet janl j|jk}ias J rda a sdfdo L@F
tjk}iatj) |J % Vj nsjatj a oj~nsd}t`s bs`k tlj fs`zeo+tsztl c`zeodef c`| `b janl
:>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 54/130
Bdfzsj >%:7 J|jk}ias+^RK~% A bjv ’tsade‛ j|jk}ias~ vdtl tljds t`} ojtjntd`e~ `e tlj ]A^NAI R@N
tj~t+~jt% E`tj tlat janl j|jk}ias—~ L@F la~ dt~ `ve odkje~d`e~% E`tj ai~` l`v janl ojtjnt`s d~ ~}jndfine`t gz~t t` tlj tsade—~ `sdjetatd`e) czt jrje t` tlj t}j `b tsade%
j|jk}ias vdtl a njii ~dwj `b < }d|ji~ z~def a ~dwdef ljzsd~tdn vldnl attjk}t~ t` sj}sj~jet
janl j|jk}ias vdtl s`zfli 1== njii~% De~tjao `b vas}def janl j|jk}ias t` a nae`ednai
bsakj) vj ijt janl j|jk}ias ojfiej dt~ `ve L@F odkje~d`e~ sj~}jntdef tlj a~}jnt satd`
`b dt~ c`zeodef c`| ,~jj tlj oj~nsd}td`e `b tld~ }s`nj~~ de A}}jeod| ~jntd`e C%3$% Vj
nsjatj ejfatdrj ~ak}ij~ `b tlj ~akj odkje~d`e~ a~ |J c j|tsantdef ejfatdrj vdeo`v~)
E J ) bs`k dkafj~ e`t n`etadedef ae `cgjnt~ bs`k tlj j|jk}ias—~ natjf`s%
Janl J|jk}ias+^RK) ,vJ ) cJ $) tsdj~ t` ~j}asatj |J bs`k aii vdeo`v~ de E J c tlj
iasfj~t }`~~dcij kasfde de tlj L@F bjatzsj ~}anj% Ijasedef tlj vjdflt rjnt s vJ
ak`zet~ t` `}tdkdwdef tlj b`ii`vdef n`erj| `cgjntdrj7
™J ,v) c$ ; yyvyy3 ( N 1l,v_ |J ( c$ ( N 3|∌E J
l,∓v_ | ∓ c$ ,>%3$
Vj z~j tlj ldefj i`~~ bzentd`e l,|$ ; ka|,=) 1 ∓ |$) vldnl aii`v~ z~ t` z~j tlj
laso+ejfatdrj kdedef a}}s`anl t` n`}j vdtl kdiid`e~ `b ejfatdrj vdeo`v~ cjnaz~j tlj
~`iztd`e `ei oj}jeo~ `e a ~kaii ~jt `b ejfatdrj ~z}}`st rjnt`s~%
Bdfzsj >%> `bbjs~ a rd~zai n`k}asd~`e `b tlj }s`}`~jo J|jk}ias+^RK kjtl`o afade~t
tv` aitjseatdrj~ b`s tlj ta~h `b ojtjntdef tj~t+~jt katnlj~ b`s a ~defij j|jk}ias) a ~e`v+
:?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 55/130
n`rjsjo cdnnij% _lj fis~t s`v ~l`v~ a ~dk}ij ejasj~t+ejdflc`s a}}s`anl% _lj ~jn`eo
s`v ~l`v~ tlj `zt}zt `b `zs }s`}`~jo J|jk}ias+^RK% E`tj tlj ~zctij nlaefj~ de tlj
ijasejo L@F rjnt`s v) kahdef dt b`nz~ k`sj `e tlj cdnnij% _lj tldso s`v ~l`v~ tlj
`zt}zt `b ijasedef a od~taenj bzentd`e) satljs tlae a idejas nia~~dfijs% B`s tld~) vj a}}idjo
tlj ~defij+}`~dtdrj J|jk}ias+^RK bsakjv`sh de tlj ’od~taenj+t`+j|jk}ias‛ rjnt`s ~}anj)
vdtl tlj j|jk}ias cjdef }ianjo at tlj `sdfde ,tld~ d~ n`enj}tzaii ~dkdias t` XKaid~djvdnw
aeo Jbs`~) 3==<) Bs`kj aeo Kaidh) 3==2[$% Vj `c~jsrjo tlat tlj njetjsjo+at+j|jk}ias
n`e~tsadet kaoj tlj od~taenj bzentd`e ij~~ }`vjsbzi tlae tlj idejas nia~~dfijs ,~jj Sj~zit~
~jntd`e$% Bdfzsj >%: ~l`v~ a bjv J|jk}ias+^RK~ bs`k tlj ’tsade‛ natjf`s ai`ef vdtl
tljds t`} ojtjntd`e~ `e tlj tj~t+~jt% E`tj l`v ~}jndfin janl ojtjnt`s d~ „ e`t gz~t t` tlj
tsade—~ `sdjetatd`e) czt jrje tlj t}j `b tsade%
>%1%1 Sjiatd`e~ld} t` I`nai Od~taenj Bzentd`e~
^denj J|jk}ias+^RK~ asj tsadejo de a }js+j|jk}ias ba~ld`e) tlj cjas a ~ts`ef sj+
~jkciaenj t` tlj I`nai Od~taenj Bzentd`e~ ojfiejo de Nla}tjs 3% L`vjrjs) tljsj asj
~`kj hj odbbjsjenj~ vldnl asj v`stlvldij t` j|}iade de ojtadi% Zeidhj od~taenj bzen+
td`e~ XBs`kj aeo Kaidh) 3==2) Kaid~djvdnw aeo Jbs`~) 3==<[) vldnl sjqzdsj a ~jt `b
}`~dtdrj j|ak}ij~ ozsdef ijasedef) J|jk}ias+ RK~ `ei sjqzdsj a ~defij }`~dtdrj de+
~taenj% Vj aoodtd`eaii z~j kdiid`e~ `b dkafj vdeo`v~ a~ tlj ejfatdrj~ z~jo t` tsade
vj% _lj ~ldbt t`vaso~ z~def ij~~ }`~dtdrj oata aeo k`sj ejfatdrj oata d~ a e`tacij
nlaefj de }ldi`~`}l b`s ijasedef }js+j|jk}ias ~dkdiasdt kja~zsj~% C rdstzj `b cjdef
a l}js}iaej) tlj J|jk}ias+^RK ~dkdiasdt bzentd`e d~ e`t sjqzdsjo t` cj njetjsjo at
tlj j|jk}ias aeo aii`v~ z~ t` cjttjs aoa}t t` tlj tsadedef oata% Z~def a ~defij }`~dtdrj
de~taenj) vj acaeo`e tlj ejjo b`s ae iatjet rasdacij~ ,vldnl vjsj z~jo t` jen`oj
a rasdacij ezkcjs `b bsdjeo~ ozsdef od~taenj bzentd`e ijasedef de ^jntd`e 3$) aeo tlj
sj~zitdef }s`cijk d~ n`erj|% _ld~ kjae~ tlat tlj `}tdkdwatd`e }s`cijk der`irjo d~ kznl
::
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 56/130
ba~tjs t` ~`irj ,c z~def ldfli `}tdkdwjo ^RK }anhafj~ tzejo b`s iasfj+~naij }s`cijk~$
aeo vj o` e`t larj t` v`ss ac`zt i`nai kdedka ,a laiikash `b n`erj|dt$% K`sj
dk}`staeti) sjqzdsdef a ~defij }`~dtdrj de~taenj kjae~ tlat `zs oj}jeojenj `e natjf`s
iacji~ d~ kznl i`vjs tlae tlj od~taenj bzentd`e~ de ^jntd`e 3 ,vldnl sjqzdsjo tlj z~j
`b natjf`sdj~ t` ojfiej tlj }`~dtdrj nia~~$% _ld~ d~ ae dk}`staet ~tj} de tlj odsjntd`e `b
n`k}ijtji k`rdef ava bs`k natjf`sdj~ ozsdef ijasedef%
Cjnaz~j J|jk}ias+^RK~ nae cj detjs}sjtjo a~ j|jk}ias+~}jndfin ~dkdiasdt kja~zsj~)
`zs bsakjv`sh ~lasj~ ~`kj ~dkdiasdtdj~ vdtl od~taenj+ijasedef a}}s`anlj~) de }astdnzias
tl`~j tlat ijase }js+j|jk}ias od~taenj bzentd`e~ ,j%f%) XBs`kj aeo Kaidh) 3==2) Kai+
d~djvdnw aeo Jbs`~) 3==<[$% L`vjrjs) tlj nszndai odbbjsjenj cjtvjje a }js+j|jk}ias
nia~~dfijs aeo a }js+j|jk}ias od~taenj bzentd`e d~ tlat tlj iattjs b`snj~ tlj j|jk}ias
dt~jib t` larj tlj ka|dkaii attadeacij ~dkdiasdt% Ae J|jk}ias+^RK la~ kznl k`sj
bsjjo`k de ojfiedef tlj ojnd~d`e c`zeoas) aeo d~ cjttjs acij t` den`s}`satj de}zt bs`k
tlj ejfatdrj ~ak}ij~ ,~jj Bdfzsj >%> b`s a n`k}asd~`e) t` cj od~nz~~jo iatjs$%
>%1%3 Detjs}sjtdef ^RK~ tsadejo vdtl a ~defij }`~dtdrj de~taenj
_lj Sj}sj~jetjs _lj`sjk XValca) 146>[ tjii~ z~ tlat tlj ~`iztd`e t` ae ^RK }s`cijk
,a t}j `b I3+sjfziasdwjo ijasedef }s`cijk$ vdii tahj tlj b`ii`vdef b`sk7
vj ; αj|j (d∌E j
αd|d ,>%>$
Ljsj) αj d~ tlj ozai rasdacij a~~`ndatjo vdtl tlj ~defij }`~dtdrj de~taenj) aeo αd d~ a ozai
rasdacij a~~`ndatjo vdtl tlj ejfatdrj~% Vj larj `c~jsrjo tlat aii `b `zs J|jk}ias+^RK~
asj acij t` ~j}asatj tlj ~defij }`~dtdrj bs`k tlj ejfatdrj~) aeo tlat tlj ~z}}`st rjnt`s~
s`zfli baii `e tljds sj~}jntdrj kasfde% |j d~ `ej ~z}}`st rjnt`s vdtl αj ≌ 1) aeo vj
fjt a ~kaii ~jt `b ejfatdrj ~z}}`st rjnt`s~ vdtl αd ≌ ∓1% Aii e`e+~z}}`stdef rjnt`s~
:2
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 57/130
larj αd ; =% Deodnatdef tlj ~jt `b ejfatdrj ~z}}`st rjnt`s~ ,ai~` naiijo laso ejfatdrj~$
a~ E^R) vj fjt7
vj ≌ |j ∓ d∌E^R
|d ,>%?$
Dt e`v cjn`kj~ nijas tlat tlj J|jk}ias+^RK ijase~ a tjk}iatj vl`~j }`~dtdrj }ast
vdii sj~jkcij tlj j|jk}ias aeo tlj ejfatdrj }ast d~ a ~zk `rjs ejfatdrj ~z}}`st rjnt`s~%
>%3 Naidcsatd`e
Z~def tlj }s`njozsj ac`rj) vj tsade ae je~jkcij `b J|jk}ias+^RK) `ej b`s janl }`~dtdrj
de~taenj de tlj tsadedef ~jt% L`vjrjs) ozj t` tlj deoj}jeojet tsadedef }s`njozsj) tljds
`zt}zt~ asj e`t ejnj~~asdi n`k}atdcij% A n`kk`e ~tsatjf t` sjn`endij tlj `zt}zt~
`b kzitd}ij nia~~dfijs~ d~ t` }jsb`sk naidcsatd`e c fittdef a }s`cacdidt od~tsdcztd`e t` a
ljio+`zt ~jt `b ejfatdrj aeo }`~dtdrj ~ak}ij~ X]iatt) 1444) Bs`kj aeo Kaidh) 3==2[% _ld~
}s`njozsj d~ `btje naiijo ]iatt—~ ^naidef Aif`sdtlk) aeo fjejsaii a}}idjo t` a ljio+`zt
~jt vldnl la~ tlj ~akj od~tsdcztd`e `b ~ak}ij~ a~ tlj tsadedef+oata% L`vjrjs) de `zs
na~j) ~denj janl j|jk}ias+^RK d~ ~z}}`~jo t` fisj `ei `e rd~zaii ~dkdias j|ak}ij~) vj
naee`t ~a b`s ~zsj vldnl `b tlj ljio+`zt ~ak}ij~ ~l`zio cj n`e~dojsjo a~ }`~dtdrj~ a
}sd`sd% B`s j|ak}ij) b`s a bs`etai rdjv `b ae tsade) `ei `tljs bs`etai rdjv~ `b ~dkdias
tsade~ ~l`zio cj n`e~dojsjo a~ }`~dtdrj~% B`stzeatji) gz~t idhj ozsdef tsadedef) vlat vj
nae cj ~zsj ac`zt d~ tlat tlj nia~~dfijs ~l`zio e`t fisj `e ejfatdrj vdeo`v~% _ljsjb`sj)
vj ijt janl j|jk}ias ~jijnt dt~ `ve }`~dtdrj~ aeo tlje z~j tlj ^RK `zt}zt ~n`sj~ `e
tlj~j }`~dtdrj~) de aoodtd`e t` i`t~ `b ljio+`zt ejfatdrj~) t` naidcsatj tlj J|jk}ias+^RK%
_ `ctade janl j|jk}ias—~ naidcsatd`e }`~dtdrj~) vj sze tlj J|jk}ias+^RK `e tlj
raidoatd`e ~jt) nsjatj a ~jt `b e`e+sjozeoaet ojtjntd`e~ z~def e`e+ka|dkzk ~z}}sj~~d`e)
:6
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 58/130
aeo n`k}ztj tlj `rjsia} ~n`sj cjtvjje sj~zitdef ojtjntd`e~ aeo fs`zeo+tsztl c`zeodef+
c`|j~% Vj tsjat aii ojtjntd`e~ vldnl `rjsia} c k`sj tlae =%: vdtl fs`zeo+tsztl c`|j~
a~ }`~dtdrj~ ,tld~ d~ tlj ~taeoaso ]A^NAI R@N nsdtjsd`e b`s a ~znnj~~bzi ojtjntd`e$%
Aii ojtjntd`e~ vdtl ae `rjsia} i`vjs tlae =%3 asj tsjatjo a~ ejfatdrj~) aeo vj fit a
i`fd~tdn bzentd`e t` tlj~j ~n`sj~% E`tj tlat) aitl`zfl vj naee`t fzasaetjj tlat ldfli
`rjsia}}def n`ssjnt ojtjntd`e~ vdii deojjo cj rd~zaii ~dkdias t` tlj j|jk}ias) vdtl rjs
ldfl }s`cacdidt tlj vdii cj) ~denj tlj vjsj ldfli saehjo c tlj j|jk}ias+^RK de tlj
fis~t }ianj%
@zs naidcsatd`e ~tj} nae cj detjs}sjtjo a~ a ~dk}ij sj+~naidef aeo ~ldbtdef `b tlj
ojnd~d`e c`zeoas ,~jj Bdfzsj >%2$ „ }``si }jsb`skdef j|jk}ias~ vdii cj ~z}}sj~~jo
c lardef tljds ojnd~d`e c`zeoas k`rj t`vaso~ tlj j|jk}ias aeo vjii+}jsb`skdef
j|jk}ias~ vdii cj c``~tjo c lardef tljds ojnd~d`e c`zeoas k`rj ava bs`k tlj
j|jk}ias% Vldij tlj sj~zitdef ojnd~d`e c`zeoas d~ e` i`efjs ae `}tdkai ~`iztd`e b`s
tlj i`nai+^RK }s`cijk) jk}dsdnaii vj b`zeo tld~ }s`njozsj fsjati dk}s`rj~ tlj detjs+
j|jk}ias `sojsdef% Fdrje a ojtjntd`e | aeo tlj ijasejo ~dfk`do }asakjtjs~ ,αJ ) γJ $)
tlj naidcsatjo ojtjntd`e ~n`sj b`s j|jk}ias J d~ a~ b`ii`v~7
b ,|yvJ ) αJ ) γJ $ ;1
1 ( j∓αJ,v_ J|∓γJ$
Vldij tlj i`fd~tdn fittdef d~ }jsb`skjo deoj}jeojeti b`s janl j|jk}ias) vj b`zeo
tlat dt fdrj~ z~ a n`e~dojsacij c``~t de ojtjntd`e }jsb`skaenj `rjs z~def sav ^RK
`zt}zt ~n`sj~% At tj~t+tdkj) vj nsjatj ojtjntd`e~ bs`k janl nia~~dfijs c tlsj~l`iodef tlj
sav ^RK `zt}zt ~n`sj at ∓1 ,tlj ejfatdrj kasfde$ aeo tlje sj~naij tljk z~def janl
j|jk}ias—~ ijasejo ~dfk`do }asakjtjs~%
:<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 59/130
Bdfzsj >%27 J|jk}ias+^RK naidcsatd`e% _lj naidcsatd`e ~tj} sj~naij~ tlj ^RK ~n`sj~ czt o`j~ e`t abbjnttlj `sojsdef `b tlj katnlj~) aii`vdef z~ t` n`k}asj tlj `zt}zt~ `b kzitd}ij deoj}jeojeti+tsadejoJ|jk}ias+^RK~%
>%> J|jk}ias N`+`nnzssjenj Katsdnj~
Abtjs naidcsatd`e) vj nae nsjatj ojtjntd`e vdeo`v~ b`s janl j|jk}ias de a ~idodef+
vdeo`v ba~ld`e% A n`kk`e aeo ~dk}ij kjnlaed~k b`s ~z}}sj~~def sjozeoaet sj~}`e~j~
d~ e`e+ka|dkzk ~z}}sj~~d`e ,EK^$8 l`vjrjs) z~def EK^ odsjnti `e j|jk}ias~ kjae~
tlat kzitd}ij j|jk}ias~ vdii cj n`k}jtdef b`s ojtjntd`e~ vdeo`v~% L`vjrjs) ja~ t`
sjn`fedwj `cgjnt~ vdii `btje larj kae rd~zai a~~`ndatd`e~ nsjatjo as`zeo tlj `cgjnt
`b detjsj~t ‟ deb`skatd`e vldnl n`k}jtdtdrj EK^ n`k}ijtji od~sjfaso~% De~tjao `b
gz~t z~def tlj sav a~~`ndatd`e ~n`sj) vj }s`}`~j t` azfkjet janl ojtjntd`e vdtl ae
j|jk}ias n`etj|t ~n`sj vldnl z~j~ tlj dojetdtdj~ aeo ~n`sj~ `b ejasc ojtjntd`e~ t`
c``~t tlj sav ojtjntd`e ~n`sj% B`s janl ojtjntd`e vj fjejsatj a n`etj|t bjatzsj ~dkdias
t` XC`zsojr jt ai%) 3=1=) Bjiwje~wvaic jt ai%) 3=1=[ vldnl }``i~ de a~~`ndatd`e ~n`sj~ `b
ejasc ,`rjsia}}def$ ojtjntd`e~ aeo fjejsatj~ tlj fieai ojtjntd`e ~n`sj c a vjdfltjo
~zk `b tlj i`nai a~~`ndatd e ~n`sj aeo tlj n`etj|t ~n`sj% A ~jt `b H ojtjntd`e~ O d~
}s`oznjo c a}}idef a ~jt `b E j|jk}ias~ t` a ~defij dkafj aeo }jsb`skdef EK^
ans`~~ ojtjntd`e~ vdtl tlj ~akj j|jk}ias do% Vldij EK^ kahj~ ~zsj tlat vj tls`v
ava sjozeoaet ojtjntd`e~ bs`k tlj ~akj j|jk}ias) vj vdii ~tdii cj ijbt vdtl kae
:4
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 60/130
`rjsia}}def ojtjntd`e~ cjnaz~j kae j|jk}ias asj ~dkdias t` janl `tljs%
Vj nae tldeh `b janl ojtjntd`e a~ a tsd}ijt n`e~d~t~ `b a sjntaefzias ojtjntd`e sjfd`e
Sd) tlj j|jk}ias do jd) aeo tlj j|jk}ias—~ rd~zai a~~`ndatd`e ~n`sj ~d%
O ; x,Sd) jd) ~d${H d;1 ,>%:$
Sd ∌ ? ,>%2$
jd ∌ x1) % % % ) E { ,>%6$
~d ; v_ jd|Sd
( cjd ( 1%= ,>%<$
Cjnaz~j vj tlsj~l`iojo ojtjntd`e~ at ∓1%=) aoodef 1%= t` janl J|jk}ias+^RK—~
sav `zt}zt ~n`sj fzasaetjj~ tlat ~d ≪ =% Vj e`v ojfiej tlj j|jk}ias n`etj|t bjatzsj
b ,Sd$ ∌ E () a~~`ndatjo vdtl ojtjntd e sjfd`e Sd) a~ b`ii`v~7
b ,Sd$ ; Xb 1 b 3 % % % b E [ ,>%4$
b g ; ka|,jh; g$∤,Sh∌B d$
~h ,>%1=$
B d ; xS 7 @^ ,S)Sd$ 5 %:{ ,>%11$
^dk}i }zt) b`s a ojtjntd`e sjfd`e Sd) vj `ei n`e~dojs `tljs j|jk}ias ojtjntd`e~
de tlj ~jt B d ,tlj ~jt `b sjfd`e~ vldnl `rjsia} vdtl Sd c k`sj tlae =%:$% _lj g+tl
n`k}`ejet `b tlj j|jk}ias n`etj|t bjatzsj d~ tlje tlj ka|dkzk ~n`sj `b a ojtjntd`e
vdtl j|jk}ias do j g % _lj fieai ~n`sj b`s sjfd`e Sd d~ tlje7
~d ; k_ jdb ,Sd$ ,>%13$
K ; Xkj1 kj3 % % % kjE [ ,>%1>$
2=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 61/130
vljsj K d~ tlj E Ö E j|jk}ias n`+`nnzssjenj katsd| vldnl jen`oj~ l`v `btje
j|jk}ias~ jd aeo j g ~dkzitaej`z~i nsjatjo a n`ssjnt ojtjntd`e `e tlj tsaderai ~jt% Vj
j~~jetdaii i``} `rjs aii ojtjntd`e~ de tlj tsaderai ~jt) aeo n`zet l`v `btje j|jk}ias~
jd aeo j g }s`oznjo a n`ssjnt ojtjntd`e% @enj vj `ctade tlj fieai a~~`ndatd`e ~n`sj ~d
z~def tlj n`+`nnzssjenj katsd|) vj z~j ~taeoaso e`e+ka|dkzk ~z}}sj~~d`e t` nsjatj a
fieai) ~}as~j ~jt `b ojtjntd`e~ }js dkafj%
>%? Jraizatd`e `b J|jk}ias+^RK~
Vj jraizatj tlj J|jk}ias+^RK bsakjv`sh tv` odbbjsjet va~% Bds~t) vj n`k}asj tlj
ijasejo }js+j|jk}ias tjk}iatj~ afade~t tjk}iatj~ n`k}ztjo odsjnti bs`k tlj }`~dtdrj
de~taenj) vdtl`zt tlj z~j `b ae ejfatdrj~% ^jn`eo) vj n`k}asj tlj bzentd`eai b`sk
`b J|jk}ias+^RK~) afade~t a od~taenj bzentd`e njetjsjo as`zeo tlj j|jk}ias ,~jj
^jntd`e 3$% Vj z~j tlj ~akj j|ant }js+j|jk}ias bjatzsj sj}sj~jetatd`e de aii tlsjj na~j~)
aeo }jsb`sk tsadedef vdtl tlj ~akj }asakjtjs~ b`s tlj tv` ijasedef+ca~jo a}}s`anlj~%
E`skaidwjo L@F7 _` ~tzo tlj jbbjnt `b ijasedef) vj ejjo a va `b nsjatdef rd~zai
~dkdiasdtdj~ vdtl`zt ae ijasedef% Vj b`zeo tlat a ~dk}ij o`t }s`oznt vdtl a e`skaidwjo
tjk}iatj v`sh~ vjii a~ a ejasj~t+ejdflc`s ca~jidej% _lj ijasedef+bsjj vj d~ `ctadejo c
nsjatdef tlj =+kjae) e`skaidwjo L@F oj~nsd}t`s odsjnti bs`k tlj j|jk}ias—~ bjatzsj~
,|j ∌ B ($ a~ b`ii`v~7
vj ; |j ∓ º|1B ,>%1?$
º| ;|_ j 1B
B ,>%1:$
De tld~ jqzatd`e 1B d~ tlj B odkje~d`eai rjnt`s `b `ej~) aeo de `sojs t` cj =+kjae
21
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 62/130
tlj fieai tjk}iatj la~ c`tl }`~dtdrj aeo ejfatdrj }ast~% _lj doja cjldeo z~def tlj
e`skaidwjo L@F tjk}iatj n`kj~ bs`k `zs `c~jsratd`e tlat tlj tjk}iatj~ `ctadejo bs`k
tlj J|jk}ias+^RK aif`sdtlk asj kjae = aeo tlj }`~dtdrj }ast i``h~ rjs kznl idhj
tlj `sdfdeai sav L@F bjatzsj~% @ej detjs}sjtatd`e `b tlj e`skaidwjo L@F tjk}iatj
d~ tlj ~`iztd`e `b a ’rdstzai‛ ^RK ijasedef }s`nj~~ vljsj ejfatdrj ~z}}`st rjnt`s~ asj
zedb`ski od~tsdcztjo fsaodjet tjk}iatj~% De `tljs v`so~) tlj e`skaidwjo L@F tjk}iatj
dk}idndti a~~zkj~ tlat aii fsaodjet~ de tlj ejfatdrj v`sio asj jqzaii idhji) vldij tlj
J|jk}ias+^RK a}}s`anl o`j~ e`t kahj ae a~~zk}td`e~ ac`zt tlj ejfatdrj v`sio ,aeo
j|}idndti kdej~ tlj ejfatdrj~ bs`k a iasfj ~jt$%
L@F Od~taenj Bzentd`e~7 _` n`k}asj tlj tjk}iatj o`t+}s`oznt sj}sj~jetatd`e `b
~dkdiasdt bzentd`e~ afade~t `zs jasidjs ’njetjsjo+at+j|jk}ias‛ }ldi`~`}l) vj }jsb`skjo
j|}jsdkjet~ `e ijasedef z~def tlj odaf`eai Kalaiae`cd~ bzentd`eai b`sk `b Jqzatd`e 3%1%
_` kahj ~zsj tlat tlj `ei odbbjsjenj de ijasedef d~ tlj nlaefj `b tlj bzentd`eai b`sk)
vj }jsb`skjo ijasedef z~def tlj ~akj ~defij+}`~dtdrj+de~taenj `cgjntdrj bzentd`e a~ tlj
J|jk}ias+^RK) a~ vjii a~ tlj ~akj ~tsatjf b`s kdedef ejfatdrj vdeo`v~% _lj `ei
odbbjsjenj d~ tlat de~tjao `b a}}idef tlj ijasejo vjdflt rjnt`s de sav bjatzsj ~}anj
v_ j |d) vj a}}i dt de tlj od~taenj+t`+j|jk}ias ~}anj) v_
j ojd%
_lj bzii ]A^NAI R@N ojtjntd`e nlaiijefj sj~zit~ vdii cj od~nz~~jo iatjs) czt vj
~zkkasdwj tlj sj~zit~ ljsj b`s n`k}ijtjej~~% Z~def tlj ~akj naidcsatd`e ~tj} ,vldnl
vdii cj oj~nsdcjo de ^jntd`e >%3$ b`s aii > kjtl`o~) vj ~jj tlat E`skaidwjo L@F `ctade~
a kA] `b %11=) I`nai Od~taenj Bzentd`e~ `ctade a kA] `b %1:6) aeo J|jk}ias+^RK~
`ctade a kA] `b %14<% _lj bzii ~jt `b sj~zit~ nae cj ~jje de _acij >%1%
>%?%1 ]A^NAI R@N @cgjnt Ojtjntd e _a~h
Vj jraizatj `zs J|jk}ias+^RK bsakjv`sh `e tlj vjii+j~tacid~ljo ]A^NAI R@N cjenl+
kash ta~h `b `cgjnt ojtjntd`e% B`s `zs j|}jsdkjet~) vj z~j a ~defij ~`zsnj `b j|jk}ias~7
23
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 63/130
Bdfzsj >%67 @cgjnt Ojtjntd`e aeo A}}jasaenj _sae~bjs% Janl j|ak}ij ~l`v~ a ojtjntd`e bs`k `zsje~jkcij `b J|jk}ias+^RK~ ai`ef vdtl tlj a}}jasaenj tsae~bjssjo odsjnti bs`k tlj ~`zsnj j|jk}ias) t`ojk`e~tsatj tlj ldfl qzaidt `b rd~zai aidfekjet% C`tt`k s`v ~l`v~ `cgjnt natjf`s ojtjntd`e badizsj~%
tlj ]A^NAI R@N 3==6 oata~jt XJrjsdeflak jt ai%) 3=1=[ „ a }`}zias oata~jt z~jo t`
cjenlkash `cgjnt ojtjntd`e aif`sdtlk~% Ozsdef tsadedef) vj ijase a ~j}asatj nia~~dfijs v
b`s janl `b tlj 13) 2=< j|jk}ias~ bs`k tlj 3= natjf`sdj~ de :) =11 tsaderai dkafj~% Vj
kdej laso ejfatdrj~ bs`k `zt+`b+nia~~ dkafj~ de tlj tsade ~jt aeo }jsb`sk naidcsatd`e
z~def aii }`~dtdrj aeo ejfatdrj dkafj~ de tsaderai ,^jj ^jntd`e >%3$%
At tj~t tdkj) janl J|jk}ias+^RK nsjatj~ ojtjntd`e vdeo`v~ de a ~idodef+vdeo`v
ba~ld`e) czt de~tjao `b z~def a ~taeoaso e`e+ka|dka+~z}}sj~~d`e vj z~j ae j|jk}ias
n`+`nnzsjenj ca~jo kjnlaed~k b`s ~z}}sj~~def sjozeoaet sj~}`e~j~% B`s janl ojtjntd`e
vj fjejsatj a n`etj|t bjatzsj ~dkdias t` XC`zsojr jt ai%) 3=1=)Bjiwje~wvaic jt ai%) 3=1=[
vldnl }``i~ de tlj ^RK ~n`sj~ `b ejasc ,`rjsia}}def$ ojtjntd`e~ aeo fjejsatj~ tlj
fieai ojtjntd`e ~n`sj c a vjdfltjo ~zk `b tlj i`nai ^RK ~n`sj aeo tlj n`etj|t ~n`sj%
@enj vj `ctade tlj fieai ojtjntd`e ~n`sj) vj z~j ~taeoaso e`e+ka|dkzk ~z}}sj~~d`e t`
nsjatj a fieai) ~}as~j ~jt `b ojtjntd`e~ }js dkafj%
Vj sj}`st sj~zit~ `e tlj 3=+natjf`s ]A^NAI R@N 3==6 n`k}> `cgjnt ojtjntd`e
2>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 64/130
J|jk}ias Kjtl`o~ Fi`cai
_}j EE EE(Nai OBZE OBZE(Nai J^RK J^RK(Nai J^RK(N` NW O_ IO]K
ajs`}iaej =%==2 =%=:2 =%=?? =%123 =%=23 =%3=? =%3=< =%323 =%136 =%3<6cdnnij =%=4? =%34> =%344 =%>2? =%>>> =%?=6 =%?<= =%?=4 =%3:> =%:1=cdso =%=== =%=13 =%==2 =%==< =%=43 =%=4> =%=66 | =%==: =%==2c`at =%==: =%=>? =%=4> =%=42 =%=44 =%1== =%1?> | =%=1: =%1?:c`ttij =%=== =%==4 =%==: =%=46 =%=3: =%1=> =%1>1 | =%1=6 =%32:cz~ =%==2 =%3=6 =%1:> =%>12 =%3>2 =%>1= =%>46 =%>4> =%3=: =%>46nas =%=1= =%321 =%3=3 =%>22 =%>34 =%?=1 =%?11 =%?>3 =%3>= =%:=3nat =%=43 =%=16 =%=43 =%=43 =%=4: =%=42 =%=:3 | =%==: =%12>nlads =%==1 =%=4? =%=4: =%=4< =%=4: =%1=? =%112 | =%=31 =%12:n`v =%=43 =%111 =%=46 =%1=6 =%11< =%1?6 =%1<2 | =%13< =%122odedeftacij =%==1 =%==? =%==> =%==3 =%=12 =%=3> =%111 | =%=1? =%3?:o`f =%==? =%=>> =%=43 =%=4> =%=4? =%=46 =%=>1 | =%==? =%=:=l`s~j =%=42 =%3?> =%3:> =%3>? =%>?= =%><? =%??6 | =%133 =%?:3k`t`scdhj =%=4? =%1<< =%142 =%33> =%3<6 =%>3= =%>4? =%>6: =%1=> =%><>}js~`e =%==: =%11? =%=46 =%13= =%13? =%143 =%124 | =%1=1 =%>23}`ttjo}iaet =%=1< =%=3= =%=13 =%=>6 =%=3= =%=42 =%113 | =%=33 =%=4=~ljj} =%==4 =%134 =%=?= =%116 =%116 =%126 =%332 | =%=:2 =%16?~`ba =%==< =%==> =%=?6 =%=12 =%=4< =%11= =%16= | =%=:= =%33<tsade =%=42 =%1<> =%1<= =%361 =%3=: =%341 =%>24 =%>>? =%13= =%>?1trk`edt`s =%1?? =%14: =%334 =%34> =%331 =%>1: =%>== | =%3?< =%><?
kA] =%=>4 =%11= =%113 =%1:: =%1:= =%14< =%336 | =%=46 =%322
_acij >%17 J|jk}ias+^RK @cgjnt Ojtjntd`e Sj~zit~ `e ]A^NAI R@N 3==6% _lj > odbbjsjet i`nai~dkdiasdt bzentd`e~ vj n`k}asj afade~t asj7 EE ,E`skaidwjo L`f Ejasj~t Ejdflc`s$) OBZE ,ijasedef aodaf`eai Kalaiae`cd~ od~taenj bzentd`e$) aeo J^RK ,J|jk}ias+^RK$% B`s janl kjtl`o vj ai~` }jsb`sk
naidcsatd`e) a~ deodnatjo vdtl a (Nai) aeo a}}i tlj n`etj|tzai n`+`nnzssjenj katsd| ,(N`$ t` tlj cj~t}jsb`skdef J^RK kjtl`o% _lj tv` fi`cai kjtl`o~ asj `zs dk}ijkjetatd`e `b Oaiai+_sdff~ ,ijasedef a~defij fi`cai tjk}iatj$) aeo IO]K ,Iatjet ojb`skacij }ast k`oji$% @zs kjtl`o cjat~ `zt tlj Oaiai+_sdff~ca~jidej ans`~~ aii natjf`sdj~% @zs a}}s`anl `ctade~ a kA] `b 33%6 sdraidef tlj kA] `b 32%6 `ctadejo bs`kBjiwje~wvaic jt ai XBjiwje~wvaic jt ai%) 3=1=[ od~nsdkdeatdrji tsadejo }ast+ca~jo kd|tzsj k`oji% NW~taeo~ b`s XNlzk aeo Wd~~jskae) 3==6[ aeo IO]K ~taeo~ b`s tlj Iatjet Ojb`skacij ]ast+ca~jo K`oji`b XBjiwje~wvaic jt ai%) 3=1=[% Ozj t` n`k}ztatd eai sja~`e~) vj n`k}ztjo tlj EE ca~jidej~ `e tlj}js~`e natjf`s vdtl 13:= j|jk}ias~%
2?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 65/130
nlaiijefj% Bdfzsj >%6 ~l`v~ ~jrjsai ojtjntd`e~ ,fsjje c`|j~$ }s`oznjo c `zs J|jk}ias+
^RK bsakjv`sh% Vj ai~` ~l`v tlj ~z}js+dk}`~jo j|jk}ias ,jii`v c`|j~$ a~~`ndatjo
vdtl janl ojtjntd`e% B`ii`vdef tlj }s`t`n`i `b tlj R@N Nlaiijefj) vj jraizatj `zs
~~tjk `e a }js+natjf`s ca~d~ `e tlj tj~t ~jt) n`e~d~tdef `b ?) 4:3 dkafj~% Vj n`k}asj
tlj }jsb`skaenj `b `zs a}}s`anl ,J^RK(N`+`n$ t` ~jrjsai j|jk}ias ca~jidej~ a}ast
bs`k tlj R@N sj~zit~ sj}`stjo de XBjiwje~wvaic jt ai%) 3=1=) Nlzk aeo Wd~~jskae)
3==6[% _lj~j sj~zit~ larj cjje ~zkkasdwjo de _acij >%1 a~ Arjsafj ]sjnd~d`e }js nia~~%
@zs sj~zit~ ~l`v tlat ~taeoaso Ejasj~t Ejdflc`s 1 ,EE$ o`j~ e`t v`sh at aii% Vldij
tlj }jsb`skaenj dk}s`rj~ abtjs naidcsatd`e ,EE(Nai$) dt d~ ~tdii e`t n`k}asacij t` `tljs
a}}s`anlj~ ozj t` dt~ ianh `b k`ojidef ejfatdrj oata% Vj ai~` n`k}asjo afade~t a
od~taenj bzentd`e b`skziatd`e ~dkdias t` tlj `ej }s`}`~jo de XKaid~djvdnw aeo Jbs`~)
3==<[ czt ijasejo z~def a ~defij }`~dtdrj de~taenj% _lj sj~zit~ nijasi deodnatj tlat
tlj j|tsa n`e~tsadet ozj t` a od~taenj bzentd`e }asakjtjsdwatd`e d~ v`s~j tlae z~def a
l}js}iaej% _` ldflidflt tlj dk}`staenj `b z~def tlj n`+`nnzsjenj kjnlaed~k ac`rj)
vj ai~` sj}`st `zs sj~zit~ z~def naidcsatd`e ,J^RK(Nai$%
@e tlj ]A^NAI tj~t ~jt) `zs bzii ~~tjk `ctade~ a kjae Arjsafj ]sjnd~d`e ,kA]$
`b %336) vldnl d~ n`k}jtdtdrj vdtl vdtl Bjiwje~wvaic—~ ~tatj+`b+tlj+ast ojb`skacij }ast+
ca~jo kd|tzsj k`oji% E`tj l`vjrjs) tlat `zs ~~tjk o`j~ e`t z~j }ast~ ,tl`zfl tlj
n`zio cj ja~di aoojo$ ~` tlj n`k}asd~`e d~ e`t jetdsji bads% _ljsjb`sj) vj ai~` n`k}asj
`zs }jsb`skaenj t` Oaiai/_sdff~ ca~jidej) vldnl z~j~ a ~defij natjf`s+vd~j idejas
^RK vdtl e` }ast~) aeo attade~ a kA] `b %=46) vldnl d~ ij~~ tlae laib `b `zs~% Vj ai~`
n`k}asjo afade~t tlj ]A^NAI R@N 3==6 vdeedef jets) tlj j|jk}ias+ca~jo kjtl`o
`b Nlzk jt ai% XNlzk aeo Wd~~jskae) 3==6[) aeo b`zeo tlat `zs ~~tjk cjat~ dt `e ?
`zt `b 2 natjf`sdj~ b`s vldnl tlj ~zckdttjo sj~zit~% De XNlzk aeo Wd~~jskae) 3==6[)
1Vj j|}jsdkjetjo vdtl kzitd}ij ~dkdiasdt kjtsdn~ aeo b`zeo tlat a o`t }s`oznt vdtl a e`skaidwjo
L@F tjk}iatj v`shjo tlj cj~t% _lj e`skaidwjo L@F tjk}iatj d~ nsjatjo c ~zctsantdef a n`e~taet bs`ktlj }`~dtdrj L@F bjatzsj~ t` kahj tljk =+kjae%
2:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 66/130
tlj vdeedef jets b`s kae `b tlj natjf`sdj~ de ]A^NAI 3==6) `cgjnt natjf`sdj~ asj
sj}sj~jetjo c ~t`sdef j|jk}ias~) czt tlj zeojsidef a~~zk}td`e d~ tlat aii j|jk}ias~
bs`k tlj ~akj nia~~ ~lasj a n`kk`e od~tsdcztd e `b bjatzsj~% _lz~) Nlzk jt ai%
sjqzdsj nia~~+vd~j aeo a~}jnt+vd~j iacjidef `b tsadedef oata t` csjah z} cs`ao ca~dn+ijrji
`cgjnt natjf`sdj~ ~znl a~ nas~ det` fiejs rd~zai ~zc+natjf`sdj~ ,j%f%) nas+ijbt) nas+bs`et$%
Zeb`stzeatji tld~ t}j `b iacjidef d~ tjod`z~ aeo odbfinzit t` `ctade b`s oata~jt~ `b
~dfedfinaet ~dwj% K`sj`rjs) b`s ~`kj `cgjnt~ ,j%f%) c`ttij~) ~`nnjs caii~) fsa~~$ dt d~ e`t
nijas db a~}jnt+vd~j iacjidef d~ jrje vjii+ojfiejo%
>%: J|jk}ias+^RK Aeai~d~
De tld~ ~jntd`e) vj aeaiwj de ojtadi ~`kj }s`}jstdj~ `b `zs J|jk}ias+^RK `cgjnt
ojtjntd e ~~tjk% De }astdnzias) vj ~tzo `cgjnt ojtjntd`e }jsb`skaenj vlje z~def
odbbjsjet ejfatdrj ~jt~ ,c rasdef tljds n`etjet a~ vjii a~ tljds ~dwj$ a~ vjii a~ odbbjsjet
}`~dtdrj ~jt~ ,c rasdef tlj ezkcjs `b j|jk}ias~ t` cj z~jo at tj~t+tdkj$%
>%:%1 Denizodef ^akj+Natjf`s De~taenj~ de tlj Ejfatdrj ^jt
De tlj ia~t ~jntd`e) vj ~l`vjo tlat tlj J|jk}ias+^RK tsadedef }s`njozsj v`sh~ vdtl a
~defij }`~dtdrj de~taenj aeo kdiid`e~ `b ejfatdrj~% Vj ai~` od~nz~~jo tlat tld~ dk}idj~ tlat
tlj j|jk}ias—~ ojnd~d`e c`zeoas d~ ojfiejo) de iasfj }ast) c vlat dt d~ e`t% Cjnaz~j tlj
ejfatdrj ~jt va~ nsjatjo c jidkdeatdef dkafj~ n`etadedef ~akj+natjf`s ,`s) de+nia~~$
de~taenj~) dt oj}jeo~ `e tlj }`~dtdrj—~ natjf`s% _` ojtjskdej l`v tlj }jsb`skaenj `b
`zs ~~tjk oj}jeo~ `e jidkdeatdef ~akj+natjf`s de~taenj~ bs`k tlj ejfatdrj ~jt) vj
sj}jat tlj ]A^NAI R@N 3==6 j|}jsdkjet~ bs`k ~jntd`e >%?%1) czt tld~ tdkj vj aii`v
tlj ejfatdrj~ t` n`kj bs`k ae dkafj) denizodef de+nia~~ dkafj~% B`s janl j|jk}ias de
tsaderai) vj z~j aii `b tlj dkafj~ bs`k tlj tsade ~jt ,j|nizodef tlj j|jk}ias—~ `ve
22
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 67/130
_}j J^RK' J^RK'(Nai J^RK'(N` J^RK(N`
ajs`}iaej =%=?> =%131 =%11? =%3=<cdnnij =%312 =%>1= =%>43 =%?<=
cdso =%=43 =%=>3 =%=4: =%=66c`at =%=4? =%=4< =%1?> =%1?>
c`ttij =%=1= =%=>2 =%13? =%1>1cz~ =%1:: =%3=3 =%>3> =%>46nas =%1<: =%32? =%>?> =%?11
nat =%=4> =%=4> =%=>: =%=:3nlads =%=?4 =%=66 =%11? =%112
n`v =%116 =%13< =%14> =%1<2odedeftacij =%=11 =%=13 =%=42 =%111
o`f =%=4? =%=4: =%=:> =%=>1l`s~j =%32: =%3:< =%><1 =%??6
k`t`scdhj =%33? =%3>: =%>2= =%>4?
}js~`e =%1=3 =%13< =%123 =%124}`ttjo}iaet =%=12 =%=4> =%=2: =%113~ljj} =%=2: =%1:4 =%31= =%332~`ba =%=4? =%=3> =%131 =%16=
tsade =%1:= =%313 =%>=3 =%>24trk`edt`s =%1<: =%326 =%3<1 =%>==
kA] =%11> =%1?3 =%14: =%336
_acij >%37 J|jk}ias+^RK sj~zit~ vlje kdedef ejfatdrj~ bs`k c`tl `zt+nia~~ aeo de+nia~~ dkafj~ %
Vj azfkjet tlj J|jk}ias+^RK a}}s`anl c aii`vdef tlj ejfatdrj~ t` n`kj bs`k ae dkafj) denizodefde+nia~~ dkafj~ ,deodnatjo c J^RK'$% B`s n`k}asd~`e) tlj sdfltk`~t n`izke ,J^RK(N`$ d~ tlj `sdfdeaikjtl`o ,tahje bs`k _acij >%1$ vljsj de+nia~~ dkafj~ asj jidkdeatjo bs`k tlj ejfatdrj ~jt%
dkafj$ t` ojfiej tlj ejfatdrj ~jt% Abtjs ijasedef tlj v—~) vj }jsb`sk naidcsatd`e ,~jj
^jntd`e >%3$ aeo n`+`nnzssjenj katsd| j~tdkatd`e ,~jj ^jntd`e >%>$ tlj ~akj va a~
cjb`sj ,d%j%) ztdidwdef tlj j|jk}ias—~ natjf`s$%
A~ nae cj ~jje bs`k tlj sj~zit~ de _acij >%3) tlj fieai kA] d~ =%14:) vldnl d~ `ei
~idflti cji`v tlj =%336 kA] bs`k tlj `sdfdeai kjtl`o% _ld~ ~zffj~t~ tlat `enj tlj
ejfatdrj ~jt d~ iasfj je`zfl) tlj bjv }`tjetdaii n`ebz~def ’}`~dtdrj~‛ de tlj ejfatdrj
~jt asj e`t batai t` `zs ~~tjk% _lj }jsnjetafj os`} ~jjk~ t` cj badsi n`e~d~tjet b`s
tlj vjii+}jsb`skdef natjf`sdj~% E`tdnj tlat b`s ? natjf`sdj~ ,cdso) c`at) n`v) o`f$) tlj
J^RK'(N` sj~zit~ vjsj jqzai t` `s cjttjs tlae tlj `sdfdeai J^RK(N` kjtl`o% _lj
sj~zit~ b`s n`v) a sja~`eaci+}jsb`skdef natjf`s) jrje dk}s`rjo bs`k =%1<2 t` =%14?%
26
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 68/130
>%:%3 ^dwj `b Ejfatdrj ^jt
Cjnaz~j J|jk}ias+^RK~ kz~t cj tsadejo vdtl i`t~ `b ejfatdrj~) janl j|jk}ias—~ oata+
kdedef ~tj} sjqzdsj~ a}}s`|dkatji >= kdeztj~ `b v`sh `e a ~defij N]Z% Vldij dt d~ ja~
t` }asaiijidwj tld~ }s`njozsj z~def a niz~tjs `b n`k}ztjs) dt d~ cjejfindai t` bzstljs sjoznj
tlj tdkj `b kdedef% Sjozndef tlj ezkcjs `b ejfatdrj~ d~ `ej ja~ va t` ~}jjo+z} tlj
tsadedef }s`njozsj8 tljsjb`sj) de tld~ ~jntd`e vj a~h tlj eatzsai qzj~td`e7 l`v o`j~
`cgjnt ojtjntd`e }jsb`skaenj oj}jeo `e tlj ~dwj `b tlj ejfatdrj ~jt0 Vj }jsb`sk tld~
jraizatd`e `e 2 R@N natjf`sdj~7 cz~) n`v) odedeftacij) k`t`scdhj) ~ljj}) aeo tsade vldij
n`e~dojsdef odbbjsjet ejfatdrj ~jt ~dwj~7 3:) 3:=) 1===) aeo tlj `sdfdeai 3:==% Bdfzsj >%<~l`v~ tlj ~zkkasdwjo sj~zit~ b`s tlj tlsjj kjtl`o~7 J^RK) J^RK(Nai) aeo J^RK(N`%
Ojtadijo ezkcjs~ nae cj ~jje de _acij >%>) _acij >%?) _acij >%:) aeo _acij >%2% Vj e`tj
tlat densja~def tlj ejfatdrj ~jt ~dwj bs`k 3: t` 3:= ejfatdrj~ dkafj~ fdrj~ z~ a kznl
ldfljs c``~t tlae densja~def dt bs`k 1=== t` tlj bzii 3:== dkafj~% _ld~ ~zffj~t~ tlat
`enj janl j|jk}ias la~ ~jje a ~zbfindjet ezkcjs `b ejfatdrj~) dt~ }jsb`skaenj vdii e`t
~dfedfinaeti dk}s`rj vdtl k`sj ejfatdrj~%
2<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 69/130
= :== 1=== 1:== 3=== 3:===%=:
=%1
=%1:
=%3
=%3:
=%>
.Ejfatdrj~
A ]
]A^NAI R@N kA]' rjs~z~ Ejfatdrj ^jt ^dwj
J^RK
J^RK(Nai
J^RK(N`
Bdfzsj >%<7 Rasdef ejfatdrj ~jt ~dwj~% Vj n`k}ztj tlj }jsb`skaenj `b tlj J|jk}ias+^RK aif`sdtlk a~a kA] `rjs 2 ]A^NAI natjf`sdj~7 cz~) n`v) odedeftacij) k`t`scdhj) ~ljj}) aeo tsade% _lj |+a|d~ deodnatj~
tlj ezkcjs `b ejfatdrj dkafj~ z~jo aeo tlj +a|d~ d~ tlj ]A^NAI R@N 3==6 sj~zitdef kA] ~n`sj b`s janl`b tlj > kjtl`o~%
24
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 70/130
_}j J^RK J^RK(C J^RK(K
cz~ =%1=> =%324 =%3?<n`v =%1== =%133 =%1?4odedeftacij =%==1 =%=?< =%=42
k`t`scdhj =%12? =%3>3 =%3:>~ljj} =%=4< =%1:? =%12?tsade =%1=1 =%142 =%3:2
kA] =%=4? =%16= =%14?
_acij >%>7 J|jk}ias+^RK sj~zit~ vdtl 3: ejfatdrj~%
_}j J^RK J^RK(C J^RK(K
cz~ =%1<: =%342 =%>?<
n`v =%131 =%1:2 =%12?odedeftacij =%==: =%=1< =%113
k`t`scdhj =%3?< =%3<? =%>><~ljj} =%=:> =%12> =%14<tsade =%13? =%3:< =%>1:
kA] =%13> =%142 =%3?2
_acij >%?7 J|jk}ias+^RK sj~zit~ vdtl 3:= ejfatdrj~%
_}j J^RK J^RK(C J^RK(K
cz~ =%312 =%>=? =%><?
n`v =%1>= =%1?< =%16<odedeftacij =%=13 =%=14 =%11<k`t`scdhj =%324 =%>=3 =%>66~ljj} =%=6? =%163 =%3>4tsade =%1<? =%>=: =%>21
kA] =%1?6 =%3=< =%362
_acij >%:7 J|jk}ias+^RK sj~zit~ vdtl 1=== ejfatdrj~
_}j J^RK J^RK(C J^RK(K
cz~ =%3>2 =%>1= =%>46
n`v =%11< =%1?6 =%1<2
odedeftacij =%=12 =%=3> =%111k`t`scdhj =%3<6 =%>3= =%>4?~ljj} =%116 =%126 =%332tsade =%3=: =%341 =%>24
kA] =%12> =%31= =%3<1
_acij >%27 J|jk}ias+^RK sj~zit~ vdtl 3:== ejfatdrj~% _ld~ d~ tlj ~akj a~ tlj kade sj~zit de tlj kade_acij >%1
6=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 71/130
>%:%> Oj}jeojenj @e tlj Ezkcjs `b J|jk}ias~
^denj tlj n`k}ij|dt `b `zs Je~jkcij `b J|jk}ias+^RK~ ~naij~ idejasi vdtl tlj ezkcjs
`b j|jk}ias~) dt d~ raizacij t` ~tzo tlj `cgjnt ojtjntd`e }jsb`skaenj `b je~jkcij~ `b
rasdef ~dwj~% Vj a~h7 dt }`~~dcij t` `ctade sjaii f``o `cgjnt ojtjntd`e sj~zit~ vdtl
a ~kaii ezkcjs `b j|jk}ias~0 B`s janl `cgjnt natjf`s de R@N) vj tsade aii `b tlj
j|jk}ias~ z~def `zs bzii a}}s`anl) aeo tlje ~`st tljk z~def tlj `zt}zt `b naidcsatd`e
,~jj ^jntd`e sjb~jn7naidcsatd`e$% Naidcsatd`e ~n`sj~ janl j|jk}ias deoj}jeojeti aeo
tlz~ tlj j|jk}ias~ vldnl }s`oznj tlj iasfj~t ezkcjs `b f``o katnlj~ ,`e ljio+`zt
raidoatd`e oata$ vdii n`kj fis~t de tlj ~`stdef% Fdrje tld~ `sojsdef `b j|jk}ias~) vjn`e~dojs je~jkcij~ `b densja~def ~dwj) ~tastdef vdtl tlj ~defij cj~t j|jk}ias) aeo
densjkjetaii aoodef tlj ej|t+cj~t `ej) zetdi vj larj aii j|jk}ias~ de tlj je~jkcij%
Vj sj}`st tlj ]A^NAI R@N }jsb`skaenj b`s rasdef ezkcjs~ `b j|jk}ias~ z~def tlj
}js+j|jk}ias naidcsatd`e ~tsatjf) aeo e`t tlj bzii n`+`nnzssjenj katsd|% _lj ]A^NAI
R@N A] rjs~z~ ezkcjs `b j|jk}ias~ }i`t~ nae cj ~jje de Bdfzsj >%4% Vj ~l`v tlj sj~zit~
de tv` odbbjsjet va~7 A] r~% i`f ezkcjs `b j|jk}ias~) aeo A] r~% }jsnjet `b j|jk}ias~
de tlj natjf`s% Vj e`tj tlat tld~ nzsrj d~ b`s tlj k`~t }ast k`e`t`ednaii densja~def% A~
nae cj ~jje bs`k tld~ nzsrj) tlj }jsb`skaenj ~atzsatj~ b`s tlj~j natjf`sdj~ vlje ac`zt
laib tlj j|jk}ias~ asj nl`~je% _lj ffiat tadi `b janl nzsrj ~zffj~t~ tlat aoodef tlj ia~t
laib `b j|jk}ias~ ,kae `b vldnl asj cao$ o`j~e—t kahj `zs }jsb`skaenj os`}% _ld~
~zffj~t~ tlat `zs naidcsatd`e ~tj} v`sh~ vjii at ~z}}sj~~def cao j|jk}ias~% ^denj tlj
sze+tdkj n`k}ij|dt d~ odsjnti }s`}`std`eai t` tlj ezkcjs `b j|jk}ias~ de tlj je~jkcij)
tld~ sj~zit ~zffj~t~ tlat vj nae fjt kznl ba~tjs `cgjnt natjf`s ojtjnt`s~ c n`e~tszntdef
ae je~jkcij vdtl e` k`sj tlae laib tlj ezkcjs `b j|jk}ias~ de janl natjf`s%
61
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 72/130
1==
1=1
1=3
1=>
1=?
=
=%=:
=%1
=%1:
=%3
=%3:
=%>
=%>:
=%?
=%?:
Ezkcjs `b j|jk}ias~ ,i`f ~naij$
A ]
]A^NAI R@N A] r~% Ezkcjs `b j|jk}ias~ ,i`f ~naij$
cdnnij
cz~
nas
l`s~j
k`t`scdhj
~ljj}
tsade
= 1= 3= >= ?= := 2= 6= <= 4= 1===
=%=:
=%1
=%1:
=%3
=%3:
=%>
=%>:
=%?
=%?:
]jsnjetafj `b j|jk}ias~
A ]
]A^NAI R@N A] r~% ]jsnjetafj `b J|jk}ias~
cdnnij
cz~
nas
l`s~j
k`t`scdhj
~ljj}
tsade
Bdfzsj >%47 ]A^NAI R@N Sj~zit~ r~% Ezkcjs `b J|jk}ias~% Vj sj}`st tlj R@N A] b`s je~jkcij~ `b densja~def ~dwj% B`s janl natjf`s) a ~`stjo id~t d~ nsjatjo bs`k tlj `zt}zt `b naidcsatd`e) aeo je~jkcij~`b ~dwj 1 tls`zfl E ,E d~ tlj ezkcjs `b j|jk}ias~ de a natjf`s$ asj jraizatjo `e tlj ojtjntd`e ta~h%_lj t`} }i`t ~l`v~ }jsb`skaenj rjs~z~ tlj sav ezkcjs `b j|jk}ias~% _lj c`tt`k }i`t ~l`v~ tlj ~akjdeb`skatd`e) czt z~def a idejas ~naij) aeo j|}sj~~jo a~ a }jsnjet `b tlj t`tai ezkcjs `b j|jk}ias~ vdtldetlj natjf`s%
63
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 73/130
>%:%? _lj S`ij `b Bjatzsj Odkje~d`eaidt
Ae`tljs dk}`staet oj~dfe ojnd~d`e de `zs J|jk}ias+^RK bsakjv`sh d~ tlj odkje~d`eai+
dt `b bjatzsj~ sj}sj~jetdef janl j|jk}ias% De tld~ ~jntd`e) vj ras tlj bjatzsj odkje~d`e
c rasdef janl j|jk}ias—~ L@F tjk}iatj ~dwj% Vj sj}jat tlj ]A^NAI R@N j|}jsdkjet~
vdtl iasfjs+~dwjo tjk}iatj~ a~ vjii a~ vdtl ~kaiijs+~dwjo tjk}iatj~% @zs ojbazit tjk}iatj
~dwj d~ s`zfli 1== njii~) vdtl tlj iasfj tjk}iatj~ cjdef s`zfli 3== njii~) ~kaii tjk}iatj~
cjdef s`zfli := njii~) aeo tde `ej~ vdtl 3: njii~% _lj L@F dedtdaidwatd`e/bsakdef
}s`njozsj d~ bzii oj~nsdcjo de A}}jeod| C%3% A ~zkkas `b tlj sj~zit~ arjsafjo ans`~~
2 natjf`sdj~ nae cj ~jje de Bdfzsj >%1=% _lj bzii sj~zit tacij~ asj de _acij >%6) _acij >%<)_acij >%4) aeo _acij >%1=% Vj e`tj tlat iasfjs tjk}iatj~ ~idflti dk}s`rj sj~zit~ b`s tlj
sav J^RK kjtl`o aeo J^RK(Nai8 l`vjrjs) a tjk}iatj ~dwj `b 1== d~ tlj cj~t vlje
z~def tlj n`+`nnzssjenj katsd|% L`vjrjs) tlj aoraetafj `b z~def ~kaii tjk}iatj~ `rjs
iasfjs `ej~ d~ tlat tsadedef ~kaii tjk}iatj~ a~ vjii a~ jraizatdef tljk `e tj~t+dkafj~ d~
de fjejsai kznl ba~tjs%
6>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 74/130
3= ?= 2= <= 1== 13= 1?= 12= 1<= 3===%1
=%1:
=%3
=%3:
=%>
=%>:
L@F _jk}iatj ^dwj
k A ]
kA] rjs~z~ L@F _jk}iatj ^dwj
J^RK
J^RK(Nai
J^RK(N`
Bdfzsj >%1=7 ]A^NAI R@N Sj~zit~ b`s Rasdef L@F _jk}iatj ^dwj~% Vj n`k}ztj tlj }jsb`skaenj `b
tlj J|jk}ias+^RK aif`sdtlk a~ a kA] `rjs 2 ]A^NAI natjf`sdj~7 cz~) n`v) odedeftacij) k`t`scdhj)~ljj}) aeo tsade% _lj |+a|d~ deodnatj~ tlj ~dwj `b tlj L@F tjk}iatj~ z~jo aeo tlj +a|d~ d~ tlj ]A^NAI
R@N 3==6 sj~zitdef kA] ~n`sj b`s janl `b tlj > kjtl`o~%
6?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 75/130
_}j J^RK J^RK(Nai J^RK(N`
cz~ =%1== =%13: =%1<?n`v =%1>6 =%134 =%31<odedeftacij =%==1 =%==1 =%==2
k`t`scdhj =%1<= =%162 =%3:2~ljj} =%1=> =%13= =%3=?tsade =%=4> =%=42 =%11?
kA] =%1=3 =%1=< =%12?
_acij >%67 J|jk}ias+^RK sj~zit~ vdtl tde L@F tjk}iatj~ ,f`ai~dwj7 3:) ka|odk7 :$%
_}j J^RK J^RK(Nai J^RK(N`
cz~ =%163 =%31: =%>12n`v =%1?4 =%122 =%33>odedeftacij =%==6 =%=4> =%=:>k`t`scdhj =%3:4 =%324 =%>62~ljj} =%11> =%1:6 =%3>3tsade =%111 =%116 =%1>>
kA] =%1>: =%16= =%333
_acij >%<7 J|jk}ias+^RK sj~zit~ vdtl ~kaiijs L@F tjk}iatj~ ,f`ai~dwj7 :=) ka|odk7 <$%
_}j J^RK J^RK(Nai J^RK(N`
cz~ =%3>2 =%>1= =%>46n`v =%11< =%1?6 =%1<2
odedeftacij =%=12 =%=3> =%111k`t`scdhj =%3<6 =%>3= =%>4?~ljj} =%116 =%126 =%332tsade =%3=: =%341 =%>24
kA] =%12> =%31= =%3<1
_acij >%47 J|jk}ias+^RK sj~zit~ vdtl ojbazit+~dwjo L@F tjk}iatj~ ,f`ai~dwj7 1==) ka|odk7 13$%
_}j J^RK J^RK(Nai J^RK(N`
cz~ =%3>3 =%>:3 =%>4?
n`v =%1>3 =%1:: =%12?odedeftacij =%=4> =%=42 =%1=3k`t`scdhj =%>=< =%>1= =%>6>~ljj} =%=61 =%1<= =%316tsade =%3=2 =%>=< =%>>6
kA] =%16? =%3>> =%32?
_acij >%1=7 J|jk}ias+^RK sj~zit~ vdtl iasfj L@F tjk}iatj~ ,f`ai~dwj7 3==) ka|odk7 1:$%
6:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 76/130
]ast DD7 Cj`eo @cgjnt Ojtjntd`e
Dt abb`so~ ae dkkjodatj ~tj}) l`vjrjs) t` a~~`ndatdrj deoj|def) tlj ca~dn doja `b
vldnl d~ a }s`rd~d`e vljsjc ae dtjk ka cj naz~jo at vdii t` ~jijnt dkkjodatji
aeo azt`katdnaii ae`tljs% _ld~ d~ tlj j~~jetdai bjatzsj `b tlj kjkj|% _lj }s`nj~~`b tdef tv` dtjk~ t`fjtljs d~ tlj dk}`staet tldef%
Raeejras Cz~l) A~ Vj Ka _ldeh ,14?:$
62
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 77/130
Vj larj) z} zetdi e`v) ojr`tjo `zs attjetd`e t` nsjatdef rd~zai a~~`ndatd`e~) `s de
tlj v`so~ `b Raeejras Cz~l) ’tdef tv` dtjk~ t`fjtljs%‛ Vj ojk`e~tsatjo tlat tlj
rd~zai a~~`ndatd`e~ }s`oznjo c tlj J|jk}ias+^RK aif`sdtlk nae cj z~jo b`s `cgjnt
natjf`s ojtjntd`e) kznl idhj `tljs natjf`s+ca~jo ojtjntd`e ~~tjk~ XOaiai aeo _sdff~)
3==:) Bjiwje~wvaic jt ai%) 3=1=) Nlzk aeo Wd~~jskae) 3==6[% L`vjrjs) tlj njetsai
cjejfit `b `zs a}}s`anl d~ tlat vj asj acij t` j~tacid~l a odsjnt ideh cjtvjje a ojtjntd`e
aeo a ~defij tsadedef j|jk}ias~% De Nla}tjs ?) vj ~l`vna~j tlj qzaidt `b `zs rd~zai
a~~`ndatd`e~ aeo ojk`e~tsatj tlat tlj aidfekjet cjtvjje k`~t ojtjntd`e~ aeo tljds
a~~`ndatjo j|jk}ias d~ f``o je`zfl t` ijt z~ tsae~bjs ae aradiacij j|jk}ias kjta+oata
,j%f%) ~jfkjetatd`e) fj`kjts) >O k`oji$ odsjnti `et` tlj ojtjntd`e% Vj }jsb`sk ae
jraizatd`e `e tlj ]A^NAI R@N 3==6 oata~jt) n`e~dojs tlj tlsjj tsae~bjs+ta~h~ ,d%j%)
~jfkjetatd`e tsae~bjs) fj`kjts tsae~bjs) aeo sjiatjo `cgjnt }sdkdef$) aeo rd~zaidwj tlj
j|jk}ias+j|jk}ias ~dkdiasdt ~tszntzsj a~ a fsa}l% @zs kjta+oata tsae~bjs f`j~ cj`eo
vlat d~ sjqzdsjo de k`~t `cgjnt ojtjntd`e ta~h~ ,d%j%) a natjf`s+iacjijo c`zeodef c`|$)
aeo tlj sj~zitdef detjs}sjtatd`e~ ,j%f%) `cgjnt }`~j) `cgjnt fj`kjts) `cgjnt attsdcztj~$
nae tlje cj z~jo a~ }ast `b `rjsaii ~njej zeojs~taeodef%
De Nla}tjs : vj n`e~dojs a n`etj|tzai `cgjnt }sjodntd`e ta~h XKaid~djvdnw aeo Jbs`~)
3==4[ vldnl vj tanhij c b`skdef a fsa}l `rjs j|jk}ias~ aeo tljds ~}atdai sjiatd`e~ld}~)
vldnl vj naii tlj Rd~zai Kjkj| K`oji% Vj a}}i `zs ’natjf`s+bsjj‛ Rd~zai Kjkj|
K`oji t` a rasdaet `b Aet`ed` _`ssaica—~ N`etj|t Nlaiijefj) vljsj tlj f`ai d~ t` }sjodnt
tlj a}}jasaenj `b a ldooje `cgjnt ~`iji ca~jo `e tlj ldooje sjfd`e—~ ~}atdai sjiatd`e~ld}
t` a ~jt `b rd~dcij `cgjnt~ de tlj ~njej% Vj jraizatj `zs Rd~zai Kjkj| K`oji `e tlj
N`etj|t Nlaiijefj ta~h aeo ~l`v ae dk}s`rjkjet `rjs natjf`s+ca~jo ca~jidej~%
66
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 78/130
Nla}tjs ?
A~~`ndatd`e aeo Kjta+oata _sae~bjs
Vj larj aisjao ~l`ve tlat J|jk}ias+^RK~ asj ldfli+tzejo t` tlj a}}jasaenj `b tlj
~`zsnj j|jk}ias aeo tlat vj asj acij t` `ctade ldfl qzaidt n`ssj~}`eojenj~ cjtvjje
j|jk}ias~ aeo ojtjntd`e vdeo`v~% ^znl ldfl qzaidt aidfekjet kjae~ tlat J|jk}ias+
^RK~ nae cj z~jo b`s k`sj tlae c`zeodef c`| }sjodntd`e ‟ ae kjta+oata a~~`ndatjo
vdtl j|jk}ias~ nae cj tsae~bjssjo `et` tlj ojtjntd`e vdeo`v% _lj laiikash `b `zs
a}}s`anl d~ tlat vj o` e`t sjqzdsj a n`k}ij| dkafj aidfekjet }s`nj~~ ,zeidhj XCjsf
jt ai%) 3==:) Idz jt ai%) 3==4[$ ‟ tlj j|jk}ias+~}jndfin ojtjnt`s aisjao o`j~ aii tlj
v`sh! Vj n`e~dojs kjta+oata de tlj b`sk `b a }d|ji+vd~j iacjidef ~znl a~ a ~jfkjetatd`e
`s a fj`kjtsdn iacjidef aeo vj ~dk}i tsae~bjs tlj j|jk}ias+aidfejo kjta+oata `et`
tlj ojtjntd`e z~def tlj tsae~b skatd e j~tdkatjo cjtvjje tlj ~`zsnj j|jk}ias aeo tlj
ojtjntd`e vdeo`v ,a ~naias ~naij aeo a 3O tsae~iatd`e$%
De tld~ nla}tjs vj }sj~jet sj~zit~ `e a}}idef ae je~jkcij `b J|jk}ias+^RK~
,a~ ojfiejo de Nla}tjs >$ t` a ~jt `b kjta+oata tsae~bjs ta~h~7 ~jfkjetatd`e tsae~bjs)
fj`kjts tsae~bjs) >O k`oji tsae~bjs) a~ vjii a~ sjiatjo `cgjnt }sdkdef% B`s tlj tsae~bjs
a}}idnatd`e~ vj z~j tlj J|jk}ias+^RK kjtl`o vdtl naidcsatd`e cjnaz~j jrje tl`zfl
z~def tlj j|jk}ias n`+`nnzsjenj katsd| c``~t~ `cgjnt ojtjntd`e }jsb`skaenj) dt z~j~
6<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 79/130
J|jk}ias
Kjta+oata
A}}jasaenj
^jfkjetatd`e
J|jk}iasOjtjnt`sv
A}}jasaenj
J|jk}ias
A}}jasaenj
Kjta+oata Kjta+oata
^jfkjetatd`e ^jfkjetatd`e
Ojtjnt`sv Ojtjnt`sv
Bdfzsj ?%17 ^jfkjetatd`e _sae~bjs% @cgjnt ~jfkjetatd`e~ asj tsae~bjssjo bs`k tlj j|jk}ias odsjnti `et` tlj ojtjntd`e vdeo`v%
kzitd}ij `rjsia}}def j|jk}ias~ t` ~n`sj vdeo`v~ ,at tlj n`~t `b b`sfjttdef vldnl ~defij
a~~`ndatd`e d~ tlj cj~t$% Naidcsatd`e }s`oznj~ kznl ldfljs qzaidt aidfekjet~ cjnaz~j
ojtjntd`e~ asj ~n`sjo deoj}jeojeti%
?%1 ^jfkjetatd`e aeo Fj`kjts _sae~bjs
B`s tlj ta~h `b ~jfkjetatd`e) tlj f`ai d~ t` j~tdkatj vldnl }d|ji~ cji`ef t` a fdrje
`cgjnt aeo vldnl o` e`t% Bdfzsj ?%1 ~l`v~ ~`kj qzaidtatdrj ~jfkjetatd`e tsae~bjs
j|ak}ij~ `e a vdoj rasdjt `b `cgjnt nia~~j~% B`s qzaetdtatdrj jraizatd e) vj a~hjo
iacjijs~ t` ~jfkjet aeo fj`kjtsdnaii aee`tatj aii `b tlj de~taenj~ de tlj ’cz~‛ natjf`s
de tlj ]A^NAI R@N 3==6 oata~jt% B`s tlj ~jfkjetatd`e ta~h) `zs kjtl`o }jsb`sk~ at
a }d|jivd~j annzsan `b 4=%2*% B`s fj`kjts j~tdkatd`e) tlj f`ai d~ t` a~~dfe iacji~ t`
}d|ji~ deodnatdef kjkcjs~ld} t` `ej `b > ’ijbt)‛ ’bs`et)‛ aeo ’sdflt‛ o`kdeaet `sdjetatd`e
nia~~j~ XL`djk jt ai%) 3==:[% Vj n`k}asj `zs J|jk}ias+^RK ~~tjk afade~t tv`
ca~jidej~7 ,a$ L`djk—~ }sj+tsadejo fjejsdn fj`kjtsdn nia~~ j~tdkatd`e aif`sdtlk XL`djk
jt ai%) 3==:[8 ,c$ Z~def XBjiwje~wvaic jt ai%) 3=1=[ t` ojtjnt `cgjnt~ b`ii`vjo c ~dk}ij
EE t` nsjatj a~~`ndatd`e~% Vj `ctade a 23%>* }d|jivd~j iacjidef annzsan z~def `zs
J|jk}ias+^RK a}}s`anl a~ n`k}asjo t` tlj ?>%=* `ctadejo z~def XL`djk jt ai%) 3==:[
64
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 80/130
Kjta+oataKjta+oata Kjta+oata
J|jk}ias J|jk}ias J|jk}iasOjtjnt`s v Ojtjnt`s v Ojtjnt`s v
Fj`kjts Fj`kjts Fj`kjts
A}}jasaenj A}}jasaenj A}}jasaenj
Bdfzsj ?%37 Qzaidtatdrj Fj`kjts _sae~bjs% Vj tsae~bjs fj`kjtsdn iacjidef bs`k cz~ j|jk}ias~ `et`n`ssj~}`eodef ojtjntd`e~%
aeo :1%=* z~def XBjiwje~wvaic jt ai%) 3=1=[(EE% _ld~ nijasi ~l`v~ tlat vldij `zs
tsae~bjs d~ ~dk}ij) dt d~ ojfiedtji e`t tsdrdai a~ dt sjidj~ `e `ctadedef ~ts`ef aidfekjet
cjtvjje tlj j|jk}ias aeo tlj ojtjntd`e ,~jj qzaidtatdrj sj~zit~ de Bdfzsj ?%3$% Fi`cai
kjtl`o~ badi t` fjejsatj ~znl aidfekjet~) ijaodef t` kznl i`vjs }jsb`skaenj%
?%3 >O K`oji _sae~bjs
Vj aee`tatjo a ~zc~jt `b nlads j|jk}ias~ vdtl >O k`oji~ bs`k F``fij—~ >O Vasjl`z~j
,aeo aidfejo vdtl F``fij ^hjtnl+Z} >O k`oji+t`+dkafj aidfekjet t``i$% Fdrje a
~defij j|jk}ias) iacjijs~ vjsj a~hjo t` fieo tlj k`~t rd~zaii ~dkdias k`oji de tlj >O
Vasjl`z~j b`s tlat de~taenj aeo }jsb`sk tlj aidfekjet% Ozj t` tlj ldfl qzaidt `b `zs
azt`katdnaii+fjejsatjo a~~`ndatd`e~) vj vjsj acij t` ~dk}i tsae~bjs tlj j|jk}ias+
aidfejo >O k`oji odsjnti `et` tlj ojtjntd`e vdeo`v vdtl`zt ae aoodtd`eai aidfekjet)
~jj Bdfzsj ?%>%
<=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 81/130
Bdfzsj ?%>7 >O K`oji tsae~bjs% De janl `b tlj~j > j|ak}ij~) tlj fsjje c`| de tlj t`} dkafj ~l`v~ tljojtjntd`e vdeo`v) aeo tlj c`tt`k ~l`v~ tlj azt`katdnaii tsae~bjssjo >O k`oji%
<1
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 82/130
Bdfzsj ?%?7 Sjiatjo @cgjnt ]sdkdef% A cdnnij/k`t`scdhj/l`s~j j|jk}ias d~ z~jo t` }sjodnt c`zeodefc`| b`s ’}js~`e‛%
?%> Sjiatjo @cgjnt ]sdkdef
J|jk}ias~ `btje ~l`v ae detjs}ia `b kzitd}ij `cgjnt~) tlz~ ae `tljs `cgjnt~ vldnl
~zbfindjeti `rjsia} vdtl tlj j|jk}ias nae cj rdjvjo a~ aoodtd`eai kjta+oata cji`efdef
t` tlj j|jk}ias% _ld~ ~zffj~t~ z~def ojtjnt`s~ `b `ej natjf`s t` lji} ’}sdkj‛ `cgjnt~
`b ae`tljs natjf`s% Vj i``h at tlj b`ii`vdef ta~h7 }sjodntdef a c`zeodef c`| b`s
’}js~`e‛ fdrje a ojtjntd`e `b natjf`s \ ) vljsj \ d~ jdtljs a l`s~j) k`t`scdhj) `s
cdnnij ,~jj Bdfzsj ?%? b`s qzaidtatdrj sj~zit~$% Vj ~a tlat a }js~`e d~ ’sdodef‛ ae \ db dt~ `rjsia} vdtl tlj \ c`zeodef c`| ~zbfindjet ,@^ 5 %1$% Vj qzaetdtatdrji jraizatjo
tlj }js~`e }sjodntd`e }jsb`skaenj aeo n`k}asjo afade~t a ca~jidej vldnl }sjodnt~ a
}js~`e }sj~jenj ca~jo `e kag`sdt r`tdef% @zs kjtl`o n`e~dojsaci `zt}jsb`sk~ tlj
ca~jidej ,63%?2* a~ n`k}asjo t` :<%26* b`s tlj ca~jidej$) ~zffj~tdef tlat `zs j|jk}ias
a~~`ndatd`e~ }s`rdoj f``o aidfekjet `b j|jk}ias~ a~ vjii a~ tljds sjiatjo `cgjnt~% _lj
sj~zit~ nae cj ~jje de _acij ?%1%
<3
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 83/130
Natjf`s Kag`sdt R`tdef XKaid~djvdnw jt ai%) 3=11[
cdnnij 2>%?* 63%<*k`t`scdhj :=%=* 26%?*
l`s~j 23%2* 66%3*
_acij ?%17 D~ tljsj a }js~`e sdodef tld~ l`s~j0 Vj }sjodnt bs`k `zs cdnnij) k`t`scdhj) aeo l`s~jojtjnt`s~ vljtljs tljsj d~ a }js~`e sdodef tlj `cgjnt% @zs a}}s`anl d~ cjttjs tlae tlj kag`sdt r`tjca~jidej) ~zffj~tdef tlat j|jk}ias~ asj z~jbzi at }sjodntdef ejasc) sjiatjo `cgjnt~%
?%? J|jk}ias De+}adetdef
Vlje tlj oata~jt o`j~ e`t n`kj jqzd}}jo vdtl ae kjta+oata cj`eo `cgjnt iacji~
aeo c`zeodef c`|j~) dt d~ e`t }`~~dcij t` }jsb`sk ae hdeo `b tsae~bjs% De ~znl a na~j)
vj nsjatj ae ’detjs}sjtatd`e‛ rda a ~dk}ij `cgjnt de+}adetdef z~def tlj j|jk}ias—~ sav
a}}jasaenj) a~ ~l`ve Bdfzsj ?%: aeo Bdfzsj ?%2% De janl j|ak}ij vj ~l`v tlj ~`zsnj
j|jk}ias ai`ef vdtl dt~ ijasejo tjk}iatj vj `e tlj ijbt) aeo tlj ojtjntd`e vdeo`v aeo
j|jk}ias de+}adetdef `e tlj sdflt% E`tj l`v `zs j|jk}ias de+}adetdef sj~zit~ ~l`vna~j
tlj qzaidt `b `zs azt`katdn a~~`ndatd`e~) jt afade% ^`kj badizsj~ `b `zs de+}adetdef nae
cj ~jje de Bdfzsj ?%6 aeo Bdfzsj ?%<% E`tj tlat qzdtj `btje vlje vj }s`oznj a badizsj
,ojfiejo c natjf`s kd~katnl$) asj kd~tahj~ asj `btje cjtvjje ~jkaetdnaii sjijraet
natjf`sdj~ ,j%f%) a n`v kd~tahje a~ a ~ljj}) `s a l`s~j kd~tahje a~ a o`f$% Jrje vlje
vj j~tdkatj tlj vs`ef `cgjnt natjf`s) vj nae ~tdii `btje }sjodnt tlj `cgjnt—~ }`~j qzdtj
vjii%
<>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 84/130
v11<
Ojtjntd`e De+}adetdef
v33
Ojtjntd`e De+}adetdef
v1:= Ojtjntd`e De+}adetdef
v>4: Ojtjntd`e De+}adetdef
Bdfzsj ?%:7 ^znnj~~bzi J|jk}ias _sae~bjs~ 1% Vj ~`kj `b tlj t`} j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI
R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%
<?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 85/130
v3=?
Ojtjntd`e De+}adetdef
v3>=
Ojtjntd`e De+}adetdef
v<3<
Ojtjntd`e De+}adetdef
v12:
Ojtjntd`e De+}adetdef
Bdfzsj ?%27 ^znnj~~bzi J|jk}ias _sae~bjs~ 3% Vj ~`kj `b tlj t`} j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI
R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%
<:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 86/130
v3<3
Ojtjntd`e De+}adetdef
v:63
Ojtjntd`e De+}adetdef
v3:<
Ojtjntd`e De+}adetdef
v1=6 Ojtjntd`e De+}adetdef
Bdfzsj ?%67 J|jk}ias _sae~bjs Badizsj~ 1% Vj ~`kj `b tlj t`} badijo j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI
R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%
<2
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 87/130
v3:= Ojtjntd`e De+}adetdef
v?:>
Ojtjntd`e De+}adetdef
v?21
Ojtjntd`e De+}adetdef
v3=6
Ojtjntd`e De+}adetdef
Bdfzsj ?%<7 J|jk}ias _sae~bjs Badizsj~ 3% Vj ~`kj `b tlj t`} badijo j|jk}ias tsae~bjs~ bs`k tlj ]A^NAI R@N 3==6 tj~t~jt% Janl j|ak}ij ~l`v~7 ijasejo j|jk}ias tjk}iatjvj aeo j|jk}ias a}}jasaenj)ojtjntd`e vdeo`v de tj~t dkafj) j|jk}ias a}}jasaenj `rjsia%
<6
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 88/130
?%: Rd~zaidwdef J|jk}ias+J|jk}ias A~~`ndatd`e~
’%%%vj ~jj a n`k}idnatjo ejtv`sh `b ~dkdiasdtdj~ `rjsia}}def aeo nsd~~+ns`~~def‛
Izovdf Vdttfje~tjde
De tld~ ~jntd`e) vj nsjatj a rd~zaidwatd`e `b tlj j|jk}ias+j|jk}ias rd~zai a~~`ndatd`e
~tszntzsj deoznjo c tlj J|jk}ias+^RK aif`sdtlk `e tlj ]A^NAI R@N 3==6 oata~jt%
@zs rd~zaidwatd`e d~ a ~`st `b ’Rd~zai Kjkj|‛ fsa}l) vldnl vj rd~zaidwj z~def tlj }`}zias
`}je ~`zsnj fsa}l rd~zaidwatd`e idcsas Fsa}lrdw XJii~`e jt ai%) 3==1[% Vj ai~` w``k
det` }`std`e~ `b tld~ fsa}l t` aeaiwj tlj ’rd~zai tsadi‛ ~dkdiasdt ~tszntzsj as`zeo a
}astdnzias j|jk}ias%
Vldij tlj n`enj}t `b a ejtv`sh `b `rjsia}}def ~dkdiasdtdj~ n`kj~ bs`k Vdttfje~tjde—~
bakdi sj~jkciaenj~ XVdttfje~tjde) 14:>[) tlj doja `b a rd~zai tsadi d~ odsjnti tahje bs`k
Cz~l—~ ’A~ Vj Ka _ldeh%‛ _lj aoraetafj `b tldehdef ac`zt j|jk}ias~ de tjsk~ `b rd~zai
tsadi~ d~ tlat a tsadi aii`v~ b`s a ~defij `cgjnt de~taenj t` }astdnd}atj de kzitd}ij rd~zai
tsadi~% _` nsjatj a fsa}l) vj n`eejnt tv` j|jk}ias~ vdtl ae jofj db c`tl j|jk}ias~ fisj
`e janl `tljs vdtl a naidcsatjo ~n`sj fsjatjs tlae =%6 aeo db tljds c`zeodef c`| `rjsia}
d~ fsjatjs tlae =%:% Vldij tld~ d~ a satljs ~tsdnt sjqzdsjkjet b`s nsjatdef j|jk}ias+
j|jk}ias jofj~) vjahjs n`e~tsadet~ nsjatj fsa}l~ vldnl asj laso t` rd~zaidwj de tv`
odkje~d`e~% B`s tlj ~ahj `b niasdt) vj nl`~j tlj~j }asakjtjs~ jrje tl`zfl tlj o` e`t
aiva~ sj~zit de a n`eejntjo fsa}l% Vj od~}ia fsa}l~ b`s ~jrjsai ]A^NAI R@N `cgjnt
natjf`sdj~ z~def tlj J|jk}ias+^RK aif`sdtlk% _lj~j ’Rd~zai Kjkj|j~‛ nae cj ~jje de
tlj b`ii`vdef }afj~7 tsade ,~jj Bdfzsj ?%4$) k`t`scdhj ,~jj Bdfzsj ?%1=$) aeo ajs`}iaej
,~jj Bdfzsj ?%11$%
Dt d~ ai~` dk}`staet t` e`tj tlat kae `b tlj doja~ cjldeo Cz~l—~ kjkj| asj ~i`vi
~tastdef t` a}}jas de n`k}ztjs rd~d`e% @ej ldfli sjijraet a}}s`anl d~ tlj ]l`t`cd`~
aif`sdtlk XHjkjikanljs+^lidwjskae jt ai%) 3=11[ b`s eardfatdef a iasfj n`iijntd`e `b
<<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 89/130
dkafj~ `b tlj ~akj }js~`e% C nsjatdef ideh~ cjtvjje rd~zaii ~dkdias }l`t`~ `b tlj ~akj
}js~`e—~ banj) dt d~ }`~~dcij t` sjeojs a k`rdj vldnl ~zkkasdwj~ tlj rd~zai jr`iztd`e
`b a ~defij }js~`e) jrje nsjatdef n`k}jiidef aedkatd`e~ `b tlj banj a~ dt tsae~dtd`e~
bs`k odbbjsjet ~tatj~ ,j%f%) a k`rdj vldnl tsae~dtd`e~ bs`k a }js~`e ~kdidef t` tlj ~akj
}js~`e bs`vedef$% Vldij tld~ a}}s`anl la~ kaoj dt~ va det` tlj }`}zias Banj K`rdj~
bjatzsj `b F``fij—~ ]dna~a }l`t`+~lasdef ~`btvasj) dt d~ dk}`staet t` sjaidwj tlat dt osav~
dt~ ~znnj~~ bs`k tlj katzsdt `b banj ojtjntd`e aeo banj sjn`fedtd`e tjnle`i`f% _lj hj
nlaiijefj d~ de j|tjeodef tlj~j doja~ t` e`e+banj o`kade~%
<4
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 90/130
_sade Rd~zai Kjkj| Fsa}l
1=| w``k
Bdfzsj ?%47 _sade N`enj}t Rd~zai Kjkj|% _lj ~dkdiasdt fsa}l d~ nsjatjo c a}}idef janl J|jk}ias+^RK t` de+nia~~ dkafj~%
4=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 91/130
K`t`scdhj Rd~zai Kjkj| Fsa}l
1=| w``k1=| w``k
Bdfzsj ?%1=7 K`t`scdhj N`enj}t Rd~zai Kjkj|% _lj ~dkdiasdt fsa}l d~ nsjatjo c a}}idef janlJ|jk}ias+^RK t` de+nia~~ dkafj~%
41
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 92/130
1=| w``k1=| w``k
Ajs`}iaej Rd~zai Kjkj| Fsa}l
Bdfzsj ?%117 Ajs`}iaej N`enj}t Rd~zai Kjkj|% _lj ~dkdiasdt fsa}l d~ nsjatjo c a}}idef janlJ|jk}ias+^RK t` de+nia~~ dkafj~%
43
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 93/130
Nla}tjs :
N`etj|tzai @cgjnt ]sjodntd`e
L`v bas nae `z f` cjb`sj szeedef ae `cgjnt ojtjnt`s0
Aet`ed` _`ssaica
De sjai ~njej~ n`k}`~jo `b kae odbbjsjet `cgjnt~) tlj ~}atdai n`efifzsatd`e `b `ej
`cgjnt nae bandidtatj sjn`fedtd`e `b sjiatjo `cgjnt~ XCas aeo Ziikae) 1442[) aeo qzdtj
`btje akcdfzdtdj~ de sjn`fedtd`e naee`t cj sj~`irjo vdtl`zt i``hdef cj`eo tlj ~}atdai
j|tjet `b tlj `cgjnt de qzj~td e% _lz~) aif`sdtlk~ vldnl g`deti sjn`fedwj kae `cgjnt~
at `enj c tahdef ann`zet `b n`etj|tzai sjiatd`e~ld}~ larj cjje qzdtj }`}zias% Vldij
jasi ~~tjk~ sjidjo `e laeo+n`ojo szij~ b`s detjs+`cgjnt n`etj|t ,j%f%) XLae~`e aeo
Sd~jkae) 146<) ^tsat aeo Bd~nlijs) 1441[$) k`sj k`ojse a}}s`anlj~ t}dnaii }jsb`sk
debjsjenj de a }s`cacdid~tdn fsa}ldnai k`oji vdtl sj~}jnt t` natjf`sdj~ vljsj `cgjnt
detjsantd`e~ asj k`ojijo a~ ldfljs `sojs }`tjetdai~ XLj jt ai%) 3==?) Hzkas aeo Ljcjst)
3==:) ^l`tt`e jt ai%) 3==2) Sacde`rdnl jt ai%) 3==6) Faiijfzdii`~ jt ai%) 3==<) ]asdhl jt ai%)
3==<) Sz~~jii jt ai%) 3==6[% @ej dk}`staet dk}idndt a~~zk}td`e kaoj c aii ~znl k`oji~
d~ tlat detjsantd`e~ cjtvjje `cgjnt de~taenj~ nae cj aojqzatji k`ojijo a~ sjiatd`e~ld}~
cjtvjje lzkae+ojfiejo `cgjnt natjf`sdj~%
De tld~ nla}tjs vj nlaiijefj tld~ ’natjf`s a~~zk}td`e‛ b`s `cgjnt+`cgjnt detjsantd`e~
4>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 94/130
aeo }s`}`~j a e`rji natjf`s+bsjj a}}s`anl b`s k`ojidef `cgjnt sjiatd`e~ld}~% Vj
}s`}`~j a ejv bsakjv`sh) tlj Rd~zai Kjkj| K`oji) b`s sj}sj~jetdef aeo sja~`edef ac`zt
`cgjnt dojetdtdj~ aeo tljds n`etj|tzai sjiatd`e~ld}~ de ae j|jk}ias+ca~jo) e`e+}asakjtsdn
va% Vj jraizatj `zs k`oji `e Aet`ed _ ssaica—~ }s`}`~jo N`etj|t Nlaiijefj X_`ssaica)
3==>a[ afade~t a ca~jidej natjf`s+ca~jo ~~tjk%
@zs ~tastdef }`det d~ Raeejras Cz~l—~ `c~jsratd`e tlat ~tsdnt natjf`sdnai deoj|def `b
n`enj}t~ la~ ~jrjsj idkdtatd`e~ XCz~l) 14?:[ ,~jj od~nz~~d`e de Nla}tjs 1$% Ae`tljs
k`tdratd`e d~ K`~lj Cas vl` cjidjrj~ tlat }sjodntd`e }ia~ a hj s`ij de tlj lzkae
csade XCas) 3==4[% Acaeo`edef sdfdo `cgjnt natjf`sdj~) vj jkcsanj Cz~l—~ aeo Cas—~
cjidjb de tlj }sdkas s`ij `b a~~`ndatd`e~) czt zeidhj Cz~l) vj adk t` od~n`rjs tlj~j
a~~`ndatd`e~ azt`katdnaii bs`k tlj oata% At tlj n`sj `b `zs k`oji d~ ae j|jk}ias+
ca~jo sj}sj~jetatd`e `b `cgjnt~ XE`~`b~h) 14<2) Kaid~djvdnw aeo Jbs`~) 3==<[% _lj
Rd~zai Kjkj| nae tlje cj tl`zflt `b a~ a ra~t fsa}l) vdtl e`oj~ sj}sj~jetdef aii tlj
`cgjnt de~taenj~ de tlj oata~jt) aeo asn~ sj}sj~jetdef tlj odbbjsjet t}j~ `b a~~`ndatd`e~
cjtvjje tljk ,Bdfzsj :%1$% _ljsj asj tv` t}j~ `b asn~ de `zs k`oji) jen`odef tv`
odbbjsjet sjiatd`e~ld}~ cjtvjje `cgjnt~7 1$ rd~zai ~dkdiasdt ,j%f% tld~ nas i``h~ idhj tlat
nas$) aeo 3$ n`etj|tzai a~~`ndatd`e~ ,j%f% tld~ nas d~ ej|t t` tld~ czdiodef$%
@enj tlj fsa}l d~ czdit) dt nae cj z~jo t` detjs}sjt a e`rji dkafj ,Bdfzsj :%1) ijbt$ c
fis~t n`eejntdef ~jfkjet~ vdtlde tlj dkafj vdtl ~dkdias j|jk}ias~) aeo tlje }s`}afatdef
n`etj|tzai deb`skatd`e cjtvjje tlj~j j|jk}ias~ tls`zfl tlj fsa}l% Vlje ae j|jk}ias
fjt~ antdratjo) rd~zaii ~dkdias j|jk}ias~ a~ vjii a~ `tljs n`etj|tzaii sjijraet `cgjnt~
fjt antdratjo a~ vjii% _ld~ va) j|jk}ias+t`+j|jk}ias ~dkdiasdt de tlj Kjkj| fsa}l nae
~jsrj a~ Cz~l—~ ’tsadi~‛ t` ideh n`enj}t~ t`fjtljs de a e`e+}asakjtsdn) qzjs+oj}jeojet
va) vdtl`zt tlj z~j `b }sjojfiejo natjf`sdj~% B`s j|ak}ij) de Bdfzsj :%1) vj ~l`zio cj
acij t` debjs tlat a nas ~jje bs`k tlj sjas `btje n`+`nnzs~ vdtl ae `cidqzj czdiodef vaii
,czt e`t a bs`etai vaii$ „ ~`kjtldef vldnl natjf`s+ca~jo k`oji~ v`zio cj laso+}sj~~jo
4?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 95/130
Bdfzsj :%17 _lj Rd~zai Kjkj| fsa}l jen`oj~ `cgjnt ~dkdiasdt ,~`ido cianh jofj$ aeo ~}atdai n`etj|t
,o`ttjo sjo jofj$ cjtvjje }ads~ `b `cgjnt j|jk}ias~% A ~}atdai n`etj|t bjatzsj d~ ~t`sjo b`s janl n`etj|tjofj% _lj Kjkj| fsa}l nae cj z~jo t` detjs}sjt a ejv dkafj c a~~`ndatdef dkafj ~jfkjet~ vdtl
j|jk}ias~ de tlj fsa}l ,`saefj jofj~$ aeo }s`}afatdef tlj deb`skatd`e% Bdfzsj cj~t rdjvjo de n`i`s%
t` anldjrj%
B`skaii) vj ojfiej tlj Rd~zai Kjkj| K`oji a~ a fsa}l F ; ,R) J ) J N ) xO{) xb {$
n`e~d~tdef `b E `cgjnt j|jk}ias e`oj~ R ) ~dkdiasdt jofj~ J ) n`etj|t jofj~ J N ) E
}js+j|jk}ias ~dkdiasdt bzentd`e~ xO{) aeo tlj ~}atdai bjatzsj~ xb { a~~`ndatjo vdtl janl
n`etj|t jofj%
:%1 Czdiodef a IacjiKj Rd~zai Kjkj| Fsa}l
Vj j|tsant a iasfj oataca~j `b j|jk}ias `cgjnt~ aeo tljds fs`zeo+tsztl ~jfkjetatd`e
ka~h~ bs`k tlj IacjiKj XSz~~jii jt ai%) 3==<[ oata~jt aeo ijase tlj ~tszntzsj `b tlj
Rd~zai Kjkj| de ae `bffidej ~jttdef% Vj z~j `cgjnt~ bs`k tlj >= k`~t bsjqzjeti `nnzssdef
natjf`sdj~ de IacjiKj% ^dkdiasdt jofj~ asj nsjatjo z~def tlj }js+j|jk}ias od~taenj
bzentd`e ijasedef bsakjv`sh `b XKaid~djvdnw aeo Jbs`~) 3==<[) aeo n`etj|t jofj~ asj
nsjatjo janl tdkj tv` j|jk}ias~ asj `c~jsrjo de tlj ~akj dkafj% Vj larj a t`tai
`b E ; <6) <=3 j|jk}ias~ de tlj Rd~zai Kjkj|) yJ y ; 362) 6<3 ~dkdiasdt jofj~) aeo
yJ N y ; 4<4) 1=2 n`etj|t jofj~%
Vj z~j tlj }js+j|jk}ias od~taenj+bzentd`e ijasedef aif`sdtlk `b XKaid~djvdnw aeo
4:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 96/130
a~~`ndatd`e~
0
nas
o``s
s`ao
vdeo`v
vdeo`v
tsjj
nas
tsjj
vdeo`v
vljjivljji0
Natjf`s
J~tdkatd`e
nas}js~`evljjis`aovdeo`vbjenjczdiodef
~dojvaiho``stsjj
Bdfzsj :%37 _ ssaica—~ N`etj|t Nlaiijefj7 ’L`v bas nae `z f` vdtl`zt szeedef a i`nai `cgjntojtjnt`s0‛ _lj ta~h d~ t` sja~`e ac`zt tlj dojetdt `b tlj ldooje `cgjnt ,oje`tjo c a ’0‛$ vdtl`zt i`nai
deb`skatd`e% De `zs natjf`s+bsjj Rd~zai Kjkj| k`oji) `cgjnt }sjodntd`e~ asj fjejsatjo de tlj b`sk `b j|jk}ias a~~`ndatd`e~ b`s tlj ldooje `cgjnt% De a natjf`s+ca~jo k`oji) tlj natjf`s `b tlj ldooje `cgjnt
d~ odsjnti j~tdkatjo%
Jbs`~) 3==<[) a~ oj~nsdcjo de Nla}tjs 3) t` ijase tlj `cgjnt ~dkdiasdt jofj~% Vj nsjatj
a ~dkdiasdt jofj cjtvjje tv` j|jk}ias~ db tlj asj ojjkjo ~dkdias c janl `tljs~—
od~taenj bzentd`e~% Vj z~j a fi|jo θ ; %====1 aeo ώ ; 1== b`s aii j|jk}ias~ ,~jj
Jqzatd`e 3%2$%
Vlje tv` `cgjnt~ `nnzs de~doj a ~defij dkafj) vj jen`oj tljds 3+O ~}atdai sjiatd`e+
~ld} det` a n`etj|t bjatzsj rjnt`s b ∌ 1= ,rd~zaidwjo a~ sjo o`ttjo jofj~ de Bdfzsj :%1$%
_lj n`etj|t bjatzsj rjnt`s jen`oj~ sjiatdrj `rjsia}) sjiatdrj od~}ianjkjet) sjiatdrj ~naij)
aeo sjiatdrj ljdflt `b tlj c`tt`k+k`~t }d|ji cjtvjje tv` j|jk}ias sjfd`e~ de a ~defij
dkafj% _ld~ bjatzsj na}tzsj~ tlj ~}atdai sjiatd`e~ld} cjtvjje tv` sjfd`e~ aeo o`j~
e`t tahj det` ann`zet ae a}}jasaenj deb`skatd`e ‟ dt d~ a fjejsaidwatd`e `b tlj ~}atdai
bjatzsj~ z~jo de XFaiijfzdii`~ jt ai%) 3==<[% Vj kja~zsj tlj ~dkdiasdt cjtvjje tv`
n`etj|t bjatzsj~ z~def a Faz~~dae hjseji7 H ,b ) b $ ; j∓α1yy b ∓ b yy3 vdtl α1 ; 1%=%
:%3 _ ssaica—~ N`etj|t Nlaiijefj
_lj detzdtd`e tlat vj v`zio idhj t` jraizatj d~ tlat kae z~jbzi sjfziasdtdj~ `b tlj rd~zai
v`sio asj i`~t vlje ojaidef ~`iji vdtl natjf`sdj~ ,j%f%) tlj ~doj rdjv `b a czdiodef
~l`zio a~~`ndatj k`sj vdtl a ~doj rdjv `b a nas tlae a bs`etai rdjv `b a nas$% _lj hj
42
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 97/130
k`tdratd`e cjldeo tlj Rd~zai Kjkj| d~ tlat n`etj|t ~l`zio oj}jeo `e tlj a}}jasaenj `b
ae `cgjnt aeo e`t gz~t tlj natjf`s dt cji`ef~ t`% De `sojs t` tj~t tld~ l}`tlj~d~ afade~t
tlj n`kk`ei ljio }santdnj `b ac~tsantdef ava a}}jasaenj det` natjf`sdj~) vj ejjo a
sdnl jraizatd`e oata~jt a~ vjii a~ a kjaedefbzi jraizatd`e ta~h%
Vj b`zeo tlat tlj N`etj|t Nlaiijefj X_ ssaica) 3==>a[ sjnjeti }s`}`~jo c Aet`ed`
_ ssaica fit~ `zs ejjo~ }jsbjnti% _lj jraizatd`e ta~h d~ de~}dsjo c tlj qzj~td e7
’L`v bas nae `z f` vdtl`zt szeedef ae `cgjnt ojtjnt`s0‛ _lj f`ai d~ t` sjn`fedwj a
~defij `cgjnt de tlj dkafj vdtl`zt }jjhdef at }d|ji~ cji`efdef t` tlat `cgjnt% _`ssaica
}sj~jetjo ae aif`sdtlk b`s }sjodntdef tlj natjf`s aeo ~naij `b ae `cgjnt z~def `ei
n`etj|tzai deb`skatd`e X_`ssaica) 3==>c[) czt ld~ e`td`e `b n`etj|t d~ ~njej+njetjsjo
,d%j) tlj a}}jasaenj `b tlj jetdsj dkafj d~ z~jo b`s }sjodntd`e$% ^denj tlj n`etj|t vj
vd~l ~tzo d~ `cgjnt+njetjsjo) vj z~j ae `cgjnt+njetjsjo b`skziatd`e `b tlj N`etj|t
Nlaiijefj% Vldij dt d~ e`t nijas db tlj ac~`iztj }jsb`skaenj ezkcjs~ `e tlj N`etj|t
Nlaiijefj asj rjs kjaedefbzi de tljk~jirj~) vj bjji tlat dt d~ ae dojai ta~h b`s ~tzodef
`cgjnt+njetjsjo n`etj|t aeo tlj s`ij `b natjf`sdwatd`e a~~zk}td`e~ de ~znl k`oji~%
De `zs rasdaet `b tlj N`etj|t Nlaiijefj) tlj f`ai d~ t` }sjodnt tlj natjf`s `b aldooje `cgjnt td ~`iji ca~jo `e dt~ ~}atdai sjiatd`e~ld}~ t` ~`kj }s`rdojo `cgjnt~ ‟
vdtl`zt z~def tlj }d|ji~ cji`efdef t` tlj ldooje `cgjnt at aii% B`s `zs ~tzo) vj z~j
kaezaii }s`rdojo sjfd`e~ aeo natjf`s iacji~ `b H ~z}}`stdef `cgjnt~ de~doj a ~defij
dkafj% Vj sjbjs t` tlj dojetdtdj~ `b tlj H ~z}}`stdef `cgjnt~ de tlj dkafj a~ xt1) % % % ) tH {
,vljsj t ∌ x1) % % % ) yN y{$ aeo tlj ~jt `b H 3O ~}atdai sjiatd`e~ld} bjatzsj~ cjtvjje janl
~z}}`stdef `cgjnt aeo tlj ldooje `cgjnt a~ xb d1) % % % ) b dH {%
46
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 98/130
:%> Debjsjenj de tlj Rd~zai Kjkj| K`oji
De tld~ ~jntd`e) vj j|}iade l`v t` z~j tlj Rd~zai Kjkj| fsa}l ,azt`katdnaii n`e+
~tszntjo bs`k oata$ t` }jsb`sk debjsjenj b`s tlj N`etj|t Nlaiijefj ldooje+`cgjnt
}sjodntd`e ta~h% E`t kahdef tlj ’natjf`s a~~zk}td e)‛ tlj k`oji d~ ojfiejo vdtl
sj~}jnt t` j|jk}ias a~~`ndatd`e~ b`s tlj ldooje `cgjnt% Debjsjenj de tlj k`oji sjtzse~ a
n`k}atdcdidt ~n`sj cjtvjje jrjs j|jk}ias aeo tlj ldooje `cgjnt) aeo nae cj tl`zfl
`b a~ sjtzsedef ae `sojsjo id~t `b j|jk}ias a~~`ndatd`e~% Ozj t` tlj eatzsj `b j|jk}ias
a~~`ndatd`e~ a~ `}}`~jo t` natjf`s a~~dfekjet~) a ~z}}`stdef `cgjnt nae cj a~~`ndatjo
vdtl kzitd}ij j|jk}ias~ a~ `}}`~jo t` a ~defij natjf`s% Vj nsjatj ~`bt j|jk}ias a~~`nd+
atd`e~ cjtvjje janl `b tlj ~z}}`stdef `cgjnt~ aeo tlj j|jk}ias~ de tlj Rd~zai Kjkj|
z~def tlj ~dkdiasdt bzentd`e~ xO{ ,~jj ^jntd`e :%1$%
x^ 1) % % % ) ^ H { asj tlj a}}jasaenj bjatzsj~ b`s tlj H ~z}}`stdef `cgjnt~% Aa g d~ tlj
abfiedt cjtvjje j|jk}ias a de tlj Rd~zai Kjkj| aeo tlj g+tl ~z}}`stdef `cgjnt aeo d~
nsjatjo c jraizatdef ^ g zeojs a—~ od~taenj bzentd e Aa g ; j∓Oa,^ g$% ά,jd) j g) b dg$ d~ tlj
}adsvd~j n`k}atdcdidt cjtvjje j|jk}ias jd aeo j g zeojs tlj ~}atdai bjatzsj b dg % Ijt V ac
cj tlj aoganjen katsd| sj}sj~jetatd`e `b tlj ~dkdiasdt jofj~ ,V zr ; X,z) r$ ∌ J [$%
Debjsjenj de tlj Rd~zai Kjkj| K`oji d~ o`ej c `}tdkdwdef tlj b`ii`vdef n`eodtd`eai
od~tsdcztd`e vldnl ~n`sj~ tlj a~~dfekjet `b ae ascdtsas j|jk}ias jd t` tlj ldooje `cgjnt
ca~jo `e n`etj|tzai sjiatd`e~7
},jdyA1) % % % ) AH ) b d1) % % % ) b dH $ ∛H
g;1
E a;1
Aa g ά,jd) ja) b dg$ ,:%1$
i`f ά,jd) j g) b dg$ ;
,z)r$∌J N
V dzV grH ,b dg) b zr$,z)r$∌J N
V dzV gr,:%3$
4<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 99/130
_lj sja~`e b`s tlj ~zkkatd`e de~doj Jqzatd`e :%3 d~ tlat dt affsjfatj~ n`etj|tzai de+
tjsantd`e~ bs`k ~dkdias j|jk}ias~% C o`def tld~) vj jbbjntdrji ’oje~db‛ tlj n`etj|tzai
detjsantd`e~ de tlj Rd~zai Kjkj|% Ae detjs}sjtatd`e `b tld~ oje~dfinatd`e }s`njozsj d~ tlat
vj asj nsjatdef a hjseji oje~dt j~tdkat`s b`s ae ascdtsas }ads `b j|jk}ias~ ,jd) j g$ rda
a vjdfltjo ~zk `b hjseji~ }ianjo at n`etj|t bjatzsj~ de tlj oata ~jt xb zr{ 7 ,z) r$ ∌ J N
vljsj tlj vjdflt~ V dzV gr kja~zsj rd~zai ~dkdiasdt cjtvjje }ads~ ,jd) j g$ aeo ,jz) jr$ %
Vj j|}jsdkjetjo vdtl z~def a ~defij hjseji) ά,jd) j gy b dg$ ; H ,b dg ) b jd)jg $) aeo b`zeo
tlat tlj detjfsatd`e `b kzitd}ij bjatzsj~ rda tlj oje~dfinatd`e oj~nsdcjo ac`rj d~ a hj
defsjodjet b`s ~znnj~~bzi Rd~zai Kjkj| debjsjenj%
Bdeaii) abtjs }jsb`skdef debjsjenj de tlj Rd~zai Kjkj| K`oji) vj asj ijbt vdtl a
~n`sj b`s janl j|jk}ias% At tld~ ~tafj) a~ bas a~ `zs k`oji d~ n`enjsejo) tlj sjn`fedtd e
la~ aisjao cjje }jsb`skjo% L`vjrjs) ~denj tlj ta~h vj asj jraizatjo `e d~ natjf`s+
ca~jo) vj n`kcdej tlj sjtzsejo j|jk}ias~ det` a r`tj b`s natjf`sdj~ z~def Iznj—~
A|d`k `b Nl`dnj XKjode aeo ^nlabbjs) 146<[ vldnl arjsafj~ tlj j|jk}ias sj~}`e~j~
}js+natjf`s%
N`IA+ca~jo ]asakjtsdn K`oji
Vj v`zio idhj t` jraizatj tlj Rd~zai Kjkj| k`oji afade~t a k`sj tsaodtd`eai) natjf`s+
ca~jo bsakjv`sh vdtl }asakjtsdn detjs+natjf`s sjiatd`e~ld}~% @ej `b tlj k`~t sjnjet
aeo ~znnj~~bzi a}}s`anlj~ d~ tlj N`IA k`oji XFaiijfzdii`~ jt ai%) 3==<[% N`IA ijase~ a
~jt `b }asakjtjs~ b`s janl }ads `b natjf`sdj~ vldnl n`ssj~}`eo t` sjiatdrj ~tsjeftl~ `b
tlj b`zs odbbjsjet t`})ac`rj)cji`v)de~doj ~}atdai sjiatd`e~ld}~% De tlj na~j `b ojaidef
vdtl natjf`sdj~ odsjnti vj n`e~dojs a n`eodtd`eai od~tsdcztd`e `rjs tlj natjf`s `b tlj
ldooje `cgjnt td tlat bant`s~ a~ a ~tas fsa}l vdtl H ijarj~ ,vdtl tlj ldooje `cgjnt
cjdef n`eejntjo t` aii tlj ~z}}`stdef `cgjnt~$% μ asj k`oji }asakjtjs~) ά d~ a }adsvd~j
}`tjetdai tlat kja~zsj~ tlj n`k}atdcdidt `b tv` natjf`sdj~ vdtl a ~}jndfijo ~}atdai
44
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 100/130
sjiatd`e~ld}) aeo W d~ a e`skaidwatd`e n`e~taet ~znl tlat tlj n`eodtd`eai od~tsdcztd`e
~zk~ t` 1%
},tdyt1) % % % ) tH ) b d1) % % % ) b dH )μ$ ; 1W
H g;1
ά,td) t g) b dg )μ$ ,:%>$
B`ii`vdef XFaiijfzdii`~ jt ai%) 3==<[) vj z~j a bjatzsj bzentd`e l,b $ tlat n`k}ztj~
tlj abfiedt cjtvjje bjatzsj b aeo a ~jt `b }s`t`t}dnai ~}atdai sjiatd`e~ld}~% Vj azt`kat+
dnaii fieo ] }s`t`t}dnai ~}atdai sjiatd`e~ld}~ c niz~tjsdef aii ~}atdai bjatzsj rjnt`s~ xb {
de tlj tsadedef ~jt rda tlj }`}zias H+kjae~ aif`sdtlk% Ijt l,b $ ∌ ] cj tlj e`skaidwjo
rjnt`s `b abfiedtdj~ t` niz~tjs njetjs~ xn1) % % % )n] {%μ
d~ tlj ~jt `b aii }asakjtjs~ de tld~k`oji) vdtl μ,td) t g$ ∌ ] cjdef tlj }asakjtjs~ a~~`ndatjo vdtl tlj }ads `b natjf`sdj~
,td) t g$%
i`f ά,td) t g) b dg )μ$ ; Xl,b dg$_ [ μ,td) t g$ ,:%?$
ld,b $ ∛ j∓αyy b ∓ndyy3 ,:%:$
Vj tsdjo z~def tlj b`zs }s`t`t}dnai sjiatd`e~ld}~ n`ssj~}`eodef t` ac`rj) cji`v)
de~doj) aeo `zt~doj a~ de XFaiijfzdii`~ jt ai%) 3==<[) czt b`zeo tlat z~def H+kjae~ vdtl
~dfedfinaeti iasfjs ezkcjs `b }s`t`t}j~ ] ; >= }s`oznjo ~z}jsd`s sj~zit~% B`s ijasedef
μ) vj b`zeo tlj ka|dkzk idhjidl``o μ z~def fsaodjet oj~njet% _lj tsadedef `cgjntdrj
bzentd`e va~ `}tdkdwjo t` kdkdn vlat la}}je~ ozsdef tj~tdef `e tlj N`etj|t Nlaiijefj
ta~h% ^denj tlj od~tsdcztd`e~ b`s tlj N`etj|t Nlaiijefj ta~h asj ojfiejo vdtl sj~}jnt t` a
~defij natjf`s rasdacij ,~jj Jqzatd`e :%>$) vj n`zio n`k}ztj tlj }astdtd`e bzentd`e
odsjnti aeo odo e`t sjqzdsj ae a}}s`|dkatd`e~ a~ de XFaiijfzdii`~ jt ai%) 3==<[ ,vldnl
sjqzdsjo tsadedef de a i``} fsa}l$%
1==
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 101/130
Sjoznjo HOJ Kjkj| K`oji
^denj tlj Rd~zai Kjkj| K`oji aeo tlj N`IA+de~}dsjo k`oji kahj odbbjsjet a~~zk}td`e~
vdtl sj~}jnt t` `cgjnt~ ,natjf`s+ca~jo r~% j|jk}ias+ca~jo$ aeo n`etj|t ,}asakjtsdn
r~% e`e}asakjtsdn$) vj bjji dt v`zio ai~` cj z~jbzi t` j|akdej a lcsdo k`oji) ozccjo
tlj Sjoznjo HOJ Kjkj| K`oji) vldnl z~j~ a e`e}asakjtsdn k`oji `b n`etj|t ca~jo
`e Hjseji Oje~dt J~tdkatd e,HOJ$ czt `}jsatj~ `e `cgjnt natjf`sdj~% _lj Sjoznjo
HOJ Kjkj| K`oji d~ nsjatjo c n`iia}~def aii j|jk}ias~ cji`efdef t` a ~defij natjf`s
det` bzii+n`eejntjo n`k}`ejet~ vldnl nae cj tl`zflt `b a~ aoodef natjf`sdj~ det` tlj
Rd~zai Kjkj| fsa}l% Dojetdtdj~ cjtvjje deodrdozai j|jk}ias~ asj i`~t) aeo tlz~ vji`~j tlj fiej ojtadi~ `b a ~}atdai n`etj|t% C b`skdef natjf`sdj~) vj nae e` i`efjs ~a a
}astdnzias ~}atdai sjiatd`e~ld} d~ cjtvjje a cizj ~doj rdjv `b a nas aeo ae `cidqzj csdnh
czdiodef) vj nae `ei ~a dt d~ a sjiatd`e~ld} cjtvjje a nas aeo a czdiodef% E`v tlat
vj asj ijbt vdtl ae ze`sojsjo caf `b ~}atdai sjiatd`e~ld}~ xb { cjtvjje tv` natjf`sdj~)
vj ejjo a va t` kja~zsj n`k}atdcdidt cjtvjje a ejvi `c~jsrjo b aeo tlj ~t`sjo
sjiatd`e~ld}~%
Vj z~j tlj ~akj b`sk `b tlj N`etj|t Nlaiijefj n`eodtd`eai od~tsdcztd`e a~ de
Jqzatd`e :%>% Vj z~j a Hjseji Oje~dt J~tdkat`s,HOJ$ b`s jrjs }ads `b natjf`sdj~)
aeo tlj }`tjetdai ά nae cj tl`zflt `b a~ a katsd| `b ~znl j~tdkat`s~% _lj z~j `b
e`e}asakjtsdn }`tjetdai~ de fsa}ldnai k`oji~ la~ cjje aisjao j|}i`sjo de tlj o`kade
`b tj|tzsj aeai~d~ X]afjt aeo I`ef~tabb) 144<[% οdg d~ tlj Hs`ejnhjs ojita bzentd`e%
i`f ά,td) t g) b dg$ ; ,z)r$∌J N οdzοgrH ,b dg) b zr$,z)r$∌J N
οdzοgr,:%2$
_lj Sjoznjo Kjkj| k`oji) cjdef natjf`s+ca~jo aeo e`e}asakjtsdn) affsjfatj~ tlj
~}atdai sjiatd`e~ld}~ ans`~~ kae odbbjsjet }ads~ `b j|jk}ias~ bs`k tv` natjf`sdj~% Vldij
vj z~jo a fi|jo hjseji H vldnl kja~zsj~ od~taenj d~`ts`}dnaii ans`~~ tlj odkje~d`e~
1=1
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 102/130
Rd~zai Kjkj|
} j s ~ e
n a s t s j j
v d e o v
l j a o c z d i o d e f
~ h v a i i
s a o ~ d o j v a i h
~ d f e n l a d s
o s k ` z e t a d e
t a c i j b i s
~ t s j j t i d f l t
i a k } } i a e t
} i j c a i n e
v l j j i
t j | t f s a ~ ~
n i z k e
} a e j t s a ~ l
c i d e o f s z e o
a s k
}js~`e
nastsjj
vdeo`vljao
czdiodef~hvaiis`ao
~dojvaih~dfe
nladso``s
k`zetadetacij
bi``s~tsjjtidflt
iak}}iaet}`ij
cain`evljji
tj|tfsa~~
n`izke}aejtsa~lcideo
fs`zeoask
HOJ
} j s ~ e
n a s t s j j
v d e o ` v
l j a o c z d i o d e f
~ h v a i i
s a o ~ d o j v a i h
~ d f e n l a d s
o ` s k ` z e t a d e
t a c i j b i s
~ t s j j t i d f l t
i a k } } i a e t
} i j c a i n e
v l j j i
t j | t f s a ~ ~
n i z k e
} a e j t s a ~ l
c i d e o f s z e o
a s k
}js~`enas
tsjjvdeo`v
ljaoczdiodef
~hvaiis`ao
~dojvaih~dfenladso``s
k`zetadetacijbi``s
~tsjjtidfltiak}}iaet}`ij
cain`evljjitj|t
fsa~~n`izke
}aejtsa~lcideo
fs`zeo
ask
N`IA
} j s ~ e
n a s t s j j
v d e o v
l j a o c z d i o d e f
~ h v a i i
s a o ~ d o j v a i h
~ d f e n l a d s
o s k ` z e t a d e
t a c i j b i s
~ t s j j t i d f l t
i a k } } i a e t
} i j c a i n e
v l j j i
t j | t f s a ~ ~
n i z k e
} a e j t s a ~ l
c i d e o f s z e o
a s k
}js~`enastsjj
vdeo`v
ljaoczdiodef
~hvaiis`ao
~dojvaih~dfenladso``s
k`zetadetacijbi``s
~tsjjtidfltiak}
}iaet}`ij
cain`evljjitj|t
fsa~~n`izke
}aejtsa~lcideo
fs`zeoask
= =%1 =%3 =%> =%? =%: =%2 =%6 =%< =%4 1=
=%1
=%3
=%>
=%?
=%:
=%2
=%6
=%<
=%4
1
Sjnaii
] s j n d ~ d ` e
N`etj|t Nlaiijefj ]sjodntd`e N`ebdojenj
Rd~zaiKjkj|
HOJ
N`IA
=
=%3
=%?
=%2
=%<
1
N`etj|t Nlaiijefj Sjn`fedtd`e Annzsan b`s >= Natjf`sdj~
} j s ~ e
n a s t s j j
v d e o v
l j a o c z d i o d e f
~ h v a i i
s a o ~ d o j v a i h
~ d f e n l a d s
o s k z e t a d e
t a c i j b i s
~ t s j j t i d f l t
i a k } } i a e t
} i j c a i n e
v l j j i
t j | t f s a ~ ~
n i z k e
} a e j t s a ~ l
c i d e o f s z e o
a s k
Rd~zai Kjkj|
HOJ
N`IA
Bdfzsj :%>7 N`etj|t Nlaiijefj Jraizatd`e a%$ N`etj|t Nlaiijefj n`ebz~d`e katsdnj~ b`s tlj > kjtl`o~7 Rd~zai Kjkj|) HOJ) aeo N`IA% c%$ Sjn`fedtd`e ]sjnd~d`e rjs~z~ Sjnaii vlje tlsj~l`iodef `zt}zt ca~jo
`e n`efiojenj% n$ ^doj c ~doj n`k}asd~`e `b tlj > kjtl`o~— annzsandj~ b`s >= natjf`sdj~%
`b b ) tlj aoraetafj `b ~znl a e`e}asakjtsdn a}}s`anl d~ tlat vdtl je`zfl oata tlj
}astdnziasdtdj~ `b H o` e`t kattjs% Vj ai~` j|}jsdkjetjo vdtl a Ejasj~t Ejdflc`s ca~jo
k`oji) czt b`zeo tlj Hjseji Oje~dt J~tdkatd`e a}}s`anl t` cj ~z}jsd`s%
:%? N`etj|t Nlaiijefj Jraizatd`e7 Rd~zai Kjkj| rjs~z~
Natjf`sdj~
B`s tlj N`etj|t Nlaiijefj jraizatd`e) vj z~j 3== saeo`ki ~jijntjo oje~ji iacjijo
dkafj~ bs`k IacjiKj XSz~~jii jt ai%) 3==<[% @zs tj~t~jt n`etade~ >=?< t`tai `cgjnt~ bs`k
>= odbbjsjet natjf`sdj~% B`s ae dkafj vdtl H `cgjnt~) vj ~`irj H N`etj|t Nlaiijefj
}s`cijk~ vdtl `ej ldooje `cgjnt aeo H +1 ~z}}`stdef `cgjnt~% Qzaidtatdrj sj~zit~ `e tld~
}sjodntd`e ta~h nae cj ~jje de Bdfzsj :%?%
Vj jraizatj tlj }jsb`skaenj `b `zs Rd~zai Kjkj| k`oji) tlj Sjoznjo Kjkj|
HOJ k`oji) aeo tlj N`IA+de~}dsjo k`oji vdtl sj~}jnt t` natjf`sdwatd`e }jsb`skaenj
,n`ebz~d`e katsdnj~ nae cj ~jje de t`} ijbt `b Bdfzsj :%>$% _lj `rjsaii sjn`fedtd`e
1=3
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 103/130
tacij
ffi``s
vaii
o``s
ffi``s
tacij
o``s
vaii
~dojvaih
nas
s`ao
}js~`e
Rd~zai Kjkj| J|jk}ias ]sjodntd`e~De}zt Dkafj ( Ldooje Sjfd`e Natjf`sdwatd`e Sj~zit~
R d ~ z a i
K j k
j |
H O J
N I A
R d ~ z a i
K j k
j |
H O J
N I A
R d ~ z a i
K j k
j |
H O J
N I A
R d ~ z a i
K j k j |
H O J
N I A
R d ~ z a i
K j k
j |
H O J
N I A
R d ~ z a i
K j k
j |
H O J
N I A
R d ~ z a i
K j k j |
H O J
N I A
Bdfzsj :%?7 Qzaidtatdrj Sj~zit~ `e tlj N`etj|t Nlaiijefj% J|jk}ias }sjodntd e~ asj bs`k tlj Rd~zaiKjkj| k`oji aeo natjf`sdwatd`e sj~zit~ asj bs`k tlj Rd~zai Kjkj| k`oji) tlj HOJ K`oji) aeoN`IA XFaiijfzdii`~ jt ai%) 3==<[%
1=>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 104/130
annzsan `b tlj Rd~zai Kjkj| K`oji) Sjoznjo Kjkj| K`oji) aeo N`IA asj %:36)
%?>=) aeo %?:6 sj~}jntdrji% E`tj tlat tlj Rd~zai Kjkj| K`oji }jsb`sk~ ~dfedfinaeti
cjttjs tlae tlj ca~jidej~% _ahdef a ni`~js i``h at tlj }js+natjf`s annzsandj~ `b tlj
tlsjj kjtl`o~ ,~jj c`tt`k `b Bdfzsj :%>$) vj ~jj tlat tlj N`IA+ca~jo kjtl`o badi~ `e
kae natjf`sdj~% _lj arjsafj }js+natjf`s sjn`fedtd`e annzsandj~ `b tlj tlsjj kjtl`o~
asj7 %:>?) %?:?) aeo %31>% _lj Rd~zai Kjkj| K`oji ~tdii }jsb sk~ tlj cj~t) czt vj
~jj a ~dfedfinaet os`} de }jsb`skaenj b`s tlj natjf`s+ca~jo N`IA kjtl`o% N`IA d~
cda~jo t`vaso~ tlj }`}zias natjf`sdj~) sjtzsedef tlj k`~t bsjqzjeti `nnzssdef natjf`s
,vdeo`v$ qzdtj `btje% @rjsaii) tlj Rd~zai Kjkj| K`oji anldjrj~ tlj cj~t }jsb`skaenj
b`s 31 `zt `b tlj >= natjf`sdj~%
De aoodtd`e) vj }i`t }sjnd~d`e sjnaii nzsrj~ b`s janl `b tlj tlsjj kjtl`o~ t` ojtjskdej
db ldfl n`efiojenj sjtzsejo c janl k`oji d~ n`ssjiatjo vdtl ldfl sjn`fedtd`e satj~ ,t`}
sdflt `b Bdfzsj :%>$% _lj Rd~zai Kjkj| k`oji la~ tlj k`~t ~dfedfinaet ldfl+}sjnd~d`e
i`v+sjnaii sjfdkj) ~zffj~tdef tlat dt~ n`efiojenj d~ a f``o kja~zsj `b ~znnj~~% _lj
sjiatdrji ffiat nzsrj b`s tlj N`IA kjtl`o d~ sjiatjo t` tlj }s`cijk `b `rjsn`k}je~atd`e
b`s }`}zias nia~~j~ a~ kjetd`ejo ac`rj% _lj od~tsdcztd`e~ sjtzsejo c N`IA tjeo t`
ojfjejsatj t` a ~defij e`e+wjs` raizj ,k`~t `btje `e `ej `b tlj }`}zias natjf`sdj~
~znl a~ vdeo`v$% _ld~ d~ vl tlj ka|dkzk }s`cacdidt sjtzsejo c N`IA d~ e`t a f``o
kja~zsj `b n`efiojenj%
Vj ai~` ojk`e~tsatj tlj }`vjs `b tlj Rd~zai Kjkj| t` }sjodnt a}}jasaenj ~`iji
ca~jo `e n`etj|tzai detjsantd`e~ vdtl `tljs `cgjnt~ aeo tljds rd~zai a}}jasaenj% _lj
kdooij s`v `b Bdfzsj :%? ojk`e~tsatj~ ~`kj `b tlj~j a~~`ndatd`e~% E`tj l`v de s`v 1)
a }iaz~dcij rdjv}`det d~ ~jijntjo satljs tlae gz~t a saeo`k nas% De s`v > vj ~jj tlat tlj
a}}jasaenj `b ~e`v `e `ej k`zetade ~zffj~t~ tlat tlj `tljs }`std`e `b tlj dkafj ai~`
n`etade~ a ~e`v k`zetade% De ~zkkas) vj }sj~jetjo a natjf`s+bsjj Rd~zai Kjkj|
K`oji aeo a}}idjo dt t` tlj ta~h `b n`etj|tzai `cgjnt sjn`fedtd`e vdtlde tlj j|}jsdkjetai
1=?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 105/130
bsakjv`sh `b tlj N`etj|t Nlaiijefj% @zs j|}jsdkjet~ n`efisk `zs detzdtd`e tlat k`rdef
cj`eo natjf`sdj~ d~ cjejfindai b`s dk}s`rjo k`ojidef `b sjiatd`e~ld}~ cjtvjje `cgjnt~%
:%?%1 B`ii`v+z} V`sh
^denj dt~ dedtdai }zcidnatd`e de 3==4) ~jrjsai detjsj~tdef a}}s`anlj~ larj cjje }zcid~ljo
vldnl asj jdtljs deffizjenjo c `zs Rd~zai Kjkj| v`sh) `s asj rjs sjijraet% ]djts`
]js`ea—~ jbb`st t` `sfaedwj rd~zai he`vijofj de tlj b`sk `b a Rd~d}joda X]js`ea) 3=1=[
d~ k`sj ahde t` a rd~zai jenni`}joda tlae a rd~zai vjc `b n`enj}t~8 l`vjrjs) dt d~
ejrjstljij~~ k`tdratjo c Raeejras Cz~l—~ doja~ sjfasodef tlj kjkj| aeo tlz~ ldfli
sjijraet t` `zs a}}s`anl%
Sjfasodef tlj z~j `b `cgjnt+fsa}l~ de rd~d`e) tlj v`sh `b XBd~ljs aeo Laesalae)
3=1=[ z~j~ a ~dkdias natjf`s+bsjj k`oji czt a}}idj~ dt t` tlj }s`cijk `b >O k`oji
~jasnl vlje fdrje tlj a}}jasaenj~ `b a ~kaii ~jt `b ~z}}`stdef >O `cgjnt~% XIjj aeo
Fsazkae) 3=1=[ larj ~znnj~~bzii z~jo `cgjnt fsa}l~ aeo fsa}l+ca~jo oj~nsd}t`s~ b`s
ze~z}jsrd~jo `cgjnt natjf`s od~n`rjs%
1=:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 106/130
Nla}tjs 2
N`eniz~d`e7 _`vaso tlj Rd~zai Kjkj|
De ld~ od~~jstatd`e) vj ojk`e~tsatjo tlj kae aoraetafj~ `b }`~def tlj sjn`fedtd`e
}s`cijk a~ `cgjnt a~~`ndatd`e a~ `}}`~jo t` tlj t}dnai natjf`s+ca~jo `cgjnt eakdef
}asaodfk% Vj }sj~jetjo ae j|jk}ias+ca~jo `cgjnt sjn`fedtd e bsakjv sh) ~l`vna~jo
tlj }`vjs `b }js+j|jk}ias ~dkdiasdt kja~zsj~ b`s nsjatdef ldfl+qzaidt rd~zai a~~`nd+
atd`e~) a~ vjii a~ jraizatjo `zs je~jkcij `b J|jk}ias+^RK a}}s`anl `e tlj }`}zias
]A^NAI R@N `cgjnt ojtjntd`e cjenlkash ta~h% E`t `ei o`j~ `zs kjtl`o }jsb`sk
n`k}jtdtdrji vdtl ~tatj+`b+tlj+ast kjtl`o~) czt dt nae cj a}}idjo t` kjta+oata tsae~bjs
ta~h~ vldnl asj laso t` tanhij vdtl natjf`s+ca~jo k`oji~% Cji`v vj `ztidej ~jrjsai
hj aoraetafj~ `b `zs bsakjv`sh `rjs natjf`s+ca~jo a}}s`anlj~7
1% N`enj}tzai ~dk}idndt7 De~tjao `b ijasedef `ej n`k}ij| natjf`s+~}jndfin k`oji)
vj ijase a ~j}asatj ~dk}ij tjk}iatj+ca~jo ojtjnt`s b`s janl j|jk}ias de `zs
oata~jt% Janl `b `zs ijasedef }s`cijk~ der`irj~ a ~defij }`~dtdrj de~taenj aeo d~
n`erj|) tlz~ aii`vdef z~ t` z~j jbfindjet `}tdkdwatd`e idcsasdj~ ozsdef ijasedef%
3% Kjta+oata tsae~bjs7 @zs ojtjntd`e~ }s`oznj ldfl qzaidt aidfekjet~ vldnl aii`v
z~ t` tsae~bjs ae kjta+oata a~~`ndatjo vdtl tlj j|jk}ias odsjnti `et` tlj
ojtjntd`e vdeo`v) vdtl`zt ae aoodtd`eai dtjsatdrj aidfekjet ~tj}% _ld~ ~l`v~
1=2
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 107/130
tlat `enj vj na~t tlj sjn`fedtd`e }s`cijk de tjsk~ `b a~~`ndatd`e) vj asj acij t`
}s`oznj kznl k`sj tlae gz~t a natjf s+iacjijo c`zeodef c`|% ]s`cijk~ ~znl a~
~jfkjetatd`e aeo fj`kjts tsae~bjs nae cj ja~di laeoijo c `zs j|jk}ias+ca~jo
a}}s`anl%
>% ]asaiijidwacdidt/Ja~di j|tjeoacij7 Cjnaz~j `zs j|jk}ias~ asj tsadejo deoj+
}jeojeti) dt d~ }`~~dcij t` od~tsdcztj ijasedef ans`~~ a niz~tjs aeo tlje ~dk}i
n`enatjeatj tlj sj~zitdef J|jk}ias+^RK~ t` nsjatj `ej iasfj je~jkcij% Cjnaz~j
`b tlj kdedkai detjsantd`e ans`~~ j|jk}ias~) vj nae ’degjnt he`vijofj‛ det` `zs
je~jkcij `b J|jk}ias+^RK~ c azfkjetdef dt vdtl aoodtd`eai j|jk}ias~) vdtl`zt
lardef t` sj+tsade jrjs j|jk}ias%
?% Dkafj/`cgjnt katnldef7 Vj larj ~l`ve `zs bsakjv`sh nae cj z~jo b`s katnldef
`cgjnt~ a~ vjii a~ ns`~~+o`kade dkafj katnldef) ~zffj~tdef tlat `tljs a}}idnatd`e~)
sjqzdsdef tljds `ve e`td`e `b rd~zai ~dkdiasdt) n`zio cjejfit bs`k `zs j|jk}ias+
ca~jo a}}s`anl%
Fdrje tlj }`vjs `b j|jk}ias+ca~jo rd~zai a~~`ndatd`e~) n`z}ijo vdtl c`tl densja~jo
n`k}ztdef ~}jjo~ aeo dk}s`rjo annj~~ t` iasfj+~naij n`k}ztatd`eai sj~`zsnj~) dt d~
}jsla}~ tlj sdflt tdkj b`s rd~d`e sj~jasnljs~ t` ~jsd`z~i n`e~dojs Raeejras Cz~l—~ doja~
sjfasodef a kjk`s+ca~jo k`oji `b tl`zflt7 ’Dt `}jsatj~ c a~~`ndatd`e% Vdtl `ej dtjk
de dt~ fsa~}) dt ~ea}~ de~taeti t` tlj ej|t tlat d~ ~zffj~tjo c tlj a~~`ndatd`e `b tl`zflt~)
de ann`soaenj vdtl ~`kj detsdnatj vjc `b tsadi~ nassdjo c tlj njii~ `b tlj csade%‛ XCz~l)
14?:[ @zs v`sh `e tlj j|jk}ias+ca~jo Rd~zai Kjkj| K`oji ~zffj~t~ tlat tljsj d~
kjsdt t` tlj doja `b ae detsdnatj vjc `b rd~zai tsadi~% A~ vj larj ~l`ve de Nla}tjs :)
dt d~ }`~~dcij t` nsjatj ~znl a rd~zai ejtv`sh) vdtl j|jk}ias~ a~ e`oj~ aeo j|jk}ias
sjiatd`e~ld}~ a~ jofj~% C a}}idef `zs Rd~zai Kjkj| K`oji t` a n`etj|tzai `cgjnt
}sjodntd`e ta~h) vj vjsj acij t` }sjodnt tlj a}}jasaenj `b a ldooje `cgjnt ~`iji ca~jo
1=6
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 108/130
`e dt~ ~}atdai sjiatd`e~ld}~ t` a ~jt `b `c~jsrjo `cgjnt~ de tlj dkafj% _lz~ jkcsandef
a kjkj|+ca~jo va `b tldehdef ac`zt tlj rd~zai v`sio jetadi~ tldehdef ac`zt rd~zai
he`vijofj a~ l}js+idehjo rd~zai he`vijofj satljs tlae a~ natjf`s+ca~jo he`vijofj%
^znl a natjf`s+bsjj rdjv `b tlj v`sio la~ tlj nlaenj t` nlaefj tlj va vj aoosj~~
tlj }s`cijk `b dkafj zeojs~taeodef) kznl idhj tlj va F``fij aeo Vdhd}joda nlaefjo
tlj va vj tldeh ac`zt he`vijofj%
Bdeaii) tljsj d~ fs`vdef jrdojenj tlat tlj doja~ sjfasodef rd~zai a~~`ndatd`e~ asj ljsj
t` ~ta% Fdrje tlj s`ij vldnl aeai`fdj~) a~~`ndatd`e~ aeo }sjodntd`e~ }ia de n`fedtdrj
ejzs`~ndjenj XCas) 3==4[) tlj sjnjet ~ndjetdfin fieodef~ vldnl ~zffj~t tlat lzkae~—
rd~zai kjk`s d~ iasfjs tlae }sjrd`z~i tl`zflt XCsao jt ai%) 3==4[) a~ vjii a~ tlj
~znnj~~ `b he`vijofj f`def odfdtai ,j%f%) Vdhd}joda) Banjc``h) F``fij$) vj bjji tlat
tlj tdkj d~ sdflt t` ~tast jkcsandef tlj }`vjs `b rd~zai a~~`ndatd`e~% Vj l`}j tlat
`zs tlj~d~ kahj~ a ~ts`ef na~j b`s j|jk}ias+ca~jo sj}sj~jetatd`e~ aeo tlj ~tsjeftl `b
}js+j|jk}ias ~dkdiasdt bzentd`e~ ‟ vj l`}j tlat tlj ~znnj~~ `b `zs J|jk}ias+^RK
bsakjv`sh aeo Rd~zai Kjkj| K`oji vdii k`tdratj tlj ej|t fjejsatd`e `b sj~jasnljs~ t`
qzj~td`e tlj ejnj~~dt `b sjidef `e `cgjnt natjf`sdj~ b`s `cgjnt/~njej detjs}sjtatd e%
1=<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 109/130
A}}jeod| A
Ns`~~+O`kade Dkafj Katnldef
Bdfzsj A%17 Ns`~~+O`kade Dkafj Katnldef7 Vj asj detjsj~tjo de ojfiedef rd~zai ~dkdiasdt cjtvjjedkafj~ ans`~~ odbbjsjet o`kade~) ~znl a~ }l`t`~ tahje `rjs odbbjsjet ~ja~`e~ aeo idfltdef) }adetdef~)
~hjtnlj~) jtn% Vlat kahj~ tld~ nlaiijefdef d~ tlat tlj rd~zai n`etjet d~ `ei ~dkdias `e tlj ldfljs ~njej
ijrji) czt qzdtj od~~dkdias `e tlj }d|ji ijrji%
De tld~ tlj~d~) vj ~zkkasdwj tlj sj~zit~ `b ae a}}idnatd`e `b tlj J|jk}ias+^RK
bsakjv`sh t` tlj }s`cijk `b ns`~~+o`kade katnldef X^lsdra~tara jt ai%) 3=11[% Vj
csdjffi od~nz~~ tlj }s`cijk aeo `ei ~l`v a ~zc~jt `b `zs sj~zit~ b`s ~jrjsai odbbjsjet
ns`~~+o`kade a}}idnatd`e~%
_lj njetsai jijkjet n`kk`e t` kae dkafj+katnldef a}}idnatd`e~ de n`k}ztjs
fsa}ldn~ d~ ~jasnldef a iasfj oata~jt t` fieo rd~zaii+~dkdias katnlj~ t` a fdrje qzjs ‟
cj dt ae dkafj }atnl) a bzii dkafj) `s a ~}atd`+tjk}`sai ci`nh% L`vjrjs) ojfiedef a f``o
rd~zai ~dkdiasdt kjtsdn t` z~j b`s katnldef nae `btje cj ~zs}sd~defi odbfinzit% Fsaetjo)
de kae ~dtzatd`e~ vljsj tlj oata d~ sja~`eaci l`k`fjej`z~ ,j%f%) odbbjsjet }atnlj~
vdtlde tlj ~akj tj|tzsj dkafj XJbs`~ aeo Bsjjkae) 3==1[) `s odbbjsjet bsakj~ vdtlde
1=4
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 110/130
tlj ~akj rdoj` X^nl`oi jt ai%) 3===[$) a ~dk}ij }d|ji+vd~j) ~zk+`b+~qzasjo+odbbjsjenj~
katnldef v`sh~ qzdtj vjii% Czt vlat ac`zt tlj na~j~ vlje tlj rd~zai n`etjet d~ `ei
~dkdias `e tlj ldfljs ~njej ijrji) czt qzdtj od~~dkdias `e tlj }d|ji ijrji) ~znl a~ de
Bdfzsj A%1% B`s de~taenj) kjtl`o~ tlat z~j ~njej katnldef ,j%f%) XLa~ aeo Jbs`~)
3==6) Oaij jt ai%) 3==4[$ `btje ejjo t` katnl dkafj~ ans`~~ odbbjsjet diizkdeatd`e~)
odbbjsjet ~ja~`e~) odbbjsjet nakjsa~) jtn% Idhjvd~j) sj+tj|tzsdef ae dkafj de tlj ~tij `b
a }adetdef XLjstwkaee jt ai%) 3==1) Jbs`~ aeo Bsjjkae) 3==1[ sjqzdsj~ kahdef rd~zai
n`ssj~}`eojenj cjtvjje tv` rjs odbbjsjet o`kade~ ‟ }l`t`~ aeo }adetdef~% _lj ns`~~+
o`kade katnldef d~ jrje k`sj nsdtdnai b`s a}}idnatd`e~ ~znl a~ ^hjtnl3]l`t` XNlje
jt ai%) 3==4[ aeo NF3Sjai XG`le~`e jt ai%) 3=1=[) vldnl adk t` csdef o`kade~ a~
odbbjsjet a~ ~hjtnlj~ aeo NF sjeojsdef~ det` n`ssj~}`eojenj vdtl eatzsai }l`t`fsa}l~%
De aii `b tlj~j na~j~) }d|ji+vd~j katnldef basj~ j|tsjkji }``si) cjnaz~j }jsnj}tzaii
~kaii odbbjsjenj~ nae sj~zit de ascdtsas iasfj }d|ji+vd~j odbbjsjenj~% Vlat d~ ejjojo d~ a
rd~zai kjtsdn tlat nae ~l`v ~`kj s`cz~tej~~ t` ~kaii) zedk}`staet rd~zai odbbjsjenj~)
jt ~tdii na}tzsj tlj dk}`staet rd~zai ~tszntzsj~ tlat kahj tv` dkafj~ a}}jas ~dkdias%
_ld~ d~ }sjnd~ji vlat kahj~ tld~ }s`cijk ~` odbfinzit ‟ tlj rd~zai ~dkdiasdt aif`sdtlk
~`kjl`v ejjo~ t` he`v vldnl rd~zai ~tszntzsj~ asj dk}`staet b`s a lzkae `c~jsrjs
aeo vldnl asj e`t%
Vj ~l`v tlat a e`rji rjs~d`e `b tlj J|jk}ias+^RK aif`sdtlk }s`rdoj~ a rjs ~dk}ij)
jt ~zs}sd~defi jbbjntdrj a}}s`anl t` rd~zai katnldef vldnl d~ }astdnziasi vjii+~zdtjo
b`s katnldef dkafj~ ans`~~ odbbjsjet o`kade~% Fdrje ae dkafj sj}sj~jetjo c ~`kj
bjatzsj~ tlj adk d~ t` b`nz~ tlj katnldef `e tlj bjatzsj~ tlat asj tlj k`~t rd~zaii
dk}`staet b`s tld~ }astdnzias dkafj% _lj njetsai doja d~ z~j tlj e`td`e `b ’oata+osdrje
zedqzjej~~‛% Vj l}`tlj~dwj tlat tlj dk}`staet }ast~ `b tlj dkafj~ asj tl`~j tlat
asj k`sj zedqzj `s sasj vdtlde tlj rd~zai v`sio ,sj}sj~jetjo ljsj c a iasfj oata~jt$%
E`tj tlat ~denj tlj ~akj i`nai bjatzsj~ n`zio sj}sj~jet rjs odbbjsjet rd~zai n`etjet
11=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 111/130
Bdfzsj A%37 ^hjtnl aeo ]adetdef Katnldef @zs a}}s`anl v`sh~ vjii ans`~~ odbbjsjet rd~zai o`kade~denizodef ~hjtnlj~ aeo }adetdef~
oj}jeodef `b n`etj|t) `zs e`td`e `b zedqzjej~~ la~ t` cj ~njej+oj}jeojet) d%j%) janl
qzjs ~njej ojndoj~ vlat—~ tlj cj~t va t` vjdflt dt~ n`e~tdtzjet }ast~% C b`nz~def
`e tlj fi`caii ~aidjet }ast~ `b tlj dkafj) tlj a}}s`anl nae cj ~znnj~~bzii z~jo b`s
fjejsdn ns`~~+o`kade katnldef) vdtl`zt kahdef ae o`kade+~}jndfin nlaefj~) a~ ~l`ve
`e Bdfzsj A%3%
Vj `}jsatd`eaidwj tld~ oata+osdrje zedqzjej~~ c z~def tlj J|jk}ias+^RK bsakj+
v`sh) czt de a }zsji natjf`s+bsjj kaeejs% B`s a qzjs dkafj) vj tsade ae J|jk}ias+
^RK t` od~n`rjs vldnl }ast~ `b ae dkafj asj k`~t od~nsdkdeatdrj de sjiatd`e~ld} t` tlj
sj~t `b tlj oata~jt% _lz~ de tld~ ~njeasd`) tlj J|jk}ias+^RK—~ ejfatdrj ~jt o`j~ e`t z~j
ae nia~~+deb`skatd`e ,vldnl va~ sjqzdsjo de XKaid~djvdnw jt ai%) 3=11[$%
Bdfzsj A%> ~l`v ~`kj j|ak}ij ~hjtnl qzjsdj~ aeo tlj n`ssj~}`eodef t`} 2 sjtsdjrai
sj~zit~ b`s `zs a}}s`anl aeo tlj ca~jidej~ ‟ dt nae cj ~jje tlat `zs a}}s`anl e`t `ei
`zt}jsb`sk~ aii `b tlj ca~jidej~) czt sjtzse~ dkafj~ ~l`vdef tlj tasfjt `cgjnt de a rjs
~dkdias }`~j aeo rdjv}`det a~ tlj qzjs ~hjtnl% Qzaidtatdrj j|ak}ij~ `b }adetdef+t`+
dkafj katnldef nae cj ~jje de Bdfzsj A%?% Qzaidtatdrj }adetdef3F]^ j|ak}ij~ nae cj
~jje de Bdfzsj A%:%
111
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 112/130
Bdfzsj A%>7 ^hjtnl3Dkafj Qzaidtatdrj J|ak}ij% @zs a}}s`anl z~def tlj J|jk}ias+^RK aif`sdtlk d~n`k}asjo t` ~jrjsai }`}zias tjnledqzj~ b`s katnldef dkafj~%
Bdfzsj A%?7 ]adetdef3Dkafj Qzaidtatdrj J|ak}ij% @zs a}}s`anl z~def tlj J|jk}ias+^RK aif`sdtlk d~n`k}asjo t` ~jrjsai }`}zias tjnledqzj~ b`s katnldef dkafj~%
Bdfzsj A%:7 ]adetdef3F]^ Qzaidtatdrj J|ak}ij~% De tlj~j tv` j|ak}ij~) vj ~l`v tlj de}zt }adetdef a~
vjii a~ tlj t`} > katnlj~ z~def FD^_ aeo `zs a}}s`anl% Vj od~}ia F]^ i`naidwatd`e sj~zit~ a~ a oje~dt ka} `rjsiado `e tlj fi`cj%
113
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 113/130
A}}jeod| C
J|jk}ias+^RK~7 Ae @}je ^`zsnj
Dk}ijkjetatd`e
Vj csdjffi oj~nsdcj tlj dk}ijkjetatd`e `b tlj J|jk}ias+^RK aif`sdtlk z~jo t` tsade
tl`z~aeo~ `b j|jk}ias~ ans`~~ a niz~tjs `b 3==( kanldej~) de a Ka}Sjoznj+idhj ba~ld`e%
Vldij vj ijase v—~ b`s a iasfj ezkcjs `b j|jk}ias~) janl j|jk}ias—~ ijasedef }s`cijk
aeo naidcsatd`e nae cj ~`irjo deoj}jeojeti aii`vdef b`s ja~ }asaiijidwatd`e% _lj Katiac
~`zsnj n`oj `b tlj J|jk}ias+^RK bsakjv`sh ,Nla}tjs > aeo Nla}tjs ?$ d~ aradiacij
a~ a }zcidn) `}je+~`zsnj }s`gjnt `e FdtLzc% FdtLzc d~ a n`oj ~lasdef vjc~dtj vdtl
kae k`ojse ~`ndai+ejtv`shdef bjatzsj~ czdit+de% _lj }ldi`~`}l cjldeo l`~tdef `}je+
~`zsnj }s`gjnt~ `e FdtLzc d~ tlat dt kahj~ n`iiac`satd`e aeo detjsantd`e kznl ja~djs
tlae z}i`aodef a tascaii `e a vjc~dtj% FdtLzc kahj~ dt ja~ t` z~j a }s`gjnt) tsanh dt~
}s`fsj~~) aeo ai~` n`etsdcztj ae czf+fi|j~ aeo jelaenjkjet~ b`s tlj n`k}ztjs rd~d`e
n`kkzedt t` jeg`%
1% J|jk}ias+^RK ]s`gjnt7 ltt}7//vvv%n~%nkz%joz/utkaid~dj/}s`gjnt~/dnnr11/
3% J|jk}ias+^RK N`oj7 ltt}~7//fdtlzc%n`k/qzaet`kc`ej/j|jk}ias~rk
11>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 114/130
_lj J|jk}ias+^RK ~~tjk sjqzdsj~ annj~~ t` a niz~tjs vdtl a ~lasjo ejtv`sh osdrj% _`
cj acij t` sze `e a rasdjt `b niz~tjs~) tlj ca~dn }sdend}ij vldnl vj jk}i` d~ tlat `b a
~defij }s`nj~~ vldnl n`zio cj sj}idnatjo E tdkj~% _ljsj d~ `ej kade osdrjs }s`fsak)
vldnl vlje i``}def `rjs j|jk}ias~) z~j~ i`nhfiij~ t` ~jj db ae j|jk}ias la~ aisjao cjje
tsadejo% Db ae j|jk}ias la~ e`t cjje tsadejo) tlj nzssjet }s`nj~~ fsac~ tlj j|jk}ias~
aeo vsdtj~ a i`nhfiij t` od~h ‟ ae `tljs }s`nj~~ vdii ~dk}i ~hd} `rjs ae aisjao fied~ljo
j|jk}ias `s a i`nhjo j|jk}ias%
C%1 J|jk}ias+^RK _sadedef ]asakjtjs~
Vj z~j idc~rk XNlaef aeo Ide) 3==1[ t` tsade janl j|jk}ias—~ v% Vj aitjseatj cjtvjje
ijasedef tlj vjdflt~ fdrje ae antdrj ~jt `b ejfatdrj vdeo`v~) aeo kdedef aoodtd`eai
ejfatdrj vdeo`v~ z~def tlj nzssjet v a~ de XBjiwje~wvaic jt ai%) 3=1=[% Vj z~j tlj
~akj sjfziasdwatd`e }asakjtjs N 1 ; =%: aeo N 3 ; %=1 b`s aii j|jk}ias~) czt b`zeo `zs
vjdflt rjnt`s~ t` cj s`cz~t t` a vdoj saefj `b N ~) j~}jndaii ~denj tlj asj sj+~naijo
ozsdef naidcsatd`e% Vj z~j 3:== a~ tlj ka|dkzk ezkcjs `b ejfatdrj~ t` kdej) vldnl
n`ssj~}`eo~ t` a ~defij }a~~ `rjs tlj ejfatdrj~% Vj n`e~dojs dkafj~ a~ vjii a~ tljds
ijbt+sdflt ffid}}jo n`zetjs}ast~ b`s c`tl tsadedef aeo tj~tdef%
C%3 ^idodef Vdeo`v~ aeo Bjatzsj ]sakdo~
@ej `b tlj cdffj~t osavcanh~ `b ~jfkjetatd`e+ca~jo ’~jfkjet+tlje+sjn`fedwj‛ tjnl+
edqzj~ d~ tlat tlj sjqzdsj ~dfedfinaet n`k}ztatd`eai sj~`zsnj~ t` fjejsatj naeodoatj
~jfkjet~ ‟ `enj a ~jfkjetatd`e d~ n`k}ztjo b`s ae dkafj aeo bjatzsj n`k}ztatd`e
la~ cjje a}}idjo t` tlj sj~zitdef ~jfkjet~) dt d~ t}dnaii vd~j t` ~t`sj tlj sj~zit~
`e laso+od~h t` ar`do zeejnj~~asdi szeedef tlj ~jfkjetatd`e aif`sdtlk a ~jn`eo
11?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 115/130
tdkj% ^jfkjetatd`e+ca~jo a}}s`anl }sjnizoj njstade a}}idnatd`e~ ~znl a~ sjai+tdkj /
detjsantdrj+tdkj a}}idnatd`e~ ,ozj t` ~}jjo `b n`k}ztdef a ~jfkjetatd`e$) a~ vjii a~
a}}idef tljk t` oata~jt~ `b kdiid`e~ `b dkafj~ ,ozj t` ~t`safj ~}anj sjqzdsjkjet~$%
_lj ~jn`eo ~l`stn`kdef `b z~def dkafj ~jfkjetatd`e b`s `cgjnt sjn`fedtd`e d~ tlat dt
dets`oznj~ a ~jn`eo }ianj vljsj `cgjnt sjn`fedtd`e n`zio badi% C badidef t` fs`z} }d|ji~
det` sja~`eacij nlzeh~ vdtl ~zbfindjet ~}atdai ~z}}`st b`s sjn`fedtd`e) ~jfkjetatd`e+
ca~jo a}}s`anlj~ nae ~dk}i badi at }s`ozndef ae sja~`eacij ~jfkjet~ b`s ~`kj
odbfinzit+t +~jfkjet `cgjnt~% Vlat tld~ kjae~ d~ tlat jrje db vj vjsj acij t` `ctade
dojai }js+j|jk}ias ~dkdiasdt kja~zsj~) vj kdflt ~tdii e`t cj acij t` sjn`fedwj jrjs
~defij `cgjnt oj}dntjo de a ~njej% _ld~ d~ a ~jrjsj idkdtatd`e) j~}jndaii cjnaz~j a}}idef
}js+j|jk}ias ~dkdiasdt kja~zsj~ ,vldnl vj larj aisjao ~}jet a n`e~dojsacij tdkj
tsadedef$ d~ ~dfedfinaeti ba~tjs tlae fjejsatdef tlj antzai ~jfkjet~% Vlat tld~ ~zffj~t~
d~ tlat vj ~l`zio fjejsatj k`sj ~jfkjet~ de ij~~ tdkj ‟ db tlj }js+j|jk}ias ~dkdiasdt
bzentd`e~ asj f``o je`zfl) tlje tlj n`zio ojndoj vldnl ~jfkjet~ t` hjj}%
A~ vdii cj ~jje de tld~ A}}jeod|) L@F+ca~jo ~idodef vdeo`v a}}s`anlj~ `bbjs a
~dfedfinaet ~}jjoz} `rjs tljds ~jfkjetatd`e+ca~jo n`zetjs}ast~% _lj ~}jjoz} d~ ~`~dfedfinaet tlat vj nae }s`nj~~ kae k`sj sjfd`e~ }js dkafj) tlz~ aiijrdatdef `zs
v`ss tlat ~jfkjetatd`e kdflt n`k}ijtji kd~~ ~`kj `cgjnt~% Vj fis~t `ztidej tlj
kzitd~naij L@F bjatzsj+}sakdo) tlje }sj~jet tlj tv` odbbjsjet va~ de vldnl vj nae
}jsb`sk i`naidwatd`e vdtlde tlj }sakdo% _lj ciawdef+ba~t ojtjntd`e ~tsatjf ca~jo `e
n`er`iztd`e sjqzdsj~ rjs idttij kjk`s `rjsljao aeo d~ ~zdtacij b`s ojaidef vdtl a ~kaii
ezkcjs `b j|jk}ias~ ,j%f%) tlj kdedef ~tafj de J|jk}ias+^RK ijasedef$% Zeb`stzeatji)
z~def ~idodef+vdeo`v n`er`iztd`e b`s a iasfj ezkcjs `b j|jk}ias~ nae ~tdii cj ~i`v% @zs
n`etsdcztd`e de tld~ nla}tjs d~ tlj ci`nh+katnldef kjtl`o `b ojtjntd`e vldnl sjqzdsj~ a
~defij katsd|+kzitd}idnatd`e aeo d~ kznl ba~tjs vlje i`naidwdef tl`z~aeo~ `b j|jk}ias~
de~doj a ~defij dkafj ,j%f%) a}}idef ae je~jkcij `b J|jk}ias+^RK~ ozsdef tj~tdef$%
11:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 116/130
Kzitd~naij L@F ]sakdo
Ld~t`fsak~ `b @sdjetjo Fsaodjet~ ,L@F$ asj ~`kj `b tlj k`~t vdoji z~jo oj~nsd}+
t`s~ ozj t` tljds ~}jjo `b n`k}ztatd`e~) deljsjet s`cz~tej~~ t` ~idflt `cgjnt rasda+
td`e~/ojb`skatd`e~) aeo acdidt t` na}tzsj a n`as~j ,jt sdfdo$ ~}atdai ia`zt `b bjatzsj~%
Vldij dedtdaii dets`oznjo b`s }joj~tsdae ojtjntd`e XOaiai aeo _sdff~) 3==:[ z~def a idejas
^RK) L@F d~ ai~` z~jo c tlj ~tatj+`b+tlj+ast ojb`skacij }ast k`oji `b XBjiwje~wvaic
jt ai%) 3=1=[%
Satljs tlae deoj}jeojeti n`k}ztdef L@F oj~nsd}t`s~ b`s odbbjsjet sjfd`e~ de ae
dkafj) L@F d~ k`~t `btje n`k}ztjo de a }sakdo+idhj ba~ld`e% Dkafj }sakdo~ X Aoji~`e
jt ai%) 1441[ asj a nia~~dnai tjnledqzj~ de rd~d`e) aeo asj nsdtdnai de kahdef L@F
n`k}ztatd`e~ ba~t% @ej `b tlj laiikash~ `b z~def bjatzsj }sakdo~ b`s i`naidwatd`e d~ tlat
vj o` e`t sjqzdsj j|}idndti i``}def `rjs sjfd`e~ de tlj dkafj aeo deoj}jeojeti naiidef
a bjatzsj~,sjfd`e$ bzentd`e% ^denj L@F bjatzsj~ asj a n`enatjeatd`e `b ~kaiijs L@F
njii~) tv` ldfli `rjsia}}def naeodoatj sjfd`e~ bs`k tlj ~akj ijrji `b tlj }sakdo
vdii ~lasj kae `b tlj ~akj ci`nh~% De `tljs v`so~) dt ~zbfinj~ t` n`k}ztj tlj L@F
oj~nsd}t`s b`s janl njii de ae dkafj `ei `enj) tlje n`enatjeatdef odbbjsjet njii~ vlje
vj vaet oj~nsd}t`s~ n`k}ztjo b`s odbbjsjet sjfd`e~% Vj z~j 1= ijrji~ }js `ntarj vlje
nsjatdef tlj }sakdo aeo z~j 1%= a~ tlj fiej~t dkafj ~naij%
Sj}sj~jetdef @cgjnt~ a~ L@F _jk}iatj~
Vj z~j tlj L@F oj~nsd}t`s bs`k XBjiwje~wvaic jt ai%) 3=1=[ vldnl z~j~ B ; >1 ezkcjs~
t` sj}sj~jet janl njii) aeo a ci`nh ~dwj `b < }d|ji~ ,~cde ; <$% Vlje dedtdaidwdef
j|jk}ias~) de~tjao `b vas}def janl j|jk}ias t` a nae`ednai bsakj ,a~ d~ ejnj~~as de
k`e`idtldn kjtl`o~ XOaiai aeo _sdff~) 3==:[$) vj ijt janl j|jk}ias ojfiej dt~ `ve
L@F odkje~d`e~ bs`k tlj fs`zeo+tsztl c`zeodef c`|—~ a~}jnt satd`% Z~def a ~defij
112
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 117/130
}`~dtdrj de~taenj kjae~ tlat vj o` e`t ejjo t` v`ss ac`zt tlj aidfekjet `b kzitd}ij
de~taenj~ ,vldnl kz~t cj aoosj~~jo de k`e`idtldn nia~~dfijs~ c ae aidfekjet ~tsatjf
ahde t` tlj iatjet s``t+fiitjs z}oatj ~tj} `b Iatjet ^RK~ XBjiwje~wvaic jt ai%) 3==<[$%
Vlje dt d~ ejnj~~as t` j|tsant bjatzsj~ `rjs a tasfjt sjfd`e) }s`rdojo a~ a c`zeodef
c`|) vj ~dk}i n`k}ztj tlj dkafj+vdoj kzitd~naij L@F }sakdo aeo ~jasnl b`s a ~idnj
`b tlj }sakdo vl`~j ~}atdai j|tjet cj~t katnlj~ tlj tasfjt c`zeodef c`| ,ann`sodef t`
`rjsia} ~n`sj$% ^denj vj n`zio j|tsant kzitd}ij sjntaefzias sjfd`e~ ,bs`k odbbjsjet ijrji~
`b tlj }sakdo$ t` sj}sj~jet tlj j|jk}ias) dt d~ ai~` ejnj~~as t` }s`rdoj aoodtd`eai n`e+
~tsadet~ sjfasodef tlj ~dwj `b tlj oj~dsjo L@F odkje~d`e~% Vj z~j tv` ~dk}ij ljzsd~tdn~
b`s nl``~def l`v t` bsakj ae j|jk}ias7 fieodef a tjk}iatj vldnl n`k}sd~j~ s`zfli
f`ai~dwj njii~) aeo a ka|dkzk njii odkje~d`e) ka|njiiodkje~d`e% De tlj L@F j|}jsd+
kjet~ z~jo tls`zfl`zt tld~ tlj~d~) vj z~j f`ai~dwj;1== aeo ka|njiiodkje~d`e;13%
_ld~ kjae~ tlat vj i``h b`s a }sakdo ~idnj vldnl la~ ldfl `rjsia} vdtl tlj fs`zeo+
tsztl c`zeodef c`|) n`k}sd~j~ s`zfli 1== njii~ aeo la~ a ka|dkzk njii odkje~d`e `b
13% B`s j|ak}ij) db ae j|jk}ias d~ sj}sj~jetjo vdtl a 1= Ö 1= L@F oj~nsd}t`s) tlje tlj
t`tai ezkcjs `b odkje~d`e~ de tlj oj~nsd}t`s vdii cj >1== ,a~~zkdef B ; >1$%
I`naidwdef a ~defij j|jk}ias rda n`er`iztd`e
B`s tlj }zs}`~j `b i`naidwdef a ~defij j|jk}ias) sj}sj~jetjo c a tjk}iatj v) vdtlde tlj
L@F }sakdo) vj a}}i a n`er`iztd`e ans`~~ janl ijrji `b tlj }sakdo deoj}jeojeti%
Fdrje a ijrji `b tlj }sakdo) sj}sj~jetjo a~ a K Ö E Ö B katsd| `b ezkcjs~ ,B cjdef
tlj odkje~d`eaidt `b tlj oj~nsd}t`s$) aeo v cjdef `b ~dwj K Ö E Ö B ) vj }jsb`sk a
n`er`iztd`e vldnl sj~zit~ de a ,K ∓ K $ Ö ,E ∓ E $ katsd| `b ~n`sj~% _ld~ kjtl`o `b
i`naidwatd`e d~ rjs ba~t ~denj dt o`j~ e`t sjqzdsj tlj nsjatd`e `b ae j|}idndt bjatzsj katsd|
‟ tlj `rjsia}}def L@F bjatzsj~ `ei ejjo t` cj ~t`sjo de }sakdo b`sk% Vj z~j tld~
n`er`iztd`e+ca~jo ~tsatjf ozsdef tlj laso ejfatdrj kdedef ~tj} `b tlj J|jk}ias+^RK
116
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 118/130
aif`sdtlk cjnaz~j tsadedef d~ }jsb`skjo deoj}jeojeti b`s janl j|jk}ias%
A}}idef Od~taenj Bzentd`e~ rda _v` N`er`iztd`e~
N`er`iztd`e nae ai~` cj z~jo b`s i`naidwdef j|jk}ias~ sj}sj~jetjo c od~taenj bzentd`e~)
aitl`zfl tlj n`k}ij|dt d~ k`sj tlae tvdnj tlat `b idejas tjk}iatj~ ,cjnaz~j tlj
n`er`iztd`e kz~t cj a}}idjo tvdnj aeo a ~qzasjo L@F }sakdo kz~t cj n`k}ztjo$% A
~qzasjo L@F }sakdo d~ `ctadejo c ~qzasdef aii `b tlj jijkjet~ de tlj `sdfdeai L@F
}sakdo% _` ~jj vldnl tv` n`er`iztd`e~ kz~t cj }jsb`skjo) n`e~dojs tlj odaf`eai
Kalaiae`cd~ od~taenj bzentd`e b`sk ,~akj a~ a~ ojfiejo de Jqzatd`e 3%1$7
Oj,|d$ ; ,|j ∓ |d$_ V j,|j ∓ |d$ ,C%1$
vj ; odaf,V j$ ,C%3$
Vj nae }jsb`sk ~`kj kaed}ziatd`e~ ,vldnl j|}i`dt tlj bant tlat V j d~ odaf`eai$ t`
vsdtj tlj od~taenj bzentd`e a~ b`ii`v~7
Oj,|d$ ; |_ j V j|j ( |_
d V j|d ∓ 3|_ j V j|d ,C%>$
Oj,|d$ ; H ( v_ j |d ( v_
j |d ,C%?$
H ; |_ j V j|j ,C%:$
|_ d V j|d ; v_
j |d ,C%2$
|dXh[ ; ,|dXh[$
3
,C%6$
vj ; ∓3V j|j ,C%<$
A~ nae cj ~jje de Jqzatd`e C%?) tlj fis~t }ast `b tlj od~taenj bzentd`e jraizatd`e) H )
d~ deoj}jeojet `b tlj tj~t vdeo`v |d aeo ejjo~ t` cj n`k}ztjo `ei `enj) aeo nae cj
11<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 119/130
nanljo b`s bztzsj z~j% _lj }ast `b tlj od~taenj bzentd`e jraizatd`e vldnl oj}jeo~ `e
tlj tj~t+vdeo`v—~ bjatzsj~ nae cj cs`hje o`ve det` tv` n`er`iztd`e~7 tlj fis~t z~def
tlj vjdflt rjnt`s vj) czt a}}idjo t` tlj ~qzasjo L@F }sakdo
|d) aeo tlj ~jn`eo z~def
vj aeo a}}idjo t` tlj sav L@F }sakdo%
I`naidwdef kae j|jk}ias~ rda a ~defij katsd| kzitd}idnatd`e
De tlj na~j `b a}}idef tl`z~aeo~ `b j|jk}ias~ t` tlj L@F }sakdo j|tsantjo bs`k a
~defij dkafj) dt d~ cjejfindai t` j|}idndti n`e~tsznt tlj O Ö E bjatzsj+katsd|) vljsj O
d~ tlj t`tai odkje~d`eaidt `b tlj L@F tjk}iatj) aeo E d~ tlj t`tai ezkcjs `b naeodoatj
c`zeodef c`|j~ j|tsantjo bs`k a ~defij dkafj ,ans`~~ aii }sakdo ijrji~$% Vj naii tld~
a}}s`anl tlj ci`nh+katnldef kjtl`o b`s ~idodef+vdeo`v i`naidwatd`e% _lj hj de~dflt
vldnl aii`v~ z~ t` o` tld~ d~ tlj sjaidwatd`e tlat tlj sj~zit `b a}}idef ae K Ö E L@F
tjk}iatj d~ jqzdraijet t` a}}idef ae jeiasfjo) =+}aoojo) ,K (∂K $Ö,E (∂E $ tjk}iatj
vljsj ∂K aeo ∂E deodnatj l`v kznl t` }ao% Cjnaz~j a}}idnatd`e `b a tjk}iatj d~ a
~dk}ij o`t+}s`oznt) aoodef wjs`~ vdii e`t nlaefj tlj sj~zit ,a~ i`ef a~ vj ai~` }ao tlj
L@F }sakdo vdtl =~ t` ann`zet b`s c`zeoas jbbjnt~$% Vj j|}i`dt tlj bant tlat vj z~jo
ka|njiiodkje~d`e; 13 ozsdef j|jk}ias dedtdaidwatd`e ,~jj A}}jeod| C%3$% Aii `b `zs
j|jk}ias~ nae tlje cj =+}aoojo det` a n`kk`e 13 Ö 13 bsakj ,d%j%) O ; ??2?$%
Fdrje a ejv dkafj) vj fis~t j|tsant aii 13 Ö 13 vdeo`v~ bs`k janl ijrji `b dt~
=+}aoojo L@F }sakdo% De }santdnj vj fjt E ≌ 3=) === od~tdent vdeo`v~ vlje z~def
~cde; < aeo z~def dkafj~ vldnl n`k}sd~j a}}s`|dkatji >== Ö ?== }d|ji~% Aii `b `zs
j|jk}ias~ nae e`v cj a}}idjo t` tld~ dkafj cjnaz~j tlj larj ai~` cjje =+}aoojo t`
tlj nae`ednai 13 Ö 13 bsakj% De tlj na~j `b z~def idejas tjk}iatj~) a~ d~ tlj na~j vdtl
tlj J|jk}ias+^RK aif`sdtlk) aii `b `zs j|jk}ias~ nae cj a}}idjo rda a ~defij katsd|
kzitd}idnatd`e ,jrje db tlj vjsj dedtdaii `b odbbjsjet odkje~d`e~$% Vlje a}}idef
od~taenj bzentd`e~ ,d%j%) odaf`eai Kalaiae`cd~ kjtsdn~$ de a ~dkdias ci`nh+ba~ld`e ,a~
114
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 120/130
va~ ejnj~~as b`s tlj j|}jsdkjet~ de ^jntd`e >%?$) vj ejjo t` c`tl =+}ao tlj j|jk}ias—~
bjatzsj~ aeo ijasejo od~taenj bzentd`e vjdflt~ t` fit tlj n`kk`e 13 Ö 13 bsakj%
Vldij tlj ci`nh+katnldef kjtl`o~ tahj~ k`sj n`k}ztjs kjk`s t` ~t`sj tlj
ldfli+sjozeoaet bjatzsj+katsd| aeo tahj~ a}}s`|dkatji ? ~jn`eo~ t` n`k}ztj }js
dkafj) tld~ n`k}ztatd`e d~ deoj}jeojet `b tlj ezkcjs `b j|jk}ias~% _lj ~defij katsd|+
kzitd}idnatd e d~ qzdtj ba~t aeo }s`rdoj~ a ~dfedfinaet ~}jjoz} `rjs n`er`iztd`e vlje
tlj ezkcjs `b j|jk}ias~ d~ k`ojsatji iasfj% Vj b`zeo tlat b`s z} t` := j|jk}ias~)
tlj n`er`iztd`e+ca~jo kjtl`o d~ ba~tjs) vldij b`s k`sj tlae := j|jk}ias~) tlj ~defij+
katsd|+kzitd}idnatd`e kjtl`o d~ ba~tjs%
C%> J|jk}ias+^RK sze+tdkj n`k}ij|dt
_lj sze+tdkj n`k}ij|dt `b `zs a}}s`anl at tj~t tdkj ~naij~ idejasi vdtl tlj ezkcjs
`b }`~dtdrj de~taenj~ ,czt zeidhj hjseji+^RK kjtl`o~) e`t tlj ejfatdrj~$% L`vjrjs) de
}santdnj) tlj c`ttijejnh a}}jas~ t` cj }js+dkafj ta~h~ ,i`aodef) n`k}ztdef L@F }sakdo
jtn%$ „ tlj antzai }js+de~taenj n`k}ztatd`e d~ gz~t a ~defij o`t+}s`oznt) vldnl nae cj
o`ej j|tsjkji ba~t% B`s ae arjsafj ]A^NAI nia~~ ,∹ >== tsadedef j|ak}ij~ djiodef
∹ >== ~j}asatj nia~~dfijs~$ `zs kjtl`o d~ `ei 2 tdkj~ ~i`vjs tlae a natjf`s+ca~jo
kjtl`o ~znl a~ XBjiwje~wvaic jt ai%) 3=1=[% K`sj fjejsaii) cjnaz~j `b tlj i`ef+tadijo
od~tsdcztd`e `b `cgjnt~ de tlj v`sio ,1=* `b `cgjnt~ `ve 4=* `b j|jk}ias~ X_`ssaica
jt ai%) 3==<[$) tlj j|tsa n`~t `b z~def j|jk}ias~ r~% natjf`sdj~ vdii fsjati odkded~l a~
tlj ezkcjs `b natjf`sdj~ densja~j~%
13=
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 121/130
Cdcid`fsa}l
XAoji~`e) 3==1[ Aoji~`e) J% ,3==1$% @e ~jjdef ~tzbb7 tlj }jsnj}td`e `b katjsdai~ c
lzkae~ aeo kanldej~% De ]s`n% ^]DJ %
XAoji~`e jt ai%) 1441[ Aoji~`e) J% L%) ^dk`enjiid) J% ]%) aeo Bsjjkae) V% _% ,1441$%
]sakdo~ aeo kzitd~naij sj}sj~jetatd`e~% ]s`n% 1>tl Jzs`}jae N`ebjsjenj `e Rd~zai
]jsnj}td`e%
XAla jt ai%) 1441[ Ala) O% V%) Hdcijs) O%) aeo Aicjst) K% H% ,1441$% De~taenj+ca~jo
ijasedef aif`sdtlk~% Kanldej Ijasedef) 2%
XAsd~t tij) [ Asd~t tij% Natjf`sdj~%
XAthj~`e jt ai%) 1446[ Athj~`e) N%) K``sj) A%) aeo ^nlaai) ^% ,1446$% I`naii vjdfltjo
ijasedef% Astdfindai Detjiidfjenj Sjrdjv%
XCas) 3==6[ Cas) K% ,3==6$% _lj }s`antdrj csade7 Z~def aeai`fdj~ aeo a~~`ndatd`e~ t`
fjejsatj }sjodntd`e~% _sjeo~ de N`fedtdrj ^ndjenj%
XCas) 3==4[ Cas) K% ,3==4$% _lj }s`antdrj csade7 kjk`s b`s }sjodntd`e~% ]ldi`~`}ldnai
_sae~antd`e~ `b tlj S`ai ^`ndjt C) >2?713>:„13?>%
XCas aeo Ziikae) 1442[ Cas) K% aeo Ziikae) ^% ,1442$% ^}atdai n`etj|t de sjn`fedtd e%
]jsnj}td`e) 3:7>?>„>:3%
131
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 122/130
XCa~sd) 1443[ Ca~sd) S% ,1443$% Sjn`fedtd`e c }s`t`t}j~% Detjseatd`eai G`zseai `b
N`k}ztjs Rd~d`e) 14714„1?6%
XCji`efdj jt ai%) 3==3[ Cji`efdj) ^%) Kaidh) G%) aeo ]zwdnla) G% ,3==3$% ^la}j katnldef
aeo `cgjnt sjn`fedtd`e z~def ~la}j n`etj|t~% ]AKD %
XCjsf jt ai%) 3==:[ Cjsf) A% N%) Cjsf) _% I%) aeo Kaidh) G% ,3==:$% ^la}j katnldef aeo
`cgjnt sjn`fedtd`e z~def i`v od~t`std`e n`ssj~}`eojenj% NR]S%
XCjshjij) 161=[ Cjshjij) F% ,161=$% _sjatd~j N`enjsedef tlj ]sdend}ij~ `b Lzkae
He`vijofj%
XC`zsojr jt ai%) 3=1=[ C`zsojr) I%) Kagd) ^%) Cs`|) _%) aeo Kaidh) G% ,3=1=$% Ojtjntdef
}j`}ij z~def kztzaii n`e~d~tjet }`~jijt antdratd`e~% JNNR %
XCsao jt ai%) 3==4[ Csao) _%) H`ehij) _%) Airasjw) F%) aeo @idra) A% ,3==4$% Rd~zai
i`ef+tjsk kjk`s la~ a ka~~drj ~t`safj na}andt b`s `cgjnt ojtadi~% ]s`njjodef~ `b
tlj Eatd`eai Anaojk `b ^ndjenj~) Z^A%
XCz~l) 14?:[ Cz~l) R% ,14?:$% A~ vj ka tldeh% _lj Atiaetdn K`etli %
XNlaef aeo Ide) 3==1[ Nlaef) N%+N% aeo Ide) N%+G% ,3==1$% IDC^RK7 a idcsas b`s ~z}+
}`st rjnt`s kanldej~% ^`btvasj aradiacij at ltt}7//vvv%n~dj%etz%joz%tv/ungide/
idc~rk%
XNla}jiij) 3==6[ Nla}jiij) @% ,3==6$% _sadedef a ~z}}`st rjnt`s kanldej de tlj }sdkai%
Ejzsai N`k}ztatd`e) 14711::„116<%
XNlje jt ai%) 3==4[ Nlje) _%) Nljef) K%+K%) _ae) ]%) ^lakds) A%) aeo Lz) ^%+K% ,3==4$%
^hjtnl3}l`t`7 detjsejt dkafj k`etafj% ANK _sae~% Fsa}l%) 3<%
XNlje jt ai%) 3==1[ Nlje) T%) Wl`z) \%) ) aeo Lzaef) _% ^% ,3==1$% @ej+nia~~ ~rk b`s
ijasedef de dkafj sjtsdjrai% DND]%
133
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 123/130
XNlzk aeo Wd~~jskae) 3==6[ Nlzk) @% aeo Wd~~jskae) A% ,3==6$% Ae j|jk}ias k`oji
b`s ijasedef `cgjnt nia~~j~% NR]S%
XN`kaedndz aeo Kjjs) 3==3[ N`kaedndz) O% aeo Kjjs) ]% ,3==3$% Kjae ~ldbt7 A s`cz~ta}}s`anl t`vaso bjatzsj ~}anj aeai~d~% ]AKD ) 3?,:$72=>„214%
XOaiai aeo _sdff~) 3==:[ Oaiai) E% aeo _sdff~) C% ,3==:$% Ld~t`fsak~ `b `sdjetjo fsaod+
jet~ b`s lzkae ojtjntd`e% NR]S%
XOaij jt ai%) 3==4[ Oaij) H%) G`le~`e) K% H%) ^zeharaiid) H%) Katz~dh) V%) aeo ]fi~tjs) L%
,3==4$% Dkafj sj~t`satd`e z~def `eidej }l`t n`iijntd`e~% De Detjseatd`eai N`ebjsjenj
`e N`k}ztjs Rd~d`e%
XJojikae) 144:[ Jojikae) ^% ,144:$% Sj}sj~jetatd`e) ~dkdiasdt aeo tlj nl`sz~ `b
}s`t`t}j~% Kdeo~ aeo Kanldej~%
XJbs`~ aeo Bsjjkae) 3==1[ Jbs`~) A% A% aeo Bsjjkae) V% _% ,3==1$% Dkafj qzditdef b`s
tj|tzsj ~etlj~d~ aeo tsae~bjs% ]s`njjodef~ `b ^DFFSA]L 3==1%
XJii~`e jt ai%) 3==1[ Jii~`e) G%) Fae~ejs) J% S%) H`zt~`fi`~) J%) E`stl) ^% N%) aeo V``olzii)
F% ,3==1$% Fsa}lrdw + `}je ~`zsnj fsa}l osavdef t``i~% De Fsa}l Osavdef) }afj~
?<>„?<?%
XJrjsdeflak jt ai%) 3=1=[ Jrjsdeflak) K%) F``i) I% R%) Vdiidak~) N% H% D%) Vdee) G%) aeo
Wd~~jskae) A% ,3=1=$% _lj }a~nai rd~zai `cgjnt nia~~j~ ,r`n$ nlaiijefj% DGNR %
XBjiwje~wvaic jt ai%) 3==<[ Bjiwje~wvaic) ]%) KnAiij~tjs) O%) aeo Sakaeae) O% ,3==<$%
A od~nsdkdeatdrji tsadejo) kzitd~naij) ojb`skacij }ast k`oji% NR]S%
XBjiwje~wvaic jt ai%) 3=1=[ Bjiwje~wvaic) ]% B%) Fds~nldnh) S% C%) KnNaiij~tjs) O%) aeo
Sakaeae) O% ,3=1=$% @cgjnt ojtjntd`e vdtl od~nsdkdeatdrji tsadejo }ast ca~jo
k`oji~% ]AKD %
13>
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 124/130
XBd~ljs aeo Laesalae) 3=1=[ Bd~ljs) K% aeo Laesalae) ]% ,3=1=$% N`etj|t+ca~jo
~jasnl b`s >o k`oji~% De ANK ^DFFSA]L A~da 3=1= }a}js~) ^DFFSA]L A^DA —1=)
}afj~ 1<371„1<371=% ANK%
XBs`kj aeo Kaidh) 3==2[ Bs`kj) A% aeo Kaidh) G% ,3==2$% Dkafj sjtsdjrai aeo sjn`fed+
td`e z~def i`nai od~taenj bzentd`e~% ED]^%
XBs`kj jt ai%) 3==6[ Bs`kj) A%) ^defjs) T%) ^la) B%) aeo Kaidh) G% ,3==6$% Ijasedef
fi`caii+n`e~d~tjet i`nai od~taenj bzentd`e~ b`s ~la}j+ca~jo dkafj sjtsdjrai aeo
nia~~dfinatd`e% DNNR %
XBs`kj) 3==6[ Bs`kj) A% I% ,3==6$% Ijasedef I`nai Od~taenj Bzentd`e~ b`s J|jk}ias+
Ca~jo @cgjnt Sjn`fedtd`e% ]lO tlj~d~) JJN^ Oj}astkjet) Zedrjs~dt `b Naidb`seda)
Cjshjij%
XFaiijfzdii`~ jt ai%) 3==<[ Faiijfzdii`~) N%) Sacde`rdnl) A%) aeo Cji`efdj) ^% ,3==<$%
@cgjnt natjf`sdwatd`e z~def n`+`nnzssjenj) i`natd`e aeo a}}jasaenj% JNNR %
XFz aeo Sje) 3=1=[ Fz) N% aeo Sje) \% ,3=1=$% Od~nsdkdeatdrj kd|tzsj+`b+tjk}iatj~
b`s rdjv}`det nia~~dfinatd`e% JNNR %
XLaijr jt ai%) 3==4[ Laijr) A%) E`srdf) ]%) aeo ]jsjdsa) B% ,3==4$% _lj zesja~`eacij
jbbjntdrjej~~ `b oata% Detjiidfjet ^~tjk~%
XLae~`e aeo Sd~jkae) 146<[ Lae~`e) A% aeo Sd~jkae) J% ,146<$% Rd~d`e~7 A n`k}ztjs
~~tjk b`s detjs}sjtdef ~njej~% N`k}ztjs Rd~d`e ^~tjk~) }afj~ >=>„>>>%
XLa~ aeo Jbs`~) 3==6[ La~) G% aeo Jbs`~) A% A% ,3==6$% ^njej n`k}ijtd`e z~def
kdiid`e~ `b }l`t`fsa}l~% ANK _sae~antd`e~ `e Fsa}ldn~ ,^DFFSA]L 3==6$) 32%
XLj jt ai%) 3==?[ Lj) \%) Wjkji) S% ^%) aeo Nassjdsa+]js}de¿ae) K% ¿ A% ,3==?$% Kzitd~naij
n`eodtd`eai saeo`k fijio~ b`s dkafj iacjidef% NR]S) }afj~ 24:„6=3%
13?
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 125/130
XLjstwkaee jt ai%) 3==1[ Ljstwkaee) A%) Gan`c~) N%) @idrjs) E%) Nzsij~~) C%) aeo ^aij~de)
O% ,3==1$% Dkafj aeai`fdj~% De ]s`njjodef~ `b ^DFFSA]L 3==1%
XL`djk jt ai%) 3==:[ L`djk) O%) Jbs`~) A% A%) aeo Ljcjst) K% ,3==:$% Azt`katdn }l`t`}`}+z}% ANK _sae~antd`e~ `e Fsa}ldn~ ,^DFFSA]L 3==:$) 3?,>$%
XLzkj) 16>4[ Lzkj) O% ,16>4$% A _sjatd~j `b Lzkae Eatzsj%
XG`le~`e jt ai%) 3=1=[ G`le~`e) K% H%) Oaij) H%) Ardoae) ^%) ]fi~tjs) L%) Bsjjkae) V% _%)
aeo Katz~dh) V% ,3=1=$% NF3sjai7 Dk}s`rdef tlj sjaid~k `b n`k}ztjs fjejsatjo
dkafj~ z~def a iasfj n`iijntd`e `b }l`t`fsa}l~% DJJJ _sae~antd`e~ `e Rd~zaidwatd`e
aeo N`k}ztjs Fsa}ldn~%
XHjkjikanljs+^lidwjskae jt ai%) 3=11[ Hjkjikanljs+^lidwjskae) D%) ^ljnltkae) J%)
Fasf) S%) aeo ^jdtw) ^% K% ,3=11$% J|}i`sdef }l`t`cd`~% ^DFFSA]L %
XHzkas aeo Ljcjst) 3==:[ Hzkas) ^% aeo Ljcjst) K% ,3==:$% A ldjsasnldnai fijio bsakj+
v`sh b`s zedfijo n`etj|t+ca~jo nia~~dfinatd`e% DNNR %
XIad jt ai%) 3=11[ Iad) H%) C`) I%) Sje) \%) aeo B`|) O% ,3=11$% ^}as~j od~taenj ijasedef
b`s `cgjnt sjn`fedtd`e n`kcdedef sfc aeo oj}tl deb`skatd`e%
XIah`bb) 14<6[ Iah`bb) F% ,14<6$% V`kje) Bdsj) aeo Oaefjs`z~ _ldef~7 Vlat Natjf`sdj~
Sjrjai Ac`zt tlj Kdeo%
XIjj aeo Fsazkae) 3=1=[ Ijj) T% G% aeo Fsazkae) H% ,3=1=$% @cgjnt+fsa}l~ b`s n`etj|t+
avasj natjf`s od~n`rjs% N`k}ztjs Rd~d`e aeo ]attjse Sjn`fedtd`e) DJJJ N`k}ztjs
^`ndjt N`ebjsjenj `e) =71„<%
XIjdcj jt ai%) 3==?[ Ijdcj) C%) Ij`easod~) A%) aeo ^nldjij) C% ,3==?$% N`kcdejo `c+
gjnt natjf`sdwatd`e aeo ~jfkjetatd`e vdtl ae dk}idndt ~la}j k`oji% V`sh~l`} `e
^tatd~tdnai Ijasedef de N`k}ztjs Rd~d`e) JNNR %
13:
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 126/130
XId jt ai%) 3=1=[ Id) B%) Nassjdsa) G%) aeo ^kdenld~j~nz) N% ,3=1=$% @cgjnt sjn`fedtd`e a~
saehdef l`id~tdn fifzsj+fs`zeo l}`tlj~j~% NR]S%
XId jt ai%) 3=11[ Id) T%) Fz) I%) aeo Haeaoj) _% ,3=11$% S`cz~ti aidfedef a ~la}j k`ojiaeo dt~ a}}idnatd`e t` nas aidfekjet `b zehe`ve }`~j% ]attjse Aeai~d~ aeo Kanldej
Detjiidfjenj) DJJJ _sae~antd`e~ `e) ]],44$71%
XIdz jt ai%) 3==4[ Idz) N%) Tzje) G%) aeo _`ssaica) A% ,3==4$% E`e}asakjtsdn ~njej
}as~def7 Iacji tsae~bjs rda oje~j ~njej aidfekjet% NR]S%
XI`nhj) 12<4[ I`nhj) G% ,12<4$% Ae J~~a N`enjsedef Lzkae Zeojs~taeodef%
XKaid~djvdnw aeo Jbs`~) 3==6[ Kaid~djvdnw) _% aeo Jbs ~) A% A% ,3==6$% Dk}s`rdef
~}atdai ~z}}`st b`s `cgjnt~ rda kzitd}ij ~jfkjetatd`e~% CKRN %
XKaid~djvdnw aeo Jbs`~) 3==<[ Kaid~djvdnw) _% aeo Jbs`~) A% A% ,3==<$% Sjn`fedtd`e c
a~~`ndatd`e rda ijasedef }js+j|jk}ias od~taenj~% NR]S%
XKaid~djvdnw aeo Jbs`~) 3==4[ Kaid~djvdnw) _% aeo Jbs`~) A% A% ,3==4$% Cj`eo natj+
f`sdj~7 _lj rd~zai kjkj| k`oji b`s sja~`edef ac`zt `cgjnt sjiatd e~ld}~% ED]^%
XKaid~djvdnw jt ai%) 3=11[ Kaid~djvdnw) _%) Fz}ta) A%) aeo Jbs`~) A% A% ,3=11$% Je~jkcij
`b j|jk}ias+~rk~ b`s `cgjnt ojtjntd`e aeo cj`eo% DNNR %
XKjode aeo ^nlabbjs) 146<[ Kjode) O% I% aeo ^nlabbjs) K% ,146<$% N`etj|t tlj`s `b
nia~~dfinatd`e ijasedef% ]~nl`i`fdnai Sjrdjv) <:73=6„3><%
XKzs}l) 3==3[ Kzs}l) F% I% ,3==3$% _lj Cdf C``h `b N`enj}t~% _lj KD_ ]sj~~%
XE`~`b~h) 14<2[ E`~`b~h) S% K% ,14<2$% Attjetd`e) ~dkdiasdt) aeo tlj dojetdfinatd`e+
natjf`sdwatd`e sjiatd`e~ld}% G`zseai `b J|}jsdkjetai ]~nl`i`f7 Fjejsai) 11:,1$7>4„
:6%
132
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 127/130
X]afjt aeo I`ef~tabb) 144<[ ]afjt) S% aeo I`ef~tabb) D% O% ,144<$% _j|tzsj ~etlj~d~ rda
a e`enaz~ai e`e}asakjtsdn kzitd~naij kash`r saeo`k fijio% DJJJ _sae~antd`e~ `e
Dkafj ]s`nj~~def%
X]asdhl jt ai%) 3==<[ ]asdhl) O%) Wdtednh) N% I%) aeo Nlje) _% ,3==<$% Bs`k a}}jasaenj
t` n`etj|t+ca~jo sjn`fedtd`e7 Oje~j iacjidef de ~kaii dkafj~% NR]S%
X]ash jt ai%) 3=1=[ ]ash) O%) Sakaeae) O%) aeo B`vihj~) N% ,3=1=$% Kzitdsj~`iztd e
k`oji~ b`s `cgjnt ojtjntd`e% JNNR %
X]js`ea) 3=1=[ ]js`ea) ]% ,3=1=$% Rd~d`e `b a rd~d}joda% ]s`njjodef~ `b tlj DJJJ %
X]iatt) 1444[ ]iatt) G% N% ,1444$% ]s`cacdid~tdn `zt}zt~ b`s ~z}}`st rjnt`s kanldej~ aeo
n`k}asd~`e~ t` sjfziasdwjo idhjidl``o kjtl`o~% Aoraenj~ de Iasfj Kasfde Nia~~dfijs~%
XSacde`rdnl jt ai%) 3==6[ Sacde`rdnl) A%) Rjoaiod) A%) Faiijfzdii`~) N%) Vdjv`sa) J%) aeo
Cji`efdj) ^% ,3==6$% @cgjnt~ de n`etj|t% DNNR %
XSakaeae aeo Cahjs) 3=11[ Sakaeae) O% aeo Cahjs) ^% ,3=11$% I`nai od~taenj bzen+
td`e~7 A ta|`e`k) ejv afi`sdtlk~) aeo ae jraizatd`e%
XSje jt ai%) 3==:[ Sje) I%) ]atsdnh) A%) Jbs`~) A% A%) L`ofde~) G% H%) aeo Sjlf) G% K%
,3==:$% A oata+osdrje a}}s`anl t` qzaetdbdef eatzsai lzkae k`td`e% ANK _sae~an+
td`e~ `e Fsa}ldn~ ,^DFFSA]L 3==:$) 3?,>$71=4=„1=46%
XS`~nl) 146<[ S`~nl) J% ,146<$% ]sdend}ij~ `b natjf`sdwatd`e% N`fedtd`e aeo Natjf`sdwa+
td`e) }afj~ 36„?<%
XS`~nl aeo Kjsrd~) 146:[ S`~nl) J% aeo Kjsrd~) N% C% ,146:$% Bakdi sj~jkciaenj~7
^tzodj~ de tlj detjseai ~tszntzsj `b natjf`sdj~% N`fedtdrj ]~nl`i`f ) 67:6>„2=:%
136
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 128/130
XSz~~jii jt ai%) 3==2[ Sz~~jii) C%) Jbs`~) A% A%) ^drdn) G%) Bsjjkae) V%) aeo Wd~~jskae) A%
,3==2$% Z~def kzitd}ij ~jfkjetatd`e~ t` od~n`rjs `cgjnt~ aeo tljds j|tjet de dkafj
n`iijntd`e~% NR]S%
XSz~~jii jt ai%) 3==4[ Sz~~jii) C%) Jbs`~) A% A%) ^drdn) G%) Bsjjkae) V% _%) aeo Wd~~jskae)
A% ,3==4$% ^jfkjetdef ~njej~ c katnldef dkafj n`k}`~dtj~% ED]^%
XSz~~jii jt ai%) 3==<[ Sz~~jii) C%) _`ssaica) A%) Kzs}l) H%) ) aeo Bsjjkae) V% _% ,3==<$%
Iacjikj7 a oataca~j aeo vjc+ca~jo t``i b`s dkafj aee`tatd`e% DGNR %
XSz~~jii jt ai%) 3==6[ Sz~~jii) C% N%) _`ssaica) A%) Idz) N%) Bjsfz~) S%) aeo Bsjjkae) V% _%
,3==6$% @cgjnt sjn`fedtd`e c ~njej aidfekjet% ED]^%
X^arasj~j aeo Bjd+Bjd) 3==6[ ^arasj~j) ^% aeo Bjd+Bjd) I% ,3==6$% >o fjejsdn `cgjnt
natjf`sdwatd`e) i`naidwatd`e aeo }`~j j~tdkatd`e% DNNR %
X^nlaeh) 14<>[ ^nlaeh) S% N% ,14<>$% Oeakdn Kjk`s7 A _lj`s `b Sjkdeodef aeo
Ijasedef de N`k}ztjs~ aeo ]j`}ij% Nakcsdofj Zedrjs~dt ]sj~~%
X^nl`oi jt ai%) 3===[ ^nl`oi) A%) ^wjid~hd) S%) ^aij~de) O% L%) aeo J~~a) D% ,3===$% Rdoj`
tj|tzsj~% De ]s`njjodef~ `b ^DFFSA]L —==%
X^nlih`}b jt ai%) 3==1[ ^nlih`}b) C%) ]iatt) G% N%) ^lavj+_ai`s) G%) ^k`ia) A% G%) aeo
Vdiidak~`e) S% N% ,3==1$% J~tdkatdef tlj ~z}}`st `b a ldfl+odkje~d`eai od~tsdcztd`e%
Ejzsai N`k}ztatd`e%
X^ld aeo Kaidh) 3===[ ^ld) G% aeo Kaidh) G% ,3===$% E`skaidwjo nzt~ aeo dkafj ~jf+
kjetatd`e% ]AKD ) 33,<$%
X^ldsh) 3==:[ ^ldsh) N% ,3==:$% @et`i`f d~ `rjssatjo7 Natjf`sdj~) ideh~) aeo taf~%
13<
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 129/130
X^l`tt`e jt ai%) 3==2[ ^l`tt`e) G%) Vdee) G% K%) S`tljs) N%) aeo Nsdkded~d) A% ,3==2$%
_j|t`ec``~t7 G`det a}}jasaenj) ~la}j aeo n`etj|t k`ojidef b`s kzitd+nia~~ `cgjnt
sjn`fedtd`e aeo ~jfkjetatd`e% JNNR %
X^lsdra~tara jt ai%) 3=11[ ^lsdra~tara) A%) Kaid~djvdnw) _%) Fz}ta) A%) aeo Jbs`~) A% A%
,3=11$% Oata+osdrje rd~zai ~dkdiasdt b`s ns`~~+o`kade dkafj katnldef% ^zckdttjo t`
^DFFSA]L A~da%
X^drdn aeo Wd~~jskae) 3==>[ ^drdn) G% aeo Wd~~jskae) A% ,3==>$% Rdoj` f``fij7 A tj|t
sjtsdjrai a}}s`anl t` `cgjnt katnldef de rdoj`~% De DNNR ) }afj~ 1?6=„) Va~ldeft`e)
ON) Z^A% DJJJ N`k}ztjs ^`ndjt%
X^hafj~tao) 1442[ ^hafj~tao) ]% ,1442$% _lj kdeo—~ kanldej~7 _lj tzsdef kanldej)
tlj kjkj|) aeo tlj }js~`eai n`k}ztjs% ^JKD@_DNA%
X^tsat aeo Bd~nlijs) 1441[ ^tsat) _% aeo Bd~nlijs) K% ,1441$% N`etj|t+ca~jo rd~d e7
Sjn`fedwdef `cgjnt~ z~def deb skatd`e bs`k c`tl 3+o aeo >+o dkafjs% ]AKD ) 1>71=:=„
1=2:%
X_l`ka~ jt ai%) 3==4[ _l`ka~) A%) Bjssasd) R%) Ijdcj) C%) _ztjiaas~) _%) aeo F``i) I% G% R%
,3==4$% ^la}j+bs`k+sjn`fedtd`e7 Sjn`fedtd`e jeacij~ kjta+oata tsae~bjs% NRDZ %
X_`ssaica) 3==>a[ _ ssaica) A% ,3==>a$% _lj n`etj|t nlaiijefj%
ltt}7//vjc%kdt%joz/t`ssaica/vvv/nas~AeoBanj~DeN`etj|t%ltki %
X_`ssaica) 3==>c[ _`ssaica) A% ,3==>c$% N`etj|tzai }sdkdef b`s `cgjnt ojtjntd`e% Detjs+
eatd`eai G`zseai `b N`k}ztjs Rd~d`e) :>7124„141%
X_`ssaica jt ai%) 3==<[ _`ssaica) A%) Bjsfz~) S%) aeo Bsjjkae) V% _% ,3==<$% <= kdiid`e
tde dkafj~7 a iasfj oata~jt b`s e`e+}asakjtsdn `cgjnt aeo ~njej sjn`fedtd`e% ]AKD )
>=%
134
8/3/2019 CMU PhD Thesis: Exemplar-Based Representations for Object Detection, Association and Beyond - Malisiewicz - Unk…
http://slidepdf.com/reader/full/cmu-phd-thesis-exemplar-based-representations-for-object-detection-association 130/130
XRjoaiod jt ai%) 3==4[ Rjoaiod) A%) Fzi~lae) R%) Raska) K%) aeo Wd~~jskae) A% ,3==4$%
Kzitd}ij hjseji~ b`s `cgjnt ojtjntd e% DNNR %
XValca) 146>[ Valca) F% ,146>$% A nia~~ `b a}}s`|dkatj ~`iztd`e~ t` idejas `}jsat`sjqzatd`e~% G`zseai `b A}}s`|dkatd`e _lj`s ) 4,1$721 „ 66%
XVjdecjsfjs) 3==6[ Vjdecjsfjs) O% ,3==6$% Jrjstldef d~ kd~njiiaej`z~ 7 tlj }`vjs `b
tlj ejv odfdtai od~`sojs% Ejv T`sh 7 _dkj~ C``h~%
XVdttfje~tjde) 14:>[ Vdttfje~tjde) I% ,14:>$% ]ldi`~`}ldnai Derj~tdfatd`e~% Cianhvjii
]zcid~ldef%
X\da` jt ai%) 3=1=[ \da`) G%) La~) G%) Jldefjs) H%) @idra) A%) aeo _`ssaica) A% ,3=1=$%
^ze oataca~j7 Iasfj+~naij ~njej sjn`fedtd`e bs`k accj t` w``% NR]S%
X\z jt ai%) 3==2[ \z) I%) Nsakkjs) H%) aeo ^nlzzskae~) O% ,3==2$% S`cz~t ~z}}`st
rjnt`s kanldej tsadedef rda n`erj| `ztidjs aciatd`e% AAAD %
XWlaef jt ai%) 3==2[ Wlaef) L%) Cjsf) A%) Kadsj) K%) aeo Kaidh) G% ,3==2$% ^RK+HEE7
Od~nsdkdeatdrj ejasj~t ejdflc`s nia~~dfinatd`e b`s rd~zai natjf`s sjn`fedtd`e% NR]S)