2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP...
Transcript of 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP...
![Page 1: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/1.jpg)
2ndDetection / Segmentation Challenge
Yin Cui, Tsung-Yi Lin, Matteo Ruggero Ronchi, Genevieve Patterson
ImageNet and COCO Visual Recognition Challenges WorkshopSunday, October 9th, ECCV 2016
![Page 2: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/2.jpg)
/ 542
Workshop Organizers
Yin CuiCornell Tech
Matteo Ruggero RonchiCaltech
Genevieve PattersonBrown University
Michael MaireSerge BelongieLubomir BourdevRoss GirshickJames HaysPietro PeronaLarry ZitnickPiotr Dollár
Workshop Advisors:Deva RamananPietro PeronaMichael MaireLubomir BourdevSerge BelongieMatteo Ruggero RonchiGenevieve PattersonYin Cui
Award Committee:
Tsung-Yi LinCornell Tech
![Page 3: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/3.jpg)
/ 543
Outline
1. Download MS COCO Train / Val set
Participate in challenge?
Yes
No 3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
2. Develop the algorithm
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 4: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/4.jpg)
/ 543
Outline
1. Download MS COCO Train / Val set
Participate in challenge?
Yes
No 3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
2. Develop the algorithm
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 5: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/5.jpg)
/ 544
• 80 object categories • 200k images• 1.2M instances (350k people)• Every instance segmented
Available for download atmscoco.org
COCO Dataset
![Page 6: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/6.jpg)
/ 544
• 80 object categories • 200k images• 1.2M instances (350k people)• Every instance segmented
Available for download atmscoco.org
COCO Dataset
• 106k people with keypoints
![Page 7: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/7.jpg)
/ 545
Available for download atmscoco.org/external
COCO 3rd Party Datasets
![Page 8: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/8.jpg)
/ 546
1. Download MS COCO Train / Val set
Participate in challenge?
Yes
No
2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 9: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/9.jpg)
/ 546
1. Download MS COCO Train / Val set
Participate in challenge?
Yes
No
2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 10: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/10.jpg)
/ 547
Shout-out to previous algorithms!
![Page 11: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/11.jpg)
/ 547
Shout-out to previous algorithms!
![Page 12: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/12.jpg)
/ 548
1. Download MS COCO Train / Val set
Participate in challenge?
Yes
No
2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 13: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/13.jpg)
/ 548
1. Download MS COCO Train / Val set
Participate in challenge?
Yes
No
2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 14: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/14.jpg)
/ 549
Challenges at ECCV 2016
![Page 15: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/15.jpg)
/ 5410
Participate in challenge?
Yes
No
1. Download MS COCO Train / Val set 2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 16: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/16.jpg)
/ 5410
Participate in challenge?
Yes
No
1. Download MS COCO Train / Val set 2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 17: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/17.jpg)
/ 5411
MS COCO Test Sets
![Page 18: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/18.jpg)
/ 5411
The 2015/2016 MS COCO Test set consists of ~80k test images.
MS COCO Test Sets
![Page 19: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/19.jpg)
/ 5411
The 2015/2016 MS COCO Test set consists of ~80k test images.
Test-dev (development) Debugging, Validation and Ablation Studies. Allows unlimited submission to the evaluation server.
MS COCO Test Sets
![Page 20: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/20.jpg)
/ 5411
The 2015/2016 MS COCO Test set consists of ~80k test images.
Test-dev (development) Debugging, Validation and Ablation Studies. Allows unlimited submission to the evaluation server.
Test-standard (publications) Used to score entries for the Public Leaderboard.
MS COCO Test Sets
![Page 21: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/21.jpg)
/ 5411
The 2015/2016 MS COCO Test set consists of ~80k test images.
Test-dev (development) Debugging, Validation and Ablation Studies. Allows unlimited submission to the evaluation server.
Test-standard (publications) Used to score entries for the Public Leaderboard.
Test-challenge (competitions) Used to score workshop competition.
MS COCO Test Sets
![Page 22: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/22.jpg)
/ 5411
The 2015/2016 MS COCO Test set consists of ~80k test images.
Test-dev (development) Debugging, Validation and Ablation Studies. Allows unlimited submission to the evaluation server.
Test-standard (publications) Used to score entries for the Public Leaderboard.
Test-challenge (competitions) Used to score workshop competition.
Test-reserve (security) Used to estimate overfitting. Scores on this set are never released.
MS COCO Test Sets
![Page 23: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/23.jpg)
/ 5412
Participate in challenge?
Yes
No
1. Download MS COCO Train / Val set 2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 24: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/24.jpg)
/ 5412
Participate in challenge?
Yes
No
1. Download MS COCO Train / Val set 2. Develop the algorithm
Outline
3. Download MS COCO Test-Dev
3. Download MS COCO Test-Full
4. Upload to CodaLab (unlimited)
4. Upload to CodaLab (5 times max)
![Page 25: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/25.jpg)
/ 5413
Evaluation Server Usage
Submissions to all test sets
![Page 26: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/26.jpg)
/ 5414
Evaluation Metrics
![Page 27: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/27.jpg)
/ 5415
Evaluation Metrics
• AP is averaged over multiple IoU values between 0.5 and 0.95.
Challenges Score: AP• More comprehensive metric than
the traditional AP at a fixed IoU value (0.5 for PASCAL).
![Page 28: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/28.jpg)
/ 54
• AP is averaged over instance size:• small (A < 32 x 32)• medium (32x 32 < A < 96 x 96)• large (A > 96 x 96)
16
Evaluation Metrics
A<32x32
32x32<A<96x96
A>96x96Other Scores: Size AP
![Page 29: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/29.jpg)
/ 5417
Evaluation Metrics
Other Scores: AR
• Measures the maximum recall over a fixed number of detections allowed in the image of 1, 10, 100.
• AR is averaged over small (A < 32 x 32), medium (32x 32 < A < 96 x 96) and large (A > 96 x 96) instances of objects.
![Page 30: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/30.jpg)
/ 5418
Evaluation Ambiguity
Which one is better?
![Page 31: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/31.jpg)
/ 5418
Evaluation Ambiguity
Which one is better?
![Page 32: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/32.jpg)
/ 5418
Evaluation Ambiguity
Which one is better?
Ground-Truth BBox
![Page 33: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/33.jpg)
/ 5418
Evaluation Ambiguity
Which one is better?
Detection BBoxGround-Truth BBox
![Page 34: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/34.jpg)
/ 5418
Evaluation Ambiguity
IoU = 0.5 IoU = 0.7 IoU = 0.95
Which one is better?
Detection BBoxGround-Truth BBox
![Page 35: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/35.jpg)
/ 5419
COCO Challenges Results
![Page 36: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/36.jpg)
/ 5420
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
![Page 37: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/37.jpg)
/ 5420
0%
10%
20%
30%
40%
50%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
Fast
R-CNN*
(VGG-16)
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
![Page 38: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/38.jpg)
/ 5420
0%
10%
20%
30%
40%
50%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
Fast
R-CNN*
(VGG-16)
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
(*) Performance on Test-Dev
![Page 39: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/39.jpg)
/ 5420
0%
10%
20%
30%
40%
50%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
Fast
R-CNN*
(VGG-16)
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
(*) Performance on Test-Dev
![Page 40: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/40.jpg)
/ 5420
0%
10%
20%
30%
40%
50%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
Fast
R-CNN*
(VGG-16)
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
(*) Performance on Test-Dev (**) 2015 Winner
![Page 41: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/41.jpg)
/ 5420
0%
10%
20%
30%
40%
50%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
Fast
R-CNN*
(VGG-16)
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
(*) Performance on Test-Dev
+22% absolute+110% relative
(**) 2015 Winner
![Page 42: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/42.jpg)
/ 5420
0%
10%
20%
30%
40%
50%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
Fast
R-CNN*
(VGG-16)
Bounding Boxes Leaderboard (I)
COCO AP (over all IoU)
(*) Performance on Test-Dev
+22% absolute+110% relative
(**) 2015 Winner
+4.2% absolute+11.2% relative
![Page 43: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/43.jpg)
/ 5421
Segmentation Leaderboard (I)
COCO AP (over all IoU)
![Page 44: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/44.jpg)
/ 5421
0%
10%
20%
30%
40%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
Segmentation Leaderboard (I)
COCO AP (over all IoU)
![Page 45: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/45.jpg)
/ 5421
0%
10%
20%
30%
40%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
Segmentation Leaderboard (I)
COCO AP (over all IoU)
(**) 2015 Winner
![Page 46: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/46.jpg)
/ 5421
0%
10%
20%
30%
40%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
Segmentation Leaderboard (I)
COCO AP (over all IoU)
(**) 2015 Winner
![Page 47: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/47.jpg)
/ 5421
0%
10%
20%
30%
40%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
Segmentation Leaderboard (I)
COCO AP (over all IoU)
(**) 2015 Winner
![Page 48: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/48.jpg)
/ 5421
0%
10%
20%
30%
40%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
Segmentation Leaderboard (I)
COCO AP (over all IoU)
(**) 2015 Winner
+9.1% absolute+32.3% relative
![Page 49: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/49.jpg)
/ 5421
0%
10%
20%
30%
40%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
Segmentation Leaderboard (I)
COCO AP (over all IoU)
COCO AP for segmentation winner trails the one for bbox detection by ~4%:
• Last year the gap was ~10%• Localization is harder
for segmentation
(**) 2015 Winner
+9.1% absolute+32.3% relative
![Page 50: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/50.jpg)
/ 5422
Bounding Boxes Leaderboard (II)
Object Localization is improving
![Page 51: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/51.jpg)
/ 5422
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
AP_50 AP_75
Bounding Boxes Leaderboard (II)
Object Localization is improving
![Page 52: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/52.jpg)
/ 5422
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
AP_50 AP_75
Bounding Boxes Leaderboard (II)
Object Localization is improving
![Page 53: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/53.jpg)
/ 5422
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
AP_50 AP_75
Bounding Boxes Leaderboard (II)
Object Localization is improving
objects correctly detected but not well localized17% AP
![Page 54: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/54.jpg)
/ 5422
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRAVC**
Trimps-S
oush
en
Imag
ine La
b
Cmu-a2-v
gg16
ToCon
coctP
ellucid Wall
hust-
mclab
AP_50 AP_75
Bounding Boxes Leaderboard (II)
Object Localization is improving
17% AP 19% AP
![Page 55: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/55.jpg)
/ 5423
Segmentation Leaderboard (II)
Mask localization can improve
![Page 56: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/56.jpg)
/ 5423
0%
10%
20%
30%
40%
50%
60%
70%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
AP_50 AP_75
Segmentation Leaderboard (II)
Mask localization can improve
![Page 57: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/57.jpg)
/ 5423
0%
10%
20%
30%
40%
50%
60%
70%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
AP_50 AP_75
Segmentation Leaderboard (II)
Mask localization can improve
![Page 58: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/58.jpg)
/ 5423
0%
10%
20%
30%
40%
50%
60%
70%
MSRAG-R
MI
MSRAVC**
anon
ymou
s
AP_50 AP_75
Segmentation Leaderboard (II)
Mask localization can improve
20% AP
![Page 59: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/59.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
![Page 60: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/60.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
(*) 2015 Winner
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRA (seg
m)
MSRAVC (bbox
)*
AP AP_75 AP_50
![Page 61: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/61.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
COCO AP for segmentation winner trails the one for bbox detection by ~2%:
(*) 2015 Winner
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRA (seg
m)
MSRAVC (bbox
)*
AP AP_75 AP_50
![Page 62: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/62.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
COCO AP for segmentation winner trails the one for bbox detection by ~2%:
(*) 2015 Winner
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRA (seg
m)
MSRAVC (bbox
)*
AP AP_75 AP_50
![Page 63: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/63.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
COCO AP for segmentation winner trails the one for bbox detection by ~2%:
• Results in 2nd place in the bbox challenge!
(*) 2015 Winner
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRA (seg
m)
MSRAVC (bbox
)*
AP AP_75 AP_50
![Page 64: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/64.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
COCO AP for segmentation winner trails the one for bbox detection by ~2%:
• Results in 2nd place in the bbox challenge!
• Gap is about constant at multiple IoU values.
(*) 2015 Winner
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRA (seg
m)
MSRAVC (bbox
)*
AP AP_75 AP_50
![Page 65: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/65.jpg)
/ 5424
Bounding Boxes vs Segmentation
Segmentation provides great bounding boxes!
COCO AP for segmentation winner trails the one for bbox detection by ~2%:
• Results in 2nd place in the bbox challenge!
• Gap is about constant at multiple IoU values.
• Participate in Segmentation Challenge!
(*) 2015 Winner
0%
10%
20%
30%
40%
50%
60%
70%
G-RMI
MSRA (seg
m)
MSRAVC (bbox
)*
AP AP_75 AP_50
![Page 66: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/66.jpg)
/ 5425
Performance Breakdown (I)
COCO AP varies across supercategories and size
![Page 67: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/67.jpg)
/ 5425
0%
10%
20%
30%
40%
50%
60%
anim
alou
tdoo
rve
hicle
elect
roni
cpe
rson
appl
iance
furn
iture
spor
tsfo
odin
door
kitch
enac
cess
ory
Performance Breakdown (I)
COCO AP varies across supercategories and size
![Page 68: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/68.jpg)
/ 5425
0%
10%
20%
30%
40%
50%
60%
anim
alou
tdoo
rve
hicle
elect
roni
cpe
rson
appl
iance
furn
iture
spor
tsfo
odin
door
kitch
enac
cess
ory
Performance Breakdown (I)
COCO AP varies across supercategories and size
![Page 69: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/69.jpg)
/ 5425
0%
10%
20%
30%
40%
50%
60%
anim
alou
tdoo
rve
hicle
elect
roni
cpe
rson
appl
iance
furn
iture
spor
tsfo
odin
door
kitch
enac
cess
ory
Performance Breakdown (I)
COCO AP varies across supercategories and size
Performance across teams improved on all supercategories
• Average AP increase of ~10%.• Average Standard Deviation
decrease of ~1%.
![Page 70: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/70.jpg)
/ 5426
Performance Breakdown (II)
Impact of size on performance
0%
15%
30%
45%
60%
2015 2016
small
medium large
![Page 71: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/71.jpg)
/ 5426
Performance Breakdown (II)
Impact of size on performance
0%
15%
30%
45%
60%
2015 2016
small
medium large
+33%
![Page 72: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/72.jpg)
/ 5426
Performance Breakdown (II)
Impact of size on performance
0%
15%
30%
45%
60%
2015 2016
small
medium large
+33%
+53%
![Page 73: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/73.jpg)
/ 5426
Performance Breakdown (II)
Impact of size on performance
0%
15%
30%
45%
60%
2015 2016
small
medium large
+33%
+53%
+118%!!
![Page 74: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/74.jpg)
/ 5427
Correlation between methods
![Page 75: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/75.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
![Page 76: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/76.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
(*) 2015 Winner
![Page 77: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/77.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
(*) 2015 Winner
![Page 78: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/78.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
(*) 2015 Winner
![Page 79: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/79.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
R2 = 0.98
(*) 2015 Winner
![Page 80: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/80.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
R2 = 0.98
(*) 2015 Winner
Segmentation
G-RMI
MSR
A
![Page 81: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/81.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
0 % 80%AP
R2 = 0.98
(*) 2015 Winner
Segmentation
G-RMI
MSR
A
![Page 82: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/82.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
0 % 80%AP
R2 = 0.98
(*) 2015 Winner
Segmentation
G-RMI
MSR
A
![Page 83: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/83.jpg)
/ 5427
Correlation between methods
How similarly do algorithms perform?
G-R
MI
MSRAVC*
Bounding Boxes
0 % 80%AP
0 %
80 %
AP
0 % 80%AP
R2 = 0.98 R2 = 0.97
(*) 2015 Winner
Segmentation
G-RMI
MSR
A
![Page 84: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/84.jpg)
/ 5428
Bounding Box Detection Errors
![Page 85: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/85.jpg)
/ 5428
Bounding Box Detection Errors
How similarly do top algorithms perform?
![Page 86: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/86.jpg)
/ 5428
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors
How similarly do top algorithms perform?
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
overall-all-all
[.456] C75[.623] C50[.686] Loc[.700] Sim[.723] Oth[.925] BG[1.00] FN
G-RMI
![Page 87: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/87.jpg)
/ 5428
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors
How similarly do top algorithms perform?
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
overall-all-all
[.456] C75[.623] C50[.686] Loc[.700] Sim[.723] Oth[.925] BG[1.00] FN
G-RMI MSRAVC*
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
overall-all-all
[.399] C75[.589] C50[.682] Loc[.695] Sim[.713] Oth[.870] BG[1.00] FN
(*) 2015 Winner
![Page 88: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/88.jpg)
/ 5429
Bounding Box Detection Errors (I)
![Page 89: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/89.jpg)
/ 5429
Bounding Box Detection Errors (I)
What type of errors are algorithms doing?
![Page 90: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/90.jpg)
/ 5429
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors (I)
![Page 91: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/91.jpg)
/ 5429
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors (I)
G-RMI MSRAVC*
(*) 2015 Winner
![Page 92: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/92.jpg)
/ 5429
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
person-person-all
[.582] C75[.812] C50[.875] Loc[.875] Sim[.886] Oth[.970] BG[1.00] FN
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
person-person-all
[.510] C75[.724] C50[.832] Loc[.832] Sim[.841] Oth[.911] BG[1.00] FN
Bounding Box Detection Errors (I)
G-RMI MSRAVC*
(*) 2015 Winner
![Page 93: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/93.jpg)
/ 5430
Bounding Box Detection Errors (II)
![Page 94: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/94.jpg)
/ 5430
Bounding Box Detection Errors (II)
What type of errors are algorithms doing?
![Page 95: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/95.jpg)
/ 5430
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors (II)
![Page 96: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/96.jpg)
/ 5430
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors (II)
G-RMI MSRAVC*
(*) 2015 Winner
![Page 97: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/97.jpg)
/ 5430
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
overall-all-small
[.244] C75[.416] C50[.506] Loc[.518] Sim[.533] Oth[.824] BG[1.00] FN
0 0.2 0.4 0.6 0.8 1recall
0
0.2
0.4
0.6
0.8
1
prec
ision
overall-all-small
[.175] C75[.343] C50[.469] Loc[.476] Sim[.484] Oth[.709] BG[1.00] FN
AP @ IoU = [0.5; 0.75]
AP @ IoU = 0.1
Super-category FP removed
Category FP removed
Background FP removed
All errors are removed
Bounding Box Detection Errors (II)
G-RMI MSRAVC*
(*) 2015 Winner
![Page 98: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/98.jpg)
/ 5431
Summary of Findings
![Page 99: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/99.jpg)
/ 5431
Summary of Findings
2016 Detection Challenge Take-aways
![Page 100: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/100.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.
2016 Detection Challenge Take-aways
![Page 101: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/101.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.• G-RMI imroved COCO AP by 4% absolute, 11% relative.
2016 Detection Challenge Take-aways
![Page 102: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/102.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.• G-RMI imroved COCO AP by 4% absolute, 11% relative.• MSRA 2016 segmentation algorithm is great on bboxes.
2016 Detection Challenge Take-aways
![Page 103: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/103.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.• G-RMI imroved COCO AP by 4% absolute, 11% relative.• MSRA 2016 segmentation algorithm is great on bboxes.• Performance on all classes has improved across entries.
2016 Detection Challenge Take-aways
![Page 104: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/104.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.• G-RMI imroved COCO AP by 4% absolute, 11% relative.• MSRA 2016 segmentation algorithm is great on bboxes.• Performance on all classes has improved across entries.• Localization improved greatly in both challenges.
2016 Detection Challenge Take-aways
![Page 105: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/105.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.• G-RMI imroved COCO AP by 4% absolute, 11% relative.• MSRA 2016 segmentation algorithm is great on bboxes.• Performance on all classes has improved across entries.• Localization improved greatly in both challenges.• High relative improvement on small object instances.
2016 Detection Challenge Take-aways
![Page 106: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/106.jpg)
/ 5431
Summary of Findings
• MSRAVC 2015 set a very high bar for performance.• G-RMI imroved COCO AP by 4% absolute, 11% relative.• MSRA 2016 segmentation algorithm is great on bboxes.• Performance on all classes has improved across entries.• Localization improved greatly in both challenges.• High relative improvement on small object instances.• False negatives are reduced, thus better recall of teams.
2016 Detection Challenge Take-aways
![Page 107: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/107.jpg)
/ 5432
Challenges Ranking
![Page 108: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/108.jpg)
/ 54
G-RMI 1st 2nd
MSRA - 1st
Trimps-Soushen 2nd -
Imagine Lab 3rd -
UofA 5th -
1026 - 3rd
32
Challenges RankingTeam BBox Segmentation
![Page 109: 2nd Detection / Segmentation ChallengeSegmentation Leaderboard (I) COCO AP (over all IoU) COCO AP for segmentation winner trails the one for bbox detection by ~4%: • Last year the](https://reader034.fdocuments.us/reader034/viewer/2022051605/600d3a2b89b62a54d328a2fd/html5/thumbnails/109.jpg)
/ 54
G-RMI 1st 2nd
MSRA - 1st
Trimps-Soushen 2nd -
Imagine Lab 3rd -
UofA 5th -
1026 - 3rd
32
Challenges Ranking
Invited Speakers:• G-RMI / Object Detection / (2:30pm - 2:45pm)• MSRA / Segmentation / (2:45pm - 3:00pm)
Team BBox Segmentation