High Level Computer Vision

Deep Learning for Computer Vision Part 3 - Segmentation

Bernt Schiele - schiele@mpi-inf.mpg.de Mario Fritz - mfritz@mpi-inf.mpg.de

https://www.mpi-inf.mpg.de/hlcv

High Level Computer Vision - July 15, 2o15

• From detection to segmentation ‣ Main Reading:

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs, Chen, Papandreou, Kokkins, Murphy, Yuille, ICLR’15

‣ Further interesting reading - Hypercolumns for object segmentation and fine-grained localization

Bharath Hariharan, Pablo Arbeláez, Ross Girshick, Jitendra Malik, CVPR’15 - Fully Concolutional Networks for Semantic Segmentation

John Long, Evan Shelhamer, Trevor Darelle, CVPR’15

2

High Level Computer Vision - July 15, 2o15 3

High Level Computer Vision - July 15, 2o15 4

High Level Computer Vision - July 15, 2o15 5

Sliding Window with ConvNet

Input Window

224

224

66

256C

classes

Conv Conv Conv Conv Conv Full Full

Feature Extractor Classifier

Sliding Window with ConvNet

Input Window

224 6256

Conv Conv Conv Conv Conv Full Full

Feature Extractor167

240

1

No need to compute two separate windowsJust one big input window, computed in a single pass

C classes

High Level Computer Vision - July 15, 2o15 8

High Level Computer Vision - July 15, 2o15 9

High Level Computer Vision - July 15, 2o15 10

High Level Computer Vision - July 15, 2o15 11

High Level Computer Vision - July 15, 2o15 12

High Level Computer Vision - July 15, 2o15

“Hole” algorithm

• skip subsampling ‣ in their case for VGG-net: after the last two max-pooling layers)

• for the next layer filter: sparsely sample the feature map with “input stride” 2 (or 4 respectively)

13

High Level Computer Vision - July 15, 2o15 14

High Level Computer Vision - July 15, 2o15

CRF - Conditional Random Field

• Energy function to be minimized

‣ with unary terms obtained from the CNN:

‣ and pairwise terms (Potts model)

- with

15

High Level Computer Vision - July 15, 2o15 16

High Level Computer Vision - July 15, 2o15 17

High Level Computer Vision - July 15, 2o15 18

High Level Computer Vision - July 15, 2o15 19

High Level Computer Vision - July 15, 2o15 20

High Level Computer Vision - July 15, 2o15

Another fully convolutional network for semantic segmentation (without CRF)

21

High Level Computer Vision - July 15, 2o15 22

High Level Computer Vision - July 15, 2o15 23