Deconvolutional

Networks

Overview

•

•

–

–

•

Motivation

•

•

–

•

Beyond Edges?

•

Two Challenges

•

•

•

Recap: Sparse Coding (Patch-based)

•

•

•

Talk Overview

•

–

–

•

–

–

•

•

Talk Overview

•

–

–

•

–

–

•

•

Single Deconvolutional Layer

•

Single Deconvolutional Layer

Single Deconvolutional Layer

Single Deconvolutional Layer

1

Single Deconvolutional Layer

1

Single Deconvolutional Layer

1

Single Deconvolutional Layer

1

Top

-do

wn

Dec

om

po

siti

on

Single Deconvolutional Layer

1

Single Deconvolutional Layer

1

Toy Example

Objective for Single Layer

Inference for Single Layer

•

•

•

•

•

•

•

•

Effect of Sparsity

Local Inhibition/Explaining Away

•

Deconvolutional Networks 23

Local Inhibition/Explaining Away

Image

Filters

Talk Overview

•

–

–

•

–

–

•

•

•

•

•

3D Max Pooling

3D Max Pooling

•

•

•

Role of Switches

•

–

–

•

–

•

Overall Architecture (1 layer)

Toy Example

Effect of Pooling

•

•

•

•

•

•

Talk Overview

•

–

–

•

–

–

•

•

Stacking the Layers

•

•

•

–

–

•

–

Overall Architecture (2 layers)

•–

–

•

–

–

–

Multi-layer Inference

Filter Learning

•

–

–

•

•

Overall Algorithm

•

•

•

•

•

Toy Input

Talk Overview

•

–

–

•

–

–

•

•

Related Work

•–

–

–

•–

–

–

–

–

–

–

Comparison: Convolutional Nets

•

•

•

•

Related Work

•

–

–

–

Talk Overview

•

–

–

•

–

–

•

•

Training Details

•

–

•

•

•

•

Model Parameters/Statistics

•

Model Reconstructions

Layer 1 Filters

•

Layer 2 Filters

•

Layer 3 filters

•

Layer 4 filters

•

Relative Size of Receptive Fields

Largest 3 activations at top layer

•

Largest 5 activations at top layer

Top-down Decomposition

•

Largest 5 activations at top layer

•

•

–

•

–

–

–

Application to Object Recognition

FeatureMapsFeatureMaps

Classification Results: Caltech 101

•

Classification Results: Caltech 256

•

Classification Results:

Transfer Learning

•

•

–

–

•

–

–

Classification/Reconstruction

Relationship

•

Effect of Sparsity

64

64.5

65

65.5

66

66.5

67

67.5

68

68.5

0 2 4 6 8 10

Cal

tech

10

1 R

eco

gnit

ion

(%

)

Number of ISTA iterations in inference

•

•

•

Analysis of Switch Settings

•

Summary

•

•

•

•

•

Model using layer-layer

reconstruction

Single Deconvolutional Layer

1

Single Deconvolutional Layer

1

Single Deconvolutional Layer

1

animal head instantiated by bear head

e.g. discontinuities, gradient

e.g. linelets, curvelets, T-junctions

e.g. contours, intermediate objects

e.g. animals, trees, rocks

Context and Hierarchy in a Probabilistic Image ModelJin & Geman (2006)

A Hierarchical Compositional System for Rapid Object Detection

Long Zhu, Alan L. Yuille, 2007.

Able to learn #parts at each level

Comparison: Convolutional Nets

LeCun et al. 1989

Deconvolutional Networks

• Top-down decomposition with convolutions in feature space.

• Non-trivial unsupervised optimization procedure involving sparsity.

Convolutional Networks

• Bottom-up filtering with convolutions in image space.

• Trained supervised requiring labeled data.

Learning a Compositional Hierarchy of Object

Structure

Fidler & Leonardis, CVPR’07; Fidler, Boben & Leonardis, CVPR 2008

The architecture

Parts model

Learned parts