MIT6870_ORSU_lecture11 Hierarchies

8/3/2019 MIT6870_ORSU_lecture11 Hierarchies

http://slidepdf.com/reader/full/mit6870orsulecture11-hierarchies 1/36

Lecture 11Hierarchies

6.870 Object Recognition and Scene Understandinghttp://people.csail.mit.edu/torralba/courses/6.870/6.870.recognition.htm

Next weekAlec Rivers

Scene Understanding Based on Object RelationshipsGokberk Cinbis

Category Level 3D Object Detection Using View-Invariant Representations

Hueihan Jhuang and Sharat Chikkerur

Video shot boundary detection using GIST representation

Jenny Yuen

Semiautomatic alignment of text and images

Nathaniel R Twarog

A Filtering Approach to Image Segmentation: Perceptual Grouping in Feature Space

Nicolas Pinto

Evaluating dense feature descriptor and multi-kernel learning for face detection/recognition

Tilke Judd and Vladimir Bychkovsky

Identify the same people in different photographs from the same event

Tom Kollar

Context-based object priors for scene understanding

Tom Ouyang

Hand-Drawn Sketch Recognition, A Vision-Based Approach

Papers due this Friday (5pm): send PDF by email

Hierarchies vs. holistic features

Although we have

seen some ³successful´

holistic methods.

Hierarchies, compositionality and

reusable parts

Compositionality refers to our evident ability to

construct hierarchical representations, whereby

constituents are used and reused in an

essentially infinite variety of relationalcompositions.

Assumption (Bienenstock, Geman): what islearnable is what is representable as a hierarchy

of more-or-less simple composition rules.

Bienenstock, Geman. Compositionality in neural systems.

Hierarchies vs. holistic features

Feature hierarchies are often inspired by the structure of the primate visual system,

which has been shown to use a hierarchy of features of increasing complexity, fromsimple local features in the primary visual cortex, to complex shapes and object

views in higher cortical areas.

S. Ullman et al.

Diagram of the visual system

Felleman and Van Essen, 1991

Modified by T. Serre from Ungerleider and Haxby, and then shamelessly copied by me.

Modified by T. Serre from Ungerleider and Haxby, and then copied by me.

IT readout

Slide by Serre

Voxel Activity ModelGoal: to predict the image seen by the observer out of a large collection of

possible images. And to do this for new images: this requires predicting f MRIactivity for unseen images.

Kay, K.N., Naselaris, T., Prenger, R.J., & Gallant, J.L. (2008). Identifying natural imagesfrom human brain activity. Nature, 452, 352-355.

Performance

D.Marr

Neocognitron

Learning is done greedily for each layer

Convolutional Neural Network

The output neurons share all the intermediate levels

Le Cun et al, 98

Hierarchical models of object recognition in cortex

Hierarchical extension of the classical paradigm of building complex cells from simple cells.

Uses same notation than Fukushima: ³S´ units performing template matching, solid lines and

³C´ units performing non-linear operations ( ³M AX´ operation, dashed lines)

Riesenhuber, M. and Poggio, T. 99

Slide by T. Serre

Learning a Compositional Hierarchy of Object Structure

Fidler & Leonardis, CVPR¶07; Fidler, Boben & Leonardis, CVPR 2008Fidler & Leonardis, CVPR¶07; Fidler, Boben & Leonardis, CVPR 2008

The architecture

Parts model

Learned parts

Fidler & Leonardis, CVPR¶07; Fidler, Boben & Leonardis, CVPR 2008Fidler & Leonardis, CVPR¶07; Fidler, Boben & Leonardis, CVPR 2008

Fidler & Leonardis, CVPR¶07Fidler & Leonardis, CVPR¶07

Fidler, Boben & Leonardis, CVPR 2008Fidler, Boben & Leonardis, CVPR 2008

Layer 2

Layer 3

Layer 4

Layer 1

LEARNLEARNhierarchical libraryhierarchical library

car motorcycle dog person

Hierarchical compositional architectureHierarchical compositional architecture

Features are shared at each layer Features are shared at each layer

Learning is done on natural imagesLearning is done on natural images

Indexing and matching detection schemeIndexing and matching detection scheme

Learned L1Learned L1 ± ± L3L3

Learned hierarchicalLearned hierarchical

vocabularyvocabulary DetectionsDetections

Fidler & Leonardis, CVPR¶07Fidler & Leonardis, CVPR¶07

Fidler, Boben & Leonardis, CVPR 2008Fidler, Boben & Leonardis, CVPR 2008

Layer 2

Layer 3

Layer 4

Layer 1

LEARNLEARNhierarchical libraryhierarchical library

car motorcycle dog person

Learned hierarchicalLearned hierarchical

vocabularyvocabulary DetectionsDetections

Hierarchical compositional architectureHierarchical compositional architecture

Features are shared at each layer Features are shared at each layer

Learning is done on natural imagesLearning is done on natural images

Biologically plausible?Biologically plausible?

Learns TLearns T-- and Land L-- junctions, different junctions, differentcurvatures, and features that graduallycurvatures, and features that graduallyincrease in complexityincrease in complexity

HDP Object Model

We learn the

number of parts. Each object

uses a different

number of parts.

The model

assumes aknown number

of object

categories.

Parts are distributions

over appearances andlocations

Sudderth et al. IJCV 2008

MIT6870_ORSU_lecture11 Hierarchies

Documents

Transcript of MIT6870_ORSU_lecture11 Hierarchies

The Celestial Hierarchies

Collaborations and Hierarchies

New Hierarchies

Week 5 - Visual Hierarchies

Multilingualism: Hierarchies, Inequalities ... · Multilingualism: Hierarchies, Inequalities, Marginalization Plurilinguisme : hiérarchies, inégalités et marginalisations Jeudi

Hierarchies of Matter

Scene Hierarchies - cs.utexas.edu

Reverse Dominance Hierarchies

Models & Hierarchies

Designing Effective Hierarchies

Optimization of cortical hierarchies with continuous scales and … · 2009-07-15 · Running head: Optimization of cortical hierarchies 1 Optimization of cortical hierarchies with

Load Hierarchies

Nettverk Software Protocol Hierarchies

Role Activation Hierarchies

Big Hierarchies, Little Hierarchies, and Flat Directions ... Hierarchies, Little Hierarchies, and Flat Directions ... it means computing it directly from Feynman diagrams only involves

Angelic Hierarchies

Overcoming Reverse Dominance Hierarchies

Hierarchies and eBay

Code Reuse Through Hierarchies

New hierarchies 0111011 3