Pulkit AgrawalY7322

BVV Sri Raj DuttY7110

Sushobhan NayakY7460

OutlineWhat is a sceneScene recognitionMethodResultsFuture WorkReferences

What is a Scene?Scene- as opposed to

‘object’ or ‘texture’

Object: when view subtends 1 to 2 meters around observer---hand distance

What is a Scene?

observer and fixated point- >5 meters

Scene Recognition2 approachesObject recognition

Global info – details and object info ignoredo Experimental

evidenceo ‘Gist’ of image

Scene RecognitionExclusive

classificationStructural

attributes- Continuous organization of scenes along semantic axes

Semantic axes2 levels:

Degree of naturalness: man-made to natural landscape

Ambiguous (building in field) pictures around center

Semantic axesNatural scenes-

degree of openness

Artificial urban scenes- degree of verticalness and horizontalness

Highways--Highways +Tall Building---Tall Buildings

Method

Information at various Scales

What do we Need ??

High Frequency ? Low Frequency ?

Both ??

Feature ExtractionImage Power Spectrum

Gabor Filters (Scale, Orientation)

Features (512 used)

Mathematical Details…Important data from Image power spectrum

Structural discriminant feature

DST=Discriminat Spectral Template- --an encoding of the discriminant structure between two image categories

‘u’ -weighted integral of power spectrum

Classification

Image = Feature Vector()

Required Classes

Linear Discriminant Analysis

Discriminating Vector (D.V)Maximum Separation b/w classes

Mathematical Details…..Image represented as Feature Vector x.m1 , m2: mean vector of feature vector of 2

classes

Mathematical Details…

gn= feature

Gn = Gabor filter

dn = through learning

Learning…Projection of Training Set

Image F.V. on D.V.

Use LDA to determine Threshold

Classifier Obtained

Learning

Work..

Artificial v/s Natural

Open v/s Non Open

ResultsArtificial v/s Natural

Artificial•80 Test Images•67 classified Correctly

Natural•80 Test Images•75 classified Correctly

89% Correct results

Result

Future WorkArrangement in semantic axesAddition of features

Depth Symmetry

Contrast Ruggedness

8 category arrangement (skyscrapers, highway, street, flat building, beach, field, mountain, forest)

Experiment with Haar and other filters

ReferenceTorralba A. & Olivia A., Semantic

Organisation of Scenes using Discriminant Structural Templates (1999)

Torralba A. & Olivia A., Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope(2001)

Olivia A., Gist of the Scenehttp://people.csail.mit.edu/torralba/code/spati

alenvelope/