Evaluation of Content-Based Image Retrieval Systems Based on Feature Extraction

Presented by

Fouzia Ashraf Khan

Motivation Exponential increase in computing power and electronic storage capacity

Exponential increase in digital image/video database sizes

Increase use of image and video:EntertainmentEducationCommercial purposes

Need abstractions for efficient and effective modeling

Continuous data, multiple features and analysis of multi-layer or multivariate data processing is difficult job.

Spatial Database Spatial Data

Displayed, manipulated and analyzed by means of spatial attribute which denotes a location on or near the surface of earth

Spatial attribute is presented in the form of co-ordinate pairs that allow position and shape of a particular feature to be measured and represented graphically.

• Spatial Database

A spatial database is a database that is optimized to store and query data related to objects in space, including points, lines and polygons

Typical databases understand various numeric and character data

Need additional functionality spatial data types - called geometry or feature.

CBIR • Content-Based Image Retrieval

- a technique that utilizes the visual content of an image, to search for similar images in large-scale image databases, according to a user’s interest

- is a robust system based upon a combination of higher-level and lower-level vision principles.

- Higher-level analysis uses perceptual organization, inference

and grouping principles to extract semantic information - Lower-level analysis employs a channel energy model to

describe image texture, shape and utilizes color histogram techniques

CBIR System Components

Three Major Components

Feature Extraction IndexingSystem Design

Challenge:

Gap between low-level features and high level user semantics

Feature Extraction Feature

A cluster of points or a boundary /region of pixels Primary Features

◦ Color

◦ Texture

◦ Shape

◦ Spatial location

• Feature extraction

Process of studying and locating areas and objects on the ground and deriving useful information from images.

Feature Extraction Methods

◦ Fuzzy approach

◦ Relevance feedback (supervised learning)

Example of a Classified Image

Color Based Feature Extraction Approaches Conventional Color Histogram (CCH)

Frequency of occurrence of every color in image. Refers probability mass function Captures the joint probabilities of intensities of color channels.

hA,B,C(a,b,c) = N. Prob(A=a, B=b, C=c), where A, B and C are the three color channels and N is the number of pixels in the image

Fuzzy Color Histogram (FCH) Considers degree of color similarity between pixels A color space with K color bins, FCH of I is defined as

F(I)=[f1,f2,…fk] where fi = and

where N = # of pixels

μij = Membership value of jth pixel to ith color bin

dij = Euclidean distance between the color of pixel jth and the ith color bin, ς = Average distance between colors in quantized color space

N

i

ijN

)(1

dijxfij

1

1)(

Color Based Feature Extraction Approaches Color Correlogram (CC)

Express how spatial correlation of pairs of colors changes with distance. A table indexed by color pairs For dth entry at location (i,j) is computed by

- Counting number of pixels of color j at a distance d from a pixel of color i, divided by the total number of pixels in the image

Color-Shape Based Method Includes area and shape information Area of each object is encoded as the number of pixels. Shape is characterized by “perimeter intercepted lengths” PIL - obtained by intercepting object perimeter with 8 line segments having 8

different orientations and passing through object center.

Comparative Study of Color Based Feature Extraction

Texture Based Feature Extraction Approaches Steerable Pyramid

◦ Basic filters are translation and rotation of a single function

◦ Filter is linear combination of basis functions

◦ Only for rotation-invariant texture retrieval

Contourlet Transform

◦ Combination of a Laplacian pyramid and directional filter bank

◦ LP provides the multi‐scale decomposition,

◦ DFB provides the multi‐directional decomposition

Gabor Wavelet

◦ Optimally achieves joint resolution in space and spatial frequency

◦ Transform dilates and rotates the two‐dimensional Gabor function.

◦ Image is convolved with each of the obtained Gabor functions

◦ Gabor function works in Fourier domain,

◦ Computationally intensive

Complex Directional Filter Bank (CDFB)

◦ Retrieval results comparable with Gabor wavelet results

◦ Shift Invariant

Comparative Study of Texture Based Feature Extraction

Shape Based Feature Extraction Approaches

Blob Detection

Detecting points , regions are either brighter or darker than surrounding

Two classes of blob detectors

- Differential method based on derivative expressions

- Methods based on local extrema of intensity

• Thresholding

Classify pixels into two categories:

– Those to which some property measured from the image falls below a threshold,

and those at which the property equals or exceeds a threshold.

– Thresholding creates a binary image : binarization

- Choosing a threshold is a critical task.

Hough Transform

- Find imperfect instances of objects within a certain class of shapes by a voting procedure

- Voting procedure is carried out in a parameter space

- Object candidates are obtained as local maxima

Template Matching

-For finding small parts of an image which match a template

-To detect edges in images.

-Simple to implement and understand, it is one of the slowest methods.

High Dimensional Indexing Techniques

To make CBIR truly scalable to large size image collections

Some Multi-dimensional Indexing Methods Self-Organizing Maps

- SOM are feed-forward, competitive artificial neural networks.

- Reduces complexity (both time and computation)

Shape-Depth Representation

-Well-defined geometric objects, a novel strategy based on a 3D embedding

-Is between contour-based and region-based representations

-A very useful one for retrieving objects/images

Fourier Transform of Shape-Depth and Indexing

- Invariant to transformations such as translation, scaling and rotation.

- Fourier transform is widely used for achieving the invariance

System Design Some factors of Design - Random browsing - Search by example - Search by sketch - Search by text (including keyword or speech) and navigation with customized image categories.

Some Current Developed CBIR Systems

QBIC - Query By Image Content

Virage

RetrievalWare

VisualSEEk and WebSEEk

Netra

Performance Evaluation of CBIR System Performance Observed Probability

Distance Measure and Retrieval Score

measure

- user is facing a target search task from a database D for an image I belonging to class C D.

- Resembles the “target testing" method

- Obtained by implementing an “ideal screener“

- A computer program which simulates the human user by examining the output of the retrieval system

- Marking the images returned by the system either as relevant (positive) or non-relevant (negative).

Important Points

CBIR has become a very active area research for two major research communities, Database Management and Computer Vision

Feature Extraction methods are easy, effective and less expensive

FCH and Gabor wavelet transform are found to yield the highest color and texture retrieval results, respectively, at the cost of higher computational complexity.

To make the content-based Image Retrieval truly scalable to large size image collections, multi-dimensional indexing techniques are highly essential.