Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use...
Transcript of Content Based Image Retrieval Techniques · 2020-05-07 · Content based Image Recognition To use...
Content Based Image Retrieval Techniques
Ambrose Tuscano ([email protected])University of Maryland Baltimore County,
CMSC 676 Information Retrieval
Introduction
Image retrieval systems aim to find similar images to a query image among an image dataset.
Represented as
• Pixels (Also called Rasters)
• Vectors
● By annotation (manual) • Text retrieval • Semantic level (good for picture with people, architectures)
● By the content (automatic) • Color, texture, shape • Vague description of picture (good for pictures of scenery and with pattern and texture)
Features in an Image
● Color : Low level, Can't specify context.
● Texture : Produce a mathematical characterisation of a repeating pattern in the image.
● Shape: Region based and Contour(outline) based.
● Local Image Features : small parts of a big image. ○ extracted from the images at salient points and dimensionality
reduced using Principal Component Analysis (PCA) transformation○ SIFT using Harris interest points
Structure
IMAGE RETRIEVAL METHODS
Text based Image RetrievalFirst annotated the images by text and then used text-based database management systems to perform image retrieval.
Text based Image Retrieval
Three Ways to go● Manually Assign Keywords to each image
● Use text associated with the images (captions, web pages)
● Analyse the image content to automatically assign keywords(Computer Vision?)
Content Based Image Recognition
● A technique which uses visual contents to search images from large scale image databases according to users' interests.
● CBIR research is mainly contributed by the computer vision community
Content based Image Recognition
To use local features for image retrieval, three different methods are available:
● Direct transfer: nearest neighbors for each of the local features of the query searched and the database images containing most of these neighbors returned.
● Local feature image distortion model (LFIDM): Compares the distances between local features from the query image to the local features of each image of the database . The images with the lowest total distances are returned.
● Histograms of local features: A reasonably large amount of local features from the database is clustered and then each database image represented by a histogram of indices of these clusters.
● Color Histogram● Color Correlogram● Color AutoCorrelogram● Color Coherence vector● Dominant Color Descriptors
● A shape is the form of an object or its external boundary, outline,or external surface, as opposed to other properties like color, texture or composition.
● Fourier Descriptors● Canny Algorithm● SIFT Descriptors● Moment Invariants● Eccentric and Axis Oriented
●
●●●●
●●●
○
●○○○
● Smoothing: Blur image to remove Noise
● FInd Gradients : Edges are marked where gradients of image have large magnitudes.
● Non-Max Suppression: Only local Maxima is marked for edges.
● Double Thresholding: Potential Edges are determined
● Hysteresis : Finally Edges which are not connected/near to many other potential edges are removed.
Texture Extraction- Motif Co-Occurrence Matric● MCM is used to represent transveral
of adjacent pixel color difference in an image.
● Each Pixel corresponds to four adjacent pixel colors
● Each image can be presented by four images of motifs of scan pattern, which can be further constructed into four two dimensional matrices of the image size.
● The attribute of the image will be computed with motifs of scan pattern and a color motif cooccurence matrix(CMCM) will be obtained
● Euclidean DIstance● Mahalanobis Distance● MInkowski Distance● Histogram Intersection Distance● Quadratic Form Distance
Techniques used by CBIR
● K-MeansK-means clustering algorithm is proposed as it improves the scalability.
● Wavelet TransformFeature vectors of images are be constructed from wavelet transformations,
which can also be utilized to distinguish images through measuring distances between feature vectors.
● Support Vector Machine: SVM classifier can be trained using training data of images marked by users .
● Neural Network:A CNN doesn’t need complex work like feature extraction to work. Having a
proper labelled data, we can train the system to learn the data features using complex layer structure.
CBIR + TBIR◦CBIR can be costly in the fact that it needs a lot of complex computations.
◦TBIR can be comparatively fast but has low precision.◦A hybrid model is currently being implemented.
◦a text-based image meta-search engine retrieves images from the Web using the text information from the Query.
◦ Techniques like matching term frequency-inverse document frequency (tf-idf) weightings and cosine similarity are used.
◦use the CBIR approach to re-filter the search results.
● X.Y. Wang,Y.J. Hong and H.Y.Yang,”An effective image retrieval scheme using color,
texture and shape features”
● Nidhi Singh ,Kanchan Singh and Ashok Sinha “A Novel Approach for Content Based Image
Retrieval”
● Yogita Mistry,and D. T. Ingole “Survey on Content Based Image Retrieval Systems”
● John Canny, “A Computational Approach to Edge Detection”
● Mussarat Yasmin, Muhammad Sharif and Sajjad Mohsin, “Use of Low Level Features for
Content Based Image Retrieval: Survey”
● N. Jhanwar, Subhasis Chaudhuri, et.al. Content based image retrieval using motif
cooccurrence matrix