Dense Color Moment: A New Discriminative Color Descriptor

Kylie Gorman, Mentor: Yang Zhang

University of Central Florida

I. Problem:

Create Robust Discriminative Color Descriptor Color Descriptor Significance

Large variations in RGB values occur due to scene accidental events

II. Previous Methods:

III. Our Approach:

V. Experiments: Color Moment

Training Data

Google Data Set: 1,100 images

Testing Data

EBay Data Set: 4 categories

12 images per color, 132 images

per category

Color Moment and Dense SIFT

Birds 200 (20 classes)

200 species/categories with 11,788 images total

Color HistogramColor Mapping

Blockwise Color Moment Feature Incorporate the spatial context

information More complete representation of

color in an image than pixel color value. We use three moments to describe mean, variance and degree of asymmetry of a color distribution.

Color Moment Calculations

Advantages:

1. “Colors” themselves are chromatic distribution. Quantized color

descriptor ,which is based on distribution, can better represent color in images.

2. Instead using color means only, we also introduce two other moments.

3. Color moment can be used as the discriminative descriptor and is effective in

a general classification problem

Training StepsCalculate feature matrix based on Color MomentsCalculate every box rather than every pixelConcatenate feature matricesCalculate PCA (Principal Component Analysis)Calculate GMM (Gaussian Mixture Model) based on PCA resultsMultiply individual feature matrices by coefficient matrixUse GMM results to calculate Fisher VectorsTrain SVMs

IV. Pipeline:

Testing StepsCalculate feature matrix of each image, isolating the object first using binary

imagesUse PCA and GMM results from training data to calculate fisher vectorsApply Fisher Vector to each individual result to obtain vectors that are the same

sizeClassify images using SVM’s from training dataCalculate Precision

V. Results:

VI. Conclusions:

CIELAB: 42% Accuracy HSV: 45% Accuracy RGB: 50% Accuracy

Classification using Color Moment performance on Google and EBay Datasets

Classification using Color Moment vs. Dense SIFT on Birds 200 Dataset

Our Color Descriptor: Accuracy = 21.3592% (110/515)

Dense SIFT: Accuracy = 21.7476% (112/515)

Color Moment and Dense SIFT Combined: Accuracy: 25.44%

Our Color Descriptor showed the same accuracy as the Dense SIFT and therefore possesses the same discriminative ability

When both features were fused, accuracy increased by 4%. This indicates that color moment representation and shape representation, like SIFT, are complementary.

Future Work Complete Color Moment and Dense SIFT with all 200 classes Increase/ Decrease Block Size Incorporate Object Detection and Image Retrieval

Acknowledgments:

Thank you to the NSF for funding the REU program for the University of Central Florida.

Also, thanks to Dr. Shah and Dr. Lobo for overseeing the program.

Mean Standard Deviation Skew

𝐸𝑖=∑𝑁

𝑗=1

( 1𝑁 𝑃 𝑖𝑗)

Color Mapping Methods

Color Histogram Method

Hue

Ligh

tness