ROTATION INVARIANT SIMULTANEOUS CLUSTERING AND DICTIONAR Y LEARNING

Yi-Chen Chen1, Challa S. Sastry2, Vishal M. Patel1, P. Jonathon Phillips3, and Rama Chellappa1

1Department of Electrical and Computer EngineeringCenter for Automation Research, University of Maryland, College Park, MD

2Department of Mathematics, Indian Institute of TechnologyHyderabad, Hyderabad, India3National Institute of Standards and Technology, Gaithersburg, MD

{chenyc08,pvishalm,rama}@umiacs.umd.edu csastry@iith.ac.in jonathon.phillips@nist.gov

ABSTRACT

In this paper, we present an approach that simultaneously clustersdatabase members and learns dictionaries from the clusters. Themethod learns dictionaries in the Radon transform domain, whileclustering in the image domain. The main feature of the proposed ap-proach is that it provides rotation invariant clustering which is usefulin Content Based Image Retrieval (CBIR). We demonstrate throughexperimental results that the proposed rotation invariantclusteringprovides better retrieval performance than the standard Gabor-basedmethod that has similar objectives.

Index Terms— Radon transform, rotation invariance, cluster-ing, dictionary learning, CBIR.

1. INTRODUCTION

In recent years, sparse representation has emerged as a powerfultool for efficiently processing data in non-traditional ways. This ismainly due to the fact that signals and images of interest tend to en-joy the property of being sparse in some dictionary [1]. These dictio-naries are often learned directly from the training data. Ithas beenobserved that learning dictionaries directly from examples usuallyleads to improved results in many practical image processing appli-cations such as restoration and classification [2]. One of the sim-plest algorithms for learning dictionaries is the K-SVD algorithm[2]. Given a set of images{xi}

ni=1, K-SVD seeks a dictionaryD

that provides the sparsest representation for each image bysolvingthe following optimization problem

(D, Γ) = arg minD,Γ

‖X −DΓ‖2F subject to∀i ‖γi‖0 ≤ T0, (1)

whereγi represents theith column ofΓ, X is the matrix whosecolumns arexi andT0 is the sparsity parameter. Here,‖A‖F de-

notes the Frobenius norm defined as‖A‖F =√

∑

ijA2

ij . The K-

SVD algorithm alternates between sparse-coding and dictionary up-date steps. In the sparse-coding step,D is fixed and the representa-tion vectorsγis are found for each examplexi. Then, the dictionaryis updated atom-by-atom in an efficient way.

While these dictionaries are often trained to obtain good recon-struction, training dictionaries with a specific discriminative crite-rion has also been considered. For instance, linear discriminant anal-ysis (LDA) based basis selection and feature extraction algorithm forclassification using wavelet packets was proposed by Etemand andChellappa [3] in the late nineties. Recently, similar algorithms for

This work was partially supported by an ONR grant N00014-12-1-0124.

Fig. 1. Left: Kimia’s database. Right: Smithsonian isolated leafdatabase with rotated images.

simultaneous sparse signal representation and discrimination havealso been proposed [4], [5], [6] ,[7], [8], and [9]. Additional tech-niques may be found within these references.

Dictionary learning techniques for unsupervised clustering havealso gained some traction in recent years. In [10], a method for si-multaneously learning a set of dictionaries that optimallyrepresenteach cluster is proposed. To improve the accuracy of sparse cod-ing, this approach was later extended by adding a block incoherenceterm in the optimization problem [11]. Some of the other sparsitymotivated subspace clustering methods include [12], [13].

Rotation invariance is an important property in many applica-tions such as image classification and retrieval where one wants toclassify or retrieve images having same content but different orienta-tion. For instance, in content based image retrieval (CBIR), imagesare retrieved from a database using features that best describe theorientation of objects in the query image. Due to the abilityof Ga-bor filters to capture directional information, they are often used toextract features for retrieval. However, the chosen directions of Ga-bor filters may not correspond to the orientation of the content in thequery image. Hence, a feature extraction method that is independentof orientation in the image is desirable. Fig. 1 shows some sampleimages from the Kimia’s object dataset and Smithsonian leafdataset,where the presence of orientation is clearly seen in the images.

In this paper, we present a rotation invariant clustering method,extending the dictionary learning and sparse representation frame-work for clustering of data. Given a database of images{xj}

Nj=1

and the number of clustersK, we learnK dictionaries for repre-senting the data. The dictionaries are learnt in the Radon transform

domain which ensures that the clustering is rotation independent.We demonstrate the effectiveness of our approach in retrieval exper-iments, where significant improvements are shown.

The organization of the paper is as follows. Our rotation invari-ant clustering framework is detailed in Section 2. We demonstrateexperimental results in Section 3 and Section 4 concludes the paperwith a brief summary and discussion.

2. SIMULTANEOUS CLUSTERING AND DICTIONARYLEARNING

In this section, we present our rotation invariant clustering method.We first discuss how Radon transform is used to detect rotationpresent in an image.

2.1. Estimating the rotation present in images

The Radon transform of a sufficiently regular continuous domaintwo variable functionx is defined as

Rθx(t) =

∫ ∞

−∞

x(t cos θ − s sin θ, t sin θ + s cos θ)ds, (2)

where(t, θ) ∈ (−∞,∞) × [0, π). If x is the rotated copy ofx byan angleθ, then a simple proof shows that their Radon transformsare related as

Rθx(t) = Rθ+θx(t), ∀t, θ. (3)

The varianceσθ of the Radon transform is found [14] to be usefulin estimating the presence of angle in images. Therefore, given theimagex, one may estimateθ from the following formula

θ = arg minθ

(

d2σθ

dθ2

)

. (4)

An application of properties of the Fourier transforms implies thatθcan be estimated from the following simple formula

θ = arg minθ

(

d2

dθ2

∫ ∞

−∞

X2(s cos θ, s sin θ)ds

)

, (5)

whereX is the Fourier transform ofx. In all practical situations,for an image of sizem × n, the Radon transform is representedas a matrix [Rθl

f(tm)], called sinogram. Usually, one takesθl = (l+0.5)π

n, l = 0, 1, . . . , n − 1 and tm = δr(k − m

2), k =

0, 1, . . . , m− 1, whereδr is the radial sampling, which is chosen toavoid aliasing error in the reconstruction.

The second image shown on the first row of Fig. 21 is a rotatedcopy of the first image by30◦. The plots on the second row are thesecond derivatives of variances of Radon transform of them.It maybe noted that the difference between the points of global minima ofboth curves is 30, coinciding with the rotation present in the sec-ond image. Consequently, the estimate presented in (5) is useful inestimating the presence of rotation in the images.

2.2. Proposed algorithm

Let {xj}Nj=1 be the database of images andK be the number of

clusters. DefineD = [D1, . . . ,DK ] as the concatenation of dictio-naries corresponding toK clusters. LetCi be the matrix containingimages as columns corresponding to clusterCi. Equipped with theabove notation and motivated by dictionary learning methods [2], werealize our objective in two steps

1http://www.flowersofpictures.com/wp-content/uploads/2011/07/Lily-Flowers.jpg

0 50 100 150 200−6

−4

−2

0

2

4x 10

6

0 50 100 150 200−4

−3

−2

−1

0

1

2

3x 10

6

Fig. 2. For the rotated images present on the first row, the plots on

the second row are theird2σθ

dθ2 . The second row plots indicate thatthe difference between the points of global minimum of both curvespreserves the rotation present in the second image.

• Cluster assignment: We start with arbitrary dictionaryD =[D1, . . . ,DK ]. Given an imagexj and its estimated orien-tation θj which is calculated from the discretized version of(5), we useRθj

xj to denote the Radon transform matrix of

the rotated version of the imagexj by −θj . In other words,Rθj

xj is an aligned Radon transform matrix ofxj . It is ob-tained by left shifting columns of the original Radon trans-form matrix ofxj by θj . Our approach considers obtainingthe sparsest representation ofRθj

xj in an appropriate dictio-naryDi from

αj = arg min

ω‖Rθj

xj − Dω‖22 s. t. ‖ω‖0 ≤ T0,

i = arg mini

‖Rθjxj − Dδi(α

j)‖22 j = 1, · · · , N,

(6)

thenxj is set to belong toCi. In (6), δi is a characteristicfunction that selects the coefficients associated with theith

dictionary.

• Dictionary update: Having obtained the initial clustersC1, C2, . . . , CK , we update the dictionariesDi using theK-SVD approach described in the previous section. The newdictionaries are obtained by solving the following optimiza-tion problem

(Di, Γi) = arg minDi,Γi

‖Ci − DiΓi‖2F s.t∀i ‖γ i‖0 ≤ T0,

satisfyingCi = DiΓi, i = 1, 2, . . . , K.

We repeat the cluster assignment and dictionary update steps tillthere is no significant change inCi. It may be noted here that thedictionaries are learnt in the Radon domain, while clustering is donein the image domain. The clustering methodology presented abovemay be summarized in terms of the following optimization problem

minDi,Ci

K∑

i=1

∑

x∈Ci

minθ

{

‖Rθx − Dδi(α)‖22 + γ‖α‖1 + µ

∣

∣

∣

∣

d2σθ

dθ2

∣

∣

∣

∣

}

,

(7)whereγ, µ > 0. Here,σθ is the variance of the first column ofRθx.The last term in (7) estimates the presence of rotation in images.

We refer to our rotation invariant clustering and dictionary learningmethod as RICD.

2.3. Application to CBIR

Once the dictionaries have been learnt for each class in Radon do-main, given a query imagexq , we estimateθq and then projectRθq

xq onto the span of the atoms in eachDi using the orthogonalprojector

ProjDi= Di(D

Ti Di)

−1D

Ti . (8)

The approximation and residual vectors can then be calculated as

Ri

θqxq = ProjDi

(

Rθqxq

)

, (9)

and

ri(Rθq

xq) = Rθqxq − R

i

θqxq = (I− ProjDi

)Rθqxq, (10)

respectively, whereI is the identity matrix. Since the dictionarylearning step in our algorithm finds the dictionary,Di, that leadsto the best representation for each member ofCi in Radon domain,we suspect‖Rθq ,ixq‖2 to be small ifxq were to be relevant to the

ith cluster and large for the other clusters. Based on this, if

d = arg min1≤i≤K

‖ri(Rθq,ixq)‖2,

we search for the relevance ofxq in Cd.

3. EXPERIMENTAL RESULTS

To demonstrate the effectiveness of our method, we present exper-imental results on various datasets [15], [16],[17]. We evaluate theperformance of our method using precision-recall curves and aver-age retrieval performance [18]. Recall and precision are defined as

Recall =Number of relavant images retrieved

Total number of relevant images, (11)

Precision =Number of relavant images retrieved

Total number of images retrieved. (12)

Recall is the portion of total relevant images retrieved whereas pre-cision indicates the capability to retrieve relevant only images. Anideal retrieval should give precision rate which always equals 1 forany recall rate. Given a certain number of retrieved images,the av-erage retrieval performance is defined as the average numberof rel-evant retrieved images over all query images of a particularclass.We compare the performance of our method with that of modifiedGabor-based approach [18] and [10]. Note that [18] uses featuresthat are rotation invariant and the method presented in [10]uses adiscriminative dictionary learning approach to clustering. We referto the method presented in [10] as dictionary-based clustering (DC).In all the experiments, the dictionaries are initialized byrandomlypartitioning the Radon transformed data intoK subsets.

3.1. Kimia database

This database consists of 216 images of 18 shapes. Each shapehas 12 different images. Fig. 1(a) shows sample images from thisdatabase. The images were resized to100 × 80. The precision-recall curves are shown in Fig. 3(a) for 18 query images belongingto different classes. Our Radon transform-based approach is able tomaintain precision rate above94% even when the recall rate reaches

90%. The modified Gabor-based method’s precision goes down to50% at approximately half recall rate. It is clearly seen from thefigure that the performance of our method is better than the othermethods. Fig. 3(b) shows 18 clusters obtained using our method. Afew atoms from the learned dictionaries from each shape are shownin Fig. 3(c).

0 20 40 60 80 10050

60

70

80

90

100

Recall (%)

Pre

cisi

on (

%)

Radon Transform with dictionaryGrayscale with dictionaryMod. Gabor

(a) (b) (c)Fig. 3. Results on Kimia’s database. (a) Precision-recall graphs. (b)Clustering results. (c) Associated learned dictionaries.

The original Kimia’s dataset contains various shapes with smallrotation. In order to create a more challenging dataset, we selectedone representative image from each of the 18 shapes. For eachse-lected shape, we created 11 in-plane rotated images with thefollow-ing angles:

18◦, 36◦

, 54◦, 72◦

, 90◦, 108◦

, 126◦, 144◦

, 162◦, 180◦

, 198◦.

The resulting dataset (shown in Fig. 4(a)) is more challengingthan the original Kimia’s dataset as it contains in-plane rotated im-ages with various angles. The precision-recall curves are shown inFig. 4(b) for 18 query images belonging to different shapes.Fig. 4(c)shows the total average retrieval performance over all shapes. Onaverage our method obtained 4.4907 out of 8 retrieved imagespershape. Whereas the DC method and Gabor-based method obtained2.0926 and 1.2778, respectively. As can be seen from the figures,our method performed the best.

0 10 20 30 40 50 600

20

40

60

80

100

Recall (%)

Pre

cisi

on (

%)

Radon Transform with dictionaryGrayscale with dictionaryMod. Gabor

2 4 6 8 10 12 14 16 180

1

2

3

4

5

6

7

8

classes of images

aver

age

num

ber

of c

orre

ct r

etrie

val i

mag

es

Average retrieval performances for Kimia database

Radon Transform with dictionaryRadon Transform with dictionary (total avg.)Grayscale with dictionaryGrayscale with dictionary (total avg.)Mod. GaborMod. Gabor (total avg.)

(a) (b) (c)

Fig. 4. Results on in-plane rotated Kimia’s database. (a) Database.(b) Precision-recall curves. (c) Average retrieval performance.

3.2. Smithsonian isolated leaf database

The original Smithsonian isolated leaf database consists of 93 differ-ent leaves [16]. We selected one representative image from each ofthe last 18 leaves. As before, for each representative image, we cre-ated 11 in-plane rotated images with the same angles as consideredin the previous experiment. Fig. 1(b) shows the resulting databasecontaining 18 different leaves with rotated images. The images wereresized to100 × 80. This database is more challenging than the

Kimia database as more shape similarities can be easily found amongdifferent leaves (for instance, 5th and 6th rows; 9th and 10th rows;11th and 13th rows in Fig. 1(b).) Fig. 5(a) shows the performance ofdifferent methods on this database. Fig. 5(b) shows the total averageretrieval performance over all shapes. On average our method ob-tained 5.7083 out of 8 retrieved images per shape. Whereas the DCmethod and Gabor-based method obtained 2.9630 and 2.5324, re-spectively. Even on this difficult dataset, our method performs betterthan the other methods.

0 20 40 60 8030

40

50

60

70

80

90

100

Recall (%)

Pre

cisi

on (

%)

RICDDCMod. Gabor

2 4 6 8 10 12 14 16 180

1

2

3

4

5

6

7

8

classes of images

aver

age

num

ber

of c

orre

ct r

etrie

val i

mag

esAverage retrieval performances for (in−plane rotated) Smithsonian database

Radon Transform with dictionaryRadon Transform with dictionary (total avg.)Grayscale with dictionaryGrayscale with dictionary (total avg.)Mod. GaborMod. Gabor (total avg.)

(a) (b)Fig. 5. Experiment with in-plane rotated Smithsonian isolated leafdatabase. (a) Precision-recall graphs. (b) Average retrieval perfor-mance.

3.3. PIE database

Even though, our method can provide rotation invariant clustering, itcan be generalized to handle other variations present in thedata byusing appropriate features. To illustrate this, we consider the frontalface images of the CMU PIE dataset. Our objective is to show the ef-fectiveness of our approach in retrieving face images in thepresenceof various illumination conditions. We use the principle componentanalysis (PCA) features for this experiment. This databaseconsistsof 68 subjects, each of which contains 21 images under various il-lumination conditions. Eighteen subjects from this dataset are usedfor this experiment. All images are resized to48× 40. Fig. 6 showsthe retrieval results of our method. The top row shows 18 queryimages from 18 different subjects. For each query image, thecor-responding column shows the first six retrieved images. The falseretrievals are marked with red boxes. As can be seen from thisfig-ure, there is no false retrieval from the first rank up to the third rankmatches. In this experiment, the average retrieval performance is(1 − 8

108) × 6 = 5.5556 out of 6 images.

Fig. 6. Retrieval results on the PIE database.

4. CONCLUSION

In this paper, we have proposed a rotation invariant clustering al-gorithm suitable for such applications as content based image re-

trieval. With a view to achieving rotation invariance in clustering, themethod learns dictionaries in the Radon transform domain though,the simultaneous clustering is done in the image domain. We demon-strated the effectiveness of the proposed method for CBIR applica-tions.

5. REFERENCES

[1] P. J. Phillips, “Matching pursuit filters applied to faceidentification,”IEEE Trans. Image Process., vol. 7, no. 8, pp. 150–164, 1998.

[2] M. Aharon, M. Elad, and A. M. Bruckstein, “The k-svd: an algorithmfor designing of overcomplete dictionaries for sparse representation,”IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311–4322, 2006.

[3] K. Etemand and R. Chellappa, “Separability-based multiscale basis se-lection and feature extraction for signal and image classification,” IEEETransactions on Image Processing, vol. 7, no. 10, pp. 1453–1465, Oct.1998.

[4] F. Rodriguez and G. Sapiro, “Sparse representations forimage clas-sification: Learning discriminative and reconstructive non-parametricdictionaries,”Tech. Report, University of Minnesota, Dec. 2007.

[5] K. Huang and S. Aviyente, “Sparse representation for signal classifica-tion,” NIPS, vol. 19, pp. 609–616, 2007.

[6] Q. Zhang and B. Li, “Discriminative k-svd for dictionarylearningin face recognition,”Proc. IEEE Conf. Computer Vision and PatternRecognition, pp. 2691–2698, 2010.

[7] M. Ranzato, F. Haung, Y. Boureau, and Y. LeCun, “Unsupervised learn-ing of invariant feature hierarchies with applications to object recogni-tion,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.1–8, 2007.

[8] J. Mairal, F. Bach, J. Pnce, G. Sapiro, and A. Zisserman, “Discrimi-native learned dictionaries for local image analysis,”Proc. of the Con-ference on Computer Vision and Pattern Recognition, Anchorage, AL,June 2008.

[9] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman,“Superviseddictionary learning,”Advances in Neural Information Processing Sys-tems, Vancouver, B.C., Canada, Dec. 2008.

[10] P. Sprechmann and G. Sapiro, “Dictionary learning and sparse codingfor unsupervised clustering,” inICASSP, march 2010, pp. 2042 –2045.

[11] I. Ramirez, P. Sprechmann, and G. Sapiro, “Classification and clus-tering via dictionary learning with structured incoherence and sharedfeatures,” inCVPR, june 2010, pp. 3501 –3508.

[12] E. Elhamifar and R. Vidal, “Sparse subspace clustering,” in CVPR, june2009, pp. 2790 –2797.

[13] S. Rao, R. Tron, R. Vidal, and Y. Ma, “Motion segmentation via ro-bust subspace separation in the presence of outlying, incomplete, orcorrupted trajectories,” inCVPR, june 2008, pp. 1 –8.

[14] K. Jafari-Khouzani and H. Soltanian-Zadeh, “Radon transform orien-tation estimation for rotation invariant texture analysis,” IEEE Trans.Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 1004–1008, 2005.

[15] T. B. Sebastian, P. N. Klein, and B. B. Kimia, “Recognition of shapes byediting their shock graphs,”IEEE Trans. Pattern Analysis and MachineIntelligence, vol. 26, no. 5, pp. 550–571, 2004.

[16] H. Ling and D. W. Jacobs, “Shape classification using theinner-distance,” IEEE Trans. Pattern Analysis and Machine Intelligence,vol. 29, no. 2, pp. 286–299, 2007.

[17] S. Baker, “The cmu pose, illumination, and expression (PIE) database(http://www.ri.cmu.edu/projects/project418.html).”

[18] C. S. Sastry, M. Ravindranath, A. K. Pujari, and B. Deekshatulu, “Amodified Gabor function for content based image retrieval,”PatternRecognition Letters, pp. 293–300, 2007.