SPECIAL ISSUE

CFA local binary patterns for fast illuminant-invariant colortexture classification

Olivier Losson • Ludovic Macaire

Received: 3 September 2012 / Accepted: 10 November 2012

� Springer-Verlag Berlin Heidelberg 2012

Abstract This paper focuses on the classification of color

textures acquired by single-sensor color cameras under

various illuminants. Local binary patterns (LBPs) are

robust texture descriptors suited to such conditions. This

property is still improved when LBPs are computed from

the level ranks. Our main contribution is to avoid the de-

mosaicing step that is classically performed in single-sen-

sor color cameras to estimate color images from raw data.

We instead compute rank-based LBPs from the color filter

array image, in which each pixel is associated to a single

color component. Experimental results achieved on a

benchmark color texture database show the effectiveness of

the proposed approach for texture classification, and a

complexity study highlights its computational efficiency.

Keywords Local binary patterns � Color texture

classification � Illuminant invariance � Bayer color filter

array

1 Introduction

Texture classification is a particular problem of image

retrieval, closely related to appearance-based object rec-

ognition. It consists in retrieving, among a set of texture

target images grouped into classes, those that represent the

same texture as a given test image. The target images are

then ranked with respect to their similarity to the test

image, which allows to determine the class of texture to

which the test image belongs. The similarity between the

test and target images is evaluated thanks to a given sim-

ilarity measure between their texture features. These fea-

tures have to be as rich as possible to provide a high

classification accuracy. On the other hand, they have to be

as simple as possible to be computed in real time.

Palm [23] shows that the texture discrimination quality

provided by the sole analysis of luminance texture features

is improved by taking into account both the spatial

arrangement of the colors in the image plane and their

distribution in a color space. However, since color infor-

mation is represented at each pixel by three color compo-

nents (red, green, and blue), the processing time required to

extract color texture features is at least three times longer

than that required to compute luminance features. More-

over, the color formation step should be taken into account

to compare the computation times of features from gray-

level images and from color images.

Indeed, most digital color cameras are fitted with a

single sensor covered by a color filter array (CFA) which

consists of a mosaic of spectrally selective filters. Each

sensor element then samples only one of the three color

components. It provides the corresponding CFA image

pixel, where two components are missing and must be

determined to estimate this pixel’s color. This process is

commonly referred to as CFA demosaicing, and its result

as the demosaiced image. It is either performed by an

embedded electronic device or by an external software

from the raw image file.

Demosaicing methods are designed to produce ‘‘per-

ceptually pleasant’’ color images. They often use filtering

to avoid the apparition of color artifacts, and the most

efficient methods rely on the frequency domain [14].

O. Losson (&) � L. Macaire

Laboratoire LAGIS, UMR CNRS 8219, Universite

Lille 1 - Sciences et Technologies Cite Scientifique,

Batiment P2, 59655 Villeneuve d’Ascq, France

e-mail: olivier.losson@univ-lille1.fr

L. Macaire

e-mail: ludovic.macaire@univ-lille1.fr

123

J Real-Time Image Proc

DOI 10.1007/s11554-012-0302-5

Therefore, they tend to alter the local texture information

that is, however, useful to discriminate texture images.

Besides, the demosaicing step requires a computation time

that is far from negligible [15]. For these reasons, it is

relevant to consider how to avoid it in any classification

scheme of color texture images. We propose to compute

texture descriptors directly from CFA images.

Among the many texture descriptors proposed in the

literature, local binary patterns (LBPs) and their extensions

exhibit specific attractive properties [25]. They provide a

unified approach of statistical and structural characteristics

to describe textures. They result from a simple local

thresholding and their histograms are one of the most

computationally efficient features. They are able to repre-

sent textures of different granularities and are fairly robust

to illumination intensity variations. These properties make

LBP-based descriptors well suited to many industrial

applications [4, 27] and to face recognition [3, 9].

From the basic original LBP operator proposed by Ojala

et al. [20], many enhancements have been derived.

Pietikainen et al. [27] and Zhao [34] have recently sur-

veyed the various LBP-based descriptors and their appli-

cations in computer vision. Among others, the properties of

rotation invariance [6, 22] and illumination invariance

[12, 24] have been widely investigated. As a generalization

of the LBP, Tan and Triggs [33] propose local ternary patterns

(LTPs) that are less sensitive to noise in uniform regions.

However, since one LTP can be decomposed into two LBPs,

the LTP histogram is twice larger than that of LBPs.

Maenpaa and Pietikainen [16] extend LBPs to color

image analysis. They consider a joint color–texture

descriptor but conclude that handling color and texture

separately yields better results. A common approach is then

to apply LBPs to each color component image indepen-

dently, which provides three LBP histograms. To reduce

the number of LBP values, Porebski et al. [28] evaluate the

Euclidean distance between the central pixel’s color and its

neighbors’ color in the RGB color space.

These two papers show that color LBPs only provide

satisfying classification results when lighting conditions are

controlled. However, the color of pixels strongly depends

on that of the ambient light (i.e., the illuminant) used

during the image acquisition. Illuminant invariance is one

of the most important properties required by efficient

descriptors for object recognition [31]. Some LBP variants

may be worth being considered as color texture descriptors

(CTDs) in such conditions. Zhu et al. [35] assess the per-

formance of six of them for the multi-scale recognition of

visual object classes. They recommend to use a combina-

tion of the Hue-, Opponent-, and Normalized Opponent-

LBPs, respectively, extracted from the hue channel of the

HSV color space, from the three channels of the opponent

color space, and from the two channels of the normalized

opponent space. Unfortunately, this requires three times

more bins than classical LBPs.

The rest of the paper is organized as follows. In Sect. 2,

we present color LBPs and focus on their robustness

against illuminant variations. In Sect. 3, we experimentally

show that demosaicing affects LBP values of images

acquired by single-sensor color cameras under different

illuminants. This may degrade the results of LBP-based

texture classification. To achieve color texture classifica-

tion with no demosaicing step, we propose a LBP

descriptor that is directly computed from the CFA images,

which has never been proposed to our knowledge. Here-

after referred to as the CFA LBP, this new descriptor relies

on the assumption that CFA images are relevant functions

to describe the observed textures. To assess the efficiency

of color texture classification using CFA LBPs, Sect. 4

presents experimental results obtained with a benchmark

database of texture images acquired under different illu-

minants. This section evaluates the performance reached by

our approach in terms of both classification accuracy and

computation efficiency.

2 LBPs and illuminant invariance

In this section, we first introduce how LBPs are computed

and how their histograms are used to compare two color

texture images. Second, we show that these texture

descriptors are only coarsely invariant to illuminant chan-

ges. We then propose to compute LBPs from the ranks of

the levels rather than levels directly, and we assess the

resulting improvement in terms of robustness to illuminant.

2.1 LBPs of a color image

2.1.1 LBP definition

A full-color image I is composed of three color component

images Ik; k 2 fR;G;Bg: In each of them, a given pixel P

is characterized by the color component level Ik(P)

classically quantized with 256 levels for a 24-bit color

image. A three-component vector defined as IðPÞ ¼ðIRðPÞ; IGðPÞ; IBðPÞÞT is, therefore, associated with each

pixel P. The color component levels may be (sub-)quan-

tized with q \ 256 levels to provide a quantized color

image Iq in which each pixel P is characterized by three

quantized component levels Ik,q(P).

To characterize a color texture image Iq; we extract

LBPs from each color component image. The LBPs capture

the spatial interactions of the color component levels that

characterize the pixel P and the nearby pixels Q within a

J Real-Time Image Proc

123

neighborhood N d in the color component image. They are

defined as:

LBPt½Ik;q�ðPÞ ¼X

Q2N dðPÞ2f ðQÞ � sðIk;qðPÞ � Ik;qðQÞ; tÞ; ð1Þ

where sð�Þ is the unit step function defined as:

sðx; tÞ ¼ 1 if x� t0 otherwise

�ð2Þ

When all the neighbors Q 2 N dðPÞ are considered, sð�Þdefines a binary code. f(Q) is the number associated with

each neighbor Q. Since this number ranges between 0 and

jN dj � 1; the LBP values range between 0 and b - 1,

where b ¼ 2jN d j and jXj indicates the cardinal of the set X.

In this paper, we use a 8-pixel neighborhood; LBP values

then range between 0 and 255. Specifically, N d is made of

the 8 pixels located at uniform distance (also called

infinity-norm distance) d along the four main directions of

the image plane (horizontal and vertical directions, and the

two diagonals). This neighborhood can be expressed as:

N dðPÞ ¼ Q 2 IjkPQk1 ¼ d ^ h modp4¼ 0

n o; ð3Þ

where I is the image that contains P and h denotes the angle

between the horizontal unit vector and the vector PQ

defined by the pixels P and Q. The parameter d is adjusted

by the user to fit the texture granularity at best.

The original version of LBPs proposed by Ojala et al.

[20] simply compares the level of the central pixel to those

of its neighbors, which corresponds to t = 0 in Eq. (1). To

make LBPs robust against small local variations of levels,

Heikkila and Pietikainen [8] propose to threshold the dif-

ference between the levels Ik,q(P) and Ik,q(Q). The param-

eter t has to be adjusted so that LBPs are invariant against

local illumination variations, but should have small values

to keep the texture discrimination power of LBPs.

Figure 1 shows a numerical example of LBP computation

on a simple G component image whose levels range from 0 to

5 to ensure easy readability. The LBP value is computed at

the central pixel for d = 2 when t ranges from 0 to 3.

2.1.2 Texture similarity based on LBP histograms

For a given neighborhood N d and a given threshold t, the

local relationships between levels around a pixel P within

any of the three color component images are represented by

LBPt½Ik;q�ðPÞ; k 2 fR;G;Bg: Each color component image

is characterized by the b-bin normalized histogram of its

LBP values, defined as:1

h LBPt½Ik;q�� �

: ½0; b� 1� ! ½0; 1�

i!P 2 Ik;q LBPt½Ik;q�ðPÞ ¼ i

�� �� ����Ik;q�� �� .

ð4Þ

So, the quantized color image Iq is described by three

texture features, namely its three normalized LBP histo-

grams h½LBPt½Ik;q��; k 2 fR;G;Bg:The comparison of two color texture images generally

relies on their texture features. In our case, their similarity is

defined by a bin-wise comparison of their normalized LBP

histograms. Several measures have been proposed to com-

pare probability density functions such as histograms [30].

For our experiments, we use the intersection between his-

tograms [32] which is one of the simplest and fastest simi-

larity measures (we also considered the Jeffrey divergence

[30] but this measure did not improve texture classification

results significantly). The similarity between two color tex-

ture images I1 and I2 is then defined as the mean of the

pairwise intersections between their LBP histograms:

SIM I1;I2ð Þ¼1

3

X

k¼R;G;B

Inter h LBPt½Ik;q1 �

h i;

�h LBPt½Ik;q

2 �h i

;

ð5Þ

where the intersection of two histograms is expressed as:

Inter ðh1; h2Þ ¼Xb�1

i¼0

min ðh1ðiÞ; h2ðiÞÞ: ð6Þ

When the two images I1 and I2 share a similar spatial

arrangement of colors, their similarity measure value (SIM)

(a) (b)

(c) (d)

Fig. 1 LBP computation of a color component image. At the central

pixel P of IG,6 (a), for Q 2 N 2ðPÞ; the binary codes obtained are b for

t = 0 and c for t = 1..3. Considering the neighbor numbers (d), LBP

values are then LBP0[IG,6](P) =P

i=07 2i = 255 and LBPt[I

G,6](P) =

20 ? 23 ? 24 ? 26 ? 27 = 217 for t = 1…3

1 This definition ignores border effects for readability sake. Only

pixels whose neighborhood N d is fully enclosed in the image are

actually taken into account to compute the LBP histogram.

J Real-Time Image Proc

123

is close to 1. Although it does not necessarily mean that the

two images contain the same texture, we assume so. On the

other hand, a similarity measure value close to 0 means that

the two textures are significantly different.

2.2 Illuminant invariance

LBPs are by definition invariant against any monotonic

transformation of the gray levels over the entire image (i.e.,

the overall luminance) [22]. Beyond this restrictive case, it

is interesting to experimentally investigate the robustness

of LBPs against more realistic illumination changes, like

illuminant variations which correspond to changes in the

light color temperature.

For this purpose, we use the Outex texture database [21]

that contains a large number of textured surfaces acquired

by a three-CCD color camera under controlled conditions,

and is widely referred to as a benchmark. Specifically, we

use the Outex-TC-00014 classification test suite that

consists of 2,040 texture images of size 128 9 128 pixels

acquired under three very different illuminants. This data

set contains 680 images (called here the ‘‘2,856-K subset’’)

that represent textured surfaces acquired under a 2,856-K

incandescent CIE A light. It also contains 680 images

(‘‘2,300-K subset’’) of the same textures but illuminated by

a 2,300-K horizon sunlight, and 680 images (‘‘4,000-K

subset’’) of still the same textures but illuminated by a

4,000-K fluorescent TL84 light (see examples in Fig. 2).

From this data set, we form three pairs of subsets (2,856 K,

2,300 K), (2,856 K, 4,000 K), and (2,300 K, 4,000 K), then

we pairwise compare each of their images. Their similarity

(see Eq. 5) is expected to be all the closer to 1 than LBPs

are robust to illuminant variations.

Table 1a displays the LBP similarity between two

images that represent the same texture under two different

illuminants, for four values of the quantization level q and

of the threshold t, the distance parameter d being set to 2.

These values are averaged over the 3 subsets and the 680

image pairs in each subset, and are highly representative of

the results because all standard deviations range from

0:37� 10�3 to 2:31� 10�3 (hence not shown). The uni-

form and median-cut [7] quantization schemes have been

used to quantize the color component levels with

q = 8, 16, 32, and 64 levels. The threshold t used to

compute LBPs ranges between 0 and 3 to have a brief

overview of its influence [8].

This table mainly shows that the mean similarity

between two images of the same texture under different

illuminants never reaches 1, the highest value being 0.92.

These results confirm that LBPs are only coarsely invariant

against general photometric variations like light color

changes and shifts. Table 1a also shows that the median-

cut quantization provides higher similarity values than the

uniform scheme. These experiments confirm that the

quantization scheme impacts the LBP-based similarity

whatever the used threshold value. But no tested quanti-

zation scheme leads to a strict invariance of LBPs against

illuminant changes.

2.3 LBPs of level ranks

Finlayson et al. [5] show that the ranks of levels within

each color component image are modified neither by illu-

mination intensity nor illuminant changes under the diag-

onal model assumption. To improve the robustness of

LBPs against illuminant invariance, we propose to compute

them from the ranks of the color component levels rather

than directly from these levels. However, Muselet and

Macaire [19] show that these ranks are themselves not

strictly invariant to illumination changes and that their

robustness is improved by sub-quantizing the color com-

ponent levels. As well as computation considerations, this

is another motivation to quantize the color components

with q levels.

To each quantized color component image Ik;q; k 2fR;G;Bg; we associate a rank map ~Ik;q defined at each

pixel P as:

~Ik;qðPÞ ¼XIk;qðPÞ

l¼0

1� dðh½Ik;q�ðlÞÞ; ð7Þ

where dð�Þ is the Kronecker delta:

dðxÞ ¼ 1 if x ¼ 0;0 x [ 0:

�ð8Þ

The rank map, therefore, indicates the rank of each pixel’s

level in the ascending-ordered list of all the levels in the

quantized color component image. In ~Ik;q; the rank 1 is

associated to the pixel(s) of minimal level in Ik,q, and the

maximal rank (which equals the number of different levels

existing in Ik,q) is associated to the pixel(s) of maximal

level.

Following the classical LBP approach, we then thresh-

old the rank differences in a neighborhood of the rank map

to compute its LBPs. Let ~t be the rank threshold. LBPs of

level ranks LBP~t½~Ik;q� are computed by analogy with

Eq. (1). Figure 3a shows a simple numerical example of

the computation of a level rank map from the same com-

ponent image IG,6 as in Fig. 1. It details how the histogram

of IG,6 (unnormalized here for readability sake) allows us to

compute the rank map ~IG;6: Note that the rank map differs

from the original component image if and only if the his-

togram contains at least one empty bin. Figure 3b and c

detail how LBPs are computed from this rank map. Note

J Real-Time Image Proc

123

that the binary codes (hence the LBP values) at the central

pixel of IG,6 and ~IG;6 are the same when t = 1 or 2 (see

Figs. 1c, 3b), but differ when t = 3 (see Figs. 1c, 3c).

Because the rank difference is always lower than the level

difference for any pair of pixels P and Q, the rank

threshold is of greater effect than the level threshold and

should have still smaller values to preserve the discrimi-

nation power of LBPs.

At last, to compare two color images I1 and I2; we

redefine the similarity measure of Eq. (5) using the LBPs

computed from the rank map:

SIM I1; I2ð Þ ¼ 1

3

X

k¼R;G;B

Interðh½LBP~t½~Ik;q1 ��; h½LBP~t½~Ik;q

2 ��Þ:

ð9Þ

To assess the improvement brought by the rank

approach for LBP invariance against illuminant changes,

we use this new definition of similarity to compare two

images of the same texture under the previous Outex-

TC-00014 illuminant pairs (2,856 K, 2,300 K), (2,856 K,

4,000 K), and (2,300 K, 4,000 K). Table 1b shows the

mean similarity values obtained with rank levels. By

comparing Table 1a and b, we notice that the similarity is

improved for significant thresholds (t; ~t� 2), whatever the

quantization scheme applied to the color component levels.

With a fine quantization (32 levels or more), the rank

approach brings little or no improvement to LBP-based

similarity because rank values are very close to the

corresponding levels (i.e., rank maps are almost identical

to color component images). We conclude that taking into

account the ranks of the color component levels rather than

the color component levels themselves improves the

robustness of LBPs against illuminant changes, especially

with coarse level quantizations.

But let us point out that this study only evaluates the

robustness of LBPs against illuminant changes regardless

of their power to discriminate textures.

2.4 Selection of a quantization scheme

for classification

We here intend to select the quantization scheme that

should be retained for further experiments. For this

purpose, we assess the classification performance of the

Outex-TC-00014 data set. The original color images I

are quantized to provide Iq: Then, we compute the rank-

based LBP histograms and perform the classification.

The goal of texture classification is to retrieve the most

similar target images to an input test image. To classify

each of the Ntest test images, the Ntar target images are

sorted with respect to their similarity to the test image.

Each test image is then assigned to a class thanks to the one

nearest neighbor (1-NN) classifier [23, 16].

Fig. 2 Examples of Outex-TC-00014 images of four textures (from left to right canvas, tile, wood, barley rice) acquired under three different

illuminants (from top to bottom 2,856 K incandescent light, 2,300 K horizon sunlight, 4,000 K fluorescent TL84 light)

J Real-Time Image Proc

123

The 2,040 images of the Outex-TC-00014 data set

are grouped into Nclass = 68 texture classes. Each class is

represented by 30 images: Nprot = 10 prototypes acquired

under a 2,856 K incandescent CIE A light, and 20 test

images acquired under two different illuminants (i.e., 10

under a 2,300-K horizon sunlight and 10 under a 4,000-K

fluorescent TL84 light). All the class prototypes form the

subset of Ntar ¼ Nprot � Nclass ¼ 680 target images and

there are Ntest = 1,360 test images [16, 26].

Table 2 displays the classification results obtained with

different parameter values (the distance parameter d being

set to 2) when either the uniform or the median-cut quan-

tization scheme is used. It shows that the median-cut

should be used rather than the uniform scheme to quantize

the considered image data set in a classification context.

This was already noticed in the context of illuminant

invariance alone (see Table 1) but holds true when the

power of LBPs to discriminate textures is considered.

Unlike uniform quantization, the median-cut scheme

provides an image-dependent quantization (the same color

component level in different images may be quantized

differently), hence an image-dependent meaning of LBP

codes (the same neighborhood values in different images

may provide different LBP codes). The results in Table 2

show that the dependency of the quantization scheme to the

image content is not a key issue in case of illuminant

variations.

3 CFA local binary patterns

Studies about texture classification assume that the color

information is well represented by the color images, and do

not consider the impact of the acquisition device on the

texture characterization quality. Yet, for cost reasons, the

most widespread color cameras are equipped with one

(a)

(b) (c)

Fig. 3 Rank map and LBP computation from a color component

image. At the central pixel P of ~IG;6; the binary codes obtained are bfor ~t ¼ 1 or 2, and c for ~t ¼ 3: Considering the neighbor numbers of

Fig. 1d, LBP values are then LBP~t½~IG;6�ðPÞ ¼ 217 for ~t ¼ 1 or 2, and

LBP3½~IG;6�ðPÞ ¼ 8: Note that LBP0½~IG;6�ðPÞ ¼ LBP0½IG;6�ðPÞ ¼ 255

(see Fig. 1b)

Table 1 Mean LBP similarity of the Outex-TC-00014 data set

achieved by (a) the classical (see Sect. 2.2) and (b) the rank-based

(see Sect. 2.3) approaches on images representing the same texture

under different illuminants

Parameters Similarity

t q Uniform Median-cut

(a) Using LBPt [Ik,q]

0 8 0.82 0.92

16 0.83 0.92

32 0.86 0.92

64 0.89 0.92

1 8 0.83 0.92

16 0.84 0.92

32 0.86 0.92

64 0.89 0.92

2 8 0.83 0.92

16 0.84 0.91

32 0.86 0.91

64 0.89 0.91

3 8 0.83 0.90

16 0.84 0.89

32 0.86 0.89

64 0.89 0.89

Parameters Similarity

~t q Uniform Median-cut

(b) Using LBP~t½~Ik;q�0 8 0.82 0.92

16 0.83 0.92

32 0.86 0.92

64 0.89 0.92

1 8 0.83 0.92

16 0.84 0.92

32 0.86 0.92

64 0.89 0.92

2 8 0.97 0.93

16 0.92 0.92

32 0.88 0.91

64 0.86 0.91

3 8 0.99 0.94

16 0.96 0.92

32 0.91 0.91

64 0.86 0.89

J Real-Time Image Proc

123

single sensor covered with a color filter array [15]. These

devices acquire CFA images (also known as raw images)

from which the final color images must be estimated by

demosaicing, as described generically in Sect. 3.1. Such

color estimation may modify LBPs in comparison with

those extracted from images acquired by three-sensor

cameras, as shown in Sect. 3.2. This lets us expect that the

quality of these texture descriptors (and hence their clas-

sification performance) are degraded by demosaicing. To

avoid this, we propose to compute LBPs directly from CFA

images without any color estimation, as explained in Sect.

3.3. This strategy saves the processing time required by the

demosaicing step.

3.1 CFA image and demosaicing

In a CFA image ICFA, a single color component is associ-

ated with each pixel. Let ICFA be of size X 9 Y pixels,

hence defined on the 2-D regular lattice K which is the

finite subset of Z2 bounded by the least element (0, 0) and

the greatest element (X - 1,Y - 1). The CFA image

delivered by the widely-used Bayer CFA of Fig. 4a con-

tains 12

XY pixels associated with the G component, arran-

ged in a quincunx lattice, and 14

XY pixels associated with

the R or B component, arranged in a regular lattice. Each

color component k in ICFA is then defined on its own

sublattice Kk � K :

KR ¼ ðx; yÞ 2 K x is odd and y is evenj gf ; ð10aÞ

KG ¼ ðx; yÞ 2 K x and y are of same parityj gf ; ð10bÞ

KB ¼ ðx; yÞ 2 K x is even and y is oddj g:f ð10cÞLet ICFA

k be the image defined on the lattice Kk and

whose levels are those of ICFA according to the same

lattice (see Fig. 4b–d). The original CFA image ICFA can

be interpreted as the union of the three component images

ICFAk , k = R, G, B. To determine the color of each pixel P

in the demosaiced color image I; the demosaicing process

generally retains the color component available at the

same location in ICFA, and estimates the other two

components:

IðPÞ ¼ðICFAðPÞ; IGðPÞ; IBðPÞÞ if P 2 IR

CFA;ðIRðPÞ; ICFAðPÞ; IBðPÞÞ if P 2 IG

CFA;ðIRðPÞ; IGðPÞ; ICFAðPÞÞ if P 2 IB

CFA:

8<

: ð11Þ

Each color component triplet in Eq. (11) represents an

estimated color. Out of the three components at IðPÞ; the

one denoted by ICFA(P) is available at pixel P in ICFA, and

the other two among IRðPÞ; IGðPÞ; and IBðPÞ are estimated

by demosaicing because they are unavailable in ICFA.

Many demosaicing methods exist and are often time

consuming. The reader may refer to [18] for a recent

review.

(a) (b)

(c) (d)

Fig. 4 CFA image ICFA and component images ICFAk from the Bayer

color filter array. Each pixel is artificially colored with the main

spectral sensitivity of the corresponding selective filter. The notations

Rx,y, Gx,y, and Bx,y express that the respective color component is

available at the pixel of spatial coordinates (x, y)

Table 2 Correct LBP-based classification rates (%) of Outex-TC-00014 images according to the quantization scheme

~t q Uniform Median-

cut

0 8 16.10 68.31

16 33.16 67.72

32 47.35 66.91

64 60.59 65.81

1 8 20.22 69.93

16 33.53 67.87

32 47.65 67.06

64 60.96 66.54

2 8 8.46 65.88

16 12.87 69.19

32 22.65 67.87

64 38.60 65.59

3 8 3.60 60.59

16 7.35 66.69

32 13.01 63.82

64 25.81 60.00

J Real-Time Image Proc

123

3.2 LBP fidelity of demosaiced images

The evaluation of the demosaicing performance most often

uses an original full-color image I: The CFA image ICFA is

obtained by sampling I according to the Bayer CFA mosaic

of Fig. 4a. A color image I is estimated from ICFA by a

given demosaicing method, whose performance is then

evaluated as a fidelity measure between I and I: Classical

fidelity measures like the Peak Signal-to-Noise Ratio

(PSNR) are based on a pixel-wise color comparison

between I and I: Therefore, they may poorly express how

demosaicing affects texture descriptors that provide infor-

mation about the relationships between colors of neigh-

boring pixels. To quantify how demosaicing specifically

degrades the LBPs, we rather propose to evaluate the LBP-

based similarity of I and I:

To achieve this comparison, we still use the Outex-

TC-00014 texture data set as original color images. Their

component images are quantized with q levels by the

median-cut scheme to obtain Ik,q. The rank map ~Ik;q is

deduced from Ik,q and LBPs of level ranks LBP~t½~Ik;q� are

computed as described in Sect. 2.3. Each original color

image I is also sampled to ‘‘simulate’’ the CFA image ICFA.

To obtain the demosaiced color image I; ICFA is processed

by the highly efficient demosaicing method of Lian et al.

[11] that provides a very good compromise between

fidelity results and computation complexity [14]. Then, the

same scheme of quantization, rank mapping, and LBP

computation as for I is applied to I to obtain LBP~t½~Ik;q�:

Finally, we compute the similarity SIMðI; IÞ between the

original image and its demosaiced counterpart from their

rank-based LBP histograms according to Eq. (9).

Table 3a displays the mean similarity values obtained

on the 2,040 original color images of the Outex-TC-

00014 data set, for four values of the quantization level q

and of the rank threshold ~t; the distance parameter d being

still set to 2. This table shows that, whatever the parameter

values, the histograms of LBPs computed from the three

rank maps~I

k;qof the demosaiced images are at best similar

by 96 % to those computed from the rank maps ~Ik;q of the

original images. Parameter variations merely change the

similarity values.

This study highlights that demosaicing affects the LBP

values and, subsequently, may impact the results of LBP-

based texture classification. When the classification scheme

is applied to texture images acquired by single-sensor color

cameras, it is therefore desirable to avoid the demosaicing

step. The following is an attempt to use LBPs as CTDs

directly extracted from CFA images.

3.3 CFA LBPs

3.3.1 Overview

In applications that use single-sensor cameras, only CFA

images may be available. The color images are commonly

estimated by demosaicing before LBPs can be computed.

In this paper, we propose to avoid the demosaicing step and

to replace classical LBPs by specific descriptors that are

directly extracted from the CFA images.

Table 3 Mean rank-based LBP similarity between original images of

the Outex-TC-00014 data set and the corresponding (a) demosa-

iced color images (see Sect. 3.2) or (b) CFA images (see Sect. 3.3)

~t q Similarity

(a) SIMðI; IÞ0 8 0.94

16 0.94

32 0.94

64 0.94

1 8 0.94

16 0.94

32 0.94

64 0.94

2 8 0.95

16 0.94

32 0.94

64 0.94

3 8 0.96

16 0.95

32 0.94

64 0.94

(b) SIMðI; ICFAÞ0 8 0.95

16 0.94

32 0.94

64 0.94

1 8 0.95

16 0.94

32 0.94

64 0.94

2 8 0.95

16 0.95

32 0.95

64 0.95

3 8 0.97

16 0.95

32 0.95

64 0.95

The standard deviation ranges are: (a) [0.29 9 10-3, 0.42 9 10-3]

and (b) [0.10 9 10-3, 0.23 9 10-3]

J Real-Time Image Proc

123

The key idea of this approach is to analyze each com-

ponent image IkCFA; k 2 fR;G;Bg; separately to provide

specific CFA LBPs. This is justified because ranks and

LBPs are computed component-wise. Each component

image is examined independently to compute the level

ranks in the CFA image. Then, a specific neighborhood is

used to define LBPs that take into account the CFA special

structure of interleaved components.

First, a level quantization rule using median-cut is defined

for (and applied to) each component image ICFAk . By exam-

ining the sole levels of the color component k, this scheme

provides the quantized component image ICFAk,q . Then, we

compute the histogram h[ICFAk,q ] of each quantized component

image, and the corresponding rank map. Finally, the com-

ponent-wise CFA LBPs are computed from this rank map by

adapting the previous procedure followed for a color image.

The CFA LBPs are hereafter denoted as CLBP although

the same acronym is already fairly used in the LBP literature

(to denote Compound-, Color-, Completed-, or Circular

LBPs). In the general context of color texture analysis and

classification, CLBPs can be related to other approaches

thanks to Bianconi et al.’s [1] generic hierarchical taxon-

omy. Each CLBP histogram is then a color texture function

(CTF) defined on its own sublattice. These three component-

wise CTFs form a CTD of the class CTD:Spec-

tralMarginalizationIntraChannel [1].

3.3.2 Computation details

Due to the particular structure of the CFA image, the

proposed approach separately examines each component

image ICFAk which results from the component-wise sam-

pling of the CFA image ICFA.

The first step is a component-wise level quantization

that is performed marginally on each color component.

First, we determine a level quantization rule Qk;q based on

the median-cut scheme by examining the sole levels of the

color component k in ICFA. These levels are available at

pixels belonging to the lattice Kk: Then, we apply this rule

to quantize ICFAk with q levels and obtain the quantized

component image Ik;qCFA ¼ Qk;qðIk

CFAÞ; which is also defined

on the lattice Kk:

The second step is to build the CFA rank map. To

achieve this, we first compute the component-wise nor-

malized histogram of ICFAk,q as:

h½Ik;qCFA�ðlÞ ¼

jfP 2 Ik;qCFAjI

k;qCFAðPÞ ¼ lgj

jIk;qCFAj

; l ¼ 0. . .q� 1:

ð12Þ

Then, we build the component-wise rank map that is

defined at each pixel P 2 IkCFA by analogy with Eq. (7) as:

~Ik;qCFAðPÞ ¼

XIk;qCFAðPÞ

l¼0

1� dðh½Ik;qCFA�ðlÞÞ: ð13Þ

In the last step, we compute the component-wise CFA

LBPs. Let CLBP~t½~Ik;qCFA� be the (rank-based) LBP that

captures the local spatial interactions of the color

component k in the CFA image. It is defined at each

pixel P 2 IkCFA by analogy with Eq. (1) as:

CLBP~t½~Ik;qCFA�ðPÞ ¼

X

Q2N 2nðPÞ2f ðQÞ � sð~Ik;q

CFAðPÞ � ~Ik;qCFAðQÞ; ~tÞ:

ð14Þ

Several important remarks can be made about this

expression. First, note that we consider the neighborhood

N 2n; n 2 N� (see Eq. 3). This ensures that the neighbor Q is

characterized by the same color component as P in ICFA

(i.e., Q also belongs to ICFAk , see Fig. 4). For instance, the

eight neighbors of R3,2 at distance d = 2n = 2 are also red

(see Fig. 4b), those of G3,3 are green (see Fig. 4c), and

those of B2,3 are blue (see Fig. 4d) (let us recall that Rx,y

denotes that the R level is available at pixel (x, y), and

similarly for G and B). Second, note that the four closer

diagonal neighbors could have been considered for the G

component (for instance, G2,2, G4,2, G2,4, and G4,4 as

neighbors of G3,3). But using N 2n allows the definition

(14) of CFA LBPs to be fully consistent with that of LBPs

given by Eq. (1). Last, note that this expression defines

CFA LBPs as a function over the lattice Kk only (which is a

corollary of the previous remarks). Their histogram is used

as a texture feature of a CFA image, as for a full-color

image.

3.3.3 Numerical example

Figure 5 shows a numerical example of level rank map and

LBP computations from a CFA component image. The

green component is still considered, which here provides

13 available green levels for ICFAG,6 among the 5 9 5 avail-

able in IG,6 (see Fig. 1a). Note that the levels of ICFAk here

appear as if sampled from IG,6 according to KG: This is

motivated only by comparison purposes because these

images actually correspond to two distinct approaches.

Besides, Ik,q and ICFAk result from two different quantization

rules: one examines Ik to perform median-cut, the other

examines ICFAk (i.e., a half (k = G) or a quarter (k = R, B)

of the levels available in the CFA image).

To compute the rank map from the component image

histogram according to Eq. (13), only levels that actually

exist in the component image are represented as ranks. As a

consequence, the ranks computed from a color component

image and its CFA sampled counterpart may differ.

J Real-Time Image Proc

123

For instance, five pixels have level 3 in IG,6 (see Fig. 3a)

but none in ICFAk (see Fig. 5a) because none of the five

pixels belong to KG: Consequently, ~IG;6 maximal rank is 4,

whereas that of ~IG;6CFA is 3.

Such cases result in a simple shift of some ranks but

preserve the global rank order (compare for instance ~IG;6 in

Fig. 3a and ~IG;6CFA in Fig. 5a pixel-wise). Hence, CFA sam-

pling does generally not change the final LBP values

computed from these ranks. For instance, at the central

pixel P of ~IG;6 and ~IG;6CFA; the binary codes are the same for

these two component images when the rank threshold ~t

is set to 1 (see Figs. 3b, 5b), hence LBP1½~IG;6� ¼CLBP1½~IG;6

CFA�: When ~t increases, CFA sampling is more

prone to affect the LBP value. At P for instance, the binary

codes of ~IG;6 and ~IG;6CFA differ for ~t ¼ 2 (see Figs. 3b, 5c),

hence LBP2½~IG;6� 6¼ CLBP2½~IG;6CFA�: However, we have

LBP3½~IG;6� ¼ CLBP2½~IG;6CFA�: This example shows that the

parameter ~t has still greater effect and must be carefully

chosen when used to compute LBPs from CFA rank maps.

3.3.4 LBP fidelity of CFA images

We now quantify how CFA sampling degrades the LBPs,

as this was assessed for demosaicing in Sect. 3.2. To

evaluate the LBP-based fidelity of CFA images ICFA to the

reference full-color images I from which ICFA are simu-

lated, we use the same data set and parameter values as in

Table 3a.

Table 3b displays the mean similarity values provided

by the CFA-based approach according to the following

measure:2

SIMðI; ICFAÞ ¼1

3

X

k¼R;G;B

Interðh½LBP~t½~Ik;q�; h½CLBP~t½~Ik;qCFA��Þ:

ð15Þ

These results highlight that, over the 16 tested parameter

values, the LBP histograms of the CFA image are at least

similar by 94 % to those of the reference LBPs. When

compared to the results of Table 3a, they show that CFA

LBPs provide at least the same similarity values as

classical LBPs extracted from the demosaiced images,

and even slightly higher similarity values for high

thresholds.

These two approaches analyze two kinds of images: the

demosaiced image I or the CFA image ICFA. At the

expense of the demosaicing step, I has the same spatial

definition as the reference image I: However, its colors

differ because they are estimated. The color components

within ICFA are exactly those of I; but the spatial definition

of the CFA component images ICFAk are twice (k = G) or

four times (k = R, B) lower than that of Ik. The LBPs

computed from I take advantage of its full spatial definition

but differ from the reference LBPs because of color esti-

mation errors. The CFA LBPs are not affected by such

demosaicing artifacts, but are twice (k = G) or four times

(k = R, B) fewer in number than reference LBPs. In both

cases, LBP histograms differ (by about 5 %) from refer-

ence LBPs, but for two distinct reasons.

This study shows that the proposed CFA LBPs are CTDs

of potential interest, because they well represent the ori-

ginal textural information. However, nothing can be

inferred thereby about their ability to discriminate textures,

which is crucial in our context of texture classification.

This aspect is deeply investigated in the next section

together with computational considerations.

4 Experiments

This section gives some experimental insights about the

relevance of the proposed approach. Section 4.1 summa-

rizes how LBPs are computed from the quantized images.

Section 4.2 gives experimental results of texture classifi-

cation. Section 4.3 evaluates the computational effort

required by the proposed CFA-based approach in com-

parison with the classical one that uses demosaiced images.

(a)

(b) (c)

Fig. 5 Rank map and LBP computations from a CFA component

image. At the central pixel P of ~IG;6CFA; the binary codes obtained are b

for ~t ¼ 1; and c for ~t ¼ 2: Considering the neighbor numbers of Fig.

1d, LBP values are then CLBP1½~IG;6CFA�ðPÞ ¼ 217 and CLBP2½~IG;6

CFA�ðPÞ ¼ 8: Note that CLBP0½~IG;6

CFA�ðPÞ ¼ 255 (see Fig. 1b) and that

CLBP3½~IG;6CFA�ðPÞ ¼ 0

2 Histogram normalization (see Eq. 4) is crucial here because

j~IG;qCFAj ¼ 1

2j~IG;qj and j~Ik;q

CFAj ¼ 14j~Ik;qj (k = R, B).

J Real-Time Image Proc

123

4.1 Experimental scheme

The goal of texture classification is to retrieve the most

similar target images to an input test image. For this pur-

pose, the matching scheme evaluates the similarity

between a test image and a target image thanks to a mea-

sure which bin-wise compares their normalized LBP

histograms.

The computation scheme of LBP histograms that

describe a texture according to three approaches is outlined

in Fig. 6. The data set (Outex-TC-00014) used to assess

the classification performance contains full-color images

acquired by a three-CCD camera. The corresponding CFA

images that a single-sensor camera would have provided

are unavailable in practice. Thus, we simulate each CFA

image ICFA from its original full-color image I by sampling

a single color component at each pixel according to the

Bayer CFA arrangement shown in Fig. 4. This makes sense

because Outex-TC-00014 images are full-color

undemosaiced images. Three kinds of rank-based LBPs are

then computed:

• Demosaiced We compute the LBPs LBP~t½~Ik;q� from the

rank maps of the color demosaiced and quantized

image Iq (see Sect. 2.3). For this purpose, the CFA

image is demosaiced according to the method of Lian

et al. [11].

• CFA The CFA LBPs CLBP~t½~Ik;qCFA� are computed from

the rank maps of the quantized CFA component images

ICFAk (see Sect. 3.3).

• ReferenceWe also compute the LBPs LBP~t½~Ik;q� from

the rank maps of the quantized full-color image Iq;

which provides a reference for the classification

performance.

Note that the same neighborhood N 2n; n 2 N�; is prac-

tically used in all three cases to ensure that the

Fig. 6 Outline of the

computation of LBP and CFA

LBP histograms

J Real-Time Image Proc

123

classification results obtained for these different kinds of

LBPs are strictly comparable.

Once LBP histograms are computed, we use Eq. (9) to

evaluate the similarity between two color images (either

original or demosaiced). To compare two CFA images

acquired by a single-sensor color camera, we have to use

the CFA counterpart of Eq. (9) defined as:

SIMðICFA;1; ICFA;2Þ

¼ 1

3

X

k¼R;G;B

Interðh½CLBP~t½~Ik;qCFA;1��; h½CLBP~t½~Ik;q

CFA;2��Þ:

ð16Þ

4.2 Classification results

Table 4 displays the classification accuracies of the

Outex-TC-00014 data set. Several parameter values are

considered to study their influence:

• the distance parameter n is either set to 1 (see Table 4a)

or to 2 (see Table 4b);

• the threshold ~t used to compute the rank-based LBPs

ranges from 0 (classical LBPs) to 3;

• the number q of levels used to quantize the color

components are the powers of 2 from 8 to 64.

These values allow us to draw conclusions about the

different LBPs when comparing their classification per-

formance results. The three columns in each subtable

contain the correct classification rates achieved by the

considered CTDs:

• Demosaiced LBPs from the (quantized) demosaiced

image Iq provided by Lian et al.’s [11] highly efficient

method;

• CFA CFA LBPs directly computed from the CFA color

component images ICFAk,q ;

• Reference LBPs from the full-color image Iq as

reference rates.

This table shows that the proposed approach provides

better classification results than the classical one for all but

4 cases out of the 32 tested ones. On average over all

parameter values, our approach achieves 2.04 % higher

performance than the classical approach using demosaiced

images, and only 0.69 % lower performance than when the

reference original images are used. The highest classifica-

tion accuracy achieved by CFA LBPs (68.97 %) or by

LBPs extracted from the original color images (69.19 %)

are the same as that (69.5 %) obtained by Maenpaa and

Pietikainen [16] who extract LBPs directly from the color

component levels. As mentioned by these authors,

considering a large number of neighbors may better cope

with the changes caused by different illuminants. It could

be interesting to take more than eight neighbors into

account (while respecting the CFA pattern) as n increases.

Furthermore, note that the highest classification accuracy

achieved on the demosaiced color images is only 66.40 %.

We conclude that the demosaicing step degrades the clas-

sification performance and should be avoided.

4.3 Computational effort

In this subsection, we focus on the analysis of images

acquired by a single-sensor color camera which delivers

CFA images [15]. Maenpaa et al. [17] show that LBPs are

texture descriptors that are suitable to a real-time setting.

So, we propose to estimate and compare the computational

effort required by CFA LBPs on the one hand, and by LBPs

extracted from a demosaiced image on the other hand. We

also measure the processing times although they strongly

depend on the code optimization.

Table 5 summarizes the required number of elementary

operations and processing times to compute the LBPs of a

CFA image of size N = XY pixels, according to the two

examined approaches (Demosaiced and CFA).

The sole computational operations considered here are

arithmetic operations and binary tests, all of them being

equally weighted in the complexity estimation. Array

indexing (or memory access) associated with each ele-

mentary operation is not taken into account, because the

neighborhoods practically used to compute the different

kinds of LBPs are always identical in order to obtain

comparable results. Note that the number N of pixels is

much higher than the number q of levels. For simplification

purpose, we hence ignore here the processing steps that

only depend on q, like the computation of the cumulative

histograms (required by the median-cut quantization and

by Eqs. 7, 13), the median cuts of this cumulative histo-

gram, and the computation of the look-up tables between

levels and associated ranks.

Let us estimate the number of operations required to

compute a rank-based LBP histogram from one color

component image of size N pixels. The quantization step

needs the image histogram to be computed, hence requires

N additions. The rank map is also computed from this

histogram, hence only requires N operations to assign each

pixel’s rank. Array indexing being ignored here, the

number of elementary operations required to compute a

LBP (see Eq. (1)) or a CFA LBP (see Eq. (14)) at each

pixel P from a rank map only depends on the number of

considered neighbors. For each neighbor, three elementary

operations are required: a subtraction of its level from that

of P, a binary test to threshold this difference, and a mul-

tiplication by the neighbor weight. Since the neighborhood

used to compute (CFA) LBPs contains 8 pixels, computing

J Real-Time Image Proc

123

N LBPs requires 3� 8� N operations. At last, computing

the LBP histogram requires N operations.

From a CFA image, the classical approach first performs

demosaicing. The number of operations required by Lian

et al.’s [11] demosaicing method amounts to 65� N

according to its authors. Then, a LBP histogram is com-

puted for each of the three color component images whose

sizes sum up to 3� N: Our proposed approach processes

the CFA directly, hence both avoids the demosaicing step

and requires three times fewer operations to compute the

LBP histograms, because the sizes of the three color

component images sum up to N. The overall number of

elementary operations amounts to 146� N for the classical

approach and 27� N for our approach (see top rows of

Table 5).

Let us now give an insight into the processing times

required by the two approaches, and check whether these

are consistent with the number of operations. For our

experiments, the demosaicing method implementation is

available as Matlab code [11], whereas we have imple-

mented other steps as Java plugins under ImageJ [29].

Although they are not fully comparable, we indicate the

processing times required by these steps (see bottom three

rows of Table 5) on a Core Duo PC at 1.58 GHz with 4 Go

RAM. These times are averaged over 100 images of each

data set and four values of q. They show that demosaicing

is a time-consuming step that should be avoided, and that

LBPs alone are computed faster with our approach.

5 Conclusion

This paper focuses on the classification of color texture

images acquired by single-sensor color cameras under

varying scene illumination. In such conditions, we show

that thresholded LBPs based on quantized level ranks are

interesting descriptors that notably handle illuminant

variations efficiently. Our main contribution is an original

approach that considers the specific CFA pattern of each

color component independently to extract LBPs directly

from CFA images without any demosaicing. LBP-based

histograms are then computed and used as texture features

in a classification scheme.

We present experimental results obtained on a bench-

mark texture data set under several values of the parame-

ters (quantization and threshold levels, or neighboring pixel

Table 4 Correct classification rates (%) of the Outex-TC-00014data set, for two values of the distance parameter n

Parameters Demosaiced CFA Reference

~t q LBP~t½~Ik;q� CLBP~t½~Ik;q

CFA� LBP~t½~Ik;q�

(a) n = 1

0 8 64.71 68.24 68.31

16 65.22 66.40 67.72

32 64.71 67.28 66.91

64 63.75 66.40 65.81

1 8 66.40 66.54 69.93

16 65.37 68.38 67.87

32 65.66 67.57 67.06

64 64.49 66.76 66.54

2 8 0.81 65.59 65.88

16 65.44 68.97 69.19

32 65.22 67.65 67.87

64 63.38 65.51 65.59

3 8 53.53 57.87 60.59

16 62.57 65.59 66.69

32 62.87 65.15 63.82

64 59.49 60.07 60.00

(b) n = 2

0 8 62.35 63.97 66.03

16 62.28 64.26 66.25

32 61.84 65.07 65.22

64 63.46 65.81 65.66

1 8 63.82 65.07 65.51

16 62.87 64.85 65.00

32 60.96 64.19 65.59

64 61.69 64.78 66.03

2 8 61.84 66.10 66.40

16 64.04 65.44 66.76

32 63.46 64.63 65.96

64 63.09 62.94 64.41

3 8 57.21 58.60 60.07

16 61.47 63.38 65.88

32 63.09 61.47 63.53

64 62.06 60.15 61.40

For each pair ð~t; qÞ of parameter values, the best classification accu-

racy achieved by either LBPs from Iq or CFA LBPs is written in bold

face

Table 5 LBP computational complexity: number of elementary

operations (top rows) and average processing times (bottom rows) of

the LBP computation steps

Computation step Demosaiced CFA

LBPs from I LBPs from ICFA

Demosaicing 65� N 0

Quantization 3� N N

Rank map 3� N N

LBP 3� 24� N 24� N

Histogram 3� N N

Total 146� N 27� N

Demosaicing (ms) 42 0

Following steps (ms) 6 4

Total (ms) 48 4

J Real-Time Image Proc

123

distance). These results show that our approach reaches

better classification rates than when LBPs are extracted

from demosaiced color images.

By design, LBPs are fast to compute and their histo-

grams are memory-efficient descriptors. Moreover, our

approach is less time consuming than the classical one

based on demosaiced images. Such outstanding perfor-

mance can be exploited whenever color texture images

have to be classified under severe time constraints. The

LBP-based descriptor used is fairly robust to illuminant

variations and well suited to the analysis of outdoor scene

images. It seems, therefore, possible to use this pioneer

approach in any on-board low-level classification (or

color–texture segmentation) of images acquired by a sin-

gle-sensor camera, such as for mobile robots control [2] or

road detection in vehicle environment [10].

Besides, the same CFA approach could be applied with

recent LBP variants that exhibit interesting properties. In

particular, the extended local ternary patterns [13] have

been designed to be less sensitive to noise, which may be

of special interest with low-cost single-sensor cameras. But

more sophisticated descriptors like these require more

memory and processing time. Using them in a real-time

context is therefore a challenge.

References

1. Bianconi, F., Harvey, R., Southam, P., Fernandez, A.: Theoretical

and experimental comparison of different approaches for color

texture classification. J. Electron. Imaging 20(4), 043006-1–

043006-17 (2011). doi:10.1117/1.3651210

2. Blas, M.R., Agrawal, M., Sundaresan, A., Konolige, K. Fast

color/texture segmentation for outdoor robots. In: Proceedings of

the International Conference on Intelligent Robots and Systems

(IROS 2008), Nice, France, pp. 4078–4085 (2008). doi:10.1109/

IROS.2008.4651086

3. Choi, J.Y., Ro, Y.M., Plataniotis, K.N.: Color local texture fea-

tures for color face recognition. IEEE Trans. Image Process.

21(3), 1366–1380 (2012). doi:10.1109/TIP.2011.2168413

4. Fernandez, A., Ghita, O., Gonzalez, E., Bianconi, F., Whelan,

P.F.: Evaluation of robustness against rotation of LBP, CCR and

ILBP features in granite texture classification. Mach. Graph. Vis.

22(6), 913–926 (2011). doi:10.1007/s00138-010-0253-4

5. Finlayson, G.D., Hordley, S.D., Schaefer, G., Tian, G.Y.: Illu-

minant and device invariant colour using histogram equalisation.

Pattern Recogn. 38(2), 179–190 (2005). doi:10.1016/j.patcog.

2004.04.010

6. Guo, Z., Zhang, L., Zhang, D.: Rotation invariant texture clas-

sification using LBP variance (LBPV) with global matching.

Pattern Recogn 43(3), 706–719 (2010). doi:10.1016/j.patcog.

2009.08.017

7. Heckbert, P.: Color image quantization for frame buffer display.

In: Thomas, J.J. (ed.) Proceedings of the 9th Annual Confer-

ence on Computer Graphics and Interactive Techniques

(SIGGRAPH’82), pp. 297–307. ACM, New York (1982). doi:

10.1145/800064.801294

8. Heikkila, M., Pietikainen, M.: A texture-based method for mod-

eling the background and detecting moving objects. IEEE Trans.

Pattern Anal. Mach. Intel. 28(4), 657–662 (2006). doi:10.1109/

TPAMI.2006.68

9. Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Local

binary patterns and its application to facial image analysis: a

survey. IEEE Trans. Syst. Man Cybern. Part C 41(6), 765–781

(2011). doi:10.1109/TSMCC.2011.2118750

10. Kang, Y., Kidono, K., Kimura, Y., Ninomiya, Y.: Texture-based

objects recognition for vehicle environment perception using a

multiband camera. In: Advances in Visual Computing. Lecture

Notes in Computer Science, vol. 4842, pp. 582–591. Springer,

Berlin (2007). doi:10.1007/978-3-540-76856-2_57

11. Lian, N.X., Chang, L., Tan, Y.P., Zagorodnov, V.: Adaptive

filtering for color filter array demosaicking. IEEE Trans. Image

Process. 16(10), 2515–2525 (2007). doi:10.1109/TIP.2007.904

459. http://en.pudn.com/downloads142/sourcecode/graph/texture_

mapping/detail619008_en.html

12. Liao, S., Zhao, G., Kellokumpu, V., Pietikainen, M., Li, S.Z.:

Modeling pixel process with scale invariant local patterns for

background subtraction in complex scenes. In: Proceedings of the

IEEE Conference on Computer Vision and Pattern Recognition

(CVPR’10), San Francisco, CA, USA, pp. 1301–1306 (2010).

doi:10.1109/CVPR.2010.5539817

13. Liao, W.H.: Region description using extended local ternary

patterns. In: Proceedings of the 20th IEEE International Confer-

ence on Pattern Recognition (ICPR’10), Istambul, Turkey,

pp. 1003–1006 (2010)

14. Losson, O., Macaire, L., Yang, Y.: Comparison of color demo-

saicing methods. Adv. Imaging Electron Phys. 162, 173–265

(2010). doi:10.1016/S1076-5670(10)62005-8

15. Lukac, R.: Single-sensor imaging in consumer digital cameras: a

survey of recent advances and future directions. J. Real Time

Image Process. 1(1), 45–52 (2006). doi:10.1007/s11554-006-

0003-z

16. Maenpaa, T., Pietikainen, M.: Classification with color and tex-

ture: jointly or separately? Pattern Recogn. 37, 1629–1640

(2004). doi:10.1016/j.patcog.2003.11.011

17. Maenpaa, T., Viertola, J., Pietikainen, M.: Optimising colour and

texture features for real-time visual inspection. Pattern Anal.

Appl. 6(3), 169–175(2003). doi:10.1007/s10044-002-0179-1

18. Menon, D., Calvagno, G.: Color image demosaicking: an over-

view. J. Signal Process. Image Commun. 26(8–9), 518–533

(2011). doi:10.1016/j.image.2011.04.003

19. Muselet, D., Macaire, L. Histogram of fuzzy ranks for object

recognition across illumination changes. In: Proceedings of the

3rd European Conference on Colour in Graphics, Image and

Vision (CGIV’06), Leeds, UK, pp. 220–225 (2006)

20. Ojala, T., Pietikainen, M., Harwood, D.: A comparative study of

texture measures with classification based on feature distribu-

tions. Pattern Recogn. 29(1), 51–59 (1996). doi:10.1016/0031-

3203(95)00067-4

21. Ojala, T., Maenpaa, T., Pietikainen, M., Viertola, J., Kyllonen, J.,

Huovinen, S.: Outex new framework for empirical evaluation of

texture analysis algorithms. In: Proceedings of the 16th Interna-

tional Conference on Pattern Recognition (ICPR’02), vol. 1,

pp. 701–706 (2002a). http://www.outex.oulu.fi/temp/

22. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-

scale and rotation invariant texture classification with local binary

patterns. IEEE Trans. Pattern Anal. Mach. Intel. 24(7), 971–987

(2002b). doi:10.1109/TPAMI.2002.1017623

23. Palm, C.: Color texture classification by integrative co-occur-

rence matrices. Pattern Recogn. 37(5), 965–976 (2004). doi:

10.1016/j.patcog.2003.09.010

24. Pan, H., Xia, S.Y., Jin, L.Z., Xia, L.Z.: Illumination invariant face

recognition based on improved local binary pattern. In:

J Real-Time Image Proc

123

Proceedings of the 30th Chinese Control Conference

(CCC’2011), Yantai, China, pp. 3268–3272 (2011)

25. Pietikainen, M., Maenpaa, T.: Texture analysis with local binary

patterns. In: Chen, C.H., Wang, P.S.P. (eds) The Handbook of

Pattern Recognition and Computer Vision, 3rd edn., pp. 197–216.

World Scientific Publishing, Singapore (2005). doi:10.1142/

9789812775320_0011

26. Pietikainen, M., Maenpaa, T., Viertola, J. Color texture classifi-

cation with color histograms and local binary patterns. In: Pro-

ceedings of the 2nd International Workshop on Texture Analysis

and Synthesis (TEXTURE 2002), Copenhagen, Denmark,

pp. 109–112 (2002)

27. Pietikainen, M., Hadid, A., Zhao, G., Ahonen, T.: Computer

vision using local binary patterns. In: Computational Imaging and

Vision, vol. 40. Springer, London (2011). doi:10.1007/978-0-

85729-748-8

28. Porebski, A., Vandenbroucke, N., Macaire, L.: Haralick feature

extraction from LBP images for color texture classification. In:

Proceedings of the International Workshops on Image Processing

Theory, Tools and Applications (IPTA’08), Sousse, Tunisie,

pp. 1–8 (2008). doi:10.1109/IPTA.2008.4743780

29. Rasband, W.S.: Image J. US National Institutes of Health,

Bethesda (1997–2012). http://imagej.nih.gov/ij/

30. Rubner, Y., Puzicha, J., Tomasi, C., Buhmann, J.M.: Empirical

evaluation of dissimilarity measures for color and texture.

Comput. Vis. Image Underst. 84(1), 25–43 (2001). doi:10.1006/

cviu.2001.0934

31. van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating

color descriptors for object and scene recognition. IEEE Trans.

Pattern Anal. Mach. Intel. 32(9), 1582–1596 (2010). doi:10.1109/

TPAMI.2009.154

32. Swain, M.J., Ballard, D.H.: Color indexing. Int. J. Comput. Vis.

7(1), 11–32 (1991). doi:10.1007/BF00130487

33. Tan, X., Triggs, B.: Enhanced local texture feature sets for face

recognition under difficult lighting conditions. IEEE Trans.

Image Process. 19(6), 1635–1650 (2010). doi:10.1109/TIP.2010.

2042645

34. Zhao, Y.: Theories and applications of LBP: a survey. In: Huang,

D.S., Gan, Y., Gupta, P., Gromiha, M. (eds) Advanced Intelligent

Computing Theories and Applications. With Aspects of Artificial

Intelligence, Lecture Notes in Computer Science, vol. 6839,

pp. 112–120. Springer, Berlin (2012). doi:10.1007/978-3-642-

25944-9_15

35. Zhu, C., Bichot, C.E., Chen, L.: Multi-scale color local binary

patterns for visual object classes recognition. In: Proceedings of

the 20th IEEE International Conference on Pattern Recognition

(ICPR 2010), Istambul, Turkey, pp. 3065–3068 (2010). doi:

10.1109/ICPR.2010.751

Author Biographies

Olivier Losson received his

M.S. (Engineer) degree in 1994

from the Ecole Centrale of Lille,

France, and his Ph.D. degree in

Computer Science and Control

from the University of Lille 1 in

2000. He is an Associate Pro-

fessor at the Department of

Computer Science, Electronics,

Electrical Engineering and

Automatics, and a member of

the LAGIS Laboratory (Labo-

ratoire d’Automatique, Genie

Informatique et Signal) of Lille

1. His research interests focus

on color image processing, chiefly image demosaicing, color image

segmentation, and color texture representation.

Ludovic Macaire received his

M.S. (Engineer) degree in

computer science from the UTC

Engineering school of Compie-

gne, France, in 1988 and his

Ph.D. in computer science and

Control from the University of

Lille 1 in 1992. He is presently a

Full Professor in the LAGIS

Laboratory at the Universityof

Lille 1. His research interests

include color representation,

color image analysis applied to

segmentation and retrieval.

J Real-Time Image Proc

123