IEEE TRANSACTIONS ON MULTIMEDIA, VOL. X, NO. X, JANUARY ... · IEEE TRANSACTIONS ON MULTIMEDIA,...

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. X, NO. X, JANUARY 2012 1

Edge-preserving Texture Suppression FilterBased on Joint Filtering Schemes

Zhuo Su, Student Member, IEEE, Xiaonan Luo, Zhengjie Deng, Yun Liang, and Zhen Ji, Member, IEEE

Abstract—Obtaining a texture-smoothing and edge-preservingfiltered output is significant to image decomposition. Althoughthe edge and the texture have salient difference in human vision,automatically distinguishing them is a difficult task, for theyhave similar intensity difference or gradient response. The state-of-the-art edge-preserving smoothing (EPS) based decompositionapproaches are hard to obtain a satisfactory result. We proposea novel edge-preserving texture suppression filter, exploiting thejoint bilateral filter as a bridge to achieve the purpose of bothproperties of texture-smoothing and edge-preserving. We developthe iterative asymmetric sampling and the local linear modelto produce the degenerative image to suppress the texture, andapply the edge correction operator to achieve edge-preserving. Anefficient accelerating implementation is introduced to improve theperformance of filtering response. The experiments demonstratethat our filter produces satisfactory outputs with both propertiesof texture-smoothing and edge-preserving, while compared withthe results of other popular EPS approaches in signal, visual andtime analysis. Finally, we extend our filter to a variety of imageprocessing applications.

Index Terms—texture suppression, edge preserving, oscillation,image smoothing, degenerative scheme.

I. INTRODUCTION

In image filtering, how to distinguish edge, texture andsmoothing transition is challenging and significant to specifiedapplications. Classical linear filtering (e.g. Gaussian filtering)can smooth an image effectively but cause serious edgeblurring [1], see Fig. 1(b). The state-of-the-art edge-preservingsmoothing (EPS) approaches [2], [3], [5], [8] emphasize onthe importance of edge sharpness, successfully applied tomany applications, e.g. tone mapping, detail manipulation,non-photorealistic rendering, etc. However, if the input im-age contains various textures, these EPS approaches wouldmistreat some textures as edges and preserve them instead

Copyright (c) 2012 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to [email protected].

This work is supported by the NSFC-Guangdong Joint Fund (U0935004,U1135003), the National Key Basic Research and Development Program ofChina 973 (2013CB329505), the National Key Technology R&D Program(2011BAH27B01), the National Science Fund of China (61232011, 61262050,61202293), and the Scholarship Award for Excellent Doctoral Student grantedby Ministry of Education 2012.

Z. Su and X. Luo are with the National Engineering Research Center of Dig-ital Life, State-Province Joint Laboratory of Digital Home Interactive Appli-cations, School of Information Science & Technology, Sun Yat-sen University,Guangzhou, China. (E-mail: [email protected]; [email protected]).

Z. Deng is with the School of Information Science and Technology, HainanNormal University, Haikou, China (E-mail: [email protected])

Y. Liang is with the School of Information, South China AgriculturalUniversity, Guangzhou, China (E-mail: [email protected])

Z. Ji is with the Department of Computer Science, Shenzhen University,Shenzhen, China (E-mail: [email protected])

of smoothing, see Fig. 1(c)-(e). Paris et al. [9] pointed outthat these approaches usually depend on the variance of pixelintensity [10], gradient magnitude [3], or extreme value [11]to preserve the edge. Because this problem would produceserious inference to specified applications, suppressing thetextures in EPS filtering is necessary. Subr and Farbman etal. exploited the weighted least squares (WLS) optimizationframework [3], and successively constructed the envelop oflocal extrema [12] and diffusion map [13] to solve this problemand further demonstrated the applicability. In this paper, wepropose a novel edge-preserving texture suppression filter. Ittakes the joint bilateral filter (JBF) [14], [15] as a bridge,combining different filtering schemes, to suppress the texturesto the most extent with retaining the edge sharpness. Ourfiltered output owns both properties of texture-smoothing andedge-preserving. In the following, we summarize the state-of-the-art EPS approaches and give an overview of our solution.

A. Related Work

The bilateral-based filter is first presented by Tomasi andManduchi [10], which is a classical and effective edge-preserving smoothing filter. A popular formulation is to con-struct the coefficients of Gaussian low-pass filter according tothe spatial and the range distances of pixels. Then a non-linearfiltering mask is composed to implement the translate-variantspatial filter [9]. Since then, some extensions of the bilateralfilter are presented. Choudhury et al. [16] presented a trilateralfilter to improve the filtering effect in high contrast cases.Eisemann et al. [15] and Petschnigg et al. [14] successivelyintroduced cross/joint bilateral filter by modifying the inputof range function. Takeda et al. [17] presented the high-orderbilateral filter by applying kernel regression theory. And Baeket al. [8] summarized the bilateral-based filter as a spatiallyvarying high dimensional Gaussian filter.

On the other hand, considering the naıve implementation ofbilateral filter is time consuming, some accelerating methodswere presented. Durand and Dorsey [18] accelerated BF byusing a piecewise-linear approximation in the intensity domainand appropriate sub-sampling. Pham and Vliet [19] exploitedthe separability of Gaussian to apply 1D BF to each spatialdirection. Paris et al. [2], [20], [21] developed a data structurecalled bilateral grid (BG) and took advantage of some signalprocessing results to accelerate the filtering process. Adams etal. [22], [23] successively presented two novel data structuresbased on the Gaussian KD-tree and permutohedral lattice tofurther decrease the time and memory cost. Weiss [24] applied3D histograms which were described by square box spatial

2 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. X, NO. X, JANUARY 2012

(a) (b) (c) (d) (e)

(f) (g) (h) (i) (j)

Fig. 1. The performance of the state-of-the-art image smoothing approaches applied to Barbara image with two local magnified oscillation regions: a portionof scarf and a portion of tablecloth. (a) Original Barbara image. (b) Gaussian filtering (r=15, σ=5) [1]. (c) Hybrid median filtering (r=15) [1]. (d) Paris’sbilateral filtering (σS=5, σR=0.2) [2]. (e) Farbman’s weighted least squares filtering (α=1.8, λ=0.35) [3]. (f) Chao’s improved anisotropic diffusion (T=200,K=15) [4]. (g) Kass’s local mode filtering (r=19, σ=20) [5]. (h) Fattal’s edge-avoiding wavelets filtering (α=1, j=4) [6]. (i) Xu’s L0 gradient minimizationfiltering (λ=0.04, κ=2) [7]. (j) Our JIAS filtering (σR=0.05, λ=0.01, t=3). The definition of the parameters is clarified in the corresponding references. Andthe detailed explanation is presented in Sec. IV. C.

kernel for acceleration. Yoshizawa et al. [25] used the FastGauss Transform (FGT) to accelerate the computation. Porikliand Yang et al. [26]–[28] further demonstrated the existingO(1) BF and developed a variety of spatial kernels andrange kernels. Recently, some real-time implementations wereproposed with recursive approximation [29], domain transform[30], and adaptive manifolds [31].

The optimization-based filter is an effective edge-preservingsmoothing approach. Farbman et al. [3] proposed an edge-preserving multi-scale decomposition based on the weightedleast squares optimization framework. Bhat et al. [32] formu-lated the edge-preserving smoothing problem in a variationalframework and solved a 2D version of the screened Poissonequation. Subr et al. [12] considered an oscillatory model oftexture regions and removed image details by identifying andfitting envelopes to local extreme intensities. Subsequently,Farbman et al. [13] introduced the idea of replacing Eu-clidean distances with diffusion distances which were cal-culated through edge-aware diffusion maps. Bhat et al. [33]further summarized a gradient-domain solution frameworkcalled GradientShop, which was based on weighted leastsquares optimization as well. He et al. [34] presented anexplicit filter with the guided image based on the local linearmodel. And recently, Xu et al. [7] presented a novel imagesmoothing framework by solving L0 gradient minimization.

The diffusion-based filter is known as the partial differ-ential equations based filter, and anisotropic diffusion [35]is a famous approach for edge-preserving smoothing, whichconsiders the local gradient information of each pixel in the

image. Weickert et al. [36] proposed two efficient numer-ical approaches to accelerate the diffusion, which were inaccordance with the fully discrete scale-space framework andbased on an additive operator splitting (AOS). Chao et al. [4],[37] proposed a diffusion model incorporating both the localgradient and the gray level variance to preserve edges and finedetails.

The histogram-based edge-preserving filter is first intro-duced by Weijer and Boomgaard [11]. They proposed alocal mode filtering that was motivated from both the localhistogram with tonal scale and the robust statistics viewpoint.Felsberg et al. [38] presented an approach using B-splinelook-up tables for the histogram-based smoothing filter, calledchannel smoothing. Subsequently, Kass and Solomon [5] alsoprovided another way of smoothed local histogram filter foredge-preserving smoothing operation.

The decomposition-based filter is a novel approach formulti-scale edge-preserving smoothing. Fattal [6] presented anew family of second-generation wavelets constructed by a ro-bust data-prediction lifting scheme. Paris et al. [39] presenteda local Laplacian filter for edge-aware image processing onthe basis of Laplacian pyramid [1].

However, most of the above edge-preserving smoothingapproaches have the drawback that they would mistreat partialtextures as edges, resulting in the incomplete results of neitherpreserving nor smoothing. We choose some state-of-the-artapproaches and demonstrate their performance in Fig. 1.

SU et al.: EDGE-PRESERVING TEXTURE SUPPRESSION FILTER BASED ON JOINT FILTERING SCHEMES 3

B. Overview of Our Approach

As illustrated in Fig. 1, the state-of-the-art EPS approachesare hard to produce a satisfactory filtered output if an imagecontains textures. However, considering the result of Gaussianfiltering (GF), we can see an obvious smoothing for thetexture. That is, although the linear filtering produces edgeblurring, it can perform a sound texture suppression at thesame time. In other words, if the texture-smoothing character-istic of linear filtering and the edge-preserving characteristicof EPS approach can be combined in union, we can obtainboth properties of texture-smoothing and edge-preserving.However, edge preservation and texture suppression is anincompatible operation, so a simple additive combination isnot feasible. In our opinion, exploiting the framework of jointbilateral filter (JBF) [9], we can achieve our purpose elegantly.The pipeline of our approach is shown in Fig. 2.

JBF imports a joint image to improve the filtered effect.Because of this characteristic, achieving both properties oftexture-smoothing and edge-preserving becomes feasible. Buthow to construct a suitable joint image is a key problem. Inone hand, to achieve the texture suppression, we develop twoschemes based on iterative asymmetric sampling and locallinear model. In the other hand, to prevent the edge degen-eration caused by texture suppression, we adopt a gradientminimization to correct the edge. In addition, we exploit Pariset al.’s high dimensional linear convolution [2] to promotethe computational efficiency. In section IV, we will discussthe parameter setting and the relationship. Through the exper-iments in signal, visual, and time analysis, we demonstrate thatour approach achieves the purpose even better than previousEPS approaches. Furthermore, some image applications arepresented in our experiments as well, including edge detec-tion, detail enhancement, texture copy, tone mapping, non-photorealistic rendering, etc.

Our main contributions are as follows:(1) Propose a novel filter with both properties of texture-smoothing and edge-preserving.(2) Develop the iterative asymmetric sampling and the locallinear model degenerative schemes to suppress the texture.(3) Exploit the postprocessing operators for the edge correctionand the filtering acceleration.(4) Demonstrate the performance of our edge-preserving tex-ture suppression filter and illustrate the applications with thefiltered results.

II. JBF FRAMEWORK WITH JOINT FILTERING SCHEMES

Joint bilateral filter (JBF) is an extension of bilateral filter(BF), which was introduced in [14], [15]. The major distinctionbetween BF and JBF is that the latter one takes some specifiedimage E, called joint image, related with the input image Ias the input data in the range function of BF. The formulationis the following

IJ(p) =

∑p,q∈S

GS (∥p− q∥)GR (|E(p)− E(q)|) I(q)∑p,q∈S

GS (∥p− q∥)GR (|E(p)− E(q)|),

(1)

Input Image

Joint Bilateral

Filtering

Degenerative Image

Output Image

Construct

Spatial Weights

Construct Range Weights

Gradient Minimization

Apply Degenerative

Schemes

Edge Correction

Filtering

Fig. 2. The pipeline of our approach, including three major stages: imagedegeneration, edge correction and JBF filtering. Essentially, taking advantageof the JBF framework [9] as a bridge, we obtain both properties of texture-smoothing and edge-preserving.

where S and R denote the spatial domain and the rangedomain in the image space, respectively. The pixel p is thecenter of S and the neighbor pixel q = p + k, wherek = [kx, ky] with −r ≤ kx ≤ r, −r ≤ ky ≤ r, and r denotesthe radius of spatial domain S. In addition, both the spatialand the range filtering function are constructed by Gaussianbasis function G(x)=exp

(−x2/2σ2

), where the spatial σS

controls the filtering smoothness, and the range σR controlsthe sensitivity of the edge preservation.

Because of the flexibility in choosing the joint image E,JBF (1) is an ideal framework to achieve both propertiesof texture-smoothing and edge-preserving. He et al. [34]proposed a guided image filter (GuiF) with the performancesimilar with JBF. Ideally, constructing some suitable weightedconstraints in the optimization framework [3], [12] can alsoachieve aforementioned task. However, in our experiments,we found that the approximate fitting gradient, which wouldmore possibly lead to total gray-level derivation, can not beincomparable to JBF, see gray arrow in Fig. 7(a).

In the viewpoint of image decomposition, an image canbe decomposed into low- and high-frequency signals, i.e.I = IL + IH . The low-frequency signal IL can be producedby applying the linear filter h to the original image in thespatial domain, i.e. IL = h ⊗ I , where ⊗ is the spatialconvolution operator. Since the textures are the high-frequencysignals, the texture suppression is a process of smoothing high-frequency signal, such as removing all high-frequency signals.After filtered, the output low-frequency signal is said thedegenerative image (corresponding to the original image) andthe textures are suppressed as a consequent result. Consideringthe degenerative image D as a specified joint image, we willfocus on its construction in the following.

A. Iterative Asymmetric Sampling Degenerative Scheme

Signal sampling is an accelerating scheme usually applied inEPS filtering, e.g. Paris’s bilateral filtering [2] and local Lapla-cian filter [39], Chen’s bilateral grid [21], Fattal’s multiscaledecomposition [40], etc. Besides acceleration, sampling oper-ation can control the amount of information. Intuitively, imagedownsampling is a process of reducing information, and theupsampling adding. From the viewpoint of signal sampling,


(a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 3. (a) Input image. (e) The filtered results of JIAS (top) and JLLM (bottom) correspond to two marked regions in (a). From (b) to (d) are the results ofJIAS approach in different steps. (b) The degenerative image is constructed by IAS scheme (t=6). (c) The edge correction for the result of IAS (λ=1e-3). (d)The final result of JIAS which is produced by JBF (σR=0.05). And from (f) to (h) are the results of JLLM approach (σR=0.05, ε=0.25, λ=8e-4). Note thesemagnified local subfigures. In the degenerative images (b) and (f), the textures are smoothed effectively, but the edges are blurred. By the edge correction,this phenomenon would be improved, but some blocks-like effects appear in (c) and (g). Through the final JBF operator, we obtain a sound tradeoff betweentexture smoothing and edge preserving.

restoring an image from low-resolution to high-resolution is anill-conditioned problem that merely regains the salient edges.Although the edge and the texture have similar performancein variance of intensity or gradient, textures usually presenta gathering of variance in local regions. As illustrated ingray signal of Fig. 7, the obvious difference between edgesand textures is that textures have some salient oscillations.We can conclude that the textures can not be restored if thesampling scope is over the range of oscillations. Exploitingthis conclusion, we develop an asymmetric sampling approach.For flexibility, we introduce the iteration similar with Levand Zucker’s approach [41]. Therefore, the course of iterativeasymmetric sampling (IAS) is formulated as

Lt =(g ⊗ It

)↓d, It+1 = Lt↑d

−1

, (2)

where ↑ and ↓ are the discrete up- and down-samplingoperations with a sampling rate d−1 and d. t denotes theiterative times. g is a smoothing operator to improve the texturesuppression and anti-aliasing. In our implementation, we applythe bicubic and the bilinear sampling as the up- and down-sampling operations, respectively. And g is the same with thespatial filtering GS in JBF. Finally, the output D = It+1

is taken as the degenerative image. The pseudo code of thisscheme is given in Algorithm 1. And the outputs are shownin Fig. 3(b).

B. Local Linear Model Degenerative Scheme

Although IAS can achieve texture suppression, it is timeconsuming with the increasing of iterative times or image sizes

Algorithm 1: Iterative Asymmetric Sampling Degenerative Scheme

Input: I: image, t: iterative timesOutput: D: degenerative image1: s0 = I(p) initial the sampling image2: for i = 0 to t do3: li = (g ⊗ si) ↓ downsample the input scalar image by

gaussian pyramid4: hi = li ↑ upsample the low-resolution image5: si+1 = hi update the degenerative image6: end for7: D = si+1 final output image8: return

(see Tab. II). He et al. [34] used the linear regression of localpatches and the implementation of box filter, to improve thefiltering efficiency. Here, we exploit their implementation toconstruct the degenerative image. For each pixel p, we obtainthe degenerative image Dp by setting all the pixel values tobe zero except the pixels with the indices in the patch Ωp andthose pixels are defined as the following

Dp(q) = µpI(q) + ηp,∀q ∈ Ωp. (3)

According to the solution of the linear equation, we obtainthe explicit expression about linear coefficients µp and ηp [34]

µp =|ω|−1∑

q∈ΩpI(q)2 − I(p)2

σ2p + ε

, ηp = I(p)− µpI(p),

(4)where σ2

p denotes the variance on the pixel intensities in thepatch Ωp, |ω| denotes the number of the pixels of the patch Ωp,


and I(p)=|ω|−1∑q∈zΩp

I(q) denotes the mean value of I re-stricted to the patch Ωp. The smoothing factor ε is used to con-trol the texture suppression. The local linear model (LLM) canbe extended to the entire image, when µp=|ω|−1∑

q∈Ωpµq,

ηp=|ω|−1∑q∈Ωp

ηq, which overcomes the overlap of patches.Then, the joint image E=D=µpI(p) + ηp. The pseudo codeof this scheme is given in Algorithm 2. And the outputs areshown in Fig. 3(f).

Algorithm 2: Local Linear Model Degenerative Scheme

Input: I: image, ε: regular factor, r: filtered kernel radiusOutput: D: degenerative image1: for all p in I do2: Ip ← mean of I by box filter in the kernel Ωp with r

3: I2p ← mean of I2 by box filter in the kernel Ωp with r4: σ2

p ← variance of pixel intensity of the kernel Ωp

5: var Ip = I2p − I2p variance of I in each patch6: µp = var Ip/(σ2

p + ε) linear coefficient7: ηp = I − µp · I linear coefficient8: end for9: for all p in I do10: µp ← mean of all µp

11: ηp ← mean of all ηp12: D(p) = µp · I(p) + ηp local linear model for input image13: end for14: return

III. EDGE CORRECTION AND FILTERING ACCELERATION

The textures can be suppressed by taking the degenerativeschemes, however, the side effect of edge blurring appears aswell. To reduce the edge degeneration, we adopt a gradient-reconstructing operation to correct the salient sharp edges. Inaddition, JBF is a nonlinear filter whose brute-force implemen-tation is time consuming; therefore, we exploit the bilateralgrid implementation [2] to accelerate our filtering process.

A. Edge Correction Using Gradient MinimizationSince the edges will be blurred in the degenerative pro-

cedure, to obtain the edge-preserving effect, the edge ofdegenerative image which is corresponded to the original inputimage should be corrected to prevent the blurring. From thevisual effect, sharp variance of neighbor pixels reflects themost obvious edge feature. Therefore, we try to reconstructthe sharp edges in the degenerative image to overcome theserious distortions which are caused by degeneration. But re-constructing the step edges is a challenging problem. Recently,Xu et al. [7] proposed an image smoothing approach via L0

gradient minimization (L0GM), which counts the non-zerogradients and considers the edge-preserving smoothing as aL0-norm regularized optimization problem. This approach isnot suitable to the texture suppression because it merely treatsthe non-edge regions as constants. As illustrated in Fig. 1(i),that would lead to obvious intensity offset. However, inspiredby Xu’s approach, we can take advantage of the propertyof L0-norm to describe the feature of sharp edges, and useit to reconstruct the salient edge in the degenerative image.Considering the solved image D as a 2D spatial signal, thecount of sharp edges is defined as the following [7]

C(D) = # |∂xI(p)|+ |∂yI(p)| = 0 , (5)

where #· denotes the operator counting the number ofp that satisfies the non-zero gradients, that is the L0-normof gradient. ∂x and ∂y denote the x- and y-direction first-order forward difference operators, respectively. According tothe above equation (5), we will minimize the following L0

optimization equation [7]

minD

∑∥∥∥D − I∥∥∥2 + λ · C(D)

, (6)

where∑∥∥∥D − I

∥∥∥2 is used to preserve the similarity of theimage structure. λ is used to control the number of step edges.The smaller λ is, the more step edges are contained in theoutput. They give a special alternating optimization strategywith half-quadratic splitting [42] to approximately solve theabove equation [7]. We demonstrate the effectiveness of edgecorrection based on L0GM in Fig. 4. As illustrated in Fig.4(d), compared with the 1D signal of the original step edge,degeneration and correction, the signal of the corrected resultregains the sharpness of edge.

(a) (b) (c) (d)

Fig. 4. Edge correction. (a) Input. (b) Edge degeneration (ED). (c) Resultof Edge Correction (EC). (d) 1D signal comparison. The degenerative edgeis corrected to recover the sharpness comparable to the original input.

B. Acceleration Using High Dimensional Linear Convolution

In our implementation, we consider applying BG [2], [21] toaccelerate JBF, for it can be efficiently implemented on GPUand work seamlessly for color images. In this approach, thesignal sampling is exploited to convert the two dimensionalnon-linear BF into a high dimensional linear convolutionto accelerate the filtering process. By adding an additionaldimension ξ, a new coordinate (q, ξ) is defined for each pixelof the input image I . Therefore, the equation (1) can berewritten in the following three dimensional linear convolution(

W (p)IJ(p)

W (p)

)= gσS ,σR ⊗

(w(q, ξ)i(q, ξ)

w(q, ξ)

), (7)

where gσS ,σRis defined as the high-dimensional separable

Gaussian convolution, and w(·) and i(·) denote the 3Dweighted functions and intensity function in the product spaceS × R, respectively. W (p) is the 2D normalized weightedfunctions defined in S. By the nonlinear slicing and divisionoperations [2], the output of JBF with degenerative image Dis the following

IJ(p) =wJ(p, D(p))iJ(p, D(p))

wJ(p, D(p)). (8)


IV. PERFORMANCE AND DISCUSSION

In this section, we firstly discuss the parameter settings inour approaches. Then, we analyze the relationship with thestate-of-the-art EPS approaches and give out the comparisonsfor the texture suppression problem. We demonstrate theeffectiveness of our approaches of joint iterative asymmetricsampling (JIAS) and joint local linear model (JLLM) in signalperformance, visual effect, and time efficiency. All the exper-iments are tested on PC with Intel I5-2450M 2.5GHz CPU,NVIDIA 610M, 4GB DDR3 Ram, and MATLAB R2010a.

SSIM Value

(a)

SSIM Value

(b)

Fig. 5. Parameters relationship. (a) JIAS (σR, t, λ). (b) JLLM (σR, ε, λ).We measure the filtered outputs by Wang’s SSIM [43].

(a) (b) (c) (d)

Fig. 6. Compared with median filters. (a) Pinwheel iamge. (b) Photoshop me-dian filering. (c) Kass’ median filtering [5], (d) Our JIAS filtering (σR=0.05,t=3, λ=1e-4).

A. Parameters and Relationship

Our joint filtering schemes refer to JBF, IAS, LLM andL0GM approaches, and each operation has its correspondingparameters. How to assign the parameters appropriately isa key problem in our JIAS and JLLM approaches. Firstly,we can adopt the adaptive scheme in terms of the imagesize to determine the radius of spatial filter in JBF, IAS andLLM. In our experiments, this radius is evaluated by round offmin(width, height)/102. For the corresponding spatial σS inJBF and IAS, we specify them in min(width, height)/102

as well. The default sampling rate d is 1/2 according tothe pattern of Gaussian pyramid. In the stage of EC, κ=2recommended by Xu’s implementation [7] for the naturalimage. After the above adjustment, we reduce the parametersof JIAS and JLLM to triple (σR, t, λ) and (σR, ε, λ). t and εare used to smooth the texture, λ is used to correct the edge,and σR is a balance factor between the texture smoothing andedge preserving. To analyze the relationship of the parameters,we take SSIM [43] to measure the filtered results of JIAS(σR=0.01) and JLLM (σR=0.05) with 100 groups parameters,respectively. The measurement records are shown in Fig. 5.

In our experiments, we can obtain the satisfactory output byadjusting (t, λ) or (ε, λ) only. Fig. 3 shows the output in eachstage with the corresponding parameters. And Fig. 8 shows thefiltered output which corresponds to the parameter assignmentsin Tab. I.

Compared with the state-of-the-art approaches [2], [3],[7], Our JIAS and JLLM adopt the joint filtering schemeswhich use JBF as a bridge to integrate all advantages ofvarious approaches together. In edge preserving, our approachintegrates the intensity estimation of BF and gradient countingof L0GM to be aware of the edge; in texture smoothing, bothIAS and LLM schemes intend to construct the band-limitedlow-pass filters. We distinguish the edge and the texture bytheir geometry characteristics in spatial domain, rather than thescale of intensity or gradient variance. In our opinion, althoughboth the edge and the texture are the intensity variance, theyhave obvious distinction in spatial distribution. The edge canbe represented as the sole intensity variance, while the texturecan be represented as the neighbor dense intensity variance inthe spatial domain. For example, the scarf and the clothes ofBarbara in Fig. 1, and the 1D signal in Fig. 7.

Either the sampling operation of IAS or the local fittingof LLM can suppress the texture in the spatial domain, thatis, the texture is removed by the change of spatial distancesamong textures. The removed textures can not be restored,but the salient edge can be corrected in terms of the blurringedge structure (see Fig. 4). On the basis of this property, wecan adjust the parameter settings to distinguish the textureand the edge, and balance the texture smoothing and edgepreserving. Fig. 1 shows the comparison of various approachesand demonstrates our approach assembles kinds of advantages.

Median filter (MF) is a non-linear filter that has a soundnoise reduction [24]. But in dealing with the texture, itwould produce some artifacts. Kass et al. [5] pointed outthis problem in their work and proposed an improved medianfilter to overcome this defect; however, it would produce thegray transition apparently. We compare our JIAS output withPhotoshop’s MF and Kass’ MF in Fig. 6.

B. Signal Analysis

Fig. 7 provides an obvious comparison with state-of-the-art approaches and reflects the advantages of our approaches.Usually, the step-like shape is the salient edge, the oscillationis the texture, and the slope is the intensity transition. The512x512 size Barbara image is chosen as the tested image.Because the 256th pixel row contains the sharp step, multi-scale oscillations and slope signal, it is suitable for analyzingthe filtering performance. The original signal is plotted in graycurve in Fig. 7. The degenerative result (blue), corrected result(purple) and the final result (red) of JIAS and JLLM are plottedas well, and their corresponding images are presented in Fig.3. We magnify each subfigure in detail.

Fig. 7(a) shows the comparison of our JIAS (σR=0.05, t=6,λ=0.001) and WLS (α=1.8, λ=0.35). Both WLS and our JIAShave sound texture suppression in Block a1. But WLS has anintensity offset problem. Note that the result of WLS is slightlylower than that of JIAS, and has an obvious deviation at the


(a)

(b)

Fig. 7. Comparison of the 256th row pixel intensity of images in Fig. 3. (a) Plots of 1D signal of input image (gray), WLS (green), IAS (blue), correctededge (purple), and our JIAS approach (red). (b) Compare the outputs of BF (green) and JLLM (red). Through the magnified windows, we note that our JIASand JLLM approaches have a sound performance in both the texture suppression and edge preservation.

TABLE ITHE PARAMETER ASSIGNMENTS ARE RECORDED CORRESPONDING TO THE FILTERED OUTPUTS IN FIG. 8.

Noise Stripe Chessboard Ripple Leaf Bark Weave Crystal Status Lena

GF (r, σ) (15, 5) (15, 3) (15, 1.5) (15, 5) (15, 7) (15, 3) (15, 3) (15, 5) (15, 5) (15, 4)

MF (n) (15) (5) (15) (25) (15) (5) (15) (9) (25) (13)

BF (σS , σR) (10, 0.8) (8,0.1) (4, 5e-2) (6, 0.2) (8, 0.5) (8, 0.2) (15, 0.5) (8, 0.5) (8, 0.2) (8, 0.2)

WLS (α, λ) (1.8, 1.5) (1.8, 0.35) (1.2, 0.25) (1.2, 2) (1.8, 3) (1.2, 2) (1.8, 2) (1.8, 0.5) (1.8, 0.35) (1.8, 0.35)

GuiF (r, ε) (4, 0.5) (4, 1e-2) (8, 1e-2) (8, 0.2) (8, 0.1) (8, 0.1) (6, 0.15) (4, 0.16) (4, 4e-2) (4, 4e-2)

IAD (T,K) (300, 800) (200, 50) (100, 5) (200, 500) (200, 200) (200, 20) (100, 500) (100, 50) (200, 50) (100, 15)

L0GM (λ, k) (1,2) (1e-2, 2) (5e-3, 2) (0.5, 2) (0.1, 2) (1e-2, 2) (0.2, 2) (5e-2, 2) (2e-2, 2) (2e-2, 2)

JIAS (σR, t, λ) (0.1, 10,1e-2)

(1e-2, 3,1e-3)

(1e-2, 3,1e-3)

(1e-2, 5,5e-3)

(1e-2, 5,5e-3)

(1e-2, 3,1e-2)

(5e-2, 5,5e-3)

(5e-2, 5,1e-2)

(1e-2, 3,1e-2)

(1e-2, 3,1e-2)

JLLM (σR, ε, λ) (0.1, 0.4,1e-2)

(1e-2, 1e-2,1e-2)

(1e-2, 1e-2,1e-3)

(1e-2, 0.4,1e-3)

(1e-2, 0.165e-3)

(1e-2, 1e-2,1e-2)

(5e-2, 0.2,5e-3)

(5e-2, 4e-2,1e-2)

(5e-2, 1e-2,1e-2)

(5e-2, 0.1,1e-2)

intensity transition (gray arrow). This variance would lead tochanging the tone appearance in visual (see Fig. 1(e)), andmay cause unexpected influence in some applications, e.g. tonemapping (see Fig. 13(c)). The comparable edge-preservingeffect of WLS and JIAS is presented in Block a2. But inBlock a3, the result of WLS appears unexpectedly incompletesmoothing in the strong oscillation. By contrast, the result ofJIAS approach obtains a sound texture smoothing.

Fig. 7(b) shows the comparison of our JLLM (σR=0.05,ε=0.0005, λ=0.3) and BF (σS=4, σR=0.05). The result of BFpresents obvious edge preserving in Block b2, but the textureswhich should be smoothed are mistreated as edges in Block b1and b3. This mistreated output is usually difficult for manipula-tion, and produces irregular ruins in visual, as illustrated in Fig.1(d). With our LLM scheme, the texture is suppressed, alongwith an obvious edge blurring. However, with edge correction,our JLLM achieves the texture smoothing as well as edgepreserving.

C. Visual Analysis

To analyze the visual performance of the state-of-the-artapproaches and our approaches, we choose Gaussian filter(GF), median filter (MF), Paris’s bilateral filter (BF) [2],Farbman’s weighted least squares filter (WLS) [3], He’s guidedfilter (GuiF) [34], Chao’s improved anisotropic diffusion (IAD)[4], Kass’s local mode filtering (LMF) [5], Fattal’s edge-avoiding wavelets filtering (EAW) [6] and Xu’s L0 gradientminimization (L0GM) [7], and present the comparisons in Fig.1, Fig. 8 and Fig. 9.

In Fig. 1, we choose the Barbara image to test the compar-ative approaches individually because it contains complicatedtextures and edges. In Fig. 1(b), Gaussian filter [1] producesa sound texture smoothing but causes serious edge blurring.In Fig. 1(c), median filter [1] produces strange artifact whichseems like that in Fig. 6(b). From (d) to (i), the textures aresmoothed incompletely, although the edges are well preserved.Note that, WLS, IAD, LMF and L0GM produce over-flattedregions in the filtered output. Our JIAS balances the texture


(a) (b) (c) (d)

Inpu

tG

FM

FB

FW

LS

Gui

FIA

DL

0GM

JIA

SJL

LM

Fig. 8. Compare the results of the state-of-the-art EPS approaches with those of our JLLM and JIAS on different types of images (first row). There areartifacts (a), textures (b), gray (c) and color natural images (d). We extract eight samples as inputs from these four types of images (two for each), and marktheir positions by red and blue boxes in the corresponding images. In the extracted subfigures, from top to bottom, are the results of Gaussian filter (GF) [1],median filter (MF) [1], bilateral filter (BF) [10], weighted least squares filter (WLS) [3], guided filter (GuiF) [34], improved anisotropic diffusion (IAD) [4],L0 gradient minimization (L0GM) [7], and our JIAS and JLMM.


JIA

S(σ

R=

0.01)

(a) (b) (c) (d)

JLL

M(σ

R=

0.05

)

(e) (f) (g) (h)

Fig. 9. The visual performance with different parameter settings. We emphasize on the effect of the variance w.r.t. parameters t, ε and λ. The results ofJIAS are shown in the top row. (a) t=3, λ=0.01. (b) t=3, λ=0.05. (c) t=9, λ=0.01. (d) t=9, λ=0.05. And the results of JLLM are shown in the bottom row. (e)ε=0.02, λ=0.01. (f) ε=0.02, λ=0.05. (g) ε=0.1, λ=0.01. (h) ε=0.1, λ=0.05. We can see various smoothing in Lena’s hairs and well-preserving in the shoulders.

smoothing and the edge preserving and avoid the defects ofthe above comparative approaches.

We test 10 samples with a variety of groups of parametersin Fig. 8. The tested samples are divided into 4 types,including artifacts, textures, gray and color natural images.Fig. 8 illustrates the filtered results of extracted subfigures,and Tab. I lists all the corresponding parameter assignments.

Artifacts in Fig. 8(a) are made up of noise (top-left), stripewith grey transition (top-right), chessboard (bottom-left) andripple (bottom-right). GF, IAD, JIAS and JLLM present asound noise reduction. MF, WLS, and LOGM do not removethe extreme (black) noise, and BF and GuiF have a fewresidual artifacts. Except GF, other approaches are good atedge preserving in the stripe and chessboard. For the ripple,MF, BF, WLS and GuiF produce the irregular filtered outputs.IAD and L0GM have similar outputs which flat the largergaps. Our JIAS and JLLM flat not only the larger gaps butalso the smaller ones.

Textures in Fig. 8(b) are made up of branches and leaves(top-left), bark (top-right), weave (bottom-left) and crystal(bottom-right). The results of MF, BF, WLS and GuiF pro-duce some neither texture smoothing nor edge preservingphenomenon, which seems like a slight haze over the output.The results of IAD and L0GM are similar, containing a fewtrivial patches. In our results, all the details are removed, sothey seems smoother than those of IAD and L0GM.

Gray (Statue) and color (Lena) natural tested images areshown in Fig. 8(c) and (d), respectively. MF, BF, WLS andGuiF produce the output still with the haze phenomenon, andIAD and L0GM produce some trivial patches in the filteredoutput, but our JIAS and JLLM do not.

In Fig. 9, we present our JIAS and JLLM with differentparameter settings. According to the range of the parametersetting in Fig. 5, we adjust the smooth factor t (JIAS) andε (JLLM) and their edge corrected factor λ, and observethe performance of the output. Note that, with the parametervarying, the Lena’s hairs are smoothed and the boundary ofthe shoulder is preserved separately.

TABLE IITIME COST OF EACH STAGE IN OUR JIAS AND JLLM WITH DIFFERENTSIZES OF 8-BIT GRAY IMAGES. ALL THE RECORDS ARE EVALUATED IN

MATLAB 2010A (UNIT: SEC.).

Param. (256)2 (512)2 (1024)2 (1600)2 (2048)2

IASt=3t=5t=10

0.1940.2320.257

0.2660.3290.471

0.6800.9481.601

1.6492.8015.444

4.3187.11513.90

LLMε=0.4ε=0.16

0.0350.031

0.1970.193

0.8140.801

2.0101.944

3.4753.451

ECλ=1e-2λ=1e-3

0.3310.367

1.1921.449

4.4264.790

10.9611.98

17.8819.87

JBFσR=0.1σR=1e-3

0.1130.115

0.2020.215

0.6900.700

1.5681.615

2.5432.455

JIASσR=0.1t=5

λ=1e-20.674 1.811 6.141 15.40 27.34

JLLMσR=5e-2ε=0.1λ=1e-2

0.492 1.552 5.858 14.52 23.97

D. Time Analysis

To demonstrate the efficiency of our approaches, we recordthe time cost of each step in our JIAS and JLLM with specific


sizes of Barbara image and parameter assignments. This timerecord is under MATLAB 2010a, and Tab. II shows the recordsin detail. We found that LLM is more efficient than IAS (t>3),especially as the iterative times and image sizes increase. Inaddition, the major cost is in the edge correction. As mentionedpreviously, we adopt Chen’s bilateral grid [21] as our JBFimplementation. Since we only apply the framework of JBF,our approach does not depend on the specific data structureor algorithms. Generally speaking, an improved acceleratingimplementation is feasible, e.g. the O(1) approaches [27]. Forthe medium size image (<1 million pixels), our JIAS andJLLM can obtain a sound time response in processing.

V. APPLICATIONS

Image decomposition based on edge-preserving smoothingis a fundamental processing for many image applications. Pariset al. [9] summarized a variety of applications on the basisof bilateral filter in their course, including denoising, high-dynamic-range tone mapping, data fusion, etc. And Farbmanet al. [3] took advantage of the multi-scale operators tofurther improve the ability of edge-preserving decomposition,and applied it in detail enhancement, etc. A sound suppres-sion of local textures during edge-preserving smoothing hassure benefit in applications. Subr et al. [12] demonstrateda local texture suppression based image decomposition forsome applications. Inspired by their work, we apply our JIASand JLLM approaches in different texture-related applications,including edge detection, detail enhancement, texture transfer,and so on.

(a) (b) (c)

(d) (e) (f)

Fig. 10. Edge detection. All the subfigures are from the results of Fig. 1and the results are obtained by the Canny operator [1] with threshold 0.1.(a) A portion of Barbara image. (b) Most of textures are treated as edges.Some residual textures are retained in (c) to (e), which are obtained by theBF, WLS, and LE approaches, respectively. The result of our JIAS approachhas clearer edges and less residual textures in (f).

A. Edge Detection

Detecting salient edges is a basic operation in image pro-cessing and computer vision. Although lots of classical edgedetection approaches have been proposed [1], it is still a hardwork to detect the edges exactly. The main reason is that edges

and textures have similar characteristics of intensity variance,see Fig. 10(a). If we apply edge detection for the originalimage, lots of textures are treated as edges and retained, seeFig. 10(b). Applying the popular edge-preserving smoothingapproaches, to a certain extent, highlights some salient edges,but causes the interference of residual textures as well. Thedetected results of BF [10] and WLS [3] are presented in Fig.10(c) and (d), respectively. As pointed by the red arrows, someresidual textures are treated as edges obviously. Subr et al. [12]proposed a WLS-based local extrema approach to suppressthe textures, eliminating the interference of textures during thedetection, see Fig. 10(e). Unfortunately, some residual texturesstill lie near the edges. With our JIAS, the degenerative imagesuppresses most textures, and the edge correction and JBFfiltering eliminate the residual textures near the salient edges.Therefore, a clear and edge-continuous result is obtained, seeFig. 10(f).

B. Noise Reduction

Besides the edge-preserving property, EPS filter also has asound noise reduction. We took the Pepper image as the testedsample, adding the zero-mean Gaussian noise with σ=0.05 [1].In our experiment, we found that the haze effect appeared inthe output of BF when we increased the value of σS . Andsome extreme noise would be retained in the outputs of WLSand IAD. Our JIAS can remove the added Gaussian noise butthe appearance tends to be non-photorealistic. Fig. 11 showsthe comparison of noise reduction.

C. Multi-scale Detail Enhancement

Edge-preserving smoothing is an important operation forimage detail manipulation. Especially, the multi-scale de-composition can provide more subtle results in the detailenhancement. On the basis of WLS optimization framework[3], Farbman et al. emphasized on the ability of detailmanipulation which is based on the edge-preserving multi-scale decomposition. Subsequently, Fattal took advantage ofthe second-generation wavelets to construct the edge-avoidingdecomposition [6], and obtained a sound enhanced effect. Surbet al. [12] considered the property of local extreme suppressionto manipulate the details. And Paris et al. [39] applied themulti-scale local Laplacian filter to be aware of the detailsfor manipulation. Inspired by the multi-scale decomposition,our JLLM and JIAS approaches can be also applied to detailmanipulation. Fig. 12(b) is the fine-scale filtered results of Fig.12(a) by JLLM (left) and JIAS (right), respectively. The resultsare displayed in pseudo color visualization. Note that, theedges are well preserved and the textures are well smoothed.The detail enhanced results of JLLM and JIAS are shown inFig. 12(c) and (d), respectively.

D. Tone mapping

The high-dynamic-range (HDR) image tone mapping isan important application in image enhancement. Durand andDorsey [18] firstly applied the bilateral filter to tone map-ping, demonstrating the effectiveness of the edge-preserving


(a) (b) (c)

(d) (e) (f)

Fig. 11. Noise reduction. (a) Input. (b) Zero-mean Gaussian noised image(σ = 0.05). (c) BGF (σS = 15, σR = 0.5) [21]. (d) WLS (α = 1.8, λ =3.5) [3]. (e) Anisotropic diffusion (T = 600,K = 120) [4]. (f) JIAS (σR =0.5, t = 3, λ = 0.003).

(a) (b)

(c) (d)

Fig. 12. Multi-scale detail manipulation. (a) Input image is obtained fromKodak’s true color dataset. (b) Fine-scale filtered images are obtained byJLLM (left) and JIAS (right) approaches, which are mapped by pseudo color.And the detail enhancements of our JLLM and JIAS approaches are in (c)and (d), respectively. All the operations are under the YUV color space.

smoothing approach for the HDR image. But limited by theGaussian property of BF, the halo artifacts appear in theresults of bilateral-based approaches commonly, see Fig. 13(b).Choudhury et al. [16] proposed a trilateral filter to improvethe unsatisfactory effects. Subsequently, Farbman et al. [3]exploited the gradient constraint property of WLS optimizationframework to alleviate the halo artifacts. Furthermore, He etal. [34] proposed a guided filter (GuiF) that not only hasthe bilateral-like property but also has the similar gradientconstraint property of WLS. Fig. 13(c) and (d) show theresults of WLS-based and GuiF-based tone mapping. Notethat, the reduction of intensity in the WLS-based mappingis still obvious, although the gamma correction can improvethe intensity. The result of GuiF is more colorful than that ofWLS-based approach, but a slight halo appears and the toneshifts. Recently, Paris et al. [39] applied the local Laplacian

(a) (b) (c)

(d) (e) (f)

Fig. 13. High dynamic range tone mapping. (a) Input over-exposure image.(b) The result of bilateral-based approach [18] has obvious halo artifacts. (c)The result of WLS approach [3] has obvious intensity reduction. (d) The resultof Guided Filter [34] has slight halo artifacts near the textures. (e) The resultof LLF approach [39]. (f) The result of our JIAS approach has no halo artifactnear the textures and demonstrates a sound detail appearance.

filter (LLF) to balance the tone and detail, and obtained asound effect. Fig. 13(e) shows the result of LLF withoutdetail enhancement. With our JIAS, the halos are overcomecompletely. And the result is more colorful than those of bothWLS and LLF, see Fig. 13(f).

E. Texture Transfer

For most ancient oil paintings, some cracks appear as his-torical trace. These cracks have the characteristics of textures,w.r.t. netty, dense and subtle. In the digital edition of theancient painting, these cracks are usually eliminated with theshrinkage of image size, leading to loss of the historicalsense. Therefore, how to enhance or restore the historicalsense of a painting is an interesting topic. Actually, we canconsider separating these netty textures from the originalimage, and then transfer them to the non-cracked image tocreate the historical sense. Inspired by the structure-texturedecomposition [44], we implement this transfer process. InFig. 14, we exploit the local texture suppression of our JLLMapproach, separate the structure and texture component, andobtain the texture transfer result. Fig. 14(a) shows the portionof netty textures from Mona Lisa painting. The extractedimage is filtered by our JLLM approach, and the texture detailis obtained by subtracting with the original image. Fig. 14(c)shows the filtered result (left) and the texture detail (right).By putting the separated textures to the Chardin’s work (Fig.14(b)), the final result are obtained in Fig. 14(d).


(a) (b)

(c) (d)

Fig. 14. Texture transfer. We transfer the cracks from a part of Mona Lisa (a)to the target image (b). The filtered image by our JLLM approach is shownin the left part of (c). The netty cracks are extracted in the YIQ color space,and they are shown in the right part of (c). Finally, these cracks are appliedto the target image in (d).

(a) (b)

(c) (d)

Fig. 15. Non-photorealistic rendering. The input image is from RetargetMedataset [45]. (b) The bilateral filter based approach [46]. (c) The median filterbased approach [1]. Both the BF and MF would produce granular effect onthe spindrift and the wave in (b) and (c). On the contrary, our JLLM approachdoes not produce granular effect in (d). We recommend the electronic versionfor a high resolution.

F. Non-photorealistic Rendering

Because the edge-preserving smoothing approaches empha-size on the importance of both edge preserving and localsmoothing, they satisfy the needs of non-photorealistic ren-dering (NPR) image applications, which need to outline theedge contours and flat local regions. Winnemoller et al. [46]proposed a bilateral-based real-time image abstraction, andBhat et al. also emphasized on the NPR applications in their

(a) (b)

(c) (d)

Fig. 16. Hatching to tone. (b) The result of median filter [1]. Both theresults of local extrema [12] in (c) and our JIAS approach in (d) yield a goodestimation of the tone while preserving the edges of hatched regions. But ourresult is clearer at the thin edges than that of LE, such as the edges aroundshoulder area of the monster sculpt. And the tone appearance is more closeto the original image.

screened Poisson equation [32] and GradientShop [33]. As pre-viously mentioned, if the image contains textures, traditionaledge-preserving smoothing approaches produce unexpectedpreserving or smoothing effect, leading to serious inferenceon the applications. Illustrated as Fig. 15(b) and (c), the NPRresults of both bilateral-based and median-based approacheshave the grain-like effect in the abundant texture regions, e.g.the spindrift and the wave in Fig. 15(a). The reason of thisphenomenon is that the textures are mistreated as edges andpreserved incompletely. However, owning to the suppressionof the textures in our JLLM approach, the grain-like effect issuppressed at the same time (Fig. 15(d)).

In addition, for hatching or stippling images, it is hardto construct their NPR effects. The reason is that the con-tent of the image is consisted of sketch draws, representedas the texture-like drawing. Exploiting the traditional edge-preserving smoothing approaches, it is difficult to distinguishthe edges and the textures. The classical median filter [1] canachieve a certain texture smoothing for NPR, but the shapesof contents may be changed, see Fig. 16(b). Subr et al. tookadvantage of their local extrema (LE) approach [12] to obtaina sound NPR result, see Fig. 16(c). And we apply our JIASapproach and obtain a comparable result to that of LE.

VI. CONCLUSION AND FUTURE WORK

How to obtain both properties of texture-smoothing andedge-preserving is a challenging problem in filtering. In thispaper, we fill the gap between the linear filter and the EPSfilter by taking the JBF framework with joint filtering schemes.


We propose a novel edge-preserving texture suppression filterto obtain a satisfactory result which has both propertiesof texture-smoothing and edge-preserving. We develop aniterative asymmetric sampling and the local linear modeldegenerative schemes to suppress the textures, respectively.Then an edge correction operator is used to achieve thepurpose of edge preserving for the degenerative image. JBF isa bridge to link the properties of texture smoothing and edgepreserving, and provides a feasible way for a better balancebetween them. In addition, we apply a effective acceleratingapproach to improve the time performance.

Strength and weakness. In our JIAS and JLLM approaches,we integrate both properties of texture smoothing and edgepreserving, and avoid some complicated models to recognizethe textures explicitly. Our approach merely exploits the corre-sponding spatial characteristics of the edges and the textures.Generally speaking, the advantages of both linear filter andEPS filters are merged in a natural way. But our approachesstill have some weakness. The slight edge blurring still existsand some parameters in the specific applications are stillempirically. A failed example is shown in Fig. 17. In ourexperiments, we found that for the animal fur or hairs, thefiltered output would be unacceptable. In addition, the filteroutputs are lack of a determined quality measurement andvisual dependence as well.

(a) (b) (c)

Fig. 17. The failures in JIAS and JLLM. (a) Input image. (b) The filteredresult of JIAS (σR = 0.1, t = 20, λ = 0.001). With over-smoothing thetexture, it is likely to produce blocked artifacts. (c) The filtered result of JLLM(σR=0.05, ε=0.16, λ=8e-4). With under-smoothing the texture, the irregularresidual textures appear.

In the future work, we will consider the structure-texturedecomposition as a choice of degeneration in our pipeline.We hope to further reduce the amount and the sensitivity ofparameters in the implement. In addition, we will continueto extend our approaches to more applications, e.g. texturereplacement, etc.

REFERENCES

[1] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed.Prentice Hall, Upper Saddle River, NJ., 2008.

[2] S. Paris and F. Durand, “A fast approximation of the bilateral filter usinga signal processing approach,” International Journal of Computer Vision,vol. 81, no. 1, pp. 224–52, 2009.

[3] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, “Edge-preservingdecompositions for multi-scale tone and detail manipulation,” ACMTransactions on Graphics, vol. 27, no. 3, pp. 67–76, 2008.

[4] S. M. Chao and D. M. Tsai, “An improved anisotropic diffusion modelfor detail- and edge-preserving smoothing,” Pattern Recognition Letters,vol. 31, no. 13, pp. 2012–2023, 2010.

[5] M. Kass and J. Solomon, “Smoothed local histogram filters,” ACMTransactions on Graphics (Proceedings of ACM SIGGRAPH 2010),vol. 29, no. 4, pp. 100:1–10, 2010.

[6] R. Fattal, “Edge-avoiding wavelets and their applications,” ACM Trans-actions on Graphics (Proceedings of ACM SIGGRAPH 2009), vol. 28,no. 3, pp. 22:1–10, 2009.

[7] L. Xu, C. Lu, Y. Xu, and J. Jia, “Image smoothing via l0 gradientminimization,” in SIGGRAPH Asia 2011, 2011.

[8] J. Baek and D. Jacobs, “Accelerating spatially varying gaussian filters,”ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia2010), vol. 29, no. 6, pp. 169:1–10, 2010.

[9] S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, “Bilateral filtering:Theory and application,” Computer Graphics and Vision 2008, 2008.

[10] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and colorimages,” The sixth International Conference on Computer Vision, pp.839–846, 1998.

[11] J. V. D. Weijer and R. V. D. Boomgaard, “Local mode filtering,” inProceedings of IEEE Computer Society Conference on Computer Visionand Pattern Recognition, CVPR 2001, vol. 2, 2001, pp. 428–433.

[12] K. Subr, C. Soler, and F. Durand, “Edge-preserving multiscale imagedecomposition based on local extrema,” ACM Transactions on Graphics(Proceedings of ACM SIGGRAPH Asia 2009), vol. 28, no. 5, pp. 147:1–9, 2009.

[13] Z. Farbman, R. Fattal, and D. Lischinski, “Diffusion maps for edge-aware image editing,” ACM Transactions on Graphics (Proceedings ofACM SIGGRAPH Asia 2010), vol. 29, no. 6, pp. 145:1–10, 2010.

[14] G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, andK. Toyama, “Digital photography with flash and no-flash image pairs,”ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH2004), vol. 23, no. 3, pp. 664–672, 2004.

[15] E. Eisemann and F. Durand, “Flash photography enhancement viaintrinsic relighting,” ACM Transactions on Graphics, vol. 23, no. 3, pp.673–678, 2004.

[16] P. Choudhury and J. Tumblin, “The trilateral filter for high contrastimages and meshes,” in Proceedings of the 14th Eurographics workshopon Rendering, 2003, pp. 186–196.

[17] H. Takeda, S. Farsiu, and P. Milanfar, “Kernel regression for imageprocessing and reconstruction,” IEEE Transactions on Image Processing,vol. 16, no. 2, pp. 349–366, 2007.

[18] F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic-range images,” ACM Transactions on Graphics, vol. 21, no. 3,pp. 257–266, 2002.

[19] T. Q. Pham and L. J. V. Vliet, “Separable bilateral filtering for fastvideo preprocessing,” in IEEE International Conference on Multimediaand Expo, ICME 2005, 2005.

[20] S. Paris and F. Durand, “A fast approximation of the bilateral filter usinga signal processing approach,” Computer Vision–ECCV 2006, vol. 3954,pp. 568–580, 2006.

[21] J. Chen, S. Paris, and F. Durand, “Real-time edge-aware image process-ing with the bilateral grid,” ACM Transactions on Graphics, vol. 26,no. 3, 2007.

[22] A. Adams, N. Gelfand, J. Dolson, and M. Levoy, “Gaussian kd-trees forfast high-dimensional filtering,” ACM Transactions on Graphics, vol. 28,no. 3, pp. 21:1–12, 2009.

[23] A. Adams, J. Baek, and M. A. Davis, “Fast high-dimensional filteringusing the permutohedral lattice,” Computer Graphics Forum, vol. 29,no. 2, pp. 753–762, 2010.

[24] B. Weiss, “Fast median and bilateral filtering,” ACM Transactions onGraphics, vol. 25, no. 3, pp. 519–526, 2006.

[25] S. Yoshizawa, A. Belyaev, and H. Yokota, “Fast gauss bilateral filtering,”Computer Graphics Forum, vol. 29, no. 1, pp. 60–74, 2010.

[26] F. Porikli, “Constant time o(1) bilateral filtering,” in IEEE Conferenceon Computer Vision and Pattern Recognition (CVPR), 2008, pp. 1–8.

[27] Q. X. Yang, K. H. Tan, and N. Ahuja, “Real-time o(1) bilateral filtering,”in IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2009, pp. 557–564.

[28] Q. X. Yang, S. N. Wang, and N. Ahuja, “Svm for edge-preserving filter-ing,” in IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2010, pp. 1775–1782.

[29] Q. X. Yang, “Recursive bilateral filtering,” in ECCV, 2012, pp. 399–413.[30] E. S. L. Gastal and M. Oliveira, “Domain transform for edge-aware

image and video processing,” ACM Transactions on Graphics, vol. 30,no. 4, pp. 69:1–69:12, 2011.

[31] E. S. L. Gastal and M. M. Oliveira, “Adaptive manifolds for real-timehigh-dimensional filtering,” ACM Transactions on Graphics, vol. 31,no. 4, pp. 33:1–33:13, 2012.

[32] P. Bhat, B. Curless, M. Cohen, and C. Zitnick, “Fourier analysis of the2d screened poisson equation for gradient domain problems,” ComputerVision–ECCV 2008, pp. 114–128, 2008.


[33] P. Bhat, C. L. Zitnick, M. Cohen, and B. Curless, “Gradientshop: Agradient-domain optimization framework for image and video filtering,”ACM Transactions on Graphics, vol. 29, no. 2, pp. 1–14, 2010.

[34] K. M. He, J. Sun, and X. O. Tang, “Guided image filtering,” ComputerVision–ECCV 2010, vol. 6311, pp. 1–14, 2010.

[35] P. Perona and J. Malik, “Scale space and edge detection using anisotropicdiffusion,” IEEE Transactions on Pattern Analysis and Machine Intelli-gence, vol. 12, no. 7, pp. 629–639, 1990.

[36] J. Weickert, Anisotropic Diffusion in Image Processing. Teubner-Verlag,Stuttgart, Germany, 1998.

[37] S. M. Chao, D. M. Tsai, W. Y. Chiu, and W. C. Li, “Anisotropicdiffusion-based detail-preserving smoothing for image restoration,” inProceedings of IEEE 17th International Conference on Image Process-ing, 2010, pp. 4145–4148.

[38] M. Felsberg, P. E. Forsseen, and H. Scharr, “Channel smoothing: Effi-cient robust smoothing of low-level signal features,” IEEE Transactionson Pattern Analysis and Machine Intelligence, vol. 28, no. 2, pp. 209–222, 2006.

[39] S. Paris, S. W. Hasinoff, and J. Kautz, “Local laplacian filters: Edge-aware image processing with a laplacian pyramid,” in ACM Transactionson Graphics (Proceedings of ACM SIGGRAPH 2010), 2011.

[40] R. Fattal, M. Agrawala, and S. Rusinkiewicz, “Multiscale shape and de-tail enhancement from multi-light image collections,” ACM Transactionson Graphics, vol. 26, 2007.

[41] A. Lev, S. W. Zucker, and A. Rosenfeld, “Iterative enhancemnent ofnoisy images,” Systems, Man and Cybernetics, IEEE Transactions on,vol. 7, no. 6, pp. 435–442, 1977.

[42] Y. Wang, J. Yang, W. Yin, and Y. Zhang, “A new alternating minimiza-tion algorithm for total variation image reconstruction,” SIAM Journalon Imaging Sciences, vol. 1, no. 3, pp. 248–272, 2008.

[43] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assess-ment: from error visibility to structural similarity,” IEEE Transactionson Image Processing, vol. 13, no. 4, pp. 600–612, april 2004.

[44] J. F. Aujol, G. Gilboa, T. Chan, and S. Osher, “Structure-textureimage decomposition–modeling, algorithms and parameter selection,”International Journal of Computer Vision, vol. 67, no. 1, pp. 111–136,2006.

[45] M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparativestudy of image retargeting,” ACM Transactions on Graphics (Proceed-ings of ACM SIGGRAPH Asia 2010), vol. 29, no. 5, pp. 160:1–10, 2010.

[46] H. Winnemoller, S. C. Olsen, and B. Gooch, “Real-time video abstrac-tion,” ACM Transactions on Graphics, vol. 25, no. 3, pp. 1221–1226,2006.

Zhuo Su (S’13) is a Ph.D. candidate of the NationalEngineering Research Center of Digital Life, Schoolof Information Science and Technology, Sun Yat-sen University. He received the Master and Bache-lor degree in software engineering from School ofSoftware, Sun Yat-sen University in 2010 and 2008.His research interests include image processing andanalysis, computer vision, and computer graphics.

Xiaonan Luo is a professor of School of Infor-mation and Science Technology, Sun Yat-sen Uni-versity. He is the director of National EngineeringResearch Center of Digital Life and the director ofDigital Home Standards Committee on InteractiveApplications of China Electronics StandardizationAssociation. He won the National Science Fundfor Distinguished Young Scholars granted by theNational Nature Science Foundation of China. Hisresearch interests include image processing, com-puter graphics & CAD, mobile computing.

Zhengjie Deng received the Bachelor and Masterdegrees in computer science from the Fudan Uni-versity, the Ph.D. degree in computer applicationstechnology from the Sun Yat-sen University, in 2002,2005 and 2011 respectively. His research interestsinclude graphics deformations, image processing,computer animations and modeling.

Yun Liang received the M.S. and Ph.D. degree inInformation Science and Technology from Sun Yat-sen University in 2005 and 2011 respectively. She isa researcher in South China Agricultural Universityand the National Engineering Research Center ofDigital Life in China. Her research interests includeimage processing, computer vision and pattern anal-ysis.

Zhen Ji (M’04) received the B.E. and Ph.D. de-grees from Xi’an Jiaotong University, Xi’an, China,in 1994 and 1999, respectively. He is currently aProfessor with the Department of Computer Science,College of Computer Science and Software Engi-neering, Shenzhen University, Shenzhen, China. In2001, 2003, and 2004, he was an Academic Visitorwith the Department of Electrical Engineering andElectronics, University of Liverpool, Liverpool, U.K.Since 2002, he has been the Director of the TexasInstruments DSPs Laboratory, Shenzhen University.

His current research interests include digital image processing, computationalintelligence, bioinformatics, and digital signal processors.

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. X, NO. X, JANUARY ... · IEEE TRANSACTIONS ON MULTIMEDIA,...

Documents

Transcript of IEEE TRANSACTIONS ON MULTIMEDIA, VOL. X, NO. X, JANUARY ... · IEEE TRANSACTIONS ON MULTIMEDIA,...