Image Matting Using Linear Optimization

Image Analysis and Interpretation

Conrad Triquell

Reminder: What is image matting ?

• A composite image is treated as a linear combination of foreground and background images.

• Compositing equation:

• Where:

The alpha matte of a pixel defines the contributions from foreground/background

User interaction for Matting

1 equation 3 unknowns (grayscale images) 3 equations 7 unknowns (color images)

The image matting problem is severely under-constrained User interaction helps embed constrains into the image

Idea and how we handle the problem?

• Novel matting approach that converts the image matting problem into a simple linear optimization problem.

• 1st) F, B, X, X’ are estimated simultaneously by solving a

system of linear equations using the least square error (LSE) .

• 2nd) Alpha is obtained with the estimated F, B, X, X’ by solving another system of linear equations using LSE estimation.

Note that X, X’ are medium variables.

Formulation

• Define medium variables:

• Compositing equation becomes:

• From I, find X,X’,F and B. From X calculate alpha.

Calculating X, X’, F and B

• P denote the set of pixels of the unknown region and W be a window of size n x n centred at pixel i.

• We enforce a smoothness constraint on F and B for each color channel c= r, g or b. Our goal is to minimize the cost function:

where

• First terms of J1 and J2 are from compositing equations

• Second term are the smoothness constrains.

• Every term in J1 can be rewritten in the form:

where :

represents the values of Xc (or Bc) in the unknown region in vector form.

a=

• Then J1(Xc, Bc)

J2(X’c,Fc)

• Minimizing J1, derivative of J1 with respect to z and set derivative to 0

Therefore we have

For J2 :

• Since J1 and J2 are independent,

are also the closed form solution to minimizing

• By examining J1 and J2 we can see that the solution can also be obtained by solving the following system of over-determined linear equations using LSE estimation:

Calculating α (alpha)

• Given X, X’, F and B:

(Considered medium variable)

from which with additional constraints, the same alpha value for pixel i can be achieved.

• Combining the two equations

• Compute the mean and the variance of the components

• Now our goal is to minimize

where μ are two factors used to balance

In the first term the variance in the exponential allows

more or less deviation of from Second term enforces a smoothness contraint on alpha • J’ is similar to J1 so it can also be rewritten in matrix form

alpha are the values in the unknown region

• The derivation of D and m is similar to A and y we have done before. By minimizing we have:

• Again this result is the same as the result of the LSE estimation of the following over-determined linear equations:

• Note: matrices A and D are large size, which may cause the out-of-memory problem implementation when P is large. Fortunately, these matrices are sparse.

Experimental Result

Comparative results of alpha mattes and composed images with blue background

a) Simulated alpha b) Composed image of a fire foreground, simulated alpha and blue background c) Composed image with a background taken from e) d) Trimap f) Total errors after 2000 experiments

Quantitative evaluation on synthesized images.

Alpha matte and new composed images with new background images

Conclusion

• Novel algorithm to transform the ill-posed image matting problem into two over-determinated optimization problems.

• Closed-form solutions by enforcing the smoothless constraints and introducing two medium variables.

• Steps algorithm: 1) Foreground and background are estimated. 2) The matte is pulled with the estimated foreground and

background. Each step can be solved using linear optimitzation. • High quality image matting

Thanks!

http://www.ee.columbia.edu/~zgli/papers/MM07_Image_Matting.pdf

Reference