A Convex Optimization Approach for Depth Estimation Under Illumination Variation Wided Miled,...
-
date post
19-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of A Convex Optimization Approach for Depth Estimation Under Illumination Variation Wided Miled,...
1
A Convex Optimization Approach for DepthA Convex Optimization Approach for DepthEstimation Under Illumination VariationEstimation Under Illumination Variation
Wided Miled, Student Member, IEEE, Jean-Christophe Wided Miled, Student Member, IEEE, Jean-Christophe Pesquet, Senior Member, IEEE, and Michel ParentPesquet, Senior Member, IEEE, and Michel Parent
2
AbstractAbstract
• Illumination changes cause serious problems in many computer vision applicatIllumination changes cause serious problems in many computer vision applications. Aions. A spatially varying multiplicative model is developed to account for spatially varying multiplicative model is developed to account for brightness changes induced between brightness changes induced between left and right viewsleft and right views..
• ThThee recovery of the depth information of a scene from recovery of the depth information of a scene from stereo images is an activstereo images is an active e area of researcharea of research in computer in computer vision. The need for an accurate and dense deptvision. The need for an accurate and dense depth map arises inh map arises in many applications such as many applications such as autonomous navigation,autonomous navigation, 3-D reconst3-D reconstructionruction and and 3-D television3-D television..
2
3
I. INTRODUCTIONI. INTRODUCTION
• Feature-based methods:Feature-based methods: Extract salient features from both images, such as Extract salient features from both images, such as edgesedges, , segmentssegments, or , or curvescurves. .
An interpolation step is required if a An interpolation step is required if a densedense map is desired, but map is desired, but accurateaccurate..
• Region-based methods:Region-based methods: It have the advantage of directly generating dense disparity estimates by It have the advantage of directly generating dense disparity estimates by
correlation over local windows, but correlation over local windows, but not accuratenot accurate..
Many global stereo algorithms have, therefore, been developed based on Many global stereo algorithms have, therefore, been developed based on dynamic programmingdynamic programming, , graph cutsgraph cuts, or , or belief propagationbelief propagation. . Variational Variational approachesapproaches have also been very effective for solving the matching problem have also been very effective for solving the matching problem globallyglobally
4
II.MODEL FOR ILLUMINATION VARIATIONSII.MODEL FOR ILLUMINATION VARIATIONS
The intensity of an image pixel:The intensity of an image pixel:
IIii(s) = (s) = ρρ(s) R(s) Rii(n(s))(n(s)), for i , for i ∈∈ l,r﹛ ﹜l,r﹛ ﹜。。
Assuming that the stereo images have been rectified, so that the geometry of Assuming that the stereo images have been rectified, so that the geometry of
the cameras can be considered as horizontal epipolar, and using the the cameras can be considered as horizontal epipolar, and using the
Image Irradiance Equation:Image Irradiance Equation:
IIrr( x-u(s), y ) = v(s) I( x-u(s), y ) = v(s) Ill(s)(s)
4
5
II.MODEL FOR ILLUMINATION VARIATIONSII.MODEL FOR ILLUMINATION VARIATIONS
The disparity The disparity uu and illumination and illumination vv can be computed by can be computed by minimizingminimizing the the
following cost function based on the following cost function based on the sum of squared differences (SSD)sum of squared differences (SSD) metric: metric:
Ĵ( u, v ) = ∑s D ∈ [ v(s)Il(s) – Ir( x-u(s), y )]2 , D⊂N2
This expression is This expression is nonconvexnonconvex with respect to the displacement field with respect to the displacement field uu. Thus, . Thus,
to avoid a to avoid a nonconvex minimizationnonconvex minimization, we , we assume thatassume that IIrr is a is a differentiabledifferentiable
function and we consider a function and we consider a TaylorTaylor expansion of the nonlinear term expansion of the nonlinear term
IIrr( x-ū, y )( x-ū, y ) around an initial estimate around an initial estimate ūū as follows: as follows:
Ir( x-u, y ) ≈ Ir( x-ū, y ) - ( u-ū ) I∇ rx( x-ū, y )
6
II.MODEL FOR ILLUMINATION VARIATIONSII.MODEL FOR ILLUMINATION VARIATIONS
To simplify the notations:To simplify the notations:
Ĵ( u, v ) ≈ ∑s D ∈ [ L1(s)u(s) + L2(s)v(s) – r(s) ]2
where L1(s) = I∇ r
x( x-ū, y ), L2(s) = Il(s), r(s) = Ir( x-ū(s), y ) + ū(s)L1(s)
Our goal is to simultaneously recover u and v. Thus, setting Our goal is to simultaneously recover u and v. Thus, setting w = ( u, v)w = ( u, v)TT and and
L = [ LL = [ L1, 1, LL22]] , we end up with the following quadratic criterion to be minimized: , we end up with the following quadratic criterion to be minimized:
JJDD( w ) = ∑s D ∈ [ L(s)w(s) – r(s) ]2
7
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• FindFind
w S=∩∈ i=1m
Si
such that such that
J(w) = inf J(S)
where
J: H→]-∞,+∞] is a convex function.J: H→]-∞,+∞] is a convex function.
(S(Sii))1≤i ≤m1≤i ≤m are closed convex sets of H. are closed convex sets of H.
Constraint sets can be modelled as level sets :Constraint sets can be modelled as level sets :
∀i { 1,…,m }, S∈ i = { w H | f∈ i(w) ≤ δi }
where where
∀ ∀i { 1,…,m }, f∈i { 1,…,m }, f∈ ii:H →R is continuous convex function:H →R is continuous convex function
(δ(δii) ) 1≤i ≤m 1≤i ≤m are real-valued parameters.are real-valued parameters.
8
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• A. Global Objective Function ( 1 / 2 )A. Global Objective Function ( 1 / 2 ) The initial disparity estimate ū:The initial disparity estimate ū:
ū(x,y) = arg minu U ∈ ∑(i,j) β∈ [ βx,y(u) Il(x+i,y+j) – Ir(x+i-u,y+j) ]2
wherewhere
U N is the search disparity set⊂U N is the search disparity set⊂ 。。
βcorresponds to the matching block centered at the pixel (x,y)βcorresponds to the matching block centered at the pixel (x,y) 。。 ββx,yx,y(u) is the following least squares estimate of the illumination factor for (u) is the following least squares estimate of the illumination factor for
block β:block β:
ββx,yx,y(u)=(u)=∑(i,j) β∈ Il(x+i,y+j)Ir(x+i-u,y+j) /∑(i,j) β∈ Il(x+I,y+i)2
The initial illumination field ϋ :The initial illumination field ϋ :
ϋ(x,y) = βϋ(x,y) = βx,y x,y ( ( ū(x,y) ))
9
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• A. Global Objective Function ( 2 / 2 )A. Global Objective Function ( 2 / 2 )
JD\O(w) = ∑s D\O ∈ [ L(s)w(s) – r(s) ]2
J(w) = ∑s D\O ∈ [ L(s)w(s) – r(s) ]2 + α∑s D∈ | w(s) - ŵ(s) .2
2
where
ŵ = (ū, ϋ) is an initial estimate as described aboveϋ) is an initial estimate as described above
| .2 denotes the Euclidean norm in R denotes the Euclidean norm in R22
α is a positive constant is a positive constant
10
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• B. Convex ConstraintsB. Convex Constraints
1) Constraints on the Disparity Image: 1) Constraints on the Disparity Image: Total Variation Based Regularization:Total Variation Based Regularization:
For a differentiable analog image u defined on a spatial domain For a differentiable analog image u defined on a spatial domain Ω
TV(u) = ∫Ω| u(s) | ds∇
where
∇u denotes the gradient of u
Sa1 = { (u,v) H | TV(u) ≤ T∈ u }
where
a : stands for analog constraint sets .a : stands for analog constraint sets .
11
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• B. Convex ConstraintsB. Convex Constraints
1) Constraints on the Disparity Image:1) Constraints on the Disparity Image: Disparity Range Constraint:Disparity Range Constraint:
SSaa2 2 == { (u,v) H | u∈ min≤ u ≤ umax }
12
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• B. Convex ConstraintsB. Convex Constraints
1) Constraints on the Disparity Image:1) Constraints on the Disparity Image: Nagel–Enkelmann Based Regularization:Nagel–Enkelmann Based Regularization:
wherewhere
I denotes the 2 2 identity matrixI denotes the 2 2 identity matrix
r is chosen according to gradient norm value ranger is chosen according to gradient norm value range
| I|<<r:uniform areas, | I|>>r:edge∇ ∇| I|<<r:uniform areas, | I|>>r:edge∇ ∇
13
III. SET THEORETIC ESTIMATIONIII. SET THEORETIC ESTIMATION
• B. Convex ConstraintsB. Convex Constraints
2) Constraints on the Illumination Field:2) Constraints on the Illumination Field: Tikhonov Based Regularization:Tikhonov Based Regularization:
Illumination Range Constraint:Illumination Range Constraint:
SSaa5 5 == { (u,v) H | v∈ min≤ v ≤ vmax }
where
vmin = 0.8
vmax = 1.2
14
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
NNββis the total number of is the total number of
pixels inβpixels inβNNββis the total number of is the total number of
pixels inβpixels inβ
15
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
(x0, y0) is (128, 128)(x0, y0) is (128, 128)α is the standard deviation of the α is the standard deviation of the illumination change.illumination change.
(x0, y0) is (128, 128)(x0, y0) is (128, 128)α is the standard deviation of the α is the standard deviation of the illumination change.illumination change.
16
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
δδssis fixed to 1is fixed to 1δδssis fixed to 1is fixed to 1
17
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
18
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
19
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
20
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
21
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
22
IV. EXPERIMENTAL RESULTSIV. EXPERIMENTAL RESULTS
23
24
Thank you for your Thank you for your listening ! listening !
The more you The more you learn,learn, the more you the more you know.know. The more you The more you know,know, the more the more
you you forget.forget. The more you The more you forget, forget, the less you knowthe less you know.. 2009.09.012009.09.01