Representation and Reconstruction of 3D Shapes in Computer ... Pontes_Thesis.… ·...
Transcript of Representation and Reconstruction of 3D Shapes in Computer ... Pontes_Thesis.… ·...
Robotics
11/02/2020
QUT Verified Signature
P F
{wfp = (xfp, yfp)T | f = 1, ..., F, p = 1, ..., P} .
W =
⎡⎢⎢⎣w11 . . . w1P
wF1 . . . wFP
⎤⎥⎥⎦ .
W 2F × P
M S
W = MS .
W
2 × 3 Rf
2F × 3 P
WM
S
W
Rf 2× 3
M
S 2F × 3 3 × P
W
W = UDV T ,
D U V T
U
3× 3 D V T
W ′ = U ′D′V ′T U ′D′V ′T
‖W −W ′‖2F s.t. rank(W ′) = 3 .
M = U ′√D′ S =√D′V ′T
S
Rf
G
MS = M(GG−1)S = (MG)(G−1S) = MS .
G MG
iTf GGT if = 1
jTf GGT jf = 1
iTf GGT jf = 0 .
Q = GGT
Q
MQ Q−1S
P F
{wfp = (xfp, yfp)T | f = 1, ..., F, p = 1, ..., P}
W =
⎡⎢⎢⎣w11 . . . w1P
wF1 . . . wFP
⎤⎥⎥⎦ .
W 2F × P
M S W = MS
W
2F × 3F
R1...Rf 2× 3
3F × P
P
WM
S
W 10P × 10s × 30fps × 2(x, y) = 6, 000
3 × 10
900 × 10
K
B1...BK
cf = [cf1, cf2, ..., cfK ]
Sf =
K∑d=1
cfdBd ,
Sf 3×P f Bd 3×P
K cfd
B1, ..., BK c1, ..., cK
Wf = Rf (
K∑d=1
cfdBd) .
K
W = M(C ⊗ I3)B = ΠB ,
I3 3×3 B Π = M(C⊗ I3)
W Π
R ∈ 2×3
WΠ
R ∈ 2×3
W K
K
K
Π ∈ 2F×3K B ∈ 3K×P
3K × 3K Q
Q Π
Ri
Q
Mf = [cf1Rf ...cfKRf ] Π
Q
MS = MQQ−1S G = QQT
Qk Q
k Mk = MQk Mk, k = 1, ...,K
M Mk
kth cfk
M M = MQ
MfGkMTf = cfkcfkRfR
Tf ,
Gk = QkQTk 3K × 3K
Qk Mf
ith
MfGkMTf = c2fkI2 ,
I2 2×2 Gk
cfk
M2f−1GkMT2f−1 − M2fGkM
T2f = 0
M2f−1GkMT2f = 0 .
Gk
QHkQT Hk
Gk
W = ΠGG−1B Π = M(C ⊗ I3)
Π
Qk ∈ 3K×3K Qk = GkGTk Π2f−1:2fQkΠ
T2f−1:2f = c2fkI2
cfk Qk
Π2f−1QkΠT2f−1 = Π2fQkΠ
T2f
Π2f−1QkΠT2f = 0 .
Qk
2K2−K
A
W
vec(.) qk = vec(Qk)
vec(AXBT ) = (B ⊗A)vec(X)
[Π2f−1 ⊗ Π2f−1 − Π2f ⊗ Π2f
Π2f−1 ⊗ Π2f
]qk = Afqk = 0 ,
Π2f−1, Π2f 2f − 1th 2fth Π
(f = 1, ..., F )
Aqk = 0 ,
A = [AT1 , A
T2 , ..., A
TF ]
T qk (3K)(3K+1)/2
2F ≥ (3K)(3K +
1)/2 qk
2K2 −K
A
Qk
2K2 −K
A
{Avec(Qk) = 0} ∩ {Qk � 0} ∩ {rank(Qk) = 3} .
Qk
trace(Qk) s.t.
Qk � 0 ,
Avec(Qk) = 0 .
Qk Gk ∈ 3K×3
Gk
K
K
K(LK
)L
2×3
S F × 3P S
S K K
B S = C B
C ∈ F×L K Π = M(C ⊗I3) Π
2× 3 M 2× 3
W L
K
L
3× 2
B,Π‖W T −BΠ‖2F s.t. ‖Πi‖0,3 ≤ K, i = 1 : N/2 ,
‖Πi‖0,3 3 × 2 Π
B
Π
S
2 × 3
3K × 3K
3× 3 3L× 3L
G
Π B
G
643
G(V, E)V
E
α
α
R3
R3
(s, t, u) = o + s + t + u ,
o ,
0<s<1, 0<t<1 0<u<1 i,j,k
i,j,k = o +i
l+
j
m+
k
n,
l m n
l+1,m+1 n+1 , ,
(s, t, u) =
l∑i=0
m∑j=0
n∑k=0
Bi,l(s)Bj,m(t)Bk,n(u) i,j,k,
Bθ,n(x)
n
i,j,k i, j, k
n
Bθ,n(x) =
(n
θ
)xθ(1− x)n−θ,
(nθ
)i,j,k
= ,
∈ RN×3 ∈ R
N×M
∈ RM×3 N
M
t s
u
l,m, n=3( , , ) i,j,k
l,m, n=3
S TS
T
S( S , S) T ( T , T )
ST S ′( ′, S)
Δ
1
2‖ − ( A ⊗ 3)( +ΦΔ )‖22 +
γ
2‖ΦΔ ‖22.
∈ R3P
vec([ T ]A) [ T ]A ∈ R3×P
P A A⊗
∈ R3M vec( �) M
Δ ∈ R3M
Φ ∈ R3M×3M
x−Φ
L2
γ L2
Δ
x−, y−z− Φ
vec(·)
sdist
sdist =1
| ′|∑∈ ′
f( ,ST ; θ) +1
| T |∑∈ T
f( ,S ′; θ),
f( ,S; θ) =
⎧⎨⎩1 dist( ,S) > θ
0
θ
sIoU =V ′ ∩ VTV ′ ∪ VT
,
V ′ VT
G sdist < θdist sIoU > θIoU
θdist θIoU
G
Ω Sc( c, c)
i ∈ Ω
S( , )
= αc c +∑i∈Ω
αiic, = c,
α
G∈ R
2×P P
S
L
∈ R2×3 ∈ R
2×1
s
Δ , ,s,
1
2‖ L −
(( L ⊗ s )( +ΦΔ ) +
)‖22
+γ
2‖ΦΔ ‖22, � = 2.
L
LΔ
Φ
L2 γ
Δ , , ,
1
2‖ L −
(( L ⊗ )( +ΦΔ ) +
)‖22
+γ
2‖ΦΔ ‖22, � = s2 2, = ,
s
Lρ( , ,Δ , ,Λ) =1
2‖ L −
(( L ⊗ )( +ΦΔ ) +
)‖22
+γ
2‖ΦΔ ‖22 + 〈Λ, − 〉F +
ρ
2‖ − ‖2F ,
Λ ρ
〈., .〉F
k = Lρ( , k−1,Δ k−1, k−1,Λk−1),
� = s2 2;
k = Lρ(k, ,Δ k−1, k−1,Λk−1);
Δ k =Δ
Lρ(k, k,Δ , k−1,Λk−1);
k = Lρ(k, k,Δ k, ,Λk−1);
Λk = Λk−1 + ρ( k − k).
Lρ( , k−1,Δ k−1, k−1,Λk−1) =[(σ1 + σ2)/2
(σ1 + σ2)/2
]�,
− Λρ = [ σ1
σ2 ]� σ
− Λρ
Lρ(k, ,Δ k−1, k−1,Λk−1) =
(( L − ) � +Λ+ ρ
)( � + ρ )+,
= unvec( +ΦΔ ) L ∈ R3×|L| |L|
ΔLρ(
k, k,Δ , k−1,Λk−1) =
(Φ�( L ⊗ )�( L ⊗ )Φ+ γΦ
)+(Φ�( L ⊗ )�
(L − ( L ⊗ ) +
)).
Lρ(k, k,Δ k, ,Λk−1) =
∑l∈L( l − l)
|L| .
G
S α Sc( c, c)
Ω
S( , )
= αc c+∑
i∈Ω αiic c
ic
unvec(·)
Sc Sic i
i α
s, ,α
1
2
∑l∈L
‖ l −(s (αc[ c]l +
∑i∈Ω
αi[ic]l) +
)‖22
+ μ
N∑l=1
C(s (αc[ c]l +
∑i∈Ω
αi[ic]l) +
)+
γ
2
∑i∈Ω
α2i ,
l l l = 1, . . . , P
N Cμ γ
L2
∇αi
αi
∇αi =∑l∈L
(l −
(s (αc[ c]l +
∑i∈Ω
αi[ic]l) +
))�s [ i
c]l
+ μN∑l=1
∇C�s [ ic]l + γαi,
∇C
e[ξ]×
[·]× ∇ξ
ξ
∇ξ =∑l∈L
(l −
(s (αc[ c]l +
∑i∈Ω
αi[ic]l) +
))�
(s
∂[ξ]×ξj
(αc[ c]l +∑i∈Ω
αi[ic]l)
)
+ μ
N∑l=1
∇C�(s
∂[ξ]×ξj
(αc[ c]l +∑i∈Ω
αi[ic]l)
).
∇
∇ =∑l∈L
(l −
(s (αc[ c]l +
∑i∈Ω
αi[ic]p) +
))+ μ
N∑l=1
∇C.
s α
eRP
epose e3D
l,m, n = 3 4×4×4
sdist
sIoU sdist < θdist
sIoU > θIoU θdist θIoU
S Tsdist T
S ′ 1e−4 θdist = 1e−3
S ′
S ′
TS ′
T S ′
VT V ′ 0.1694 θIoU = 0.25
θdist θIoU
1283
eRP epose
e3D
eRP epose e3D eRP epose e3D eRP epose e3D eRP epose e3D
eRP epose e3D
S T T VT
S′ S′ V ′
ST
VT TS ′ V ′
S ′
T sdist S ′
sIoU
FFD
FFD
c Δ αG cG
Δα
c Δ
α G
G
Ω Sc( c, c)
i ∈ Ω
S( , )
= αc c +∑i∈Ω
αiic, = c,
α
G
= Φ( +Δ ),
∈ RN×3 ∈ R
N×M
∈ RM×3 N M
Δ
Φ ∈ R3M×3M
Δ
α
c Δ α
Δ α
Δ
α
{8, 16, 32} {5, 3, 3} {3, 3, 3}
{16, 8, 1} {3, 3, 5} {3, 3, 3}220× 220
2202 → 722 → 242 → 82 → 242 → 722 → 2202
∈ R2,048
8 × 3 × 3
Δ α
f : → {Δ ,α}
κ ∈ RMN M N
α
κ ∈ R126
(32×3) α
α
Gα α
256 × 192
70 30
sdist sdist
dist3D =1
| |∑∈
dist( ,S) + 1
| |∑∈
dist( , S),
S S
IoU =V ∩ VV ∪ V
,
V V
MSE ∼ t Acc( ) Prec( ) Rec( ) ∼ t MSE ∼ t dist3D IoU
Acc Prec Rec t
GG
Δα
G
G
α
α
43
Δ ∈ R32×3
α ∈ R30
κ ∈ R126
c κ
c
Δ
= Φ( + Δ )
α
= αc c +∑
i∈Ω αiic
dist3D
dist3D
IDX
FFD LC
IDXGT
dist3D IoU dist3D IoU
IDX FFD LC
IDX + LCGT
IDX + FFDGT + LCGT IDX + FFD + LCGT IDX
1e−3 1e−5
dist3D IoU dist3D IoU dist3D IoU dist3D IoU
IDX FFD LCFFD LC
dist3D IoU dist3D IoU dist3D IoU dist3D IoU
IDXFFD LC IDX
IDXGT
dist3D IoU dist3D IoU dist3D IoU dist3D IoU dist3D IoU
2.62 −3 . −2.16 −5 . −3.59 −5 . −
. −2.58 −5 . −
1.34 −3 . −
IDX FFDLC
FFDLC IDX
FFD LCGT
FFD LCGT
LC FFDGT
1e−3
Δ α
Δ TΔ
γ
c ∈ Rnv×3
∈ Znf×3, 0 ≤ Fij < nv i = [p, q, r]
p q r c = { , }c
c
λc
A B
λc(A,B) =∑a∈A b∈B
‖a− b‖2 +∑b∈B
a∈A‖b− a‖2.
λem
λem =φ:A→B
∑a∈A
‖a− φ(a)‖,
φ
IoU
(A,B) =|A ∩B||A ∪B| .
,
l,m n l+1,m+1, n+1 , ,
(s, t, u) =l∑
i=0
m∑j=0
n∑k=0
Bil(s)Bjm(t)Bkn(u) ijk,
B·N (x)
N
ijk i, j, k
= ,
∈ RM×3 ∈ R
N×3 M
N ∈ RN×M
Δ (t)
T c(t) 0 ≤ t < T
(t) = (t) (t)
Δ˜ (qt)
˜(qt) = (t)(
(t) +Δ˜ (qt)).
γ(qt)
λ0 =∑q, t
f(γ(qt)
)λc
(s(q), s(qt)
),
f
γ(qt) = (1− εγ) γ(qt)0 +
1
Tεγ ,
γ(qt)0
t εγ 0 < εγ � 1
t∗ =t
γ(qt),
c(q) = {˜(qt∗), (t∗)}.
Δ˜
f
f(γ) = γ)
f
f(γ) = − (1− γ)
λe =∑t
γ(t)(γ(t)
),
γ(t) t
λ0
λ′e = λ0 + κeλe.
κe = e−b/b0κe0,
κe0 b b0
λr =∑q,t
γ(qt)|Δ˜ (qt)|2,
| · |2
λ′r = λ0 + κrλr,
κr κr0
εe
α = 0.25
192 × 192
1 × 1
512
4 l = m = n = 3
16, 384
1, 024
192×256×3
30◦ 45◦
192× 256× 36× 8× 256
1× 1 6× 8× 643, 072512
(t) 192 + 1
3 × 43 = 192 Δ˜ (qt)
γ(qt)
10−3 β1 = 0.9 β2 = 0.999 ε = 10−8
100, 000 b0 = 10, 000
εe = 0.02
323
εγ f(γ) κe0 κr0γ
− (1− γ)γγ
z ≥ 0 3.2
323
5
13
1000× (λc λem 1− )
λc λem
5
13
λc λem
λIoU = 0.33
5
13
1000 × λc
Template0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Norm
alizedfrequency
b
w
e
r
Template0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Norm
alizedfrequency
b
w
e
r
10−1 100
X
0.0
0.2
0.4
0.6
0.8
1.0
λc<X
b
w
e
r
10−2 10−1 100
X
0.0
0.2
0.4
0.6
0.8
1.0
λc<X
b
w
e
r
10−1
100
X
0.0
0.2
0.4
0.6
0.8
1.0
λc<X
T=1
T=2
T=4
T=8
T=16
T=30
0.0 0.2 0.4 0.6 0.8 1.0
X
0.0
0.2
0.4
0.6
0.8
1.0
IoU>X
T=1
T=2
T=4
T=8
T=16
T=30
T > 1T = 1
T = 1
T ∈ {2, 4, 8, 16}N
±
Γ(t)
φ(x, t) : Ω×R �→ R
φ
Γ(t) = {x : φ(x, t) = 0} ,
φ(x, t)
Γ
φ
Γ
φ
∂Γ
∂t= vn,
Γ v
n
Γ
∂Γ
∂t= −∂E(Γ)
∂Γ= vn.
φ
Γ
∀tφ(Γ(t), t) = 0.
∂φ(Γ(t), t)
∂t= 0,
∂φ(Γ(t), t)
∂t=
∂φ
∂Γ
∂Γ
∂t+
∂φ
∂t= ∇φ
∂Γ
∂t+
∂φ
∂t,
∇φ = ∂φ∂Γ
∇φ‖∇φ‖
∂φ
∂t= −∇(φ)
∂Γ
∂t= −∇(φ)vn = −∇(φ)v(−1)
∇φ
‖∇φ‖ = v‖∇φ‖.
Γ v
φ v‖∇φ‖
φ
∂φ
∂t= v‖∇φ‖.
φ v
Γ
Γ S ⊂R3
xi ∈ Sni xi Γ S
ni
ni ∼ ±ni S m
X = {xi}mi=1 N = {ni}mi=1 dX (x)
X
d(x,X ) =y∈X
‖x− y‖2.
EX (Γ) =
(∫Γd(s,X )pds
)1/p
, 1 ≤ p ≤ ∞,
Γ ds
Lp Γ
XN
Γ
SLp Γ
N
EN (Γ) =
(∫Γ(1− |N(s) · nΓ(s)|)p ds
)1/p
, 1 ≤ p ≤ ∞,
N(s) = ni xi s
Γ
nΓ(s) =∇φ(s)
‖∇φ(s)‖ ,
EN (Γ)
EN (Γ) =
(∫Γ
(1−
∣∣∣∣N(s) · ∇φ(s)
‖∇φ(s)‖
∣∣∣∣)p
ds
)1/p
.
Γ
Earea =
∫Γds,
Evol =
∫Γds,
Γ ds
Γ
f(x) = ±y∈Γ
‖x− y‖,
f(x) > 0 x Γ
|∇f | = 1
Γ
Esdf (φ) =
∫(‖∇φ(x)‖ − 1)2dx.
n Ij
Sj = {X j ,N j}θ
I φ(I; θ)
θ
L(θ) =∑j∈D
EX j (Γ(Ij ; θ)) + α1
∑j∈D
EN j (Γ(Ij ; θ))
+ α2
∑j∈D
Esdf (φ(Ij ; θ)) + α3
∑j∈D
Earea(Γ(Ij ; θ))
+ α4
∑j∈D
Evol(Γ(Ij ; θ)).
Γ φ
I Γ(I; θ) = {x : φ(I; θ) = 0} D = {1, ..., n} α1 − α4
δ H
∑j∈D
EX j (Γ(Ij ; θ)) =
=∑j∈D
(∫R3
δ(φ(x, Ij ; θ))d(x,X j)pdx
)1/p
,
∑j∈D
EN j (Γ(Ij ; θ)) =∑j∈D
(∫R3
δ(φ(x, Ij ; θ))
(1−
∣∣∣N j(x) · ∇φ(x; Ij , θ)
‖∇φ(x, Ij ; θ)‖
∣∣∣)pdx
)1/p
,
∑j∈D
Esdf (φ(Ij ; θ)) =
∑j∈D
∫R3
(‖∇φ(x, Ij ; θ)‖ − 1)2dx,
∑j∈D
Earea(Γ(θ, Ij)) =
∑j∈D
∫R3
δ(φ(x, Ij ; θ)) dx,
∑j∈D
Evol(Γ(θ, Ij)) =
∑j∈D
∫R3
H(φ(x, Ij ; θ)) dx.
Ω
C1 C2
δ H
δε(x) =
{12ε
(1 + (πxε )
), |x| ≤ ε,
0, |x| > ε,
Hε(x) =
⎧⎪⎨⎪⎩
12
(1 + x
ε +1π (πxε )
), |x| ≤ ε,
1, x > ε,
0, x > −ε,
H ′ε(x) = δε(x)
Lε φ
φj(x) = φ(x, Ij ; θ)) dj(x)p = d(x,X j)p
Lε(θ) =∑j∈D
(∑x∈Ω
δε(φj(x))dj(x)p
)1/p
+ α1
∑j∈D
(∑x∈Ω
δε(φj(x))
(1−
∣∣∣N j(x) · ∇φj(x)
‖∇φj(x)‖
∣∣∣)p)1/p
+ α2
∑j∈D
∑x∈Ω
(‖∇φj(x)‖ − 1)2 + α3
∑j∈D
∑x∈Ω
δε(φj(x))
+ α4
∑j∈D
∑x∈Ω
Hε(φj(x)).
Lε φ Ω
∂Lε
∂φ=
∑j∈D
1
p
(∑x∈Ω
δε(φj(x))dj(x)p
) 1−ppδ′ε(φ
j(x))dj(x)p
+α1
p
∑j∈D
(∑x∈Ω
δε(φj(x))
(1−
∣∣∣N j(x) · ∇φj(x)
‖∇φj(x)‖
∣∣∣)p) 1−p
p
(δ′ε(φ
j(x))(1−
∣∣∣N j(x) · ∇φj(x)
‖∇φj(x)‖
∣∣∣)p+
δε(φj(x))
∂
∂φ
(1−
∣∣∣N j(x) · ∇φj(x)
‖∇φj(x)‖
∣∣∣)p)
+ α2
∑j∈D
∑x∈Ω
(‖∇φj(x)‖ − 1) ∇ ·(
∇φj(x)
||∇φj(x)||
)
+ α3
∑j∈D
δ′ε(φj(x)) + α4
∑j∈D
δε(φj(x)).
Lε∂Lε
∂φφ
Γ φ
500 2000
20
80/20
p = 2 ε = 0.15 α1 = 0.8 α2 = 1
α3 = α4 = 0.1
E(p) = − 1
N
N∑n=1
[pn pn + (1− pn) (1− pn)],
p N
p
10−6
A B
(A,B) =|A ∩B||A ∪B| .
0 1
P1 P2
d (P1,P2) =1
|P1|∑x∈P1
y∈P2
||x− y||+ 1
|P2|∑y∈P2
x∈P1
||y − x||.
203 303
φ Δ φ Δ
203 303
Δ
203 303
φ Δ φ Δ
203
303 Δ
Δ
O(NM) N M
203 303
2563
203 203 303 303
203 203
303 303
203 203 303 303
203 203
303 303
203 203 303 303
203 203
303 303
203 203 303 303
203 203
303 303
203 203 303 303
203 203
303 303