Polynomial Embeddings for Quadratic Algebraic Equations
Radu Balan
University of Maryland, College Park, MD 20742
Math-CS Joint Lecture, Drexel University Monday April 23, 2012
Overview
1. Introduction: Motivation and statement of problem
2. Invertibility Results in the Real Case3. The Algebraic Approach:
a. Quasi-Linear Embeddingsb. Hierarchical Embeddingsc. Numerical Analysis
4. Modified Least Square Estimator5. Theoretical Bounds: CRLB6. Performance Analysis
2/50
1. Introduction: Motivation
Byx BA=Identity
Question: What if |y| is known instead (that is, one looses the phase information) ?
?|| xAxz
Where is important: X-Ray Crystallography, Speech Processing
knowns?
Inversion of Nonlinear Transformations
y = A x
3/50
1. Introduction: Statement of the Problem
Reconstruction from magnitudes of frame coefficients
a complete set of vectors (frame) for the n-dimensional Hilbert space H (H=Cn or H=Rn).
Equivalence relation: x,yH, x~y iff there is a scalar z, |z|=1 so that x=zy(real case: x=y ; complex case: x=eiy). Let .
Define
Problems: 1. Is an injective map?2. If it is, how to invert it efficiently?
4/50
H
Rm
x(x)
f1
f2
fm
5/50
Real Case: K=RTheorem [R.B.,Casazza, Edidin, ACHA(2006)] is injective iff for any subset JF either J or F\J
spans Rn.Corollaries [2006]• if m 2n-1, and a generic frame set F, then is
injective;• if m2n-2 then for any set F, cannot be injective;• if any n-element subset of F is linearly independent,
then is injective; for m=2n-1 this is a necessary and sufficient condition.
2. Invertibility Results: Real Case (1)
6/50
Invertibility Results: Real Case (2)Real Case: K=R
Theorem [R.B.(2012)] is injective iff any one of the following equivalent conditions holds true:
• For any x,yRn, x≠0, y≠0,
• There is a constant a>0 so that for all x, RI
7/50
Invertibility Results: Real Case (3) complete set of vectors (frame) for H=Rn.
One would expect that if is injective and m>2n-1 then there is a strict subsetJ{1,2,…,m} so that π is injective, where π:RmR|J| is the restriction to J index.
However the next example shows this is not the case.
Example. Consider n=3, m=6, and F the set of columns of the following matrix
F =
Note that for any subset J of 3 columns, either J or F\J is linearly dependent.Thus is injective but removing any column makes π not injective.
8/50
3. The Algebraic Approach3.1 Quasi-Linear EmbeddingsExample (a)Consider the real case: n=3 , m=6.Frame vectors:
Need to solve a system of the form:
9/50
[1 2 2 1 2 11 4 −2 4 −2 11 −2 −4 1 4 41 −4 −6 4 12 91 2 −14 1 −14 491 0 2 0 0 1
] [𝑦1
𝑦2
𝑦3
𝑦 4
𝑦5
𝑦6
]=[4949
1964
]𝑠 𝑜𝑙𝑣𝑒 𝑓𝑜𝑟 𝑦→𝑦=[
1−121−24
]Then factor:
Thus, we obtain:
10/50
Summary of this approach:
where
11/50
Summary of this approach:
where
12/50
Example (b)Consider the real case: n=3 , m=5.Frame vectors:
Need to solve the system:
Is it possible? How?
X
13/50
|𝑥1+𝑥2+𝑥3|=2
|𝑥1+2 𝑥2−𝑥3|=3|𝑥1−𝑥2−2𝑥3|=2
|𝑥1−2𝑥2−3𝑥3|=3
|𝑥1+𝑥2−7 𝑥3|=14
→
𝑥12+2𝑥1𝑥2+2𝑥1𝑥3+𝑥2
2+2𝑥2𝑥3+𝑥32 ¿ 4
𝑥12+4 𝑥1𝑥2−2 𝑥1𝑥3+4 𝑥2
2−2𝑥2𝑥3+𝑥32 ¿ 9
𝑥12−2 𝑥1 𝑥2−4 𝑥1 𝑥3+𝑥2
2+4 𝑥2 𝑥3+4 𝑥32 ¿ 4
𝑥12−4 𝑥1𝑥2−6 𝑥1𝑥3+4 𝑥2
2+12𝑥2𝑥3+9 𝑥32 ¿ 9
𝑥12+2 𝑥1𝑥2−14 𝑥1𝑥3+𝑥2
2−14 𝑥2𝑥3+49𝑥32 ¿ 196
Let’s square again:
We obtained 5 linear equations with 6 unknown monomials.
Idea: Let’s multiply again these equations (square and cross)
New equations: : 15 equations
New variables (monomials): : 15 unknowns
14/50
Summary of this approach:
where
15/50
3.2 Hierarchical Embeddings
Primary data:
Level d embedding: ,
Identify:
Then
a homogeneous polynomial of total degree 2d.
16/50
How many monomials?
Real case: number of monomials of degree 2d in n variables:
Complex case: …Number of degree (d,d) in n variables:
Define redundancy at level d:
(R.B. [SampTA2009])
17/50
Real case:
18/50
Real case:
19/50
Real case:
20/50
Complex case:
21/50
Complex case:
22/50
Complex case:
23/50
Fundamental question: How many equations are linearly independent?
Recall: , Note:
and
The matrix is not canonical, and so is * for d>1.However its range is basis independent.
We are going to compute this range in terms of a canonical matrix.
24/50
Theorem The following hold true:1. (as a quadratic form)
2. Rows of are linearly independent iff
where is the mdxmd matrix given by
Real case:Complex case:
25/50
Let denote the Gram matrixAnd for integer p.
Theorem In either real or complex case:
Hence for d=1, the number of independent quadratics is given by:
Theorem For d=2,
Remark Note the k1=k2,l1=l2 submatrix of is 26/50
3.3. Numerical analysisResults for the complex case: random frames
n=3,m=627/50
n=3,m=6 28/50
n=16m=64
29/50
n=32m=128
30/50
n=4m=16
31/50
n=4m=14
Note 6 zero eigenvalues of instead of 5.32/50
n=4m=15
Number of zero eigenvalues = 20 =120-100, as expected.33/50
4. Modified Least Square Error Estimator
• Model: mifxd iii 1 , ,2
“Vanilla” Least-Square-Estimator (LSE):
m
iiixLSE fxdx
1
22,minargˆ
Tm
i
Tiiiii xxXffFdXFtr
, ,
:asly equivalentcriterion theRewrite
1
2
We modify this criterion in two ways:1. Replace X by a rank r positive matrix Y2. Regularize the criterion by adding a norm of Y
34/50
)ˆ(.)ˆ(.ˆ
)()(minargˆ1
2)(,0
QeigvectprincYeigvalprincx
YtrdYFtrYm
iiirYrankY
Use a rank-r factorization of Y to account for constraint:
)1(:,ˆˆ :ionfactorizat SDV
minargˆ
1,1
1
2
UxVUL
LLtrdLFLtrL Tm
iii
TRL rn
35/50
Optimization procedure
LLtrdLFLtrL Tm
iii
TRL rn
1
2minargˆ
Our approach: 1. Start with a large and decrease its value over
time;2. Replace one L in the inner quadratic term by a
previous estimate3. Penalize large successive variations.
36/50
KKtrKLKLtrLLtrdKFLtrKLJ Tt
m
i
Tt
Ttii
Tt
1
2),(
1
,minarg )()1(
ttLLJL t
tLt
Algorithm (Part I)Step I: Initialization
Step II: Iteration
Step III: Factorization
1
00)0(
,for policy adaptationan Choose0,,, Initialize
ttt
tL
)1(:,ˆ 1,1
)(
UxVULSVD
How to initialize?How to adapt?
Convergence?
37/50
Initialization
m
iii
m
ii
Tm
i
Tii
T
FdQ
dLQLtrLLtrdLFLtrLJ
L
1
1
2
1
2
:where
22)(
: small and largeFor
21 , : of eigenpairs be ,Let eeveQvQve kkkkk
rr vvvL ||| 2211)0(
Set:
m
i
r
k ik
krfv
rree1 1
20
10,2
,/... 38/50
Convergence
KKtrKLKLtrLLtrdKFLtrKLJ Tt
m
i
Tt
Ttii
Tt
1
2),(
Consider the iterative process:
)()1(
)()1(
,:
,minarg:tt
tt
ttL
t
LLJj
LLJL
Theorem Assume (t)t,(µt)t are monotonically decreasing non-negative sequences. Then (jt)t≥0 is a monotonically decreasing convergent sequence.
LKJKLJ tt ,, :Note
39/50
5. Theoretical bounds: CRLB
m
iii
iiii
mmdfxxdpoodLoglikelih
diiNmifxdModel
1
22
2
22
)log()2log(2
,2
1)|(log :
...),,0(~ ; 1 , , :
jk
jk xxxdpEI )|(log: :Matrixn InformatioFisher
2
,
m
i
Tiii fffxRRI
1
2
2 , where, 4
12
1
4CO : RItimatorUnbiasedEsVARCRLB
𝐸 [‖𝑥− �̂�‖2 ]≥ 𝜎2
4𝑡𝑟 (𝑅−1 )
𝐸 [‖𝑥 𝑥𝑇− �̂� �̂�𝑇‖2 ]≥ 𝜎2
2[‖𝑥‖2 𝑡𝑟 (𝑅−1 )+𝑥𝑇 𝑅− 1𝑥 ] 40/50
The LSE/MLE is a biased estimator. Modified CRLB for biased estimators:
xxBiasxEx
LBModifiedCRBiasBiasx
Ix
xxxxE TT
T
)( , ˆ)(
:ˆˆ 1
Asymptotically (for high SNR):
,
,41 ,
2
1
112
Idx
fRfRffxBiasm
iii
Tii
).(..44
6114
12
10
tohRRRMSE
MSE
T
MSE
41/50
6. Performance Analysis
n=3 , m=9 , d = 1 (dlevel) , r = 2 and , decrease by 5% every step w/ saturation (subspace)
Mintrace algorithm: Candes, Strohmer, Voroninski (2011)
42/50
n=3 , m=9, d=1, r=243/50
n=3 , m=9, d=1, r=244/50
n=8 , m=24, d=3, r=245/50
n=8 , m=24, d=3, r=246/50
n=8 , m=24, d=3, r=247/50
n=9 , m=27, d=3, r=248/50
n=9 , m=27, d=3, r=249/50
n=9 , m=27, d=3, r=250/50
Top Related