On Linear-Programming Decoding of Nonbinary Expander Codes · LP decoding of nonbinary expander...
Transcript of On Linear-Programming Decoding of Nonbinary Expander Codes · LP decoding of nonbinary expander...
On Linear-Programming Decoding
of Nonbinary Expander Codes
Vitaly SkachekClaude Shannon Institute
University College Dublin
Supported by SFI Grant 06/MI/006
University College CorkMay 18, 2009
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Literature Survey
[Gallager ’62]Low-Density Parity-Check (LDPC) codes.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Literature Survey
[Gallager ’62]Low-Density Parity-Check (LDPC) codes.
→ Very efficient in practice.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Literature Survey
[Gallager ’62]Low-Density Parity-Check (LDPC) codes.
→ Very efficient in practice.
→ Difficult to analize.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Literature Survey
[Gallager ’62]Low-Density Parity-Check (LDPC) codes.
→ Very efficient in practice.
→ Difficult to analize.
[Wiberg ’96] [Koetter Vontobel ’03–’05]Graph covers, pseudocodewords and pseudoweights.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Literature Survey
[Gallager ’62]Low-Density Parity-Check (LDPC) codes.
→ Very efficient in practice.
→ Difficult to analize.
[Wiberg ’96] [Koetter Vontobel ’03–’05]Graph covers, pseudocodewords and pseudoweights.
[Feldman Wainwright Karger ’03–’05]Decoding of binary LDPC codes using linear-programming.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Literature Survey
[Gallager ’62]Low-Density Parity-Check (LDPC) codes.
→ Very efficient in practice.
→ Difficult to analize.
[Wiberg ’96] [Koetter Vontobel ’03–’05]Graph covers, pseudocodewords and pseudoweights.
[Feldman Wainwright Karger ’03–’05]Decoding of binary LDPC codes using linear-programming.
[Feldman et al. ’04] [Feldman Stein ’04]LP decoding on expander codes corrects a fraction oferrors, achieves capacity.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
This Work
LP decoding of nonbinary expander codes.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
This Work
LP decoding of nonbinary expander codes.
The decoder corrects a number of errors which isapproximately a quarter of a lower bound on the minimumdistance.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
This Work
LP decoding of nonbinary expander codes.
The decoder corrects a number of errors which isapproximately a quarter of a lower bound on the minimumdistance.
We consider:
Bipartite expander graph.Two different types of constituent codes.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
This Work
LP decoding of nonbinary expander codes.
The decoder corrects a number of errors which isapproximately a quarter of a lower bound on the minimumdistance.
We consider:
Bipartite expander graph.Two different types of constituent codes.
The proof does not use a separate assumption on thesymmetry of the LP polytope.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Code Construction
[Sipser Spielman ’95] [Barg Zemor ’01–’02]
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Code Construction
[Sipser Spielman ’95] [Barg Zemor ’01–’02]
Graph G = (V, E) is a ∆-regular bipartite undirectedgraph.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Code Construction
[Sipser Spielman ’95] [Barg Zemor ’01–’02]
Graph G = (V, E) is a ∆-regular bipartite undirectedgraph.
Vertex set V = A ∪ B such that A ∩ B = ∅ and|A| = |B| = n.Edge set E of size n∆ such that every edge in E has oneendpoint in A and one endpoint in B.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Code Construction
[Sipser Spielman ’95] [Barg Zemor ’01–’02]
Graph G = (V, E) is a ∆-regular bipartite undirectedgraph.
Vertex set V = A ∪ B such that A ∩ B = ∅ and|A| = |B| = n.Edge set E of size n∆ such that every edge in E has oneendpoint in A and one endpoint in B.Linear [∆, rA∆, δA∆] and [∆, rB∆, δB∆] codes CA and CB ,respectively, over F = Fq.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Code Construction
[Sipser Spielman ’95] [Barg Zemor ’01–’02]
Graph G = (V, E) is a ∆-regular bipartite undirectedgraph.
Vertex set V = A ∪ B such that A ∩ B = ∅ and|A| = |B| = n.Edge set E of size n∆ such that every edge in E has oneendpoint in A and one endpoint in B.Linear [∆, rA∆, δA∆] and [∆, rB∆, δB∆] codes CA and CB ,respectively, over F = Fq.
C is a linear code of length |E| over F:
C =
{
c ∈ F|E| :
(c)E(v) ∈ CA for every v ∈ A and
(c)E(v) ∈ CB for every v ∈ B
}
,
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Code Construction
[Sipser Spielman ’95] [Barg Zemor ’01–’02]
Graph G = (V, E) is a ∆-regular bipartite undirectedgraph.
Vertex set V = A ∪ B such that A ∩ B = ∅ and|A| = |B| = n.Edge set E of size n∆ such that every edge in E has oneendpoint in A and one endpoint in B.Linear [∆, rA∆, δA∆] and [∆, rB∆, δB∆] codes CA and CB ,respectively, over F = Fq.
C is a linear code of length |E| over F:
C =
{
c ∈ F|E| :
(c)E(v) ∈ CA for every v ∈ A and
(c)E(v) ∈ CB for every v ∈ B
}
,
where (c)E(v) = the sub-word of c that is indexed by theset of edges incident with v.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Example
Take k = 2, ∆ = 3, n = 4.Let GA and GB be generatingmatrices of CA and CB
(respectively) overF22 = {0, 1, α, α2}:
GA =
(
1 1 11 α 0
)
,
GB =
(
1 0 10 1 α
)
.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Example
Take k = 2, ∆ = 3, n = 4.Let GA and GB be generatingmatrices of CA and CB
(respectively) overF22 = {0, 1, α, α2}:
GA =
(
1 1 11 α 0
)
,
GB =
(
1 0 10 1 α
)
. v4
v3
v2
v1
u4
u3
u2
u1
A B0α
α2
0α
α2
α2
0α
1α
0
0
α2
1
α0
α
α2
α0
α2
α0
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Eigenvalues of Expander Graph
Assume that all vertices in G have degree ∆. The largesteigenvalue of the adjacency matrix AG of G is ∆.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Eigenvalues of Expander Graph
Assume that all vertices in G have degree ∆. The largesteigenvalue of the adjacency matrix AG of G is ∆.
Let λG be the second largest eigenvalue of AG .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Eigenvalues of Expander Graph
Assume that all vertices in G have degree ∆. The largesteigenvalue of the adjacency matrix AG of G is ∆.
Let λG be the second largest eigenvalue of AG .
Lower ratios of γG = λG
∆ imply greater values ζ ofexpansion. [Alon ’86]
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Eigenvalues of Expander Graph
Assume that all vertices in G have degree ∆. The largesteigenvalue of the adjacency matrix AG of G is ∆.
Let λG be the second largest eigenvalue of AG .
Lower ratios of γG = λG
∆ imply greater values ζ ofexpansion. [Alon ’86]
Expander graph with
λG ≤ 2√
∆ − 1
is called a Ramanujan graph. Constructions are due to[Lubotsky Philips Sarnak ’88], [Margulis ’88].
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Parameters of Expander Codes
Code Rate
RC ≥ rA + rB − 1.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Parameters of Expander Codes
Code Rate
RC ≥ rA + rB − 1.
Relative Minimum Distance
δC ≥ δAδB − γG√
δAδB
1 − γG.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
General Notation
Let the codeword c = (ce)e∈E ∈ C be transmitted and theword y = (ye)e∈E ∈ F
|E| be received.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
General Notation
Let the codeword c = (ce)e∈E ∈ C be transmitted and theword y = (ye)e∈E ∈ F
|E| be received.
Define the mapping
ξ : F −→ {0, 1}q ⊂ Rq ,
byξ(α) = x = (x(ω))ω∈F ,
such that, for each ω ∈ F,
x(ω) =
{
1 if ω = α0 otherwise.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
General Notation
Let the codeword c = (ce)e∈E ∈ C be transmitted and theword y = (ye)e∈E ∈ F
|E| be received.
Define the mapping
ξ : F −→ {0, 1}q ⊂ Rq ,
byξ(α) = x = (x(ω))ω∈F ,
such that, for each ω ∈ F,
x(ω) =
{
1 if ω = α0 otherwise.
Let Ξ(c) = (ξ(ce1) | ξ(ce2) | · · · | ξ(ce|E|)).
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function
For vectors f ∈ Rq|E|, we adopt the notation
f = (f e1| f e2
| · · · | f e|E|) ,
where∀e ∈ E, f e = (f (α)
e )α∈F .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function
For vectors f ∈ Rq|E|, we adopt the notation
f = (f e1| f e2
| · · · | f e|E|) ,
where∀e ∈ E, f e = (f (α)
e )α∈F .
For all e ∈ E, α ∈ F, we use the variables f(α)e ≥ 0.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function
For vectors f ∈ Rq|E|, we adopt the notation
f = (f e1| f e2
| · · · | f e|E|) ,
where∀e ∈ E, f e = (f (α)
e )α∈F .
For all e ∈ E, α ∈ F, we use the variables f(α)e ≥ 0.
Variables wv,b for all v ∈ V and all b ∈ C(v): relative
weights of local codewords b associated with E(v).
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function
For vectors f ∈ Rq|E|, we adopt the notation
f = (f e1| f e2
| · · · | f e|E|) ,
where∀e ∈ E, f e = (f (α)
e )α∈F .
For all e ∈ E, α ∈ F, we use the variables f(α)e ≥ 0.
Variables wv,b for all v ∈ V and all b ∈ C(v): relative
weights of local codewords b associated with E(v).
The objective function is∑
e∈E
∑
α∈Fγ
(α)e f
(α)e , where γ
(α)e
is a function of the channel output.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function (cont.)
For each α ∈ F we set
γ(α)e =
{
−1 if α = ye
1 if α 6= ye.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function (cont.)
For each α ∈ F we set
γ(α)e =
{
−1 if α = ye
1 if α 6= ye.
Let f e = ξ(β) for some e ∈ E, β ∈ F. Then,
∑
α∈F
γ(α)e f (α)
e =
{
−1 if β = ye
1 if β 6= ye.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Objective Function (cont.)
For each α ∈ F we set
γ(α)e =
{
−1 if α = ye
1 if α 6= ye.
Let f e = ξ(β) for some e ∈ E, β ∈ F. Then,
∑
α∈F
γ(α)e f (α)
e =
{
−1 if β = ye
1 if β 6= ye.
Suppose now that f = Ξ(z) for some z ∈ F|E|. It follows
that∑
e∈E
∑
α∈F
γ(α)e f (α)
e + |E| = 2d(y, z) ,
where d(y, z) is the Hamming distance between y and z.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Primal Problem
Maximize∑
e∈E,α∈F
(
−γ(α)e
)
· f (α)e
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Primal Problem
Maximize∑
e∈E,α∈F
(
−γ(α)e
)
· f (α)e
subject to ∀v ∈ V :∑
b∈C(v) wv,b = 1 ;
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Primal Problem
Maximize∑
e∈E,α∈F
(
−γ(α)e
)
· f (α)e
subject to ∀v ∈ V :∑
b∈C(v) wv,b = 1 ;
∀e = {v, u} ∈ E, ∀α ∈ F : f(α)e =
∑
b∈C(v) : be=α wv,b ,
f(α)e =
∑
b∈C(u) : be=α wu,b ;
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Primal Problem
Maximize∑
e∈E,α∈F
(
−γ(α)e
)
· f (α)e
subject to ∀v ∈ V :∑
b∈C(v) wv,b = 1 ;
∀e = {v, u} ∈ E, ∀α ∈ F : f(α)e =
∑
b∈C(v) : be=α wv,b ,
f(α)e =
∑
b∈C(u) : be=α wu,b ;
∀e ∈ E, α ∈ F : f(α)e ≥ 0 ;
∀v ∈ V, b ∈ C(v) : wv,b ≥ 0 .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Witness
Dual Witness Approach [Feldman et al. ’04]
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Witness
Dual Witness Approach [Feldman et al. ’04]
The codeword c ∈ C was transmitted.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Witness
Dual Witness Approach [Feldman et al. ’04]
The codeword c ∈ C was transmitted.
There is a feasible combination of values of the variableswv,b that corresponds to c.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Witness
Dual Witness Approach [Feldman et al. ’04]
The codeword c ∈ C was transmitted.
There is a feasible combination of values of the variableswv,b that corresponds to c.
Decoding Success
The sufficient criteria for the decoding success is that thissolution is the unique optimum of the primal LP decodingproblem.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Decoding Success
Primal Polytope
Dual Polytope
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Decoding Success
Primal Polytope
Dual Polytope
Max
Min
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Decoding Success
Primal Polytope
Dual Polytope
Max
Min
Feasible Point
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Decoding Success
Primal Polytope
Dual Polytope
Max
Min
Feasible Point
Dual Problem Variables
For each ω ∈ F, e ∈ E, and v ∈ V , such that v is an
endpoint of e, there is a variable τ(ω)v,e .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Decoding Success
Primal Polytope
Dual Polytope
Max
Min
Feasible Point
Dual Problem Variables
For each ω ∈ F, e ∈ E, and v ∈ V , such that v is an
endpoint of e, there is a variable τ(ω)v,e .
For each v ∈ V , there is a variable σv.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Problem
Minimize∑
v∈V σv
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Problem
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F : τ(ω)v,e + τ
(ω)u,e ≤ γ
(ω)e ;
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Dual Problem
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F : τ(ω)v,e + τ
(ω)u,e ≤ γ
(ω)e ;
∀v ∈ V, ∀b ∈ C(v) :∑
e∈E(v) τ(be)v,e + σv ≥ 0 .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Uniqueness
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ
(ω)u,e <γ
(ω)e ;
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Uniqueness
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ
(ω)u,e <γ
(ω)e ;
∀e = {v, u} ∈ E : τ(ce)v,e + τ
(ce)u,e ≤ γ
(ce)e ;
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Uniqueness
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ
(ω)u,e <γ
(ω)e ;
∀e = {v, u} ∈ E : τ(ce)v,e + τ
(ce)u,e ≤ γ
(ce)e ;
∀v ∈ V, ∀b ∈ C(v) :∑
e∈E(v) τ(be)v,e + σv ≥ 0 .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Feasible Point for the Dual: Value Assignments
We aim at the objective value to be |E| − 2d(y, c). This can beachieved by setting, for all v ∈ V , σv = 1
2∆ − d((y)E(v), (c)E(v)).
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Feasible Point for the Dual: Value Assignments
We aim at the objective value to be |E| − 2d(y, c). This can beachieved by setting, for all v ∈ V , σv = 1
2∆ − d((y)E(v), (c)E(v)).
ω = ce ω 6= ce
ye is correct τ(ω)v,e = −1
2 τ(ω)v,e = 1
2 − ǫ
ye is in error τ(ω)v,e = 1
2 τ(ω)v,e = −5
2 − ǫ or τ(ω)v,e = 3
2depends on the structure of the error
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Uniqueness (again)
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ
(ω)u,e < γ
(ω)e ;
∀e = {v, u} ∈ E : τ(ce)v,e + τ
(ce)u,e ≤ γ
(ce)e ;
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Uniqueness (again)
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ
(ω)u,e < γ
(ω)e ;
∀e = {v, u} ∈ E : τ(ce)v,e + τ
(ce)u,e ≤ γ
(ce)e ;
∀v ∈ V, ∀b ∈ C(v) :∑
e∈E(v) τ(be)v,e ≥ −σv .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Uniqueness (again)
Minimize∑
v∈V σv
subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ
(ω)u,e < γ
(ω)e ;
∀e = {v, u} ∈ E : τ(ce)v,e + τ
(ce)u,e ≤ γ
(ce)e ;
∀v ∈ V, ∀b ∈ C(v) :∑
e∈E(v) τ(be)v,e ≥ −σv .
Here for all v ∈ V , σv = 12∆ − d((y)E(v), (c)E(v)).
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Error Orientation
Definition
The assignment of the directions to theedges of the subgraph H = (UA ∪ UB, E)is called a (ρA, ρB)-orientation if eachvertex v ∈ UA and each vertex u ∈ UB
has at most ρA∆ and ρB∆ incomingedges in E, respectively.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Error Orientation
Definition
The assignment of the directions to theedges of the subgraph H = (UA ∪ UB, E)is called a (ρA, ρB)-orientation if eachvertex v ∈ UA and each vertex u ∈ UB
has at most ρA∆ and ρB∆ incomingedges in E, respectively.
ρA∆
ρB∆
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Error Orientation
Definition
The assignment of the directions to theedges of the subgraph H = (UA ∪ UB, E)is called a (ρA, ρB)-orientation if eachvertex v ∈ UA and each vertex u ∈ UB
has at most ρA∆ and ρB∆ incomingedges in E, respectively.
Error pattern orientation: for ω 6= ce and
ye in error, the value τ(ω)v,e = −5
2 − ǫ willbe assigned if the edge e enters thevertex v.
ρA∆
ρB∆
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Error Orientation in Expander Graphs
Existence of (< 14δA, < 1
4δB)-orientation yields a sufficientlysmall number of assignments −5
2 − ǫ. This, in turn, yields that
∑
e∈E(v)
τ (be)v,e ≥ −σv .
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Error Orientation in Expander Graphs
Existence of (< 14δA, < 1
4δB)-orientation yields a sufficientlysmall number of assignments −5
2 − ǫ. This, in turn, yields that
∑
e∈E(v)
τ (be)v,e ≥ −σv .
Lemma
Let H = (UA ∪ UB, E) be a subgraph of G = (A ∪ B, E).Assume that |E| ≤ (αβ − 1
2γG)∆n for some α, β ∈ (0, 1], suchthat γG ≤ √
αβ, and 12α∆, 1
2β∆ are both integers. Then, E
contains a (β/2, α/2)-orientation.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Fraction of Correctable Errors
Theorem
Let θA > 0 (θB > 0) be the largest number such thatθA < δA (θB < δB) and 1
4θA∆ (14θB∆, respectively) is
integer.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Fraction of Correctable Errors
Theorem
Let θA > 0 (θB > 0) be the largest number such thatθA < δA (θB < δB) and 1
4θA∆ (14θB∆, respectively) is
integer.
Let C be as above, and assume that γG ≤ 12
√θAθB.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Fraction of Correctable Errors
Theorem
Let θA > 0 (θB > 0) be the largest number such thatθA < δA (θB < δB) and 1
4θA∆ (14θB∆, respectively) is
integer.
Let C be as above, and assume that γG ≤ 12
√θAθB.
Then, the LP decoder corrects any error pattern of a size lessthan or equal to (1
4θAθB − 12γG)∆n.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes
Fraction of Correctable Errors
Theorem
Let θA > 0 (θB > 0) be the largest number such thatθA < δA (θB < δB) and 1
4θA∆ (14θB∆, respectively) is
integer.
Let C be as above, and assume that γG ≤ 12
√θAθB.
Then, the LP decoder corrects any error pattern of a size lessthan or equal to (1
4θAθB − 12γG)∆n.
Remark
It is possible to improve (slightly) on the low-order term in theexpression for the number of correctable errors in the statementof the main result. In the present work, we omit this analysis.
Vitaly Skachek LP Decoding of Nonbinary Expander Codes