Post on 25-Aug-2020
Spatial Coupling -- What is it, why does it work, and what are the open challenges?
Ruediger Urbanke EPFL
!April 17th, 2014
Urbanke
Based on slides jointly developed with
Shrinivas Kudekar.
Part IV: Spatial coupling - A General Phenomenon
Outline Spatial Coupling - A General Phenomenon
• General one-dimensional systems • General BMS channels - Threshold Saturation and Universality • Multi-user communications and ISI channels
- Multi-access channels - Noisy Slepian-Wolf - Finite state channels - Many more...
• Problems beyond Communications - Compressive sensing - K-SAT
Practical Aspects and Open Questions • Universality • Windowed decoding • Rate loss • Scaling • Decoding Speed • Complexity and choice of parameters
General one-dimensional systems
General Coupled One-Dimensional Analysis
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
General Coupled One-Dimensional Analysis
Irregular Ensembles
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
�(x) =3x + 3x2 + 14x50
20�(x) = x15
General Coupled One-Dimensional Analysis
Irregular Ensembles
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
�(x) =3x + 3x2 + 14x50
20�(x) = x15
�BPuncoupled = 0.3531
General Coupled One-Dimensional Analysis
Irregular Ensembles
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
�(x) =3x + 3x2 + 14x50
20�(x) = x15
� = 0.4032
Balance of areas
�BPuncoupled = 0.3531
General Coupled One-Dimensional Analysis
Irregular Ensembles
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
�(x) =3x + 3x2 + 14x50
20�(x) = x15
� = 0.4032
Balance of areas
�BPuncoupled = 0.3531
�BPcoupled = 0.4032
General Coupled One-Dimensional Analysis
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
General Coupled One-Dimensional Analysis
Gaussian Approximation
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
�(x) = x2
�(x) = x5
General Coupled One-Dimensional Analysis
Gaussian Approximation
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
�(x) = x2
�(x) = x5
BAWGN
General Coupled One-Dimensional Analysis
Gaussian Approximation
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
hBPuncoupled = 0.4293
�(x) = x2
�(x) = x5
BAWGN
General Coupled One-Dimensional Analysis
Gaussian Approximation
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
hBPuncoupled = 0.4293
�(x) = x2
�(x) = x5
v
f(u)g(v)
u
h = 0.42915
BAWGN
General Coupled One-Dimensional Analysis
Gaussian Approximation
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
hBPuncoupled = 0.4293
�(x) = x2
�(x) = x5
v
f(u)g(v)
u u
v
g(v)f(u)
h = 0.42915 h = 0.4758
Balance of areasBAWGN
General Coupled One-Dimensional Analysis
Gaussian Approximation
Balance of areas in the EXIT chart of uncoupled ensembles gives the BP threshold of coupled systems
hBPcoupled = 0.4794
hBPuncoupled = 0.4293
�(x) = x2
�(x) = x5
v
f(u)g(v)
u u
v
g(v)f(u)
h = 0.42915 h = 0.4758
Balance of areasBAWGN
General BMS channels
Coupled Codes are Provably Capacity-Achieving under BP Decoding and they are universal with respect to all BMS channels
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Threshold Saturation
BAWGN Channel
(3,6,L) coupled code ensemble with increasing L≈ 0.4789 ⇡ hArea(l, r)
General BMS Channels - Main Statement
where the bounds are independent of the channel.
For a fixed spatially coupled ensemble with parameters (dl, dr, w, L) and a given BMS channel,(dl, dr, w, L)
hArea �O(1/⇤
w) ⇥ hBPcoupled ⇥ hMAP
coupled ⇥ hArea + O(w/L)
General BMS Channels - Main Statement
where the bounds are independent of the channel.
For a fixed spatially coupled ensemble with parameters (dl, dr, w, L) and a given BMS channel,(dl, dr, w, L)
hArea �O(1/⇤
w) ⇥ hBPcoupled ⇥ hMAP
coupled ⇥ hArea + O(w/L)
(hA ,2006 Cyril Measson)hArea
uniformly over all channelshArea deg.�⇥� hShannon
General BMS Channels - Main Statement
where the bounds are independent of the channel.
Concentration: Almost all elements of the ensemble are good for all channels.
For a fixed spatially coupled ensemble with parameters (dl, dr, w, L) and a given BMS channel,(dl, dr, w, L)
hArea �O(1/⇤
w) ⇥ hBPcoupled ⇥ hMAP
coupled ⇥ hArea + O(w/L)
(hA ,2006 Cyril Measson)hArea
uniformly over all channelshArea deg.�⇥� hShannon
Good means that we can transmit using belief propagation decoding with (block/bit) error probability at most e.�
Let C(c) denote the set of all BMS channels with capacity c and let e > 0. Then there exists a fixed spatially coupled code ensemble of rate at least c- e such that almost every code in the ensemble is good for all channels in C(c).
C(c)�
C(c)c� �
c
Most Codes are Universal
General BMS Channels - Proofs of Threshold Saturation
Potential Function Analysis:
Generalized EXIT Analysis:
http://arxiv.org/pdf/1201.2999.pdf
http://arxiv.org/pdf/1301.6111.pdf
General BMS Channels - Proofs of Threshold Saturation
Generalized EXIT Analysis:
http://arxiv.org/pdf/1201.2999.pdf
General BMS Channels - Proofs of Threshold Saturation
Generalized EXIT Analysis:
http://arxiv.org/pdf/1201.2999.pdf
General BMS Channels - Proofs of Threshold Saturation
Generalized EXIT Analysis:
http://arxiv.org/pdf/1201.2999.pdf
special FP
General BMS Channels - Proofs of Threshold Saturation
Generalized EXIT Analysis:
http://arxiv.org/pdf/1201.2999.pdf
(generalized) EXIT special FP
Multi-user Communications Finite-state channels, Noisy
Slepian-Wolf
Multi-Access Channels
Y = h1X1 + h2X2 + N
Gaussian MAC Channel
Multi-Access Channels
Y = h1X1 + h2X2 + N
Gaussian MAC Channel
[Figures from Yedla et al. ]
Multi-Access Channels
Y = h1X1 + h2X2 + N
Gaussian MAC Channel
[Figures from Yedla et al. ]
Multi-Access Channels
References
Y = h1X1 + h2X2 + N
Gaussian MAC Channel
[Figures from Yedla et al. ]
Multi-Access Channels
References
Y = h1X1 + h2X2 + N
Gaussian MAC Channel
[Figures from Yedla et al. ]
Multi-Access Channels
References
Y = h1X1 + h2X2 + N
Gaussian MAC Channel
[Figures from Yedla et al. ]
Noisy Slepian-Wolf
Correlated sources (BSC) transmitted over
BAWGNC(h) !
Symmetric channel conditions, joint iterative
decoding[Figure from Yedla et al. ]
Noisy Slepian-Wolf
Correlated sources (BSC) transmitted over
BAWGNC(h) !
Symmetric channel conditions, joint iterative
decoding
References[Figure from Yedla et al. ]
Finite-State Channels
Generalized Erasure Channel
Output of a binary input linear filter (1-D) transmitted
over erasure channel
Finite-State Channels
[Figure from Phong et al ]
Generalized Erasure Channel
Output of a binary input linear filter (1-D) transmitted
over erasure channel
Finite-State Channels
[Figure from Phong et al ]
Generalized AWGN Channel
Output of a binary input linear filter (1-D) transmitted
over AWGN channel
Finite-State Channels
References
[Figure from Phong et al ]
Generalized AWGN Channel
Output of a binary input linear filter (1-D) transmitted
over AWGN channel
Finite-State Channels
References
[Figure from Phong et al ]
Generalized AWGN Channel
Output of a binary input linear filter (1-D) transmitted
over AWGN channel
Finite-State Channels
References
[Figure from Phong et al ]
Generalized AWGN Channel
Output of a binary input linear filter (1-D) transmitted
over AWGN channel
Many more...
Weight Distribution Minimum Distance
Trapping set, Pseudocodeword
analysis
CDMA
Relay Channel
Many more...
Wiretap Channel
Rateless Coding
Lattice codes
Lossy Source Coding
Quantum Error Correction
Linear Programming Decoding
Bounded Complexity
Problems beyond Communications - Compressive Sensing and K-Satisfiability
Compressive Sensing
y = Ax + w
Compressive Sensing
y = Ax + w
=( (y
m� 1( (A
m� n( (x
n� 1
+ w ((m� 1
Compressive Sensing
y = Ax + w
=( (y
m� 1( (A
m� n( (x
n� 1
+ w ((m� 1
m < n cont. dist. withmass at zero
fX(x)1� �
x is sparse
Compressive Sensing
y = Ax + w
=( (y
m� 1( (A
m� n( (x
n� 1
+ w ((m� 1
� = k/n� = m/nRegime: m, n�⇥
(Bayesian) Compressive Sensing
(Bayesian) Compressive Sensing
m � nd(fX) + o(n)
d(fX) is the Renyi Information Dimension
(Bayesian) Compressive Sensing
m � nd(fX) + o(n)
d(fX) is the Renyi Information Dimension
d(fX) is less than for signal with sparsity equal to e� �
(Bayesian) Compressive Sensing
0 1
1
�
�
(Bayesian) Compressive Sensing
0 1
1
�
�
Optimal trade-off for sparsity and undersampling
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
Using coupled measurement matrices with message-passing decoder
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
Using coupled measurement matrices with message-passing decoder
...
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
Using coupled measurement matrices with message-passing decoder
�
⇧⇧⇧⇧⇧⇧⇧⇤
1 1 0 0 0 0 0 . . . 01 1 1 1 0 0 0 . . . 01 1 1 1 1 1 0 . . . 00 1 1 1 1 1 0 . . . 0...
......
......
......
. . . 00 0 0 0 0 0 0 . . . 1
⇥
⌃⌃⌃⌃⌃⌃⌃⌅
Band Diagonal Adjacency Matrix(Base) Measurement Matrix
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
base measurement
matrix representing
heteroscedasticity
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
does not affect overall
undersampling{�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
{�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
extra measurements
to help “kickstart” {
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
{�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
coupling structure will
then take over{�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0+
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
Approximate Message Passing (AMP) decoder
�
⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇧⇤
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0
⇥
⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌃⌅
1...1
1...1
0
0
0...
0
0+
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
[Figure from Javanmard et al ]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
References
[Figure from Javanmard et al ]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
References
[Figure from Javanmard et al ]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
References
[Figure from Javanmard et al ]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
References
[Figure from Javanmard et al ]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
References
[Figure from Javanmard et al ]
Compressive Sensing and Coupling [Kudekar, Pfister, ‘10] [Krzakala, Mezard, Sausset, Sun, Zdeborova, ‘12] [Donoho, Javanmard, Montanari, ‘12]
References
[Figure from Javanmard et al ]
Compressive Sensing - Proof of Threshold Saturation
Proof based on:
State evolution (evolution of the MSE under AMP) + Continuum analysis + Potential function analysis
K-Satisfiability -- Setup
K-Satisfiability -- Effect of Spatial Coupling
As for coding we can construct spatially coupled K-SAT formulas and we can show that for many algorithms the threshold of M/n up to which one can find satisfiable assignments is improved.
Combining this with the interpolation technique this can be used to prove better lower bounds on the SAT/UNSAT thresholds of uncoupled formulas.
Some more ...
Part III: Practical Aspects and Open Questions
Windowed Decoding
is the size of the sliding window
...
(dl, dr, L,W ) W
⇤ ⇥� ⌅W
Windowed Decoding
Windowed Decoding
[Figure reproduced from
Iyengar et al ]
(3,6,64)
Windowed Decoding
Latency equals W/L fraction of
BP decoderW/L
[Figure reproduced from
Iyengar et al ]Complexity in the waterfall region scales
like O(MW2L) as opposed to O(ML2)
(3,6,64)
Windowed Decoding
Latency equals W/L fraction of
BP decoderW/L
[Figure reproduced from
Iyengar et al ]Complexity in the waterfall region scales
like O(MW2L) as opposed to O(ML2)
Threshold goes exponentially fast in W
to BP threshold
(3,6,64)
Windowed Decoding
Latency equals W/L fraction of
BP decoderW/L
[Figure reproduced from
Iyengar et al ]
Reference
Complexity in the waterfall region scales
like O(MW2L) as opposed to O(ML2)
Threshold goes exponentially fast in W
to BP threshold
(3,6,64)
Windowed Decoding
Latency equals W/L fraction of
BP decoderW/L
[Figure reproduced from
Iyengar et al ]
Reference
Complexity in the waterfall region scales
like O(MW2L) as opposed to O(ML2)
Threshold goes exponentially fast in W
to BP threshold
(3,6,64)
Windowed Decoding
Latency equals W/L fraction of
BP decoderW/L
[Figure reproduced from
Iyengar et al ]
Reference
Complexity in the waterfall region scales
like O(MW2L) as opposed to O(ML2)
Threshold goes exponentially fast in W
to BP threshold
Vol. 56, No. 10, pp. 5274
(3,6,64)
Windowed Decoding
Latency equals W/L fraction of
BP decoderW/L
[Figure reproduced from
Iyengar et al ]
Reference
Complexity in the waterfall region scales
like O(MW2L) as opposed to O(ML2)
Threshold goes exponentially fast in W
to BP threshold
(3,6,64)
Rate-loss
Rate Loss Due to Boundary
L
dl=3 dr=6
Rate Loss Due to Boundary
L
dl=3 dr=6
Ldl=4 dr=8
Rate Loss Due to Boundary
Ldl=4 dr=8
drdl
L variable nodes
Rate Loss Due to Boundary
Ldl=4 dr=8
drdl
L variable nodes
L+ dl � 1 check nodes
Rate Loss Due to Boundary
Ldl=4 dr=8
r = 1� dldr
� dl(dl � 1)
dr
1
L
drdl
L variable nodes
L+ dl � 1 check nodes
Rate Loss Due to Boundary
Can we do better?
Rate Loss Due to Boundary
Can we do better?
Rate Loss Due to Boundary
‣ Mitigating rate-loss
Can we do better?
Rate Loss Due to Boundary
‣ Mitigating rate-loss‣ One-Sided Termination
Can we do better?
Rate Loss Due to Boundary
‣ Mitigating rate-loss‣ One-Sided Termination‣ Deletion
Ldl=4 dr=8
Mitigating rate-loss: One-sided termination [ Kudekar, Richardson, Urbanke, 2012]
Ldl=4 dr=8
combine
Mitigating rate-loss: One-sided termination [ Kudekar, Richardson, Urbanke, 2012]
Ldl=4 dr=8
dldr
L variable nodes
combine
Mitigating rate-loss: One-sided termination [ Kudekar, Richardson, Urbanke, 2012]
Ldl=4 dr=8
dldr
L variable nodes
combine
L+
dl � 1
2
check nodes
Mitigating rate-loss: One-sided termination [ Kudekar, Richardson, Urbanke, 2012]
Ldl=4 dr=8
dldr
L variable nodes
combine
L+
dl � 1
2
check nodes
r = 1� dldr
� dl(dl � 1)
2dr
1
L
Mitigating rate-loss: One-sided termination [ Kudekar, Richardson, Urbanke, 2012]
Ldl=4 dr=8
dldr
L variable nodes
combine
L+
dl � 1
2
check nodes
r = 1� dldr
� dl(dl � 1)
2dr
1
L
Mitigating rate-loss: One-sided termination [ Kudekar, Richardson, Urbanke, 2012]
Mitigation of rate-loss: Deletion [Kasai et. al. ITW 2012]
Ldl=4 dr=8
Mitigation of rate-loss: Deletion [Kasai et. al. ITW 2012]
Ldl=4 dr=8
delete!dl � 2
Mitigation of rate-loss: Deletion [Kasai et. al. ITW 2012]
Ldl=4 dr=8
delete!dl � 2
Mitigation of rate-loss: Deletion [Kasai et. al. ITW 2012]
Ldl=4 dr=8
dldr
L variable nodes
delete!dl � 2
Mitigation of rate-loss: Deletion [Kasai et. al. ITW 2012]
Ldl=4 dr=8
dldr
L variable nodes
delete!
L+ 1 check nodes
dl � 2
Mitigation of rate-loss: Deletion [Kasai et. al. ITW 2012]
Ldl=4 dr=8
dldr
L variable nodes
delete!
r = 1� dldr
� dldr
1
L
L+ 1 check nodes
dl � 2
Finite-length Scaling
Finite-Length Scaling (BEC)
gap to capacity = � +�
L
=
↵pn
pL+
�
L
= �n� 13
n = LM �R =�
L
⇠ n�0.275
optimal codes
polar codes
⇠ n� 12
PB ⇠ LpM�e�M�2
L = n13
Finite-Length Scaling (BEC) [Olmos, Urbanke, 2012]
gap to capacity = � +�
L
=
↵pn
pL+
�
L
= �n� 13
n = LM �R =�
L
⇠ n�0.275
optimal codes
polar codes
⇠ n� 12
PB ⇠ LpM�e�M�2
L = n13
Many more...
Open Questions
1. Simplify, simplify, simplify, ..., 2. Spatial coupling as a proof technique (Maxwell conjecture, better bounds on K-SAT threshold, etc.) 3. Rate loss mitigation, other ways of introducing “boundary effect”? Derive fundamental lower bounds on the rate-loss. 4. Find systematic ways of designing codes (finite-length scaling, determine wave speed, ...). 5. Further applications.
Main Message
Main Message
Coupled ensembles under BP decoding behave like uncoupled ensembles under MAP decoding.
Main Message
Coupled ensembles under BP decoding behave like uncoupled ensembles under MAP decoding.
Since coupled ensemble achieve the highest threshold they can achieve (namely the MAP threshold) under BP we speak of the threshold saturation phenomenon.
Main Message
Coupled ensembles under BP decoding behave like uncoupled ensembles under MAP decoding.
Since coupled ensemble achieve the highest threshold they can achieve (namely the MAP threshold) under BP we speak of the threshold saturation phenomenon.
By using spatial coupling we can construct codes which are capacity-achieving universally across the whole set of BMS channels.
Main Message
Coupled ensembles under BP decoding behave like uncoupled ensembles under MAP decoding.
Since coupled ensemble achieve the highest threshold they can achieve (namely the MAP threshold) under BP we speak of the threshold saturation phenomenon.
By using spatial coupling we can construct codes which are capacity-achieving universally across the whole set of BMS channels.
The basic principle is applicable to a wide range of graphical models.
Main Message
Coupled ensembles under BP decoding behave like uncoupled ensembles under MAP decoding.
Since coupled ensemble achieve the highest threshold they can achieve (namely the MAP threshold) under BP we speak of the threshold saturation phenomenon.
By using spatial coupling we can construct codes which are capacity-achieving universally across the whole set of BMS channels.
The basic principle is applicable to a wide range of graphical models.
The phenomenon of threshold saturation is closely connected to the way of how crystals grow.