Setting : Transmitting Correlated sources over 2−user MAC and IC
S1
S2
WS1S2
~~
S1S2
X1
X2
2-MAC
WY|X1X2
Y
Setting : Transmitting Correlated sources over 2−user MAC and IC
S1
S2
WS1S2
~~
X1
X2
Y1
2-IC
WY1Y2|X1X2
Y2
S1
S2
Problem Setup : Transmitting Correlated sources over 2−user MAC and IC
Shannon-theoretic study
?? Optimal coding technique ??
?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)
WY1Y2|X1X2(IC)) ??
Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.
1. A new coding technique.
2. Characterize performance, derive new sufficient conditions.
?? What is the New idea ??
Problem Setup : Transmitting Correlated sources over 2−user MAC and IC
Shannon-theoretic study
?? Optimal coding technique ??
?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)
WY1Y2|X1X2(IC)) ??
Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.
1. A new coding technique.
2. Characterize performance, derive new sufficient conditions.
?? What is the New idea ??
Problem Setup : Transmitting Correlated sources over 2−user MAC and IC
Shannon-theoretic study
?? Optimal coding technique ??
?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)
WY1Y2|X1X2(IC)) ??
Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.
1. A new coding technique.
2. Characterize performance, derive new sufficient conditions.
?? What is the New idea ??
What are the tasks involved?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs
Separation sub-optimal
Slepian-Wolf bin indices independent. Disables co-ordination.
What are the tasks involved?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs Separation sub-optimal
Slepian-Wolf bin indices independent. Disables co-ordination.
Co-ordination can be critical to communication : Toy Example
Source correlation has to be extracted for co-ordination.
Three Tasks - What is the block-length (B-L) of the codes?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs
Three Tasks - What is the block-length (B-L) of the codes?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs
05284649Source
Compression
05284649
WY|X
ChannelCoding Codeword
Three Tasks - What is the block-length (B-L) of the codes?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs
052846494634957335764Source
Compression
052846494634957335764
WY|X
ChannelCoding Codeword
Three Tasks - What is the block-length (B-L) of the codes?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs
05284649463495733576476723638 …....Source
Compression
Efficiency gets better and better ….
05284649463495733576476723638 …....
Efficiency gets better and better ….
WY|X
ChannelCoding Codeword
Three Tasks - What is the block-length (B-L) of the codes?
1. Source Compression/Coding (Lossless)
2. Channel Coding (Over MAC/IC)
3. Co-ordinate channel inputs
Large block-length is good
Large block-length is good
?? block-length ??
Co-ordinating Channel Inputs is by Extracting Correlation from Distributed Sources.
?? optimal block-length for extracting correlation from distributed sources ??
Witsenhausen [1975] Problem Setup
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
supfn,gn
=: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
Agreement on a non-trivial RVIs a measure of correlation.
supfn,gn
=: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X1X2...Xn...
Y1Y2...Yn...
IID PXY
~~
fn X( 1X2...Xn ) = A ∈ {0,1}
gn Y( 1Y2...Yn ) = B ∈ {0,1}
supfn,gn
=: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X1X2...Xn...
Y1Y2...Yn...
IID PXY
~~
fn X( 1X2...Xn ) = A ∈ {0,1}
gn Y( 1Y2...Yn ) = B ∈ {0,1}
Prob. of Agreement P (A B= )Is a measure of correlation.
supfn,gn
Binary RVs A and B non-trivial and agree with high prob.
=: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X n
Y n
{0,1}fn
gn
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
{0,1}
supfn,gn
=: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X n
Y n
{0,1}fn
gn
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
{0,1}
supfn,gn
P (fn(Xn) = gn(Y
n))
=: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X n
Y n
{0,1}fn
gn
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
{0,1}
supfn,gn
P (fn(Xn) = gn(Y
n))
=: P(n)
Best fn, gn for 1−bit agreement.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X n
Y n
{0,1}fn
gn
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
{0,1}
supfn,gn
P (fn(Xn) = gn(Y
n)) =: P(n)
Best fn, gn for 1−bit agreement.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Witsenhausen [1975] Problem Setup
X n
Y n
{0,1}fn
gn
X1X2...Xn...
Y1Y2...Yn...
IID PXY~
~
{0,1}
supfn,gn
P (fn(Xn) = gn(Y
n)) =: P(n)
Best fn, gn for 1−bit agreement
.
P(n) = Best Agreement probability for n symbol processing.
P(1) > P(2) > P(3) > · · · · · ·
Indeed, · · · a binary symmetric source · · ·
0101001010
0101001010
S1
S2
pS1S2
0.495 0.005
0.005 0.495
0 1
0
1
IID Source S1, S2 with P (S1 = S2) = 0.99
A shorter block agrees with higher prob.
P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →
n→∞1
Indeed, · · · a binary symmetric source · · ·
010100101001010
010100101001000
S1
S2
pS1S2
0.495 0.005
0.005 0.495
0 1
0
1
IID Source S1, S2 with P (S1 = S2) = 0.99
A shorter block agrees with higher prob.
P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →
n→∞1
Indeed, · · · a binary symmetric source · · ·
01010010100101010100101110
01010010100100010100101010
S1
S2
pS1S2
0.495 0.005
0.005 0.495
0 1
0
1
IID Source S1, S2 with P (S1 = S2) = 0.99
A shorter block agrees with higher prob.
P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →
n→∞1
Indeed, · · · a binary symmetric source · · ·
010100101001010101001011101100
010100101001000101001010101110
S1
S2
pS1S2
0.495 0.005
0.005 0.495
0 1
0
1
IID Source S1, S2 with P (S1 = S2) = 0.99
A shorter block agrees with higher prob.
P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →
n→∞1
Indeed, · · · a binary symmetric source · · ·
010100101001010101001011101100 ...
One among the exponentially large conditional typical seqeuences
S1
S2
pS1S2
0.495 0.005
0.005 0.495
0 1
0
1
As n → ∞
IID Source S1, S2 with P (S1 = S2) = 0.99
A shorter block agrees with higher prob.
P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →
n→∞1
Indeed, · · · a binary symmetric source · · ·
010100101001010101001011101100 ...
One among the exponentially large conditional typical seqeuences
S1
S2
pS1S2
0.495 0.005
0.005 0.495
0 1
0
1
As n → ∞
IID Source S1, S2 with P (S1 = S2) = 0.99
A shorter block agrees with higher prob.
P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →
n→∞1
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
S1S2
X1
X2
2-MAC
WY|X1X2
Y
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
S1S2
X1
X2
2-MAC
WY|X1X2
Y
01010010100100101010101101100011100101101001010
01010010100100101000101101100011100101101001010
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
X1
X2
2-MAC
WY|X1X2
01010010100100101010101101100011100101101001010
01010010100100101000101101100011100101101001010
01010101010101111000010101010100100
01110100101010101010010101010011110
Source – Chnl Mapping
Source – Chnl
Mapping
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
S1S2
X1
X2
2-MAC
WY|X1X2
Y
01010010100100101010101101100011100101101001010
01010010100100101000101101100011100101101001010
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
X1
X2
2-MAC
WY|X1X2
01010010100
01010010100
0111010
Source – Chnl Mapping
Source – Chnl
Mapping
0111010
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
X1
X2
2-MAC
WY|X1X2
01010010100100101010101101100011100101101001010
01010010100100101000101101100011100101101001010
0101010
0111010
Source – Chnl Mapping
Source – Chnl
Mapping
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
X1
X2
2-MAC
WY|X1X2
01010010100100101010101101100011100101101001010
01010010100100101000101101100011100101101001010
0111011
0111011
Source – Chnl
Mapping
Sourc
e –
Chnl
Map
ping
Communicating Distributed Correlated Sources over Networks
S1
S2
WS1S2
~~
X1
X2
2-MAC
WY|X1X2
01010010100100101010101101100011100101101001010
01010010100100101000101101100011100101101001010
10101101
10101101
Shorter Blocks
⇓
Efficient transfer of correlation.
Longer Block
⇓
Efficient Chnl Coding,Source Compression.
A tension in choice of Block-Length (B-L)
?? Coding scheme ?? Info-theoretic analysis ?? Single-lettercharacterization.
Tension in choice of Block-Length (B-L)
Optimal B-L neither too big nor too small
Fixed Block-Length (B-L) coding
A New approach in information theory.
Challenges in the Design and Information-theoretic Analysis
1. Design of Information-theretic coding scheme with shortblock-lengths.
2. Information - theoretic performance analysis requires block-length(B-L) of overall coding scheme to →∞.
3. Fixing the B-L results in multi-letter distribution. How do you get asingle-letter characterization?
?? Design ?? Analysis ?? Performance Characterization ??
Two-layer coding - one fixed B-L, one ∞− B-L
B-L ml.
0101001010010010101010110110001110010110……..1001010
Source 11 ml
Two-layer coding - one fixed B-L, one ∞− B-L
B-L ml.
0101001010010010101010110110001110010110……..1001010
010100101001001010101011011000111001011010
0101001
Source 11 ml
m
1
1 l
Two-layer coding - one fixed B-L, one ∞− B-L
B-L ml.
0101001010010010101010110110001110010110……..1001010
010100101001001010101011011000111001011010
0101001
Source 11 ml
m
1
1 l
Layer 1 code
Maps row-by row
Fixed B-L l
101010101011011101011111111100000001011010
0101001
Two-layer coding - one fixed B-L, one ∞− B-L
B-L ml.
0101001010010010101010110110001110010110……..1001010
Source 11 ml
101011111101110001011111111111010111101000
0101001
Layer 2 codeBlock mapping
B-L ml
Two-layer coding - one fixed B-L, one ∞− B-L
B-L ml.
0101001010010010101010110110001110010110……..1001010
010100101001001010101011011000111001011010
0101001
Source 11 ml
m
1
1 l
Layer 1 code
Maps row-by row
Fixed B-L l
101010101011011101011111111100000001011010
0101001
101011111101110001011111111111010111101000
0101001
Layer 2 codeBlock mapping
B-L ml
101011111101110001011111111111010111101000
0101001
MultiplexingTwo layers intoChannel inputs
Two-layer coding - one fixed B-L, one ∞− B-L
1. Fixed B-L Layer 1 transfers correlation efficiently
2. ∞− B-L Layer 2 performs source compression, channel coding.
Challenges in Information-theoretic Performance Analysis
1. Multiplexing multiple codewrds of Layer 1 with single codewrd of Layer 2.
I Shirani and Pradhan’s technique of interleaving.
2. Distribution induced by the coding scheme not the same as the chosentest channel
I Constant composition codes and bounding the divergence between therespective distributions.
Fixed Block-Length Coding
A New Framework in Information Theory
Builds on early findings of Dueck 1981, Witsenhaunsen 1975.
Builds on recent findings by Shirani Pradhan [2014] on Distributed SourceCoding.
Communicating Distributed Correlated Sources over Networks
New Coding Structure for
that strictly outperform all previous known.
Derived new sufficient conditions.
New Results on a Fundamental problem. Remained Stubborn since 1981.
A new coding theorem in a simplified form
Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2], (ii) l ∈ N, δ > 0, (iii) finite setU ,V1,V2 and pmf pUpV1pV2pX1|UV1
pX2|UV2defined on U × V × X , where pU
is a type of sequences in U l, (iv) A,B ≥ 0, ρ ∈ (0, A) such that φ ∈ [0, 0.5),
A+B ≥ (1 + δ)H(K1), and for j ∈ [2],
B +H(Sj |K1) + LSl (φ, |Sj |) < I(Vj ;Yj)− LCj (φ, |V|),
where,
φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl
1 6= Kl2)
gρ,l:=
2∑j=1
exp{−l(Er(A+ ρ, pU , pYj |U )− ρ)}, (Channel coding error)
τl,δ(K) = 2|K| exp{−2δ2p2K(a∗)l} (Prob of non-typicality)
LCj (φ, |U|) = hb(φ) + φ log |U|+ |Y||U|φ log 1
φ,
LSl (φ, |K|):=1
lhb(φ) + φ log |K|.
A new coding theorem in a simplified form
Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2], (ii) l ∈ N, δ > 0, (iii) finite setU ,V1,V2 and pmf pUpV1pV2pX1|UV1
pX2|UV2defined on U × V × X , where pU
is a type of sequences in U l, (iv) A,B ≥ 0, ρ ∈ (0, A) such that φ ∈ [0, 0.5),
A+B ≥ (1 + δ)H(K1), and for j ∈ [2],
B +H(Sj |K1) + LSl (φ, |Sj |) < I(Vj ;Yj)− LCj (φ, |V|),
where, φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl
1 6= Kl2)
gρ,l:=
2∑j=1
exp{−l(Er(A+ ρ, pU , pYj |U )− ρ)}, (Channel coding error)
τl,δ(K) = 2|K| exp{−2δ2p2K(a∗)l} (Prob of non-typicality)
LCj (φ, |U|) = hb(φ) + φ log |U|+ |Y||U|φ log 1
φ,
LSl (φ, |K|):=1
lhb(φ) + φ log |K|.
Problem Setup : Transmitting Correlated sources over 2−user MAC and IC
Shannon-theoretic study
?? Optimal coding technique ??
?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)
WY1Y2|X1X2(IC)) ??
Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.
1. A new coding technique.
2. Characterize performance, derive new sufficient conditions.
Problem Setup : Transmitting Correlated sources over 2−user MAC and IC
Shannon-theoretic study
?? Optimal coding technique ??
?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)
WY1Y2|X1X2(IC)) ??
Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.
1. A new coding technique.
2. Characterize performance, derive new sufficient conditions.
Problem Setup : Transmitting Correlated sources over 2−user MAC and IC
Shannon-theoretic study
?? Optimal coding technique ??
?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)
WY1Y2|X1X2(IC)) ??
Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.
1. A new coding technique.
2. Characterize performance, derive new sufficient conditions.
Central element of both problems
? Optimally transferring source correlation onto co-ordinated channel inputs ?
Prior work : Cover, El Gamal and Salehi (CES) [Nov. 1980] technique.Dueck’s example
[Nov. 1980](S1, S2) transmissible over MAC if
H(S1|S2) < I(X1;Y |X2, S2, U), H(S2|S1) < I(X2;Y |X1, S1, U)
H(S1, S2|K) < I(X1X2;Y |U), H(S1, S2) < I(X1X2;Y )
for a valid pmf WS1S2pUpX1|US1pX2|US2
WY |X1X2.
Prior work : Liu and Chen [Dec. 2011]
Incorporated
1. CES technique
2. Random source Partitioning [Han Costa 1987]
3. Message Splitting Via super position coding [Han-Kobayashi 1981]
into a single coding (LC) technique and derived sufficient (LC) conditions.
CES and LC techniques : A common thread/constraint
X1 − S1 − S2 −X2
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
CES and LC conditions are sub-optimal
MAC : Single-letter CES technique are (in general) sub-optimal. Dueck [Mar,1981] An ingenious coding technique designed for a particular example.
IC : LC conditions are sub-optimal. (Adapt Dueck’s argument).
This talk :
Understand why CES and LC are sub-optimal via Dueck’s example and developa new coding technique.
Begin with a very simple MAC example
S1, S2 ≡ Sources. Source alphabet = {0, 1, · · · , J}.
Channel inputs ≡ X1, X2 ∈ {0, 1, · · · , L− 1}
Channel output ≡ Y ∈ {0, 1, · · · , L− 1, ∗} .
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1.
H(S1) = hb(ε) + ε log J
MAC Channel
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J
MAC Channel
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J
MAC Channel
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J
MAC Channel
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J
MAC Channel
Source is transmissible as long as H(S1) = H(S2) ≤ logL.
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J
MAC Channel
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF
S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J
MAC Channel
Above source, is transmissible as long as
H(S1) = H(S2) ≤ suppX1X2
I(X1X2;Y ) (1)
ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.
Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.
Gacs Korner part facilitates co-ordination and efficient communication
Gacs Korner part provides considerable flexibility to co-ordinateand communicate efficiently.
H(S1) = logL
Need X1 = X2 uniform and X1 = X2.
For small ε, source highly non-uniform.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform and X1 = X2.
Source stripped off its Gacs-Korner part.
Single-Letter (S-L) coding is constrained to
X1 − S1 − S2 −X2.
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform and X1 = X2.
Source stripped off its Gacs-Korner part.
Single-Letter (S-L) coding is constrained to
X1 − S1 − S2 −X2.
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform and X1 = X2.
Source stripped off its Gacs-Korner part.
Single-Letter (S-L) coding is constrained to
X1 − S1 − S2 −X2.
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform and X1 = X2.
Source stripped off its Gacs-Korner part.
Single-Letter (S-L) coding is constrained to
X1 − S1 − S2 −X2.
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform and X1 = X2.
Source stripped off its Gacs-Korner part.
Single-Letter (S-L) coding is constrained to
X1 − S1 − S2 −X2.
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform and X1 = X2.
X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.
W1 and/or W2 non-trivial RVs reduces P (X1 = X2).
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform andX1 = X2.
X1 = g1(S1, φ), X2 = g2(S2, φ).
?Trivializing W1,W2 and requiring X1 = X2 uniform?
Pool all less likely symbols together.
What if we tweak the source just a little?
MAC Channel remains same.
Need X1 = X2 uniform andX1 = X2.
X1 = g1(S1, φ), X2 = g2(S2, φ).
?Trivializing W1,W2 and requiring X1 = X2 uniform?
Pool all less likely symbols together.
What if we tweak the source just a little?
X1 = g1(S1, φ), X2 = g2(S2, φ).
?Trivializing W1,W2 and requiring X1 = X2 uniform?
Pool all less likely symbols together.
Can get only (1− ε, ε)
What if we tweak the source just a little?
Carefully choose values for ε, θ, J, L, Dueck [1981] proves
1. Single-letter technique of CES [1980] is sub-optimal.
2. Proposes mapping fixed length blocks of source to channel codewords.Proves S1, S2 is transmissible.
Mimic conditional coding via fixed block-length coding
When S1 = S2.
Map large (typical) source sequences to codewordschosen uniformly at random.
We can ensure
X1 = X2 uniform and X1 = X2.
Mimic conditional coding via fixed block-length coding
P (S1 6= S2) = Jθ > 0.
P (Sn1 6= Sn2 ) = 1− (1− Jθ)n −→n→∞
1.
With increasing block-length n, P (Xn1 6= Xn
2 )→ 1.
Conditional coding with FIXED BLOCK-LENGTH l.
P (Sl1 6= Sl2) = 1− (1− Jθ)l ≤ lJθ.
Fixed block-length coding forces a concatenated coding scheme
Source PMF
MAC Channel
Block-length of coding is determined by ε, θ, J, L, not be the desired prob. oferror.
This fixed block-length code results in non-vanishing probability of error.
Necessitates an outer code of arbitrarily large block-length to
1. Correct for errors in the fixed block-length code.
2. Communicate rest of information.
This results in concatenated coding structure.
Part 2 and 3
Concatenated coding scheme for a general problem instance whoseinfo-theoretic performance can be characterized?
What challenges does the finite block-length code throw up?
Revisit CES scheme with common part K
Two phase coding, in the presence of GK common part K.
Phase 1: Common part Kn is decoded perfectly (owing to infinite block-length)
Phase 2: Rest of Information : Sn1 , Sn2 conditioned on common part Kn is
encoded in Xn1 , X
n2 and decoded by Rx.
Kn can be viewed as side-information available at all terminals.
What challenges does the finite block-length code throw up?
Revisit CES scheme with common part K
Two phase coding, in the presence of GK common part K.
Phase 1: Common part Kn is decoded perfectly (owing to infinite block-length)
Phase 2: Rest of Information : Sn1 , Sn2 conditioned on common part Kn is
encoded in Xn1 , X
n2 and decoded by Rx.
Kn can be viewed as side-information available at all terminals.
What challenges does the finite block-length code throw up?
Suppose S1, S2 do NOT have common part, but highly correlated.
m× l matrix encoding. l remains fixed, m→∞.
Two phase coding.
Phase 1: Rows of Km×l1 (Km×l
2 ) mapped to Um×l1 (Um×l2 ) via finiteblock-length code. Decoded erroneously as V m×l.
Phase 2: Rest of Information : Sm×l1 , Sm×l2 conditioned on common partV m×l is encoded in Xm×l
1 , Xm×l2 and decoded by Rx.
What challenges does the finite block-length code throw up?
Suppose S1, S2 do NOT have common part, but highly correlated.
m× l matrix encoding. l remains fixed, m→∞.
Two phase coding.
Phase 1: Rows of Km×l1 (Km×l
2 ) mapped to Um×l1 (Um×l2 ) via finiteblock-length code. Decoded erroneously as V m×l.
Phase 2: Rest of Information : Sm×l1 , Sm×l2 conditioned on common partV m×l is encoded in Xm×l
1 , Xm×l2 and decoded by Rx.
Multi-letter distribution induced by the fixed block-length code
Each row of matrices Um×l1 , Um×l2 , V m×l, Gm×l have a multi-letter pmf. Howdo you perform superposition coding?
Leverage the technique of interleaving suggested in Shirani and Pradhan [ISIT2014].
Note that the inner code of finite block-length is fixed upfront.
As a consequence, the rows of each of the m× l matrices are iid. However,each row has a product distribution.
Interleaving [Shirani and Pradhan 2014]
The same argument holds for each sub-vectorsE(1, λ1(i)) · · ·E(m,λm(i)) : i = 1, · · · , l.
We therefore have l iid sub-vectors E(1, λ1(i)) · · ·E(m,λm(i)) : i = 1, · · · , l.
Independent streams with Gacs-Korner common part
Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2] withP (K1 = K2) = φ+ ξ[l] = 0, (i) finite set U ,V1,V2 and pmfpUpV1pV2pX1|UV1
pX2|UV2defined on U × V × X , (iii) A,B ≥ 0,
A+B ≥ H(K1) = H(K2),
A ≤ min{I(U ;Y1), I(U ;Y2)} and for j ∈ [2],
B +H(Sj |K1) < I(Vj ;Yj).
A new coding theorem in a simplified form
Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2], (ii) l ∈ N, δ > 0, (iii) finite setU ,V1,V2 and pmf pUpV1pV2pX1|UV1
pX2|UV2defined on U × V × X , where pU
is a type of sequences in U l, (iv) A,B ≥ 0, ρ ∈ (0, A) such that φ ∈ [0, 0.5),
A+B ≥ (1 + δ)H(K1), and for j ∈ [2],
B +H(Sj |K1) + LSl (φ, |Sj |) < I(Vj ;Yj)− LCj (φ, |V|),
where,
φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl
1 6= Kl2)
gρ,l:=
2∑j=1
exp{−l(Er(A+ ρ, pU , pYj |U )− ρ)}, (Channel coding error)
τl,δ(K) = 2|K| exp{−2δ2p2K(a∗)l} (Prob of non-typicality)
LCj (φ, |U|) = hb(φ) + φ log |U|+ |Y||U|φ log 1
φ,
LSl (φ, |K|):=1
lhb(φ) + φ log |K|.
A new coding theorem in a simplified form
Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2], (ii) l ∈ N, δ > 0, (iii) finite setU ,V1,V2 and pmf pUpV1pV2pX1|UV1
pX2|UV2defined on U × V × X , where pU
is a type of sequences in U l, (iv) A,B ≥ 0, ρ ∈ (0, A) such that φ ∈ [0, 0.5),
A+B ≥ (1 + δ)H(K1), and for j ∈ [2],
B +H(Sj |K1) + LSl (φ, |Sj |) < I(Vj ;Yj)− LCj (φ, |V|),
where, φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl
1 6= Kl2)
gρ,l:=
2∑j=1
exp{−l(Er(A+ ρ, pU , pYj |U )− ρ)}, (Channel coding error)
τl,δ(K) = 2|K| exp{−2δ2p2K(a∗)l} (Prob of non-typicality)
LCj (φ, |U|) = hb(φ) + φ log |U|+ |Y||U|φ log 1
φ,
LSl (φ, |K|):=1
lhb(φ) + φ log |K|.
Finite blck-lngth inner code concatenated with an large blck-lngth outercode
Let U lj be CU codeword chosen by encoder j.
P (U l1 6= U l2) ≤ P (Kl1 6= Kl
2) + P (Kl1 /∈ T lδ(K1))
≤ 1− (1− ξ)l + 2|K| exp{−2p2K1(a∗)l} =: β
Suppose g is the prob. of error of a PTP channel decoder operating over aPTP channel pY |U .
Inner block code of finite block length
Assuming encoders agree, decoder decodes into CU -codebook with a PTPdecoder on pY |U chnl.
Inner block code of finite block length
V is decoded version of U . G(t, 1 : l) = e−1U (V (t, 1 : l)) is decoded version of
K.
Inner block code of finite block length
Encoders and decoder agree on a fraction 1− (g+β) of the rows of U -matrices.
Outer code : The approach
Encoder 1 has Encoder 2 has
S1(1 : l) · · ·S1((m− 1)l + 1 : ml) S2(1 : l) · · ·S2((m− 1)l + 1 : ml)
K1(1 : l) · · ·K1((m− 1)l + 1 : ml) K2(1 : l) · · ·K2((m− 1)l + 1 : ml)
U1(1 : l) · · ·U1((m− 1)l + 1 : ml) U2(1 : l) · · ·U2((m− 1)l + 1 : ml)
Decoder has
G(1 : l) · · ·G((m− 1)l + 1 : ml)
V (1 : l) · · ·V ((m− 1)l + 1 : ml)
If these had a single-letter form i.e., distributed as∏mlt=1 pS1K1U1S2K2U2V G,
then one could apply standard info-theoretic techniques.
Challenges
Encoder 1 has Encoder 2 has
S1(1 : l) · · ·S1((m− 1)l + 1 : ml) S2(1 : l) · · ·S2((m− 1)l + 1 : ml)
K1(1 : l) · · ·K1((m− 1)l + 1 : ml) K2(1 : l) · · ·K2((m− 1)l + 1 : ml)
U1(1 : l) · · ·U1((m− 1)l + 1 : ml) U2(1 : l) · · ·U2((m− 1)l + 1 : ml)
Decoder has
G(1 : l) · · ·G((m− 1)l + 1 : ml)
V (1 : l) · · ·V ((m− 1)l + 1 : ml)
I Not guaranteed to have a iid form.
I Even if we can extract sub-vectors that have product iid form, we do notknow the underlying pmf. Hence, cannot express rates in terms of thesepmfs.
The rows are independent and identically distributed
The rows are independent.
And identically distributed.
The rows are independent and identically distributed
The rows are independent.And identically distributed.
The rows are independent and identically distributed
The rows are independent and identically distributed.
The rows are independent and identically distributed
The rows are independent and identically distributed.
Randomly chosen co-ordinates are iid!!!
For each row t, choose column πt(1), where π1, · · · , πm are randompermutations.
Randomly chosen co-ordinates are iid!!!
For each row t, choose column πt(1), where π1, · · · , πm are randompermutations.
Randomly chosen co-ordinates are iid!!!
The rows are independent and identically distributed.
We therefore have l iid sub-vectors E(1, π1(i)) · · ·E(m,πm(i)) : i = 1, · · · , l.
Randomly chosen co-ordinates are iid!!!
Co-ordinates with same color coded together within a block in the outer code.l such blocks.
Randomly chosen co-ordinates are iid!!!
Co-ordinates with same color coded together within a block in the outer code.l such blocks.
Randomly chosen co-ordinates are iid!!!
Co-ordinates with same color coded together within a block in the outer code.l such blocks.
Randomly chosen co-ordinates are iid!!!
Co-ordinates with same color coded together within a block in the outer code.l such blocks.
The induced single-letter distributions are unknown!!!
S1(1, π1(1))S1(2, π2(1)) · · ·S1(m,πm(1))
K1(1, π1(1))K1(2, π2(1)) · · ·K1(m,πm(1))
U1(1, π1(1))U1(2, π2(1)) · · ·U1(m,πm(1))
S2(1, π1(1))S2(2, π2(1)) · · ·S2(m,πm(1)) ∼m∏t=1
pS1K1U1S2K2U2V G
K2(1, π1(1))K2(2, π2(1)) · · ·K2(m,πm(1))
U2(1, π1(1))U2(2, π2(1)) · · ·U2(m,πm(1))
V (1, π1(1))V (2, π2(1)) · · ·V (m,πm(1))
G(1, π1(1))G(2, π2(1)) · · ·G(m,πm(1))
The sufficient conditions will look like
H(S1|S2, G) < I(X1;Y X2|V S2), H(S2|S1, G) < I(X2;Y X1|V S1) · · ·
But, we do not know this pmf.
This pmf depends on the pmf of the empiricaldistribution of the inner layer code and the errors in that layer.
The induced single-letter distributions are unknown!!!
S1(1, π1(1))S1(2, π2(1)) · · ·S1(m,πm(1))
K1(1, π1(1))K1(2, π2(1)) · · ·K1(m,πm(1))
U1(1, π1(1))U1(2, π2(1)) · · ·U1(m,πm(1))
S2(1, π1(1))S2(2, π2(1)) · · ·S2(m,πm(1)) ∼m∏t=1
pS1K1U1S2K2U2V G
K2(1, π1(1))K2(2, π2(1)) · · ·K2(m,πm(1))
U2(1, π1(1))U2(2, π2(1)) · · ·U2(m,πm(1))
V (1, π1(1))V (2, π2(1)) · · ·V (m,πm(1))
G(1, π1(1))G(2, π2(1)) · · ·G(m,πm(1))
The sufficient conditions will look like
H(S1|S2, G) < I(X1;Y X2|V S2), H(S2|S1, G) < I(X2;Y X1|V S1) · · ·
But, we do not know this pmf.This pmf depends on the pmf of the empiricaldistribution of the inner layer code and the errors in that layer.
Upperbound the difference using relationship between chosen and actualpmf
Let pUpX1|US1pX2|US2
be chosen pmf.
pUj= pU since symbols of CU chosen iid pU .
P (U1 6= U2) ≤ β
pXj |Uj Sj= pXj |USj
: By choice of coding technique. One can derive an upper
bound using these relations.
Top Related