Lattices for Distributed Source Coding - Reconstruction of a …dineshk/dinesh_lattice... ·...
Transcript of Lattices for Distributed Source Coding - Reconstruction of a …dineshk/dinesh_lattice... ·...
Lattices for Distributed Source Coding -
Reconstruction of a Linear function of Jointly
Gaussian Sources
Dinesh Krithivasan and S. Sandeep Pradhan
University of Michigan, Ann Arbor
1
Presentation Overview
• Problem Formulation
• A straightforward coding scheme using random coding
• A new coding scheme using lattice coding
– Motivation for the coding scheme
– An overview of lattices and lattice coding
– A lattice based coding scheme
• Results and Extensions
• Conclusion
2
Problem Formulation
• Source (X1, X2) is a bivariate Gaussian.
• X1, X2 ∼ N (0, 1), E(X1X2) = ρ > 0.
• Encoders observe X1, X2 separately and quantize them at rates
R1 and R2.
• Decoder interested in a linear function Z , X1 − cX2.
• Lossy reconstruction to within a mean squared error distortion
D.
• Objective: Achievable rates (R1, R2) at distortion D.
3
Berger-Tung based Coding Scheme
• Encoders: Quantize X1 to W1 and X2 to W2. Transmit W1
and W2.
• Decoder: Reconstruct Z , E(Z | W1, W2).
• Can use Gaussian test channels for P (W1 | X1) and
P (W2 | X2) to derive achievable rates and distortion.
• Known to be optimal for c < 0 with Gaussian test channels.
4
Motivation for our Coding Scheme
• Korner and Marton considered lossless coding of Z = X1 ⊕ X2
• Coding Scheme if the encoding is centralized?
5
Motivation for our Coding Scheme
• Korner and Marton considered lossless coding of Z = X1 ⊕ X2
• Coding Scheme if the encoding is centralized?
– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.
– f(·) is any good source coding scheme.
5-a
Motivation for our Coding Scheme
• Korner and Marton considered lossless coding of Z = X1 ⊕ X2
• Coding Scheme if the encoding is centralized?
– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.
– f(·) is any good source coding scheme.
• Suppose f(·) distributes over ⊕?
5-b
Motivation for our Coding Scheme
• Korner and Marton considered lossless coding of Z = X1 ⊕ X2
• Coding Scheme if the encoding is centralized?
– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.
– f(·) is any good source coding scheme.
• Suppose f(·) distributes over ⊕?
– Compress X1 and X2 as f(X1) and f(X2).
– Decoder computes f(X1) ⊕ f(X2) = f(X1 ⊕ X2) = f(Z)
5-c
Motivation for our Coding Scheme
• Korner and Marton considered lossless coding of Z = X1 ⊕ X2
• Coding Scheme if the encoding is centralized?
– Compute Z = X1 ⊕ X2. Compress it to f(Z) and transmit.
– f(·) is any good source coding scheme.
• Suppose f(·) distributes over ⊕?
– Compress X1 and X2 as f(X1) and f(X2).
– Decoder computes f(X1) ⊕ f(X2) = f(X1 ⊕ X2) = f(Z)
• No difference between centralized and distributed coding.
• Choosing f(·) as a linear code will work.
5-d
An overview of Lattices
• An n dimensional lattice Λ - collection of integer combinations
of columns of a n × n generator matrix G.
• Nearest neighbor quantizer
QΛ(x) , {λ ∈ Λ: ‖ x − λ ‖≤‖ x − λ′
‖ ∀λ′
∈ Λ}.
• Quantization error : x mod Λ , x − QΛ(x).
• Voronoi region V0(Λ) , {x ∈ Rn : QΛ(x) = 0n}.
• The second moment σ2(Λ) of lattice Λ - expected value per
dimension of a random vector uniformly distributed in V0(Λ).
• Normalized second moment G(Λ) = σ2(Λ)V 2/n(Λ)
. V =∫
V0
dx.
6
Introduction to Lattice Codes
• Lattice code - a subset of the lattice points are used as
codewords.
• Have been used for both source and channel coding problems.
• Various notions of goodness:
– A good source D-code if log(2πeG(Λ)) ≤ ǫ and σ2(Λ) = D.
– A good channel σ2z -code if it achieves the Poltyrev exponent
on the unconstrained AWGN channel of variance σ2z .
7
Introduction to Lattice Codes
• Lattice code - a subset of the lattice points are used as
codewords.
• Have been used for both source and channel coding problems.
• Various notions of goodness:
– A good source D-code if log(2πeG(Λ)) ≤ ǫ and σ2(Λ) = D.
– A good channel σ2z -code if it achieves the Poltyrev exponent
on the unconstrained AWGN channel of variance σ2z .
• Why lattice codes?
7-a
Introduction to Lattice Codes
• Lattice code - a subset of the lattice points are used as
codewords.
• Have been used for both source and channel coding problems.
• Various notions of goodness:
– A good source D-code if log(2πeG(Λ)) ≤ ǫ and σ2(Λ) = D.
– A good channel σ2z -code if it achieves the Poltyrev exponent
on the unconstrained AWGN channel of variance σ2z .
• Why lattice codes?
– Lattices achieve the Gaussian rate distortion bound
– Lattice encoding distributes over the function Z = X1 − cX2
7-b
Nested Lattice Codes
• (Λ1, Λ2) is a nested lattice if Λ1 ⊂ Λ2.
• Nested lattice codes widely used in multiterminal
communications.
• Need nested lattice codes such that both Λ1 and Λ2 are good
source and channel codes.
• Such nested lattices termed “good nested lattice codes”.
• Known that such nested lattices do exist in sufficiently high
dimension.
8
The Coding Scheme
• A good nested lattice code (Λ11, Λ12, Λ2) with Λ2 ⊂ Λ11, Λ12.
• Dithers Ui ∼ Unif V0(Λ1i), i = 1, 2.
• σ2(Λ11) = q1, σ2(Λ12) =
Dσ2
Z
σ2
Z−D− q1, σ
2(Λ2) =σ4
Z
σ2
Z−D
• Note: Second encoder scales the input before encoding.
9
The Coding Scheme contd.
• Rate of a nested lattice (Λ1, Λ2) is R = 12 log σ2(Λ2)
σ2(Λ1).
• Thus, Encoder rates are
R1 =1
2log
σ4Z
q1(σ2Z − D)
, R2 =1
2log
σ4Z
Dσ2Z − q1(σ2
Z − D).
• Sum rate R1 + R2 = log2σ2
Z
Dachievable at distortion D.
10
The Coding Scheme contd.
• Rate of a nested lattice (Λ1, Λ2) is R = 12 log σ2(Λ2)
σ2(Λ1).
• Thus, Encoder rates are
R1 =1
2log
σ4Z
q1(σ2Z − D)
, R2 =1
2log
σ4Z
Dσ2Z − q1(σ2
Z − D).
• Sum rate R1 + R2 = log2σ2
Z
Dachievable at distortion D.
Theorem 1 The set of all rate-distortion tuples (R1, R2, D) that
satisfy
2−2R1 + 2−2R2 ≤
(
σ2Z
D
)−1
are achievable.
10-a
Outline of the Proof
• Key Idea: Distributive property of lattice mod operation
((x mod Λ) + y) mod Λ = (x + y) mod Λ ∀x, y
• Using this, one of the mod-Λ2 operation can be removed from
the signal path.
• Simplified but equivalent coding scheme is
• eq1and eq2
are subtractive dither quantization noises
independent of the sources.
11
Proof Outline contd.
• Effective dither is eq = eq1− eq2
with varianceDσ2
Z
σ2
Z−D.
• Decoder operation can be described as
Z =
(
σ2Z − D
σ2Z
)
((Z + eq) mod Λ2).
• But Λ2 is a goodσ4
Z
σ2
Z−Dchannel code.
• Implies that ((Z + eq) mod Λ2) = (Z + eq) with high
probability.
• Decoder reconstruction Z =(
σ2
Z−D
σ2
Z
)
(Z + eq).
• Can be checked that E(Z − Z)2 = D.
12
Comments about the Lattice Coding Scheme
• Larger rate region than the random coding scheme for some
source statistics and some range of D.
• Coding scheme here is lattice vector quantization followed by
“correlated” lattice-structured binning.
• Reconstructing a function of more than two sources
Z =∑K
i=1 ciXi
– Some sources coded together using the lattice coding
scheme. Others coded separately using Berger-Tung coding.
– Possible to present a unified rate region combining all such
strategies.
13
Comparison of the Rate Regions
• Compare sum rates for ρ = 0.8 and c = 0.8.
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40
1
2
3
4
5
6
7
Distortion D
Sum
Rat
e
Comparison between Berger−Tung and Lattice based Coding Schemes
rho = 0.8 c = 0.8
Berger−Tung Sum rate Lattice Sum rate
• Lattice based scheme - lower sum rate for small distortions.
• Time sharing between the two schemes - Better rate region
than either scheme alone.
14
Range of Values for Lower Sum Rate
• Shows where (ρ, c) should lie for lattice sum rate to be lower
than Berger-Tung sum rate for some distortion D.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
rho
c
Region where lattice scheme outperforms Berger−Tung scheme
0
0.1
0.3
0.8
1.5
• Contour marked R - Lattice sum rate lower by R units.
• Improved performance only for c > 0.
15
Conclusion
• Considered lossy reconstruction of a function of the sources in
a distributed setting.
• Presented a coding scheme of vector quantization followed by
“correlated” binning using lattices.
• Improves upon the rate region of the natural random coding
scheme.
• Currently working on extending the scheme to discrete sources
and arbitrary functions.
16