ECE442 Communications Lecture 6. Channel...

Near-optimal Channel Code H-ARQ

ECE442 CommunicationsLecture 6. Channel Coding

Husheng Li

Dept. of Electrical Engineering and Computer Science

Fall, 2012


Why Coding

1 Use redundancy to enhance robustness: if the informationbit is impair (by noise, fading, interference etc), it can stillbe recovered from redundant bits.

2 If coded in symbols of different time slots, coding is similarto time diversity, which is called repetition code. Channelcoding can also be done in frequency (e.g. in OFDMsystems).

3 Used in wireless communications, storage systems (harddisk) and so on.


Coding Gain

1 Coding gain is the difference of required energy per bit (Eb)(in dB) for coded and uncoded communication achievingthe same error rate.

2 Coding gain could be negative when Eb/N0 is very low.3 Essential problem of coding: Given energy Eb for the

transmission of each information bit, what is thescheme to achieve the optimal performance?.

4 Performance measure: bit error rate and block error rate.


Coding Gain


Basic Concepts

1 Codeword: some number of channel uses to representsome information bits.

2 Codebook: the ensemble of codewords.3 Codeword length: number of channel uses.4 Encoding: mapping from information bits to codewords.5 Decoding: mapping from received signal to information

bits.


Requirements of A Good Coding Scheme

1 Low bit/block error rate.2 Low-complexity encoder and decoder. (random coding has

optimal performance, but the decoding procedure has tolook up a huge codebook)

3 Reasonable codeword length (otherwise the delay is toolong)

4 No performance floor.


Reference Books

1 S. Lin and J. D. J. Costello, Error Control Coding, 2ndedition, Prentice-Hall, Englewood Cliffs, NJ, 2004 (generalbook about channel coding)

2 R. E. Blahut, Algebraic Codes for Data Transmission,Cambridge University Press, 2003. (a must-read book onalgebraic coding)

3 C. Heegard and S. B. Wicker, Turbo Coding, Kluwer,Boston, 1999

4 R. G. Gallager, Low Density Parity Check Codes, MITPress, Cambridge, 1963


Linear Block Codes

The simplest linear block code is the single-bit parity check.A block code generates a block of n coded bits from kinformation bits. We call it (n, k)-code. The coding rate isgiven by R = k/n.A linear block code means the sum of two codewords isstill a codeword. Or equivalently, the codewords form asubspace.


Hamming Distance

Codeword weight: number of nonzero bits.Hamming distance dij : the number of different coded bits intwo codewords ci and cj (the weight of codeword c1 − c2).


Minimum Distance

1 Minimum distance (also minimum weight of nonzerocodewords):

dmin = mini,j

dij ,

which characterizes how close the codewords are close toeach other (the closer, the worse performance)

2 Singleton bound:

dmin ≤ n − k + 1.

If equality holds, it is code maximum distance separation(MDS) code. A codeword in a MDS code is uniquelydetermined by any k elements (why?).


Capability of Error Detection and Correction

1 Error detection: at most dmin − 1 error bits can be detected(but may not be corrected).

2 Error correction: at most⌊

dmin−12

⌋error bits can be

corrected.


Generating Matrix

1 Information bits: k -dimensional row vector space;codewords: n-dimensional row vector space.

2 Generator matrix: mapping from information bit space tocodeword space, namely a k × n matrix.

3 Systematic code: if the generator matrix contains a k × kidentity submatrix (the information bits appear in klocations of codeword).

4 Encoding of information bit vector u:

c = uG


Parity Check Matrix

1 Parity check matrix H (n − k × n) of generator matrix Gsatisfies:

GHT = 0.

2 If receiving s = c + e (called senseword), where e is errorvector, multiplying H yields

sHT = uGHT + eHT = eHT ,

which is called syndrome of s.3 If syndrome is nonzero, we can claim decoding error. If

establishing a mapping between syndrome and errorvector (from an n− k -vector to an n-vector), we can correctsome error bits.


Encoder

1 Generating matrix:

G =

1 0 0 0 1 1 00 1 0 0 1 0 10 0 1 0 0 0 10 0 0 1 0 1 0

(1)


Cyclic Code

1 Cyclic code: any cyclic shift of any codeword is anothercodeword.

2 Polynomial representation (z is a shift operator)c(z) = g(z)u(z):

message: u(z) =k−1∑i=0

uiz i

generator: g(z) =n−k∑i=0

giz i

codeword: c(z) =n−1∑i=0

ciz i


Hard Decision Decoding (HDD)

1 The maximum likelihood detection of codewords is basedon a distance decoding metric.


Probability of HDD

1 The error probability is upper bounded by

Pe ≤n∑

j=t+1

(nj

)pj(1− p)n−j ,

where p is the demodulation error probability of a single bit.2 The low bound is given by

Pe ≥dmin∑

j=t+1

(dminj

)pj(1− p)d min−j ,


Soft Decoding

1 Choose the codeword with maximum a posterioriprobability:

C∗ = arg minC

p(R|C),

where R is observation.2 For AWGN,

p(R|C) ∼n∏

t=1

exp(−Rt −

√Ec(2Ct − 1)2

2σ2n

).

3 It is difficult to do exhaustive search for all codewords.


Group

1 Group: a set G with a operation ×, which satisfiesClosure: a× b ∈ G, if a ∈ G and b ∈ GAssociativity: (a× b)× c = a× (b × c)Identity element e: a× e = a, ∀a ∈ GInverse element: for any a ∈ G, there exists an a−1 ∈ Gsuch that a× a−1 = e

2 Abelian group: a× b = b × a3 Example: Invertible square matrices


Ring

1 Ring: a set G with operations × and +, which satisfy(G,+) is an abelian group with identity element 0(a× b)× c = a× (b × c)(a + b)× c = a× c + b × c and c × (a + b) = c × a + c × b

2 Commutative ring: a× b = b × a3 Example: integers (no inverse element for ×); polynomials


Finite Field

1 Field: a commutative ring such that 0 6= 1 and all nonzeroelements have inverse element for ×.

2 Algebraic channel coding is done in finite field.3 Finite field: a field with finite element.4 If a finite field has q elements, we denote it by GF(q).


Integer Based Finite Field

1 Quotient ring Zq: 0,1,2, ...,q − 1.2 Sum: a + b = mod(a + b,q)3 Product: a× b = mod(ab,q).4 Zq is a filed if and only if q is a prime number.5 Example: GF(2)

0 + 0 = 1 + 1 = 0× 0 = 0× 1 = 01 + 0 = 1× 1 = 1


Polynomial Based Finite Field

1 Denote by F [x ] a single-variable polynomial over field F .2 Quotient ring F [x ]/p(x) for a polynomial p(x):

Sum: a(x) + b(x) = mod(a(x) + b(x),p(x))Product: a(x)× b(x) = mod(a(x)b(x),p(x))

3 F [x ]/p(x) is a field if and only if p(x) is a prime polynomial.A prime polynomial means that it cannot be divided by anypolynomials except 1 and itself.


Example: GF(4)

Table : Elements in GF(4)

Polynomial Binary Integer0 00 01 01 1x 10 2

x+1 11 3

GF(4) is obtained by setting F = GF (2) (the polynomialcoefficients can only be 0 or 1) and p(x) = x2 + x + 1.


General Properties of Finite Field

In any finite field, the number of elements is a power of aprime.Any two finite fields with the same number of elements areequivalent (rigorously, isomorphic) to each other.For every finite field, there exists a primitive element αsuch that every field element except 0 can be expressedas a power of α. For example, in GF(5),

21 = 1,22 = 4,23 = 3,24 = 1

.


Reed-Solomon Code

1 Reed-Solomon code was proposed by Reed and Solomonin MIT Lincoln lab in 1960. An efficient decoding algorithmwas devised by Berlekamp, which facilitates theapplication.

2 It is applied in CD, DVD, DSL, WiMAX and so on.3 In 1999, Madhu Sudan proposed decoding algorithm

beyond error correction bound. Koetter and Vardyproposed soft decoding algorithm in 2003 (won best paperaward of IT in 2004).


Discrete Fourier Transform (DFT)

1 For a vector in Cn, v = (v1, ..., vn), we can define DiscreteFourier Transform:

Vk =n−1∑k=1

e−j2πik

n vi ;

2 Inverse DFT:

vi =1n

n−1∑k=0

e−j2πik

n Vk ;


Galois Fourier Transform (GFT)

1 In GF(q), choose one element w such that wn = 1 andwm 6= 1 if m < n. Such a w exists only when n dividesq − 1. For sequence {vr}r=0,...,n−1, define:

Vs =n−1∑r=0

w rsvr , s = 0, ...,n − 1.

2 Example: In GF(257), we can design a RS code withcodeword length 4, 8, 16, 32, 64, 128 or 256. In eachcodeword n, each element is a number in GF(257).


Inverse Galois Fourier Transform (IGFT)

1 Inverse GFT exists:

vi =1n

n−1∑j=0

w−ijVj

2 Similarity to Discrete Fourier Transform (DFT): w = e−j2πN .


Properties of GFT

1 Many properties of DFT hold in GFT:2 Cyclic convolution and multiplication1

e = f ? g ⇔ Ei = FiGi

3 Time shift and modulation{w ivi

}⇔{

V((j+1))}

{w jVj

}⇔{

v((i−1))}

1For e = f ? g, ei =∑n−1

k=0 f((i−k))gk , where ((i)) = mod(i, n)


Definition of Reed-Solomon Code

1 In GF(q), if the codeword length n divides q − 1, acodeword {cr}r=0,...,n−1 belongs to a (n, k) Reed-Solomoncode if, in the GFT domain {Cr}r=0,...,n−1, there are n − kconsecutive zeros.

2 There is an equivalent definition for Reed-Solomon codethrough generator polynomial.


Properties of Reed-Solomon Code

1 Let d = n − k + 1. There are d consecutive zero elementsin the GFT domain ({Ci}0=1,...,n−1). Suppose that they arethe last d − 1 elements {Ci}i=n−d+1,...,n−1.

2 Define polynomial:

ψ(x) =n−d∑k=0

xkCk ,

and ci =1nψ(w

−i), i = 0, ...,n − 1.3 There are at most n − d zeros for ψ(x); therefore, unless

codeword c is all zero, c has at least d nonzero elements.Then dmin ≥ d = n − k + 1. However, from singletonbound, dmin ≤ d = n − k + 1. Then dmin = n − k + 1 andReed-Solomon code is MDS.


History of Convolutional Code

1 Convolutional code was first proposed by Elias (1954),developed by Wozencraft (1957) and rediscovered byHagelbarger (1959).

2 Viterbi proposed his celebrated algorithm for harddecoding of convolutional code (essentially the same asBellman’s dynamic programming) in 1967.

3 Soft decision algorithm (BCJR algorithm) was proposed byBahl, Cocke, Jelinek and Raviv in 1974 (essentially thesame as forward-backward algorithm in HMM).

4 Convolutional code is widely used in ... for its high efficientencoding and decoding algorithms.


Diagram Representation

1 Constituent: memory, output operators.2 Constraint length ν: length of memory.3 Rate p/q: q output bits when p information bits are input.


Trellis Representation

1 There are 2ν states. Possible state transitions are labeled.2 For each combination of input information bit and state, the

outputs are labeled in the trellis.


Polynomial Representation

1 If one input information bit one time,use p (number ofoutput operators) polynomials {gi}i=1,...,p to represent thecode, where x is delay operator (like z transform). E.g.g0(x) = x2 + x + 1 and g1(x) = x2 + 1.

2 If multiple inputs, we can use polynomial matrix. Forexample

G(x) =(

x 1 01 x2 1

)


Hard Decoding

1 Demodulation output is binary, 0 and 1 (hard).2 Viterbi algorithm searches a path in the trellis with

minimum discrepancy compared with the demodulationoutput.

3 For the t-th demodulation output, each node in the trellischooses the node in the t − 1-th output having least outputdiscrepancy, inherits its path and carries the accumulateddiscrepancy.


Soft Decoding: BCJR Algorithm

1 Forward pass:

αt(m) =∑m′

αt−1(m′)γt(m′,m),

with initial condition α0(0) = 1 and α0(m) = 0 for m 6= 02 Backward pass:

βt(m) =∑m′

βt+1(m′)γt+1(m,m′),

with initial condition βn(0) = 1 and βn(m) = 0 for m 6= 03 Compute joint probability, from which we can compare a

posteriori probability:

λt(m) = αt(m)βt(m).


Comparison Between Hard and Soft Decoding

1 Soft decoding utilizes more information (demodulation withmuch ambiguity is substantially igored). Therefore, softdecoder achieves better performance.

2 For convolutional code with constraint length 3 andtransmission rate 1/3, soft decoding has 2-3dB power gain(for achieving the same decoding error probability, softdecoding needs 2-3dB less power than hard decoding).

3 Soft decoding has more computational cost than harddecoding.


State Diagram


Transfer Function

1 State equations:

Xc = D3Xa + DXb, Xb = DXc + DXd

Xd = D2Xc + D2Xd , Xe = D2Xb

2 Transfer function:

T (D) =D6

1− 2D2 = D6 + 2D8 + 4D10 + ...

minimum distance is 6; there are two paths with Hammingdistance 8.


Concatenated Codes


Storm in Coding Theory: Turbo Code

1 Before 1993, practical code only achieved performance several dBs beyondShannon capacity.

2 In ICC1993, Berrou, Glavieux, and Thitimajshima published their incrediblepaper: Near Shannon Limit error-correcting coding and decoding: Turbo-codes.

3 "Their simulation curves claimed unbelievable performance, way beyond whatwas thought possible". "The thing that blew everyone away about turbo codes isnot just that they get so close to Shannon capacity but that they’re so easy." —McEliece

4 "They said we must have made an error in our simulations" — Berrou


Rediscovery of LDPC

1 Gallager devised LDPC in his PhD thesis in 1960. It was still impractical at thattime.

2 People (Mackay, Richardson, Urbanke) rediscovered LDPC in late 1990s.3 LDPC beat turbo code and is now very close to Shannon limit (0.0045dB away,

Chung 2001).4 "A piece of 21st century coding that happened to fall in the 20th century" —

Forney5 "We’re close enough to the Shannon limit that from now on, the improvements

will only be incremental" — Tom Richardson


Turbo Encoder

1 A turbo encoder has several component encoders (e.g.convolutional code) and an interleaver (the magic part!).


Interleaver

1 An interleaver is used to permute the information bitsequence. It brings randomness to the encoder (similar toShannon’s random coding)


Turbo Decoder

1 Decoding is done in an iterative (turbo) way.2 The soft output (e.g. a posteriori probability) of one

decoder is used as the a priori probability of anotherdecoder.

3 Such a turbo principle can be applied in many other fields:turbo equalization, turbo multiuser detection and so on.


Performance


LDPC Codes

1 A low-density parity-check code is a linear code with asparse check matrix (H). It could be represented by Tannergraphs.

2 Parity check matrix:

H =

1 1 1 1 0 00 0 1 1 0 11 0 0 1 1 0


LDPC Decoder

1 Message Passing is also called Belief Propagation.2 Step 1: at each check node, the messages from variables

nodes are passed to neighboring variable nodes.3 Step 2: at each variable node, the messages from check

nodes are passed to neighboring check nodes(incorporating the channel observations).

4 Repeat above two steps.


Error Detection

1 The previous channel codes are focused on errorcorrecting. We call it forward error correcting (FEC).

2 We can use Automatic Repeat reQuest (ARQ), i.e. thereceiver sends back a NACK message to inform thetransmitter to retransmit if there is mistake in the firsttransmission.

3 This requires error detection.


CRC

1 A most popular error detection code is cyclic redundancycheck (CRC), which is based on polynomial code in GF(2).

2 Generator polynomial G(x) (for IEEE802)

G(x) = x32 + x26 + x23 + x22 + x16 + x12 + x11

+ x10 + x8 + x7 + x5 + x4 + x2 + x + 1.

3 The data is also expressed as a polynomial M(x).4 The idea is to append a checksum to the end of the data

such that the polynomial represented by the checksummedframe is divisible by G(x).


ARQ: Sliding Window


ARQ: Go Back N


ARQ: Selective repeat


Type I H-ARQ

1 Hybrid ARQ (H-ARQ) combines FEC and ARQ.2 Type I H-ARQ: The receiver does error detection and

correction simultaneously. If an error is detected, thereceiver try to correct it. If not able to correct the error, thereceiver asks the transmitter to retransmit the samecodeword and try to decoding it again.

3 Type I H-ARQ is suitable for communication systems withfairly constant level of noise and interference.


Type II H-ARQ

1 In Type II H-ARQ, if an error is detected, only errordetection information or error detection/correct informationare retransmitted, instead of the whole and samecodeword.

2 Example: (Lin-Yu H-ARQ): two codes are used. one highrate (n, k) code C0, designed for error detection only, and ahalf-rate invertible (2k , k) code C1, designed forsimultaneously error detection and correction.

3 Invertible means that if knowing only k parity-check bits,the k information bits can be uniquely determined by aninversion operation. The parity check section contains thesame amount of information as the message section.

ECE442 Communications Lecture 6. Channel...

Documents

Transcript of ECE442 Communications Lecture 6. Channel...