Chapter 10 Error-Control Coding

7/29/2019 Chapter 10 Error-Control Coding

1/50

1

Chapter 10 Error-Control Coding

Error-Control Codingis the platform ofShannon channel coding theorem.

Po-Ning [email protected] Chapter 10-2

10.1 Introduction


2/50

2


10.1 Introduction

Error correction versus error detection

There is an alternative system approach to achieve

reliable transmission other thanforward error correction(FEC).

By the system structure, it is namedautomatic-repeatrequest(ARQ), which is a combination oferrordetectionandnoiseless feedback.


10.1 Introduction

Classifications of ARQ

ARQ with stop-and-wait strategy

After the transmission of a codeword, the transmitter stops andwait for the feedback before moving onto the next block of

message bits. Continuous ARQ with pullback

The transmitter continue its transmissions until a retransmissionrequest is received, at which point, it stops and pulls back to theincorrectly transmitted codeword.

Continuous ARQ with selective repeat

Only retransmitethe codeword that are incorrectly transmitted.


3/50

3


10.3 Linear block codes

10.2 Discrete memorylesschannelshas been introducedin Chapter 9, so we omit it.

Linear code

A code is linear if any two code words in the code canbe added inmodulo-2 arithmetic to produce a third codeword in the code.

The code words of a linear code can always be obtainedthrough a linearoperation in the sense ofmodulo-2arithmetic.



For linear code, there exists ak-by-ngenerator matrixGsuch that

Generator matrixG is said to be in thecanonical formif itskrows are linearly independent.


4/50

4


What will happen if one row is linearly dependent on other rows?

Suppose

Then


Hence, the number of distinct code words is at most 2k1 (not the anticipated 2k).


5/50

5



Parity-check matrixH

The parity-check matrix of a canonical generator matrix

is an (nk)-by-nmatrix satisfying

where the columns ofH are linear independent.

Then, the code words (or error-free receptions) should

satisfy (nk) parity-check equations.



Syndromes

The receptions may be erroneous (with error patterne).

With the help of parity-check matrix, we obtain


6/50

6



In short, syndromesare all possiblesymptomsthat possibly happen atthe output of parity-check examination.

Similar to the diseasesymptomsthat can possibly be observed andexamined from outside the body.

Based on thesymptoms, the doctors diagnose what disease occursinside the body.

Based on thesymptoms, the receiver diagnoses which bit is erroneous(i.e., ill) based on the symptoms (so that the receiver can correct theill bit.)

Notably, the syndrome only depends on the error pattern, and iscompletely independent of the transmitted code word.



Properties of syndromes

Syndrome depends only on the error pattern, and not onthe transmitted code word.

All error patterns that differ by (at least) a code wordhave the same syndromes.


7/50

7


1. All elements in a cosethave the same syndrome since

2. There are 2k elements in a cosetsince

3. Cosetsare disjoint.

i.e., cosetelements are the same if, and only if, two codewords are the same. This is contrary to the generalassumption of a code book, in which all code words aredistinct.

4. The number of cosetsis therefore 2nk, namely, the numberof syndromes is 2nk.

Thus, syndromes (withnkunknowns) cannot uniquelydetermine the error pattern (withnunknowns).



Systematic code

A code is systematic if the message bits are part of thecode words.

The remaining part of a systematic code is calledparitybits.

Usually, message bits are transmitted first because thereceiver can do direct hard decisionwhen necessary.


8/50

8



For a systematic code,

Example 10.1. (n, 1) repetition codes

and



Error-correcting capabilityof a linear block code

Hamming distance (for 0-1 binary sequences)

Minimum (pair-wise) Hamming distancedmin


9/50

9


Operational meaning of minimum (pair-wise) Hammingdistance

There exists at least apair of code words ofwhich the distance isdmin.

Minimum distance decoder


Based on minimum distance decoder, if the receivedvector and the transmitted code word differs at most

namely, if the number of 1s in the error pattern is at most

this number, then no error in decision is obtained.

If the received vector and the transmitted code worddiffers at tpositions, where

then erroneous decision is possibly made (when thetransmitted code word is one of the pair that results indmin.


10/50

10


We therefore name the below quantity theerrorcorrecting capabilityof a code (not limited to linear

block code).

Theerror correcting capabilityof alinear block codecan be easily determined by the minimumHammingweightof code words.

By linearity,



Syndrome decoding for (n, k) linear block codes


11/50

11



syndromerH

Find the element in coset(rH), whose has minimum weight.This element is usually named thecoset leader.Thecoset leader in each coset is fixed and known before the reception ofr.

Syndrome decodingusing standard array for an (n, k) blockcode



Example 10.2 (7,4) Hamming codes


12/50

12



Decoding table



Appendix: The notion ofperfect code

(7, 4) Hamming code is a binary perfect code.

All the 27

=128binary sequences areconfined with the 24 =16 nonoverlappingballs of radius 1,


13/50

13



Dual code

Every (n, k) linear block code with generator matrix G

and parity-check matrix H ahs a (n, nk) dual code withgenerator matrixH and parity-check matrix G.


10.4 Polynomial codes/Cyclic codes

Polynomial expression of a linear code

Polynomial code: A special type of linear codes.

Represent a code [c0, c1, , cn1] as a code

polynomial of degreen1

whereX is called theindeterminate.


14/50

14



Generator polynomial (of a polynomial code)

The code polynomial (of a polynomial code) includes

all polynomials of degreenk, which can be divided byg(X), and hence can be expressed as



Example of a (6, 3) polynomial code

Note that since

the example on the right is not a cyclic code.


15/50

15



Property of a polynomial code


10.4 Polynomialcodes/Cyclic codes

Example of a (6, 3)polynomial code

(Continue)

encode encode


16/50

16


Encoder for systematic polynomial codes


Example of usual long division

multiply


quotientremainder


17/50

17

multiply


quotientremainder

. repeat this procedure until the last term is shifted in..


After 14 clocks


18/50

18

Another example of long division (with the shift-registers

containing not the remainder but the lower-power coefficients)




19/50

19


On for first k clocksOff for the remaining (nk) clocks

Down forfirst k clocksUp for theremaining(nk) clocks


Afterkclocks,


20/50

20



Decoder for polynomial codes

How to find the syndrome polynomial s(X) of

polynomial codes with respect to received wordpolynomial r(X)?

Recall thatsyndromesare all possiblesymptomsthatpossibly happen at the output of parity-checkexamination.

If there is no error in transmission, r(X) modg(X) =0. Indeed,s(X) =r(X) modg(X).



Relation of syndrome polynomial s(X), error polynomiale(X) and received vector polynomial r(X) for systematic

polynomial codes.


21/50

21



Relation of syndrome polynomial s(X), error polynomiale(X) and received vector polynomial r(X) for generalpolynomial codes.



Syndrome calculator


22/50

22



Example of a (6, 3)polynomial code(Continue)

encode encode



23/50

23



The decoding of a systematic code is therefore to simplyadd the syndrome polynomial to the received vectorpolynomial.



Definition of cyclic codes

Cyclic property: Any cyclic shift of a code word is alsoa code word.

Linearity property: Any sum of two code words is alsoa code word.

A cyclic code is also a polynomial code.

See Theorem 5.3 in ShuLin and Daniel J . Costello, Jr,Error Control Coding, 2nd edition, Prentice Hall, 1983


24/50


25/50

25



Parity-check polynomial of cyclic codes

After proving thatg(X) must divideXn+1, we can define

the parity-check polynomial of a cyclic code as

whereh(X) is a polynomial of degreekwithh0 =hk =1.

Since the degrees ofg(X) andh(X) are respectivelynkandk1, andgnk=hk=1, we may induce that



Multiplying both sides bya(X) yields:


26/50

26




27/50

27


Observation.

The parity-check matrix arranges its entries according to the coefficients ofthe parity-check polynomial in reverse order as contrary to the generatormatrix arranges its entries according to the coefficients of thegeneratorpolynomial.



Remarks

The generator matrix and parity-check matrix derivedpreviously are not for systematic codes.

We can manipulate these two matrices by adding theirelements of selective rows so as to make themsystematic.

Example can be found in Examples 10.3 and 10.4 in text.


28/50

28



Examples 10.3 and 10.4: Hamming code and Maximum-length code

Irreducible polynomial: A polynomial that cannot befurther factored.

All three of the above are irreducible.

Primitive polynomial: An irreducible polynomial ofdegreem, which dividesXn 1 for n=2m 1 but doesnot dividesXn 1 for n< 2m 1.

Only the first two irreducible polynomials areprimitive.



Example 10.3: (7, 4, 3) Hamming code

Any cyclic code generated by a primitive polynomialis a Hamming code of minimum distance 3.

Example 10.4: (2

m

1, m, 2m1

) maximum-length codeThe maximum-length code is a dual code of

Hamming codes.

In other words, it is a cyclic code with primitiveparity-check polynomial.

It only exists with m> 2.

It is a code of minimum distance 2m 1.


29/50

29

It is named themaximum-length codebecause the

code word length for minformation bits has beenpush to themaximum(or equivalently, the number ofcode words for code of lengthnhas been push to theminimum.) For example,


So, there are in total 7code words ifc is originallynonzero. Adding the all-zero code word gives8 =23

code words. This makes the(7, 3) maximum-length code.

So, the nonzero code word in a maximum-lengthcode is a circular shift of all the other nonzero codewords.



30/50

30


Example: Cyclic redundancy check (CRC) codes

Cyclic codes are extremely well-suited for errordetection owing to thesimplicity of its implementation,

and its superior error-detection capability.

Binary (n, k, dmin) CRC codes can detect:

all contiguous error bursts of lengthn kor less.

2nk+14 of contiguous error bursts of lengthnk+1

all combinations ofdmin1 (or fewer) errors

all error patterns with an odd number of errors ifthe generator polynomial g(X) has an even numberof nonzero coefficients.


Bose-Chaudhuri-Hocquenghem(BCH) codes

Primitive (n, k, dmin) BCH codes

The parameters satisfies

(n, k, 3) Hamming code can be described as BCH

codes.

The table in next slide illustrates the generatorpolynomial g(X) of some binary BCH codes.


31/50

31


n=block length

k=number of message bitst=number of errors that are guaranteed correctable

Reed-Solomon (RS) Codes

A subclass of nonbinary BCH codes

Encode and decode based on blocks ofm-bit symbols.

(n, k, dmin) RS codes



32/50

32


10.5 Convolutional codes

For block codes, the encoding and decoding must performin a block-by-block basis. Hence, a big buffer for the entiremessage block and code word block is required.

Instead, for the convolutional codes, since the inputs andoutputs are governed through a convolution operation offinite-order filter, only a buffer of size equal to the filterorder is demanded.

Convolution is defined based on modulo-2 arithmeticoperations.



Buffer two bits at a time


33/50

33



(n, k, m) convolutional codes

k input bits will producenoutput bits.

The most recentmk-message-bit input blocks will berecorded (buffered).

Thenoutput bits will be given by a linear combinationof the buffered input bits.

Constraint lengthof a convolutional code

Number ofk-message-bit shifts over which a singlek-message-bit block can influence the encoder output.

Based on the above definition, constraint length =m+1.



Effective code rateof a convolutional code

In practice, kmzeros are often appended at the end toclear the shift register contents.

Hence, kL message bits will producen(L+m) output

bits.Theeffective code rate is therefore given by:

SinceL is often much larger thanm,

This is named thecode rateof a convolutionalcode.


34/50

34



Polynomial expression of convolutional codes

Example 10.5 (slide 10-64)

D is used instead ofX because the flip-flop (onetime-unit delay) is often denoted by D.



Buffer two bits at a time


35/50

35



Graphical expressions of convolutionalcodes

Code tree

Code trellis

generate the same next code symbol



Code trellis (continue)

Appended two zeros to clearthe shift-register contents


36/50

36



State diagram


10.6 Maximum likelihood decoding ofconvolutional codes

Likelihood(probability) function

Maximum-likelihood decoding

For equal prior probability, ML decoding minimizes theerror rate.


37/50

37



Minimum distance decoding

For additive noise,



Example: AWGN

Example: BSC


38/50

38



Viterbi algorithm (for minimum distance decoding overcode trellis)

Optimality

2

3

2

21

22

2

32Onesurvivor path foreach intermediate node


Solid line =information 1Dashed line =information 0

Assume that all-zero sequence is transmitted.


39/50

39


Solid line =information 0Dashed line =information 1

Assume that all-zero sequence is transmitted.

It is possible that theMLcode wordor theminimum distance code

word is not equal to thetransmitted code word.



Free distance of convolutional codes

Under binary symmetric channels, a convolutional codewith free distancedfreecan correct terrors if and only ifdfree is greater than 2t.

Question: How to determinedfree?

Answer: Bysignal-flow graph.


40/50

40


State diagram


State diagram

Signal graph

Exponent ofD =Hamming weight on the branchExponent ofL =Length of the branch

Input 100 generates code word of

length 3 with weight 5.

Input 10100 generates code word oflength 5 with weight 6.

Example.

Since input 0 generates code portion 00 at statea, 100000generates code word of weight 5.

Generates noHamming weight


41/50

41

Po-Ning [email protected] Chapter 10-81Po-Ning [email protected] Chapter 10-81



42/50

42



Indeed, the distance transfer functionT(D, L) enumeratesthe number of code words that are a given distance apart.

The derivation assumes that

which is valid only when |x| < 1.

For afeedforwardconvolutional code, the aboveequation is valid.

A feedback (or recursive) convolutional code may be

subject tocatastrophic error propagation.

Catastrophic error propagation=A finite number oftransmission errors will possibly cause infinitenumber of decoding errors.

A code with potential catastrophic errorpropagation is named acatastrophic code.



43/50

43

It can be proved that a systematic convolutional code

cannot be catastrophic. Unfortunately, the free distance of systematic codes is

usually smaller than that of nonsystematic codes.


Maximum free distances attainable for convolutional codes of rate 1/2.



Asymptotic coding gain

Hard decision decoding (or equivalently binarysymmetric channel)

Section 6.3 has established that for BPSK, the hard-

decision error for each code bit is given by:

The error of convolutional codes (at high SNR) isdominated by the two code wordswhose pairwiseHamingdistance is equal todfree.


44/50

44



For your information,


45/50

45

Soft decision decoding (or equivalently binary-input

additive white Gaussian channel)The error of convolutional codes (at high SNR) is

dominated by the two code wordswhose pairwiseHamming distance is equal todfree.



(Approximate) Error probability

Based on the decision rule


46/50

46


Asymptotic Coding gain (asymptotic =at high SNR) Ga

The performance gain due to coding (i.e., theperformance gain of coded system against uncodedsystem)


(Asymptotic) Coding gain (at high SNR)


47/50

47


(Asymptotic) Coding gain (at high SNR)

Error(log-scale)

Ga

uncoded

coded


10.7 Trellis-coded modulation

In the previous section, encoding is performed separatelyfrom modulation in the transmitter, likewise for decoding

and detection in the receiver.

To attain more effective utilization of the availablebandwidth and power, coding and modulation have to betreated as a single entity, e.g., trellis-coded modulation.

Instead of selecting code words from code bit domain,we choose code words from signal constellationdomain.


48/50

48


Partitioning of 8-PSK constellation, which shows thatd0 < d1 < d2.


Partitioning of 16-QAM constellation, which shows thatd0 < d1

Chapter 10 Error-Control Coding

Documents

Transcript of Chapter 10 Error-Control Coding