Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional...

28
1 Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter Semester 2014/15 Lecturer: Giovanni Del Galdo

Transcript of Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional...

Page 1: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

1

Information Theory and Coding Convolutional Coding

SS2015 - Information Theory and Coding

Course: Information Theory and Coding

Winter Semester 2014/15

Lecturer: Giovanni Del Galdo

Page 2: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

2

• The waveform generator converts binary data to voltage levels (1 V., -1 V.) • The channel has an effect of altering the voltage that was transmitted • The input to the channel decoder is a vector of voltages rather than a vector of binary

values

Channel

e

rv

Channel Encoder

Waveform Generator

Channel Decoder

Channel v r x

v = [v1 v2 … vi … vn] e = [e1 e2 … ei … en] r = [r1 r2 … ri … rn] x = [x1 x2 … xi … xn]

0 T

0 T

+1 V.

-1 V.

vi vi=1

vi=0

xi +

zi ]-∞, ∞[

ri

Communication Background: Soft Decision

SS2015 - Information Theory and Coding

Page 3: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

3

Soft Decision Metric: Euclidian Distance

SS2015 - Information Theory and Coding

(1, 1) (-1, 1)

(1, -1) (-1, -1)

(0.4, 0.5)

(1, 1) (-1, 1)

(1, -1) (-1, -1)

(0.4, 0.5)

Hard Decision Soft Decision

Metric is the Hamming Distance: Sample Space = {0, 1, 2}

Metric is the Euclidian Distance: Sample Space = {0.61, 2.61, 2.21, 4.21}

Page 4: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

4

• Consider a terminated convolutional encoder with a generator polynomial: g = (5,7)oct

– Draw a schematic for the corresponding shift register implementation

– Draw a schematic for the corresponding state diagram

– Determine the rate and the constraint length

– Construct the trellis diagram and encode the sequence: [1 1 0 1 0 0]

– During transmission over an AWGN channel, transmission errors occurred and the sequence [-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] is received. Use the Viterbi algorithm once with Hard decision and once with Soft decision to decode the received sequence. Verify if the errors can be corrected.

Example: Soft Decision

SS2015 - Information Theory and Coding

Page 5: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

01 10

State Diagram Representation Shift-register Implementation

i/p o/p

+

+

b0 b-1

SS2015 - Information Theory and Coding 5

1/00

0/01

11

1/01

0/00

00

Page 6: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation

SS2015 - Information Theory and Coding 6

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11

11

11 11

-11 -11 -11 -11

11

11

1-1 1-1 1-1

1-1 1-1 1-1

-11 -11

11 11

-1-1 -1-1

Page 7: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Encoded sequence

SS2015 - Information Theory and Coding 7

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11

11

11 11

-11 -11 -11 -11

11

11

1-1 1-1 1-1

1-1 1-1 1-1

-11 -11

-1-1 -1-1

11 11

Encoded sequence: 1 1 1 -1 1 -1 -1 -1 -1 1 1 1

Page 8: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 8

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11

11

11 11

-11 -11 -11 -11

11

11

1-1 1-1 1-1

1-1 1-1 1-1

-11 -11

-1-1 -1-1

11 11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

5

1

4

3

5

3

2

Page 9: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 9

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11 11

-11 -11 -11 -11

11

11

1-1 1-1

1-1 1-1

-11 -11

-1-1

11 11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

Page 10: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 10

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11 11

-11 -11 -11 -11

11

11

1-1 1-1

1-1 1-1

-11 -11

-1-1

11 11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

2

3

4

4

1

4

3

3

Page 11: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 11

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11 -11

11

1-1

1-1 1-1

-11 -11

-1-1

11 11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

2

1

3

3

Page 12: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 12

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11 -11

11

1-1

1-1 1-1 1-1

-11 -11

-1-1

11 11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

2

1

3

3

2

5

2

4

Page 13: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 13

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11 -11

11

1-1

1-1

-11 -11

-1-1

11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

2

1

3

3

2

2

Page 14: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 14

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11 -11

11

1-1

1-1

-11 -11

-1-1

11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

2

1

3

3

2

2

4

2

Page 15: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Hard Decision Decoding

SS2015 - Information Theory and Coding 15

-1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11 -11

11

1-1

1-1

-11 -11

-1-1

11

0

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]

2

1

4

1

2

2

1

3

2

2

1

3

3

2

2

2

Decoded sequence: 0 1 0 1 0 0 Errors can‘t be corrected

Page 16: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 16

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11

11

11 11

-11 -11 -11 -11

11

11

1-1 1-1 1-1

1-1 1-1 1-1

-11 -11

-1-1 -1-1

11 11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

9

7.5

5

5

9.1

7.8

4.7

Page 17: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 17

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11 11

-11 -11 -11

11

1-1 1-1

1-1 1-1 1-1

-11 -11

-1-1

11 11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

5

5

4.7

Page 18: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 18

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11 11

-11 -11 -11

11

1-1 1-1

1-1 1-1 1-1

-11 -11

-1-1

11 11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

5

5

4.7

5.2

11

11.6

11.6

8.3

11.8

5.1

8.6

4.8

Page 19: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 19

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11

11

1-1

1-1 1-1 1-1

-11 -11

-1-1

11 11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

5

5

4.7

5.2

8.3

5.1

4.8

Page 20: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 20

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11

11

1-1

1-1 1-1 1-1

-11 -11

-1-1

11 11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

5

5

4.7

5.2

8.3

5.1

4.8

6.3

11.9

6.2

8.3

Page 21: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 21

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11

11

1-1

1-1 1-1

-11 -11

-1-1

11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

5

5

4.7

5.2

8.3

5.1

4.8

6.3

6.2

12.8

6.3

Page 22: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation – Soft Decision Decoding

SS2015 - Information Theory and Coding 22

-1-1 -1-1 -1-1 -1-1 -1-1

11 11

-11 -11

11

1-1

1-1 1-1

-11 -11

-1-1

11

1.7

[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]

2.3

3.1

5.7

7.1

2.9

5.1

5

5

4.7

5.2

8.3

5.1

4.8

6.3

6.2

6.3

Decoded sequence: 1 1 0 1 0 0 Errors corrected !!

Page 23: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

23

• Increasing Code Rate by Puncturing of Some of the Outputs of Convolutional Encoder

• Puncturing Rule selects the Outputs that are Eliminated

• The Construction of a Punctured Convolutional Code is that its Trellis should maintain the Same State and Transition Structure of the Base Code

Puncturing of Convolutional Codes

SS2015 - Information Theory and Coding

Page 24: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

24

Example : Puncturing of Convolutional Codes

SS2015 - Information Theory and Coding

+

+

Input OutputPuncturing Rule 1 1

1 0

c2

c1

Base Code Rate 1/2 Punctured Code Rate 2/3

Page 25: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation

SS2015 - Information Theory and Coding 25

-1-1 -1-1 -1-1 -1-1 -1-1 -1-1

11

11

11 11

-11 -11 -11 -11

11

11

1-1 1-1 1-1

1-1 1-1 1-1

-11 -11

11 11

-1-1 -1-1

Page 26: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation

SS2015 - Information Theory and Coding 26

-1-1 -1 -1-1 -1-1 -1-1 -1-1

11

11

11 11

-11 -1 -11 -11

1

11

1 1-1 1-1

1-1 1-1 1-1

-11 -11

11 11

-1-1 -1-1

Page 27: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation

SS2015 - Information Theory and Coding 27

-1-1 -1 -1-1 -1 -1-1 -1-1

11

11

11 1

-11 -1 -1 -11

1

11

1 1-1 1

1-1 1 1-1

-11 -11

1 11

-1-1 -1

Page 28: Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional Coding SS2015 - Information Theory and Coding Course: Information Theory and Coding Winter

00

01

10

11

Trellis Diagram Representation

SS2015 - Information Theory and Coding 28

-1-1 -1 -1-1 -1 -1-1 -1

11

11

11 1

-11 -1 -1 -11

1

11

1 1-1 1

1-1 1 1-1

-11 -11

1 1

-1-1 -1

Encoded sequence: 1 1 1 1 -1 -1 -1 1 1 Base Code Rate 1/2 Punctured Code Rate 2/3