
2/15

Content

o Overview of Coding

Source and channel coding definition.o Code Length and Code Efficiency

o Source Coding Techniques

Shannon-Feno coding

Huffman coding

o Channel Coding

Error Control coding ( ARQ and FEC)


3/15

The main purpose of coding is to improve the efficiency of communication sys

Coding is a procedure for mapping a given set of messages [m1,m2m

of encoded message [C1,C2CN] in such a manner that the transformaone. This is called source coding.

Main goal is to get minimum average length of code to increase the efftransmission.

It is also possible to devise codes to reduce the probability of error by dcorrecting errors (a.k.a. error control codes ) this coding is called chan


4/15

Code Length and Coding Efficiency:

Let M be number of symbols in there be N messages [m1,m2..MN] w[P(m1),P(m2)P(mN)] and Let ni be the number of symbols in i th messag

The average code length of the message is then given by :

_ N

L = ni p(ni) letters/messagei=1

_L should be minimum to have efficient transmission

Coding efficiency ,then can be defined as ;

_

= Lmin________

L


5/15

Let H(x)be the entropy of the source in bits per message, and let Log M is

information associated to each letter in bits /Letter. Then H(x)/Log (m) gives

average no. of letters per message Hence coding efficiency is :

_

= Lmin =H(x)______ ________ __

L L Log M


6/15

Classification of Codes:

Fixed length codes : code word length is fixed.

Variable Length codes: code word length is variable

Distinct codes: each code word is distinguishable from other.

Prefix free codes: a code in which no code word can be formed by adding c

another code ( in prefix free code no code word is prefix of another).

Uniquely decodable codes: A distinct code is uniquely decodable if original s

reconstructed from encoded binary sequence.

Instantaneous codes: uniquely decodable codes and prefix codes are instan

of any code word is recognizable without examining the subsequent code sy

Optimal Codes: A code is said to be Optimal if it is instantaneous and has m

length L


7/15

Shannon-Fano Encoding: Algorithm

The technique was proposed in Shannon's "A Mathematical Theory of communication", his 1948 introducing the field of information theory. The method was attributed to Fano, who later publishea technical report

Procedure of coding is as follows:

-Arrange the character set in order of decreasing probability

-While a probability class contains more than one symbol:

Divide the probability class in two so that the probabilities in the two halves anearly as possible equal

-Assign a '1' to the first probability class,

and a '0' to the second

Character

X6

X3

X4

X5

X1

X7

X2

Probability

0.25

0.2

0.15

0.15

0.1

0.1

0.05

1

0

1

0

1

0

1

0

1

0


8/15

Shannon-Fano Encoding: Example

Message x1 x2 x3 x4 x5 x6 x7 x8

Probability 0.25 0.25 0.125 0.125 0.0625 0.0625 0.0625 0.06

x1,x2,x3,x4,x5,x6,x7,x8

x1,x2 x3,x4,x5,x6,x7,x8

x1 x2 x3,x4 x5,x6,x7,x8

x3 x4 x5,x6 x7,x8

0100

1100

100

10

0 1

11

x5 x6 x7 x8

101 110 111

1101 1110 1111


9/15

Shannon-Fano Encoding: Example

Entropy

Average length of the encoding vector

Message x1 x2 x3 x4 x5 x6 x7

Probability 0.25 0.25 0.125 0.125 0.0625 0.0625 0.0625 0.

Encoding

vector

00 01 100 101 1100 1101 1110 1

1 1 1 1 1 12 log 2 log 4 log 2.75

4 4 8 8 16 16H

1 1 1

2 2 2 3 4 4 2.754 8 16

i iL P x n

H

Lette


10/15


11/15


12/15

Ch l di d l ith t l t h i If th d


13/15

Channel coding deals with error control techniques. If the da

the output of a communications system has errors that are to

frequent for the desired use, the errors can often be reduced

the use of a number of techniques.

Coding permits an increased rate of information transfer at a error rate, or a reduced error rate for a fixed transfer rate. Th

main methods of error control are:

Automatic Repeat Request (ARQ) when a receiver circuit

detects errors in a block of data, it requests that the data is

retransmitted.

Forward Error Correction (FEC) the transmitted data is enc

so that the data can correct as well as detect errors caused b

channel noise.


14/15

The two main categories of channel codes are:

Block codes a block code is a code having all its words of sam

length .a block of k information bits is encoded to give a codew

of n bits (n > k). For each sequence of k information bits, there a distinct codeword of n bits.

Examples of block codes include Hamming Codes and Cyclic

Codes.

A Cyclic Redundancy Check (CRC) code can detect any

error burst up to the length of the CRC code itself.

Convolution Codes the coded sequence of n bits depends noonly on the present k information bits but also on the previous

information bits.

These coding schemes will be covered in error control codes in details.!!!


15/15

Thank you

ICT (Source Encoding & Channel Encoding)

Documents

Transcript of ICT (Source Encoding & Channel Encoding)