DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DEPARTMENT OF INFORMATION TECHNOLOGY

IT2302 - INFORMATION THEORY AND CODING

UNIT - IPart A1. Write the Kraft-Mc million inequality for the instantaneous code.(M/J-13)Kraft-Mc million inequality:K-1

2-lk 1

K=0

The codeword length of the prefix code satisfy above equation. Here lk is the codeword length of kth symbol sk. And K is the total number of symbols in the alphabet. Let us apply the above equation to the prefix code.3

2-lk = 2-l0 + 2-l1 + 2-l2 +2-l3K=0

From the above equation observe that l0 = 1, l1 = 2, l2 = 3 and l3 =3 digits (bits). Putting these values in the above equation,

3

2-lk = 2-1 + 2-2 + 2-3 +2-3 = 1 K=0

Thus Kraft McMillian inequality is satisfied.2. State the properties of entropy.

(A/M-11)Properties of Entropy:

a. Entropy is zero if the event is sure or it is impossible H = 0 if pk = 0 or 1

b. When pk = 1/M for all the M symbols, then the symbols are equally likely for such source entropy is given as H = log2M

c. Upper bound on entropy is given as,

Hmax = log2M3. What is Shannon limit?

(N/D-13)

The ShannonHartley theorem tells the maximum rate at which information can be transmitted over a communications channel of a specified bandwidth in the presence of noise. It is an application of the noisy channel coding theorem to the archetypal case of a continuous-time analog communications channel subject to Gaussian noise.

4. State the Channel Capacity Theorem.

(N/D-12)The channel capacity of the discrete memory less channel is given as maximum average mutual information. The maximization is taken with respect to input probabilities P(xi)

C = B log2(1+S/N) bits/sec

Here B is channel bandwidth. 5. What is the relationship between uncertainity and information? (A/M-10)Uncertainity:i. It is probability of occurrence of the events.

ii. Uncertainity of event decides amount of information.

Information:

i. It is the content received due to occurrence of events.

ii. Information received in certain event is zero. But information received in rate even is maximum.6. What is a Huffman Coding?In Huffman coding a separate codeword is used for each character. The code words produced by Huffman coding gives an optimum value.7. Calculate the entropy of the source emitting symbol with probability x = 1/5, y = 1/2, z = 1/3.

8. Define uncertaintyUncertainity:

i. It is probability of occurrence of the events.

ii. Uncertainity of event decides amount of information.

9. Differentiate: Uncertainty, Information and Entropy.

(N/D-10)S. NoUncertainityInformationEntropy

1.

2.It is probability of occurrence of the events.

Uncertainity of event decides amount of information.

It is the content received due to occurrence of events.

Information received in certain event is zero. But information received in rate even is maximum.

It is the average information received due to occurrence of multiple eventsEntropy is zero if the event is sure or impossible.

10. Write the properties of mutual information.i. The mutual information is symmetric. I(X;Y) = I(X:Y)

ii. The mutual information is always positive I(X;Y) 0 11. Calculate the amount of information if Pk =1/4?Amount of information : Ik = log2 (1 / pk)

= log10 4 / Log10 2

= 2 bits12. Define Shannon Fano Code Method This is a much simpler code than the Huffman code, and is not usually used, because it is not as efficient, generally, as the Huffman code, however, this is generally combined with the Shannon Method (to produce Shannon - Fano codes). The main difference, such that I have found, is that one sorts the Shannon probabilities, though the Fano codes are not sorted.13. What is average information?Average information or Entropy is given as,

Here, pk is the probability of kth message.

M is the total number of memories generated by the source.Part B1. Explain briefly the source coding theorem.

(6)(N/D-10)

Source Coding Theorem:

1. Source symbols encoded in binary

2. The average code length must be reduced

3. Remove redundancy ) reduces bit-rate

Consider a discrete memoryless source on the alphabet

S = {s0, s1, , sk}

Let the corresponding probabilities be

{p0, p1, , pk}

and code lengths be

{l0, l1, , lk}.

Then, the average code length(average number of bits per symbol) of the source is defined as

K-1

PKlIK=0

If Lmin is the minimum possible value of L, then the coding efficiency of the source is given by

Lmin

L

2. Explain Huffman encoding algorithm.

(4)(A/M-11)

i. Arrange the given source symbols in descending order with respect to its probability

ii. If it is a binary Huffman Huffman coding add the last source values into a single unit and Once again arrange the source values n decreasing order as obtained in step 2.

iii. Continue the process until only 2 source symbols are left.

iv. Start assigning codes (0, 1) in the backward direction towards the initial stage.

v. Determination of H(S) and Average codeword length.

vi. Check the condition for validity by using source coding theorem. If the condition satisfy then calculate coding efficiency and code redundancy. 3. A discrete memory less source has an alphabet of five symbols whose probabilities of occurrence are as described here : SymbolsS0S1S2S3S4

Probability0.40.20.20.10.1

Compute the Huffman code for this source, entropy and the average codeword length of the source encoder.

(12)(A-11)

4. Explain the properties of entropy.

(6)(M/J-13)

Average information is represented by entropy. It is represented by H k log 2 (1 / pk)

Properties of entropy: i. Entropy is zero if the event is sure or it is impossible

H = 0 if pk = 0 or 1

ii. When pk = 1/M for all the M symbols, then the symbols are equally likely for such source entropy is given as H = log2M

iii. Upper bound on entropy is given as,

Hmax = log2M

5. Explain channel capacity and derive the channel capacity for binary symmetric channel.

(8)(M/J-13)Channel capacity can be expressed in terms of mutual information. Mutual information is given by equation as,

n m

I(X ; Y) = P(xi ,yj) log2 P(xi / yj) / P(xi)

I = 1 j = 1

Here P(xi ,yj) can be calculated from equation as, P(xi ,yj) = P(yj / xi) / p(yi)

From the above equation we have,

P(xi / yj) / P(xi) = P(yj / xi) / p(yi)

Hence we can write equation as,

n m

I(X ; Y) = P(yj / xi) P(xi) log2 P(yj / xi) / p(yi)

I = 1 j = 1

n

P(yi) = P(yj / xi) P(xi)

I = 1

n m

n

I(X ; Y) = P(yj / xi) P(xi) log2 P(yj / xi) / P(yj / xi) P(xi)

I = 1 j = 1

I = 1The channel capacity of the discrete memoryless channel is given as maximum average mutual information. The maximization is taken with respect to input probabilities P(xi). i.e.,

max

C = P(xi) I(X ; Y)

6. Explain mutual information and its properties.

(8)(M/J-13)Mutual Information:

Properties of Mutual Information:

7. In the messages, each letter occurs the following percentage of times:(12)(N/D-13)

LetterABCDEF

% of occurrence23201191522

(1) Calculate the entropy of this alphabet of symbols.

(2) Devise a codebook using Huffman technique and find the average codeword length.

(3) Devise a codebook using Shannon-Fano technique and find the average codeword length.

(4) Compare and comment on the results of both techniques.

8. A discrete memoryless source has five symbols X1, X2, X3, X4 and X5 with probabilities 0.4, 0.19, 0.16, 0.15 and 0.15 respectively. Calculate a Shannon Fano code for the source and code efficiency.

(16)(N/D-12)The Shannon Fano algorithm is explained in table.

MessageProbability of messagesIIIIIICode word for messagesNumber of bits per messages i.e nK

X1X2

X3

X4

X50.40.19

0.16

0.15

0.1501

1

1

10

0

1

10

1

0

10100

101

110

11113

3

3

3

The entropy (H) is given by equation as,

M

H = pk log2 (1 / pk)

K = 1

Here M = 5 and putting the values of probabilities in above equation,

H = 0.4 log2 (1/0.4) + 0.19 log2 (1/ 0.19) + 0.16 log2 (1/0.16) + 0.15 log2 (1/0.15) + 0.15 log2 (1/0.15) = 2.2281 bits / messages

The average number of bits per messages N is given by the equation as,

L 1

N = pk nk k =0

Here pk is the probability of kth messages and nk are number of bits assigned to it. Putting the values in above equation,

N = 0.4(1) + 0.19(3) + 0.16(3) + 0.15(3) + 0.15(3)

= 2.35

The code efficiency is given by equation as,

Code Efficiency = H / N

= 2.2281 / 3.35

= 0.9489. Write short notes on:

(a) Binary Communication channel

The communication system consist of transmitter, receiver and channel.The channel encoder and modulator forms a transmitter. Similarly demodulator and channel decoder forms a receiver. The modulation channel or the communication channel has analog signals through it. In other words, the signals through communication channel are analog. The examples of such channels are telephone channels, high frequency radio channels and troposcatter systems.The signals through the channels are corrupted due to noise. Hence errors are introduced in the data. Thus nature of the channel and noise limit the maximum rate of the data transfer.

Binary Communication Channel(b) Binary Symmetric channel.

The binary communication channel is said to be symmetric if P(y0/x0)=P(y1/x1)

Binary Symmetric Channel

For the above channel we can write the equation as,

DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DEPARTMENT OF INFORMATION TECHNOLOGY

IT2302 - INFORMATION THEORY AND CODING

UNIT - IIPart A1. State the advantages of Lempel-Ziv algorithm over Huffman coding. (N/D-10)

Advantages of Lempel-Ziv algorithm:

i. The codewords are formed by appending new character.ii. No separate character set is stored in the dictionary initially.iii. The data characters as well as dictionary indices are transmitted.

Advantages of Huffman coding:

i. Codes for the characters are derived.

ii. Precision of the computer does not affect coding.

iii. Huffman coding is the simple technique.

2. Compare LZW with Huffman coding.

(A/M-11)S.NoLZW CodingHuffman Coding

1.

2.Codewords are new words appearing in the string.

Complete character set is stored in the dictionary initially.Codes for the character are derived.

Precision of the computer does not affect coding.

3. What is meant by prefix property?

(A/M-11)i. In the prefix coding no codeword is prefix of other codeword.

ii. The prefix codes are uniquely decodable.

iii. The prefix codes satisfy Kraft McMillian inequality.4. State the main application of GUI.

(A/M-11)The main application of GUI are,

i. GUI facilitates smooth interface between the human users and the operating system.

ii. Parameters handling, control and monitoring is done with the help of GUI.5. Define Hamming Distance

(A/M-11)

The hamming distance between the two code vectors is equal to the number of elements in which they differ. For example, let the two code words be,

X = (101) and Y = (110)

These two code words differ in second and third bits. Therefore the banning distance between X and Y is two. 6. What is Dolby AC-1?

(M/J-13)

Dolby AC-1 is used for audio coding. It is MPEG audio coding standard. It uses psychoacoustic model at the encoder and has fixed bit allocations to each subband.

7. Compare LZ and LZW coding.

(N/D-13)S.NoLZ CodingLZW Coding

1.

2.The codewords are formed by appending new character.No separate character set is stored in the dictionary initially.

Codewords are new words appearing in the string.

Complete character set is stored in the dictionary initially.

8. What is meant by frequency and temporal masking?

(N/D-13)When ear hears the loud sound, certain time has to be passed before it hears quieter sound. This is called temporal masking.

When multiple signals are present as in the case with general audio a strong signal may reduce the level of sensitivity of the ear to other signals, which are near to it in frequency. This effect is known as frequency masking.

9. What is perceptual coding?

(N/D-12)

In perceptual coding only perceptual feature of the sound are stored. This gives high degree of compression. Human ear is not sensitive to all frequencies equally. Similarly masking of weaker signal takes place when louder signal to present nearby. These parameters are used in perceptual coding. 10. What is Dolby AC-3?

(N/D-12)

The audio signal is not given directly to bit allocation algorithm. Rather, a spectral envelop of audio signal is provided to bit allocation algorithm as well as decoder. The bit allocation is not sent to the decoder. But the decoder reconstructs the audio signals from representation of spectral envelope. Hence it doesnot need bit allocation details directly.11. What is sub-band coding?In signal processing, Sub-band coding is any form of transform coding that breaks a signal into a number of different frequency bands and encodes each one independently. This decomposition is often the first step in data compression for audio and video signals.12. Draw the block diagram for pulse code modulator?

13. Write the differences between delta modulation and DPCM?

DM encodes the input sample by only one bit. It sends the information about + or -, ie. Step rise or fall. DPCM can have more than one bit for encoding the sample, it sends the information about difference between actual sample value and predicted sample value. 14. What is adaptive DPCM?Adaptive differential pulse-code modulation (ADPCM) is a variant of differential pulse-code modulation (DPCM) that varies the size of the quantization step, to allow further reduction of the required bandwidth for a given signal-to-noise ratio.

Part B1. Explain the concepts of frequency masking and temporal masking. How they are used in perceptual coding?

(16)(N/D-10)

Frequency masking:

i. The strong signal reduces level of sensitivity of human ear to other signals which are near to it in frequency. This effect is called frequency masking.

ii. Observe that sensitivity is very good between 2 to 5 KHz.iii. This sensitivity of at 4 KHz is considered 0 dB. Other sensitivity are plotted relatively.

iv. Observe that signal A is about 40 dB. It is well above the sensitivity of human ear.

v. The same signal A of 40 dB and one more signal B at 200 Hz. The signal B is strong(about 70 dB) and it is very close to signal A. Due to presence of signal b the sensitivity as per modified sensitivity curve. Hence signal A will not be sensed. While coding the two signals A and B the signal A can be dropped. Only signal B can be coded. This reduces the bit rate and increases compression ratio. Temporal masking:

When ear hears the loud sound, certain time has to be passed before it hears quieter sound. This is called temporal masking.2. Explain in detail, Adaptive Huffman coding, with the help of an example.Adaptive Huffman coding:

(16)(N/D-10)Principle:

In this method, the transmitter and receiver develop the Huffman coding tree dynamically depending upon the character in the data stream. If the character is not present in the data stream, then it is transmitted in its uncompressed form. The receiver identifies this character and add to the coding tree. If the character is present in the coding tree, then its codeword is transmitted. Depending upon the codeword, the receiver identifies the character.Sibling property i. A tree is said to exhibit sibling property if the nodes can be listed in order of increasing weight and if every node appears adjacent to its sibling in the listii. A binary tree is a Huffman tree if and only if it obeys the sibling property

iii. Thus, maintain the sibling property during the update assure that we have a Huffman tree before and after the counts are updated

iv. If the newly incremented node now has a weight of w+1, the next higher node will have a weight of w when updating is needed

v. There may be more nodes after the next higher one that have a value of w as well. The update (swap) procedure moves up the node list till it finds the last node with a weight of w

vi. That node is swapped with the node with weight w+1

vii. In this example, D is swapped with A

viii. The next node to be incremented will be the new parent of the incremented node. In this example, it is node #6

ix. As each node is incremented, a check is performed for correct ordering. A swap is performed if necessary

Not a Huffman Tree,

Still not a Huffman tree,

Now it is a Huffman tree again,

a. Note that the code length for A changes from 3 to 2

b. And the code length for B and D changes from 3 to 4 Initialization

i. Create a Huffman tree with two symbol EOF and ESCAPE with weight being1

ii. Let C be the incoming character

iii. If C is in the Huffman tree, then transmit the Huffman code for C and update the Huffman tree, else transmit the Huffman code for ESCAPE and the plain 8-bit character for C, and then update the Huffman tree for ESCAPE and C The overflow problem iv. the weight of the root exceeds the capacity as the program progresses long enough

v. The longest possible Huffman code exceeds the range of the integer used as a stack

vi. It is needed because the encoding and decoding phases treat the Huffman tree path in reverse order

Solution vii. Recalling the weight by a factor of 1/2

viii. It can possibly reshape the tree

3. Assume that the character set and probabilities are e =0.3, n = 0.3, t=0.2, w=0.1, .= 0.1. Derive the codeword value for string newt. Explain how the decoder determines original string from the received codeword value.

(16)(A/M-11)

4. Explain adaptive Huffman coding for the message Malayalam. (8)(N/D-13)

The basic Huffman algorithm has been extended, for the following reasons:

(a) The previous algorithms require the statistical knowledge which is often not available (e.g., live audio, video).

(b) Even when it is available, it could be a heavy overhead especially when many tables had to be sent when a non-order0 model is used, i.e. taking into account the impact of the previous symbol to the probability of the current symbol (e.g., "qu" often come together, ...).

Adaptive Huffman coding was first conceived independently by Faller and Gallager [Faller 1973; Gallager 1978]. Knuth contributed improvements to the original algorithm [Knuth 1985] and the resulting algorithm is referred to as algorithm FGK. A more recent version of adaptive Huffman coding is described by Vitter [Vitter 1987]. All of these methods are defined-word schemes which determine the mapping from source messages to codewords based upon a running estimate of the source message probabilities. The code is adaptive, changing so as to remain optimal for the current estimates. In this way, the adaptive Huffman codes respond to locality. In essence, the encoder is "learning" the characteristics of the source. The decoder must learn along with the encoder by continually updating the Huffman tree so as to stay in synchronization with the encoder.

Another advantage of these systems is that they require only one pass over the data. Of course, one-pass methods are not very interesting if the number of bits they transmit is significantly greater than that of the two-pass scheme. Interestingly, the performance of these methods, in terms of number of bits transmitted, can be better than that of static Huffman coding. This does not contradict the optimality of the static method as the static method is optimal only over all methods which assume a time-invariant mapping.

In the adaptive case the only overhead is thenlgnbits needed to represent each of thendifferent source messages when they appear for the first time. (This is in fact conservative; rather than transmitting a unique code for each of thensource messages, the sender could transmit the message's position in the list of remaining messages and save a few bits in the average case.) In the static case, the source messages need to be sent as does the shape of the code tree.

5. Explain the various versions of Dolby ACs stating its merits and demerits, With neat illustrations.

(8) (N/D-13)

Versions of Dolby ACs:

i. Dolby AC 1

iii. Dolby AC 2

iv. Dolby AC 3

Dolby AC-1was the first digital coding technology introduced in1987with the beginning development of what would becomeHDTV. Later in1989, Dolby introduced AC-2,which had a better audio quality and a reduced bit-rate. Finally, in1991, Dolby introducedAC-3, which includes 5.1 channels and operates at 320kbps.AC-3, also known asDolby Digital, is a perceptual digitalaudiocoding technique that reduces the amount of data needed to produce high-quality sound. Perceptual digital audio coding takes advantage of the fact that the human ear screens out a certain amount of sound that is perceived as noise. Reducing, eliminating, or masking thisnoisesignificantly reduces the amount of data that needs to be provided. Dolby Laboratories developed two other perceptual coding systems, AC-1 and AC-2. Building upon Dolby's two previous coding systems, AC-3 was the first coding system designed specifically for multichannel digital audio. AC-3 is the sound format for digital television (DTV), digital versatile discs (DVDs), high definition television (HDTV), and digital cable and satellite transmissions.AC-3 is a 5.1 format, which means that it provides five full-bandwidth channels, front left, front right, center, surround left, and surround right. A low-frequency effect (LFE) channel is included for the sound needed for special effects and action sequences in movies. The LFE channel is one-tenth of the bandwidth of the other channels and is sometimes erroneously called the subwoofer channel. AC-3 also has a downmixing feature that ensures compatibility with devices that do not support the 5.1 format.

6. Explain Linear Predictive Coding, With neat illustrations.

(8) (N/D-13)The speech signal is filtered to no more than one half the system sampling frequency and then A/D conversion is performed. The speech is processed on a frame by frame basis where the analysis frame length can be variable. For each frame a pitch period estimation is made along with a voicing decision. A linear predictive coefficient analysis is performed to obtain an inverse model of the speech spectrum A (z). In addition a gain parameter G, representing some function of the speech energy is computed. An encoding procedure is then applied for transforming the analyzed parameters into an efficient set of transmission parameters with the goal of minimizing the degradation in the synthesized speech for a specified number of bits. Knowing the transmission frame rate and the number of bits used for each transmission parameters, one can compute a noise-free channel transmission bit rate. At the receiver, the transmitted parameters are decoded into quantized versions of the coeifficent analysis and pitch estimation parameters. An excitation signal for synthesis is then constructed from the transmitted pitch and voicing parameters. The excitation signal then drives a synthesis filter 1/A (z) corresponding to the analysis model A (z). The digital samples s^(n) are then passed through an D/A converter and low pass filtered to generate the synthetic speech s(t). Either before or after synthesis, the gain is used to match the synthetic speech energy to the actual speech energy. The digital samples are the converted to an analog signal and passed through a filter similar to the one at the input of the system.

7. Write a detailed note on :

(1) Arithmetic codei. Huffman coding has been proven the best fixed length coding method available

ii. Yet, since Huffman codes have to be an integral number of bits long, while the entropy value of a symbol may (as a matter of fact, almost always so) be a faction number, theoretical possible compressed message cannot be achieved

iii. For example, if a statistical method assign 90% probability to a given character, the optimal code size would be 0.15 bits

iv. The Huffman coding system would probably assign a 1-bit code to the symbol, which is six times longer than necessary

v. Arithmetic coding bypasses the idea of replacing an input symbol with a specific code. It replaces a stream of input symbols with a single floating-point output number

vi. To encode the first character B properly, the final coded message has to be a number greater than or equal to 0.20 and less than 0.30

vii. After the first character is encoded, the low end for the range is changed from 0.00 to 0.20 and the high end for the range is changed from 1.00 to 0.30

viii. The next character to be encoded, the letter I, owns the range 0.50 to 0.60 in the new subrange of 0.20 to 0.30

ix. So, the new encoded number will fall somewhere in the 50th to 60th percentile of the currently established.

x. Thus, this number is further restricted to 0.25 to 0.26 (2) LZW algorithm

(16) (N/D-12)i. If the message to be encoded consists of only one character, LZW outputs the code for this character; otherwise it inserts two- or multi-character, overlapping*,distinct patterns of the message to be encoded in a Dictionary. *The last character of a pattern is the first character of the next pattern.

ii. The patterns are of the form: C0C1 . . . Cn-1Cn. The prefix of a pattern consists of all the pattern characters except the last: C0C1 . . . Cn-1LZW output if the message consists of more than one character:

i. If the pattern is not the last one; output: The code for its prefix.

ii. If the pattern is the last one:

if the last pattern exists in the Dictionary; output: The code for the pattern.

If the last pattern does not exist in the Dictionary; output: code(lastPrefix) the output:code(lastCharacter)8. Explain in detail, JPEG image compression techniques.

(16) (N/D-12)JPEG is an image compression standard which was accepted as an international standard in 1992.

i. Developed by the Joint Photographic Expert Group of the ISO/IEC

ii. For coding and compression of color/gray scale images

iii. Yields acceptable compression in the 10:1 range JPEG is a lossy compression technique

iv. Based on the DCT

v. JPEG is a general image compression technique independent of

Image resolution

Image and pixel aspect ratio

Color system

Image compexity

vi. A scheme for video compression based on JPEG called Motion JPEG (MJPEG) exists JPEG is effective because of the following three observations.vii. Image data usually changes slowly across an image, especially within an 8x8 blockExperiments indicate that humans are not very sensitive to the high frequency data images

Therefore we can remove much of this data using transform coding

The image is divided up into 8x8 blocks

2D DCT is performed on each block

The DCT is performed independently for each block

This is why, when a high degree of compression is requested, JPEG gives a blocky image result

DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DEPARTMENT OF INFORMATION TECHNOLOGY

IT2302 - INFORMATION THEORY AND CODING

UNIT - IIIPart A1. Distinguish between global color table and local color table in GIF. (N/D-10)

The GIF format is used mainly with internet to represent and compress graphical images. GIF images can be transmitted and stored over the network in interlaced mode, this is very useful when images are transmitted over low bit rate channels. 2. What is the minimum frame/sec required in MPEG?

(M/J-13)

MPEG-Motion Pictures Expert Group (MPEG) MPEG was formed by the ISO to formulate a set of standards relating to a rangeof multimedia applications that involves the use of video with sound3. State the main difference between MPEG video compression algorithm and H.261.

MPEGH.261

MPEG stands for Motion Pictures Expert Group(MPEG). It was formed by ISO. MPEG has developed the standards for compression of video with audio.H.261 video compression standard has been defined by the ITU-T for the

provision of video telephony and video conferencing over an ISDN.

4. What is TIFF?

(N/D-13)

Tagged Image File Format(TIFF) is used for the transfer of images as well as digitized documents. It supports 48 bits of pixels resolutions. 16 bits are used for each of R, G, and b colours. The code number indicates particular format in TIFF. 5. What is MPEG?

(A/M-10)

MPEG stands for Motion Pictures Expert Group(MPEG). It was formed by ISO. MPEG has developed the standards for compression of video with audio. MPEG audio coders are used for compression of audio. This compression mainly uses perceptual coding. 6. What is Graphics Interchange Format(GIF)?

(A/M-10)

Graphics Interchange Format(GIF) is used for representation and compression of graphical images. Each pixels of 24-bit colour image is represented by 8-bits for each of R, G and B colours. Such image has 224 colours. The GIF uses only 256 colours from the set of 224 colours.7. Define Code EfficiencyThe code efficiency is the ratio of messages bits in a block to the transmitted bits for that block by the encoder i.e.,

Code efficiency = Message bits / Transmitted bits

= k / n8. What are the error detection and correction capabilities of hamming code?

The minimum distance (dmin) of Hamming codes is 3. Hence it can be used to detect double errors or correct single errors. Hamming codes are basically linear block codes with dmin = 39. What are Hamming codes?

Hamming codes are (n,k) liner block codes with following conditions:

i. Number of check bits q 3

ii. Block length n = 2q 1

iii. Number of message bits k = n q

iv. Minimum distance dmin = 3 10. Differentiate convolutional code from block codes.Block codes take k number of message bit simultaneously and form n bit code vector is also called block. Convolutional code takes one message bit at a time and generates two or more encoded bits. Thus convoutional codes generate a string.

11. What is constraint length for convolutional encoders?

Constraint length is the number of shifts over which the single message bit can influence the encoder output. It is expressed in terms of message bits. 12. What is the use of syndromes?

Syndrome gives an indication of errors present in received vector Y. If YHT

= 0, then there are no errors in Y and it is valid code vector. The non zero value of YHT is called syndrome. Its non zero value indicates that Y is not a valid code vector and it contains errors.

Part B1. Explain the working of JPEG encoder and decoder, with a block diagram. (N/D-10)

There are two types of JPEG algorithms.

Baseline JPEG: During decoding, this algorithm draws line until complete image is shown.

Progressive JPEG: During decoding, this JPEG algorithm draws the whole image at once, but in very poor quality. Then another layer of data is added over the previous image to improve its quality. Progressive JPEG is used for images on the web.the used can make out the image before it is fully downloaded.

2. Explain the MPEG algorithm for video encoding, with a block diagram.(16)(N/D-10)

MPEG = Motion Picture Experts GroupMPEG is a family of encoding standards for digital multimedia information

MPEG-1: a standard for storage and retrieval of moving pictures and audio on storage media (e.g., CD-ROM).

i. Layer I

ii. Layer II

iii. Layer III (aka MP3)

iv. MPEG-2: standard for digital television, including high-definition television (HDTV), and for addressing multimedia applications.

v. Advanced Audio Coding (AAC)

vi. MPEG-4: a standard for multimedia applications, with very low bit-rate audio-visual compression for those channels with very limited bandwidths (e.g., wireless channels).

vii. MPEG-7: a content representation standard for information search3. Explain Linear predictive coding.

(6)(A/M-11)Linear predictive coding(LPC) is a tool used mostly inaudio signal processingandspeech processingfor representing thespectral envelopeof adigitalsignalofspeechincompressedform, using the information of alinear predictivemodel.[1]It is one of the most powerful speech analysis techniques, and one of the most useful methods for encoding good quality speech at a low bit rate and provides extremely accurate estimates of speech parameters.

LPC analyzes the speech signal by estimating the formants, removing their effects from the speech signal, and estimating the intensity and frequency of the remaining buzz. The process of removing the formants is called inverse filtering, and the remaining signal after the subtraction of the filtered modeled signal is called the residue.

The numbers which describe the intensity and frequency of the buzz, the formants, and the residue signal, can be stored or transmitted somewhere else. LPC synthesizes the speech signal by reversing the process: use the buzz parameters and the residue to create a source signal, use the formants to create a filter (which represents the tube), and run the source through the filter, resulting in speech.

Because speech signals vary with time, this process is done on short chunks of the speech signal, which are called frames; generally 30 to 50 frames per second give intelligible speech with good compression.4. How are P-frame and B-frames encode and decode?

(10) (N/D-13)B framesThe MPEG encoder also has the option of using forward/backward interpolated prediction. These frames are commonly referred to as bi-directional interpolated prediction frames, or B frames for short. As an example of the usage of I, P, and B frames, consider a group of pictures that lasts for 6 frames, and is given as I,B,P,B,P,B,I,B,P,B,P,B. As in the previous I and P only example, I frames are coded spatially only and the P frames are forward predicted based on previous I and P frames. The B frames however, are coded based on a forward prediction from a previous I or P frame, as well as a backward prediction from a succeeding I or P frame. As such, the example sequence is processed by the encoder such that the first B frame is predicted from the first I frame and first P frame, the second B frame is predicted from the second and third P frames, and the third B frame is predicted from the third P frame and the first I frame of the next group of pictures. From this example, it can be seen that backward prediction requires that the future frames that are to be used for backward prediction be encoded and transmitted first, out of order. This process is summarized in Figure7.16. There is no defined limit to the number of consecutive B frames that may be used in a group of pictures, and of course the optimal number is application dependent. Most broadcast quality applications however, have tended to use 2 consecutive B frames (I,B,B,P,B,B,P) as the ideal trade-off between compression efficiency and video quality.

B-Frame EncodingThe main advantage of the usage of B frames is coding efficiency. In most cases, B frames will result in less bits being coded overall. Quality can also be improved in the case of moving objects that reveal hidden areas within a video sequence. Backward prediction in this case allows the encoder to make more intelligent decisions on how to encode the video within these areas. Also, since B frames are not used to predict future frames, errors generated will not be propagated further within the sequence.One disadvantage is that the frame reconstruction memory buffers within the encoder and decoder must be doubled in size to accommodate the 2 anchor frames. This is almost never an issue for the relatively expensive encoder, and in these days of inexpensive DRAM it has become much less of an issue for the decoder as well. Another disadvantage is that there will necessarily be a delay throughout the system as the frames are delivered out of order. Most one-way systems can tolerate these delays, as they are more objectionable in applications such as video conferencing systems.

P (predicted) frame

i. Coded with motion compression using past I frames or P frames

ii. Can be used as reference pictures for additional motion compensation

5. Explain the entropy encoding blocks of JPEG standard.

(6) (N/D-13)JPEG is an image compression standard which was accepted as an international standard in 1992.

a. Developed by the Joint Photographic Expert Group of the ISO/IEC

b. For coding and compression of color/gray scale images

c. Yields acceptable compression in the 10:1 range

JPEG is a lossy compression technique

d. Based on the DCT

e. JPEG is a general image compression technique independent of

i. Image resolution

ii. Image and pixel aspect ratio

iii. Color system

iv. Image compexity

f. A scheme for video compression based on JPEG called Motion JPEG (MJPEG) exists

JPEG is effective because of the following three observations

g. Image data usually changes slowly across an image, especially within an 8x8 block

i. Therefore images contain much redundancy

h. Experiments indicate that humans are not very sensitive to the high frequency data images

i. Therefore we can remove much of this data using transform codingJPEG Encoding Overview

6. Write short notes on H.261 video compression standard.

(8) (N/D-13)

H.261 video compression standard:An ITU-T H-series standard applicable to videophone or video conferencing

1. Video coding algorithm is designed for transport using the Real-time Transport Protocol (RTP)

2. Operates in real-time with limited delay

3. Transmission bit rate is at multiples of 64Kbit/s Features of H.261 Source format

Bit rate

i. The target bit rate is ~ 64Kbps to 1920Kbpsii. CIF (Common Intermediate Format) NTSC & PAL

iii. QCIF (Quarter Common Intermediate Format)iv. At 29.97 frames per second with 4:2:0 chrominance sub-sampling (Y:CB:CR)

v. Picture coded as luminance and two color difference components (Y, CB and CR)

vi. Group of blocks (GOB)

vii. Macro Block (MB)

Idea

i. Find similar parts in other images

ii. Encode where it was found (i.e. motion vector)

iii. Previous decoded image Reference image

iv. Image to code Target Image

v. Encode the residual only H.261 Coder

Techniques used:

i. Two dimensional (2-D) 8 X 8 DCT to remove intra-frame correlation

ii. Zig-zag order to scan the transform coefficients

iii. Run Length coding for zero-valued coefficients after quantization

iv. Motion estimation is applied to video sequence to improve the prediction between successive frames

v. Transmission rates control in the range of p X 64 Kbps

vi. Error resilience including synchronization and concealment technique required in transmission code, to cover up channel errors

vii. Common Intermediate Format (CIF) and Quarter CIF (QCIF) for a single solution to different video formats (NTSC / PAL)

DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DEPARTMENT OF INFORMATION TECHNOLOGY

IT2302 - INFORMATION THEORY AND CODING

UNIT - IVPart A1. State the various methods used for text compression.

(N/D-10)The various methods used for text compression.1. Lossy compression

All original data can be recovered when the file is uncompressed. 2. Lossless compression

With lossless compression, every single bit of data that was originally in the file remains after the file is uncompressed2. What is Syndrome?

(N/D-10)

Syndrome gives an indication of errors present in received vector Y if YHT = 0, then there are no errors in Y and it is valid codevector. The non zero value of YHT is called syndrome. Its non zero value indicates that Y is not a valid code vector and it contains errors. 3. Why are cyclic codes extremely well suited for error detection? (N/D-10)i. They are easy to encode

ii. They have well defined mathematical structure. Therefore efficient decoding schemes are available.

4. State the syndrome properties.

(A/M-11)The syndrome properties are,

i. Syndrome is obtained by S = YHTii. If Y = X then S = 0 i.e., no error in output

iii. If Y = X then S 0 i.e., there is an error in output.

iv. Syndrome depends upon the error pattern only, i.e., S = EHT 5. State the properties of cyclic codes.

(M/J-13)The properties of cyclic codes are,a. Linearity Property

This property states that sum of any two codewords is also a valid codeword. For example let X1 and X2 aretwo codewords. Then,

X3 = X1 + X2Here X3 is also a valid codeword. This property shows that cyclis code is also a linear code.b. Cyclic Property

Every cyclic shift of the valid code vector produces another valid codevector. Because of this property, the name cyclic is given. Consider an n-bit codevector as shown below:

X = {xn-1,xn-2,..,x1,x0}6. Write example for repetition code.

(N/D-13)A single message bit is encoded in to a block of n identical bits producing a (n, 1) block code. There are only two code words in the code.

i. all-zero code word

ii. all-one code word

7. Write the properties of a syndrome.

(N/D-13)The syndrome depends only on the error patterns and not on the transmitted code word. All error patterns that differ by a code word have the same syndrome.

8. What are the types of JPEG algorithms?There are two types of JPEG algorithms.

Baseline JPEG: During decoding, this algorithm draws line until complete image is shown.

Progressive JPEG: During decoding, this JPEG algorithm draws the whole image at once, but in very poor quality. Then another layer of data is added over the previous image to improve its quality. Progressive JPEG is used for images on the web.the used can make out the image before it is fully downloaded.

9. What is spatial frequency?

The rate of change of pixel magnitude along the scanning line is called spatial frequency.

10. Explain Run-length coding?

The runlength encoding is siplest lossless encoding techniques. It is mainly used to compress text or digitized documents. Binary data strings are better compressed by runlength encoding. Consider the binary data string

1111110000011111.

If we apply runlength coding to abouve data string, we get,

7,1; 6,0; 5,1; 3,0

Thus there are seven binary 1s, followed by six binary 0s followed by five binary 1s and so on. 11. What is arithmetic coding?

The arithmetic coding offers the oppertunity to create a code that exactly represents the frequency of any character. A single character will be defined by belonging to a specific interval. For more frequent characters larger intervals will be used contrary to rare characters. The size of those intervals is porportional to the frequency. The arithmetic coding is the most efficient procedure but its usage will be restricted by patents.

Part B1. For a linear block code, prove with example that

(i)The syndrome depends only on error pattern and not on transmitted code word. (8)

(ii) All error patterns that differ by a codeword have the same syndrome.

(8) (N/D-10)i. the number of codeworde is 2k since there are 2k distinct messages.

ii. The set of vectors {gi} are linearly independent since we must have a set of unique codewords.

iii. linearly independent vectors mean that no vector gi can be expressed as a linear combination of the other vectors.

iv. These vectors are called baises vectors of the vector space C.

v. The dimension of this vector space is the number of the basis vector which are k.

vi. Gi C( the rows of G are all legal codewords.

vii. A (n,k) block code , where Block : encoder accepts a block of message symbols and generates a block of codeword

viii. Linear : addition of any two valid codeword results in another valid codeword

Code rate

i. Code rate increases , error correcting capability decreases more bandwidth efficiency

ii. Code rate decreases , error correcting capability increases waste of bandwidth.2. Consider the generation of a (7,4) cyclic code by the generator polynomial g(x) = 1 +x + x3 .

(i) Calculate the code word for the message sequence [1001] and construct systematic generator matrix G.(8)

(ii) Draw the diagram of encoder and syndrome calculator generated by the polynomial.

(8) (A/M-11)Given data

Generator Polynomial, G(p) = p3 + p + 1Message sequence M = 1001

(i). To determine length of the coded sequence:

To degree of the generator polynomial is q = 3. Hence there are 3 check bits in coded sequence. Length of messages sequence is k = 4. Therefore total number of bits in the coded sequence will be,

N = k + q

= 4 + 3 =7

This is (7, 4) cyclic code.

(ii). To determine transmitted codeword:

The check bit polynomial is given as,

C(p) = rem [ pq M(p) / G(p) ]

The messages sequence is, M = 1001

M(p) = p3 + 0p2 + 0p + 1

Since q = 3, pq M(p) will be,

pqM(P) = p3 + 0p2 + 0p + 1

pq M(P) = p3(p3 + 0p2 + 0p + 1)

= p6 + 0p5 + 0p4 + p3

Thus the remainder is, C(p) = p2 + p

C = (1 1 0)

The systematic form of the codeword is given as,

X = (m3m2m1m0 : c2

c1c0)

= (10 0 1 : 1 1 0)

This is the required transmitted codeword.3. Explain syndrome and its properties.

(6) (M/J-13)i. Huge storage memory (and searching time) is required by standard array decoding.

ii. Define the syndrome

s = vHT = (c + e) HT = eHT

iii. The syndrome depends only on the error pattern and not on the transmitted codeword.

iv. Therefore, each coset in the array is associated with a unique syndrome.

The syndrome properties are,

i. Syndrome is obtained by S = YHTii. If Y = X then S = 0 i.e., no error in output

iii. If Y = X then S 0 i.e., there is an error in output.

iv. Syndrome depends upon the error pattern only, i.e., S = EHT 4. Explain syndrome decoding in linear block codes with example.(5) (M/J-13)

Hamming Code is a Linear Block Code. Linear Block Code means that the codeword is generated by multiplying the message vector with the generator matrix.

Minimum weight as large as possible. If minimum weight is 2t+1, capable of detecting 2t error bits and correcting t error bits.

Syndrome decoding is a more efficient method of decoding a linear code over a noisy channel. Syndrome decoding is minimum distance decoding using a reduced lookup table.

INCLUDEPICTURE "http://public.slidesharecdn.com/b/images/app_promo_close_hdpi.png?3fb03e1e12" \* MERGEFORMATINET

INCLUDEPICTURE "http://public.slidesharecdn.com/b/images/app_promo_logo.png?96c32f9916" \* MERGEFORMATINET SlideShare Android AppFree app in Google PlayInstall 5. Explain the Hamming codes with example.

(16) (N/D-12)

Hamming Codes are still widely used in computing, telecommunication, and other applications. Hamming Codes also applied in

i. Data compression

ii. Some solutions to the popular puzzle The Hat Game

iii. Block Turbo Codes

A [7,4] binary Hamming Code

i. Let our codeword be (x1 x2 x7) F27 ii. x3, x5, x6, x7 are chosen according to the message (perhaps the message itself is (x3 x5 x6 x7 )).

iii. x4 := x5 + x6 + x7 (mod 2)

iv. x2 := x3 + x6 + x7

v. x1 := x3 + x5 + x7vi. If (y1 y2 y7) is received and abc 000, then we assume the bit abc is in error and switch it. If abc=000, we assume there were no errors (so if there are three or more errors we may recover the wrong codeword).vii. For an [n, k] linear code, the generator matrix is a kn matrix for which the row space is the given code.

viii. A check matrix for an [n, k] is a generator matrix for the dual code. In other words, an (n-k)k matrix M for which Mx = 0 for all x in the code.

ix. For a given r, form an r 2r-1 matrix M, the columns of which are the binary representations (r bits long) of 1, , 2r-1.

x. The linear code for which this is the check matrix is a [2r-1, 2r-1 r] binary Hamming Code = {x=(x1 x2 x n) : MxT = 0}.

6. Explain in detail, Cyclic codes.

(16) (N/D-12)

They possess rich algebraic structure that can be utilized in a variety of ways. They have extremely concise specifications. They can be efficiently implemented using simple shift registers. Many practically important codes are cyclic. Convolution codes allow to encode streams od data (bits).

i. Hamming code is useful but there exist codes that off Polynomial representation of cyclic codes.

C(x) = Cn-1xn-1 + Cn-2xn-2 + + C1x1 + C0x0 , where, in this course, the coefficients belong to the binary field {0,1}.ii. That is, if the codeword is (1010011) (c6 first, c0 last), we write it as x6 + x4 + x + 1. iii. Addition and subtraction of polynomials Done by doing binary addition or subtraction on each bit individually, no carry and no borrow.iv. Division and multiplication of polynomials. Try divide x3 + x2 + x + 1 by x + 1.v. Same (if not larger) error control capabilities while can be implemented much simpler.vi. Cyclic code is a linear code that any cyclic shift of a codeword is still a codeword.vii. Makes encoding/decoding much simpler, no need of matrix multiplication.viii. A (n,k) cyclic code can be generated by a polynomial g(x) which has degree n-k and is a factor of xn - 1.

Call it the generator polynomial.

i. Given message bits, (mk-1m1m0 ), the code is generated simply as:

ii. In other words, C(x) can be considered as the product of m(x) and g(x).

DHANALAKSHMI COLLEGE OF ENGINEERING, CHENNAI

DEPARTMENT OF INFORMATION TECHNOLOGY

IT2302 - INFORMATION THEORY AND CODING

UNIT - VPart A1. What are conventional codes? How are they different from block codes? (N/D-10)

Convolution codes are error detecting codes used to reliably transmit digital data over unreliable communication channel system to channel noise.2. State the principle of Turbo coding.

(N/D-10)The significance of Turbo coding is,

i. High weight code wordsii. Decoder generates estimates of codewords in two stages of decoding and interleaving-deinter leaving.

iii. This is like circulation of air in turbo engine for better performance. Hence these codes called turbo codes3. What are the reasons to use an inter leaver in a turbo code?

(A/M-11)

An interleaver is a device that rearranges the ordering of sequence of symbols in a deterministic manner. The two main issues in the interleaver size and the interleaver map.

Interleaver is used to feed the encoders with permutations so that the generated redundancy sequences can be assumed independent.4. Define constraint length

(A/M-11)The constraint length (K) of a convolutional code is defined as the number of shifts a single message bit to enter the shift register and finally comes out of the encoder output.

K= M + 1

5. What are the differences between block and convolution codes? (M/J-13)

S.NoBlock codesConvolution codes

1.

2.

3.The information bits are followed by the parity bits.

There is no data dependency between blocks

Useful for data communications.The information bits are spread along the sequence

Data passes through convolutional codes in a continuous stream.Useful for low latency communications.

6. Define Constraint Length of a Convolutional Code.

(N/D-13)

Constraint length is the number of shifts over which the single message bit can influence the encoder output. It is expressed in terms of message bits.

7. What are convolutional codes?

(N/D-12)A convolutional code in which parity bits are continuously interleaved byinformation (or) message bits.8. Define Turbo code

(N/D-12)

The Parallel Concatenated Convolutional Codes(PCCC), called Turbo Codes, has solved the dilemma of structure and randomness through concatenation and interleaving respectively. The introduction of turbo codes has given most of the gain promised by the channel coding theorem.Part B1. Explain in detail, the Viterbi algorithm for decoding of convolutional codes, with a suitable example.

(16) (N/D-10)The Viterbi algorithm performs Maximum likelihood decoding.It find a path through trellis with the largest metric (minimum Hamming distance/minimum Euclidean distance).

i. At each step in the trellis, it compares the metric of all paths entering each state, and keeps only the path with the largest metric (minimum Hamming distance) together with its metric. The selected path is known as survivor path.

ii. It proceeds in the trellis by eliminating the least likely paths.

A Convolutional code is specified by three parameters (n,k,K) or (k/n,K) where

i. Rc=k/n is the rate efficiency, determining the number of data bits per coded bit.

ii. K is the size of the shift register.

iii. Constraint length = n*K, i.e. the effect of each bit have its influence on n*K bits.

Problems on Viterbi Algorithm:

i. Computational complexity increases exponentially with constraint length.

ii. The usually used Hamming distance in VA is sub-optimum and therefore lose some performance.

iii. Viterbi algorithm is a ML (optimum) algorithm if Euclidean distance is used.

Application of Viterbi Alogorithim:i. Convolutional decoding and channel trellis decoding. ii. Speech and character recognition . iii. Optical character recognition. iv. DNA sequence analysis . 2. Construct a convolution encoder for the following specifications : rate efficiency 1/2, constraint length 3, the connections from the shift register to modulo-2 adder are described by the following equations, g 1 (x) = 1 + x + x2 , g 2 (x) = 1 + x2 . Determine the output codeword for the message [10011]. (16) (A/M-11)Given:

g 1 (x) = 1 + x + x2

g 2 (x) = 1 + x2Here g 1 (x) and g 2 (x) has only two inputs. Hence we must interpret the constraint length as nM.

Here rate efficiency = k / n = 1 / 2

n = 2

Since constraint length is given as,

nM = 4

2 M = 4

M = 4 Since n = 2

Thus there are two storage locations.

g(1)(p) = 1 + p and g(2)(p) = p

m = (1 1 1 0)

m(p) = p3 + p2 + p

x(1)(p) = g(1)(p) m(p)

= (1 + p)(p3 + p2 + p)

= p4 + p

= 0 + p + 0 + 0 + p4

Xi(1) = {0 1 0 0 1}

And, x(2)(p) = g(2)(p) . m(p)

= p(p3 + p2 + p)

= p4 + p3 + p2

= 0 + 0 + p2 + p3 + p4

Xi(2)= {0 0 1 1 1}

Hence output sequence after multiplexing xi(1) and xi(2) will be

Xi = {0 0 1 0 0 1 0 1 1 1} 3. Explain in detail, Turbo decoding.

(16) (A/M-11)Definition

i. High weight code words

ii. Decoder generates estimates of codewords in two stages of decoding and interleaving-deinter leaving.

iii. This is like circulation of air in turbo engine for better performance. Hence these codes called turbo codesTurbo encoder diagram

Turbo decoderCriterion

For n probabilistic processors working together to estimate common symbols, all of them should agree on the symbols with the probabilities as a single decoder could do

4. How is dfree determine using Trellis algorithm? Explain.

(4) (N/D-13)Trellis Representation of Convolutional Code

State diagram is unfolded a function of time

Time indicated by movement towards right

Contents of shift registers make up "state" of code:i. Most recent input is most significant bit of state.

ii. Oldest input is least significant bit of state.

iii. Allowable transitions are denoted by connects between statesiv. transitions may be labeled with transmitted bits

Example of Trellis Diagramk=1, n=2, K=3 convolutional code

5. Explain in detail, convolutional code and compare it with block codes.(8)(N/D-12)Block Versus Convolutional Codes

Block codes take k input bits and produce n output bits, where k and n are large there is no data dependency between blocks useful for data communications.

Convolutional codes take a small number of input bits and produce a small number of output bits each time period.i. data passes through convolutional codes in a continuous stream

ii. useful for low- latency communicationsiii. k bits are input, n bits are output

iv. Now k & n are very small (usually k=1-3, n=2-6)

v. Input depends not only on current set of k input bits, but also on past input. The number of bits which input depends on is called the "constraint length" K.vi. Frequently, we will see that k=16. Explain in detail, Turbo codes and their uses.

(16) (N/D-12)

Turbo Codes IEEE International Comm conf 1993 in GenevaBerrou, Glavieux. : Near Shannon Limit Error-Correcting Coding: Turbo codes provided virtually error free communication at data date/power efficiencies beyond most experts though Turbo codes 30 years ago. i. Nonsystematic

ii. Nonrecursive combination of conv. Encoders Berrou et al at 1993iii. Recursive Systematiciv. Based on pseudo randomv. Works better for high rates or high level of noisevi. Return to zero sequences

Turbo Encoder

i. The k-bit block is encoded N times with different versions (order)ii. Pro the sequence remains RTZ is 1/2Nv. Randomness with 2 encoders; error pro of 10-5.Permutations are to fix dmin. Recursive Systematic Coder. Return to zero sequence. Non recursive encoder state goes to zero after v 0. RSC goes to zero with P= 1/2v if one wants to transform conv. into block code; it is automatically built in. Initial state i will repeat after encoding k .

Convolution Encoders

i. Turbo Decodingii. Turbo Decoding

For n probabilistic processors working together to estimate common symbols, all of them should agree on the symbols with the probabilities as a single decoder could do

The inputs to the decoders are the Log likelihood ratio (LLR) for the individual symbol d.

LLR value for the symbol d is defined ( Berrou) as Turbo Decoder.The SISO decoder reevaluates the LLR utilizing the local Y1 and Y2 redundancies to improve the confidence

Turbo Decoding

i. Ui : modulating bit {0,1}

ii. Yi : received bit, output of a correlator. Can take any value (soft).

iii. Turbo Decoder input is the log likelihood ratio

iv. R(ui) = log [ P(Yi|Ui=1)/(P(Yi|Ui=0)]

v. For BPSK, R(ui) =2 Yi/ (var)2vi. For each data bit, calculate the LLR given that a sequence of bit were sent Turbo Decoding

7. Explain convolutional encoder with an example.

(6) (A/M-10)

Convolutional Codes:Convolution codes are error detecting codes used to reliably transmit digital data over unreliable communication channel system to channel noise.

i. The information bits are spread along the sequence

ii. Data passes through convolutional codes in a continuous streamiii. Useful for low latency communications.Interleaving

We have assumed so far that bit errors are independent from one bit to the next

i. Both codes constructed over GF(256) (8-bits/symbol)

ii. Outer code is a (28,24) shortened RS code

iii. Inner code is a (32,28) extended RS code

iv. In between coders is a (28,4) cross-interleaver

v. Overall code rate is r = 0.75

vi. Most commercial CD players dont exploit full power of the error correction coder

vii. Inner code rate is , constraint length 7 convolutinal encoder

viii. Outer Code (255,223) RS code over GF(256) corrects any burst errors from convolutional codes

ix. Overall Code Rate is r= 0.437

x. A block interleaver held 2RS Code words

xi. Deep space channel is severely energy limited but not bandwidth limited EMBED Unknown

EMBED Unknown

EMBED Unknown

EMBED Unknown

EMBED Unknown

EMBED Unknown

_1461492119.vsd

_1461492392.vsd

_1461492467.vsd

_1461492589.vsd

_1461492439.vsd

_1461492264.vsd

Interleaving

We have assumed so far that bit errors are independent from one bit to the next In mobile radio, fading makes bursts of error likely. Interleaving is used to try to make these errors independent again

Error Control Coding , Brian D. Woerner , reproduced by: Erhan A. INCE

Depth Of Interleaving

1 1

6 2

11 3

16 4

21 5

31 7

26 6

5 29

10 30

30 34

35 35

Length

Order Bits Transmitted

Order Bits Received