Turbo Codes

Stéphan Fournier, B Eng

A thesis submitted

to the school of Graduate Studies

in the

Deparmient of Electrical and Cornputer Engineering

Royal Military College of Canada

Kingston, Ontario

In partial fdfiliment of the requirement for the degree

Master of Engineering

Monday, May 12, 1997

Copyright O 1996 by S. Fournier This thesis rnay be used within the Department of National Defence but copyright for open publication remaius the property of the author

National Library 191 of Canada Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibliographic Services services bibliographiques

395 Wellington Street 395, nie Wellington Ottawa ON KI A ON4 OttawaON K1AON4 Canada Canada

Y w r t5k Votm réference

Our I% Ntatre rdfdrence

The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive pexmettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distibute or sell reproduire, prêter, distribuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/fYm, de

reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni la thèse ni des extraits substantiels may be printed or othenvise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation.

Abstract

This thesis is about turbo block codes. The Turbo codes, first introduced

by C. Berrou, A. Glavieux, and P. Thitimajshirna in 1993 [Berrou] have been reported to

achieve near-capacity performance at low SNR. We analyze the general performance of

these codes with the help of an Input Redundancy Weight Enurnerator Function (IRWEF).

This function is a polynomial of the form: f( W. 3 = uydi, where the coefficient fj i. i

denotes the number of codewords with information weight i and parity weight j. We also

develop a MacWilliams type identity for the IRWEF. This identity allows the computation

of the IRWEF of a code fiom the RWEF of its dual and vice-versa. We also use the

IRWEF to elaborate a bound on the Bit Error Rate (BER) of block codes as well as Turbo

block codes.

In our study of Turbo block codes, we developed an IRWEF for the ensem-

ble average of al1 Fixed Turbo Codes and for al1 The-Varying Turbo Codes. These two

ensembles of Turbo codes differ in the conditions used to select their constituent codes.

Using these averaged IRWEF's, we c m bound the BER over the two ensembles. The

bounds found have been evaluated using a cornputer program and the results compared

with the ones over the ensemble of al1 syaematic block codes.

Aknowledgement

1 would like to thank my thesis adviser Dr G.E. Seguin for his guidance and his assistance throughout the course of this research. His extensive knowledge in the field of coding theory has been a solid support.

Special thanks also to Dr J. Lodge for suggesting the study of turbo codes. The time taken to open my eyes on this subject was reaily appreciated.

Un gros merci au Capt Hugues Latour pour ses conseils ainsi que son suppoa informatique et moral.

Se travail est dédié a ma femme Manon,

"In Galois fields fiil of flowers,

primitive elements dance for hours."

-S .B. Weinstein

Table of Contents

Table of contents

. * ...................................................................................................................................... 11

Abstract ....................................................................................................................... iii

Aknowledgement ....................................................................................................... iv

Vita ................................................................................................................................ vi

........................................................................................................... Table of contents vii

List of figures ................................................................................................................ xi

List of Tables ................................................................................................................ xiii

Chapter 1: Introduction .......................................................................................... 1

.................................................................................................. 1.1 Background 1

1.1.1 Thesis Objectives ............................................................................... 2

1 . 1.2 Synopsis ............................................................................................. 2

1.2 Error probability for a DMC ....................................................................... 3

1.2.1 The DMC .......................................................................................... 3

.............................. . 1.2.2 Analysis of error for the two-codewords code .... 4

1.2.2.1 Development for the AWGN chamel ..................... ... ....... 6

1.2.3 Extension to M-codewords ................................................................ 7

1.2.3.1 Development for the AWGN channel .................................. 7

1.3 Turbo codes .................................................................................................. 8

1.3.1 Histonc of Turbo codes ................................................................... 8

1.3.2 Description of a simple Turbo code ................................................ 8

1.4 Decoding of a turbo code ........................................................................... 10

1.4.1 The BAHL's A.P.P. decoding algorithm ........................................... 11

1.4.1.1 Example using a SPC code ................................................ 17

............................................... Chapter 2: Bounding the BER of a block code 21

vii

TURBO CODES

2.1 Bound for a block code ...................................... .... ............................ 21

............................... 2.1.1 General bound for the binary symmetric charme1 21

.... 2.1.2 Information Redundancy Weight Enumerator Function (IRWEF) 22

2.1.2.1 Relating BER and IRWEF ................................................ 23

................................... 2.1.2.2 Development for the AWGN channel 24

2.1.3 Example using the Hamming (7, 4) code ............................................ 25

2.2 Bound for a turbo coding scheme ................................................................. 26

....................................................................... 2.2.1 IRWEF of a turbo code 26

.................................................................. 2.2.2 Relating BER and IRWEF 29

........ 2.2.3 Example using two Hamming (7. 4) codes ............................ ...... 30

........................................... Chapter 3: Mac Williams Identiw for the IR WEF 35

3.1 Classical MacWilliams Identity .................................................................... 35

3.2 MacWilliams Identity for the IRWEF ......................................................... 36

3.3 Example of use of the MacWilliarns identity to compute the WWEF ......... 40

...................................................... Chapter 4: Turbo codes versus block codes 43

...................................................................................................... 4.1 Definition 43

4.1.1 Tirne-Varying Turbo Codes ............................................................ 44

4.1.2 Fixed Turbo Codes ....................................................................... 45

4.1.3 Systematic block codes ....................................................................... 45

4.2 IRWEF of the average of ail codes .............................................................. 45

4.2.1 Time-Varying Turbo Codes ............................................................... 45

4.2.1.1 Limitation in computing the IRWEF ..................................... 48

4.2.2 Fixed Turbo Codes ............................................................................. 49

4.2.2.1 Limitation in computing the IRWEF ..................................... 51

4.2.3 Systematic block codes ....................................................................... 51

............................................................................. 4.3 Cornparison of the BER'S 52

... V l l l

Table of Contents

chapier 5: ~onc~usions .......................................................................................... 61

5.1 Bound on the BER of Turbo Codes ........................................................... 62

5.2 Classification of Turbo Codes ................................................................. 62

5.3 Area of M e r study .................................................................................... 63

References ................................................................................................................... 65

Anna A: Matlab 's contputation of the averaged BER ............................ A - 1

A n n a B: Cprogrm for the BER of F W Turbo Codes and Systernatic . block codes ......................................................................................... B 1

. A n n a C: Matlab functions used ..................................................................... C 1

TURBO CODES

List of figures

List of figures

Figure 1.1 . Turbo encoder using two constituent codes ................................................. 9

Figure 1.2 . Simple Turbo code decoder ...................................................................... 1 1

Figure 1.3 . Viterbi's trellis decoding of a convolutioaal encoder .................................. 13

Figure 1.4 . Trellis for a (NL) block code ...................................................................... -14

Figure 1.5 . Trellis for the SPC code of equation (1.30) ................................................. 18

. ........................................*. Figure 2.1 BER for averaged versus specific permutations 33

............................................... . Figure 4.1 Generator matnx for the The-Varying case 44

........................................... Figure 4.2 . Generator matrix of a systematic block code 45

Figure 4.3 . Cornparison of codes with k = 4, 1 = 1 , r, = r, = 3 ..................... ....... 53

Figure 4.4 . Cornparison of codes with k = 4, 1 = 4, r, = r, = 3 .................................... 54

Figure 4.5 . Cornparison of codes with 1 = 2 and 8, k = 2, r, = r, =1 .............................. 55

............................................... . . Figure 4.6 Fixed vs Systematic; C= 2, k = 9, r, = r2 = 1 57

Figure 4.7 . 1= 3, r, = r, = 2, k takes the values 1, 3 and 5 ............................................. 58

Figure 4.8 . f= 2, r, = r2 = 1 , k takes the values 2, 6 and 9 .......................................... 59

Figure 4.9 . k = 4, r, = r, = 1, Ctakes the values 1, 3 and 5 ............................................. 60

List of Tables

List of Tables

Table 2 . 1 . Probabilities in the trellis of the SPC code .................................................... 18

Table 2.1 - Codeword of the Harnming (7, 4) code ......................................................... 26

................................................ Table 2.2 - Coefficients of the IRWEF of the turbo code 30

TURBO CODES

Chapter 1

Introduction

1.1 Backg round

Error correcting codes are now used in many communication systems. Tbey

add redundancy to the data in such a way that it is possible to correct some or al1 of the er-

rors introduced by the channel. The classical method used to classi@ codes is based on the

minimum Hamming distance of the code. The Hamming distance between two codewords

is the number of digits in which the two codewords differ. This minimum distance provides

a good indication of the word error correcting capability of the code. The Bit Error Rate

(BER) allows another way of measuring the quality of a code. Considering a design crite-

non for the codes that will minimize the bit enor rate leads to different codes and decoders.

Shannon's Channel Coding theorem gives a theoretical limit to the rate at

which information can be transrnitted reliably over a communication channel (capacity of

the chamel). It States that, for a given communication channel, and a given rate R less than

TLRBO CODES

the capacity of the channel, there exists a sequence of codes C,, n = 1,2, ... , al1 cf rate R

for which the corresponding probabilities of error P, tends to zero as n tends to infinity. The

proof of the channel coding theorem (see for example [Thomas]) utilizes a random coding

argument. The use of random coding suggests that we couid approach the capacity of the

channel if we could create a code with a weight distribution that approaches the distribution

obtained by random coding. Some codes which are designed for that purpose are called

random-like codes [Bat95]. For this class of codes to be efficient, a large block length is

required. With a large block length, we encounter the problem of complexity of the encod-

erfdecoder pair. To work around this problem, some form of concatenation of encoderd

decoders is used. One such technique uses a parallel concatenation of two or more convo-

lutional encoders and produces what is called a turbo code. In tbis thesis the narne turbo

code is extended to include codes which are parallet concatenations of linear block codes.

Thesis Objectives

The principal objective of this thesis is to provide a method to classi@ the

turbo codes without simulation. This is accompiished by bounding the BER of the turbo

codes according to their inûinsic characteristics. It is anticipated that with such a develop-

ment, we can find which parameters of the code affect its performance the most. The next

step will be to provide some idea of the performance of turbo codes as compared to classical

linear codes.

Synopsis

This work is divided into five chapters. In Chapter 1 we descnbe the concept

of turbo codes and provide background information on error correcting codes. The accent

is on the class of linear block codes.

In Chapter 2 we provide a bound on the BER by means of the Information

Redundancy Weight Enumerator Function (IRWEF). This function is first derived for a

classical linear code. We then elaborate an average IRWEF to descnbe the weight disni-

bution of a turbo code composed of two linear block codes. The average is over al1 possible

Cha~ter 1 :htroduct ion

interleavers that can be used between the two encoders.

In Chapter 3 a Mac Williams identity relating the IRWEF of a code to that

of its dual is developed. This identity allows for the computation of the IRWEF of a code

From the IRWEF of its duai and vice versa.

In Chapter 4 we develop bounds which can be used to compare the perfor-

mance of two difTerent ensembles of turbo codes and the ensemble of linear block codes.

Finally, in Chapter 5 we draw the conclusions of this research and present some recommen-

dations for fbrther study .

1.2 Error probability for a DMC

The DMC

A discrete memoryless channel @MC) consists of a finite input alphabet A,

a fmite output alphabet B, and a set of transition probabilities P[ y 1 x 1, x E a y E %, which

is the probability of receiving y given that x was transmitted. The memoryless property is:

for every 5 E aN, y E @ and for every positive integer N. If the output al-

phabet is continuous we replace the above conditional probabilities by conditional density

functions.

Two commonly encountered examples of a DMC appearing in the literature

are the binary symrnetric channel (BSC) and the Additive White Gaussian Channel

(AWGN).

TURBO CODES

For the BSC,

For the AWGN Channel, the noise is approximated by a random vanable

baving a zero-mean, gaussian density function:

We have the same input alphabet a= { 0 , l ) and the output alphabet II is the

set of real numbers and we have, assuming antipodal signalling where we transmit so(t) and

-s0(t) the conditional probability density function:

for y E 3, x E a E, is the energy per transmitted syrnbol and w2 is the

noise variance. The quantity E y / g is referred to as the signal to noise ratio per charnel

use. When we follow the AWGN by a hard

example Wozencraft & Jacobs [Wozencrafi]

nels.).

1.2.2 Analysis of error for the

quantizer, we fa11 back on the BSC (See for

for the details pertaining to these two chan-

two-codewords code.

For the sirnplest case of a two codeword binary code {y,, &) used over a

binary input Discrete Memoryless Channel @MC), a Maximum Likelihood Decoder

(MLD) will decode the received vector z into codeword y, if 2 belongs to 1, and into y, if

z belongs to 1,; where 1, and I2 are described as: -

and where N is the block length. Now, to evaluate the probability of error

over the DMC we can use 1, and 1, and write the following:

By definition of 1, and 12, we have, for z E 4:

The inequality (1 -9) can be applied to equation (1 -7) to yield the following

bound on the probability of error:

Now, this mm is independent of the transmitted codeword. For this reason,

we have the same bound for P[&/y,] and P[&] and we can compute a simple bound on the

total probability of error i.e. :

P[E] = P[&lyl] --Pi + P[~ly,] . Pz , so we have: - - -

where Pi is the probability that is transrnitted. Usiog the memoryless

property of the charnel we obtain:

T'ülU30 CODES

consequently, equation (1.13) becomes:

The notation w(ê) represents the Hamming weight of the k-tuple a (i.e. the

number of 1's in a. In the case of equation (1.15), the argument is the difference modulo

2 of two binary vecton and the operation yields the Hamming distance between y, and fi.

The Hamming distance between trvo k-tuples being the number of bits in which they differ.

1 2.2.1 Development for the AWGN channel

The Additive White Gaussian Noise channel (AWGN) is a communication

link where the noise distribution follows the probability density function of a zero-mean

Gaussian random variable with variance %/2. For that channel we bave that

P[E 1 y, J = P[E ( y l ] and therefore, the total probability of error is: - - P ~ E I = P [ ~ Y , I - = ~ I Y J - (1.16)

For this channel the bound (1 -1 5) is valid except that the quantity y is ob-

tained as:

where use of (1 -4) was made. However, in this case we can actually get an

exact expression for P[E] (see for example wozencraft]) which is given by:

Chapt er 1 htroduct ion

1.2.3 Extension to M-codewords

The bound (1.15) and the expression for the probability of error (1.18) de-

veloped so far can be extended to the more general case of a M-codeword code by using

the union bound. Let V = (gy,, ... ,x~-~) be the code used, then the probability of error is

the union of the probability of decoding into my codeword except the one that was trans-

mitted. Letting E, be the event that PJzJyJ > P,[zly,], the probability of error can be ex-

pressed as:

The inequality in equation (1.19) is the direct application of the union

bound. The last probability of equation (1.19) is over only two codewords and we can ap-

ply the results found in section 1.2.2 for the two-codeword code. Using equation (1.15) in

the above yields a bound on the probabiliv of error for an M-codeword code, i.e:

If the code V is a binary linear code, the bound (1.20) is the same for any

transrnitted codeword y, E V .

1.2.3.1 Development for the AWGN channel

If the code is used over the AWGN channel, the union bound argument yield

the foilowing:

So, aithough we had an exact expression for the probability of error in the

case of the two-codewords code, we have, in general to settle for an upper bound on the

M > 2 case. We also have the weaker bound:

TURBO CODES

We now have two bounds for the AWGN channel. The bound of equation

(1 -2 1) is tighter than the bound of equation (1 -22). Although the bound of equation (1.22)

is a little easier to cornpute, we will in general use the bound of equation (1 -21).

Turbo codes

1 .3.1 Historic of Turbo codes

In recent years, astonishingly good results have been achieved using large

codes that can be decoded using iterative decoding techniques [Berrou], [Lodge 11. So far

the most spectacular results have been obtained with the so called "Turbo" codes [Berrou],

which consist of a parallel concatenation scheme of two systematic convolutional encodes

with a pseudo-random interleaver between them. The excellent performance of these codes

can be partly credited to their randorn-like nature [Bat95]. Another factor that influences

these results is the soft-output decoding used inside the Turbo-decoding scheme. When us-

ing soft-output, we keep as much information about the received signal as possible. This

is doue by carrying the values of the probability associated with each transmitted symbols

instead of the estimated values across the decoding process.

A sofi-output decoder will generally be more complex than its traditional

hard quantized counterpart and this will limit its use to small codes. With the concept of

concatenated code proposed by Berrou, Glavieux and Thitimajshima perrou] it is possible

to use long codewords while using shorter intnnsic codes. Therefore it becomes possible

to combine the advantages of soft-output decoders for the "short" intrinsic codes while us-

ing rather long block length to make use of the characteristics of random-like long codes.

Description of a simple Turbo code

A turbo code typically uses more than one encoder. To maximize the ran-

Chapter 1:introduction

domness of the output of the turbo encoder, each of the parity (n-k)-tuple is obtained from

a specific pemutation of the message. In the simple case where only two encoders are

used, the turbo encoder could be represented as in Figure 1.1 : k information bits

X - ......

Permutation on k bits: P

*..... 1

XP Encoder 2 -

Figure 1 . I Turbo encoder using two constituent codes

Pursuing in this direction for illustration purposes, we will study the case of

two constituent systematic linear block codes. Encoders 1 and 2 have generator matrices

G, and G2 respectively, each of them in systematic f o m G = p:Q]. The overall output to

an input X - is - Y = (X , X Q, , X P Qz), where P is a kxk permutation mahix. Note that the

permutation is usually doue on more than one input vector. The overall system stays the

same but the permutation matrix is increased to work on many messages at one t h e . Once

the senes of Say I messages are permuted these are then encoded separately using the two

constituent encoders. Once the last of these messages has been processed, another set of 1

messages is permute and so on.

To decode, two iterations are required. We correct the information bits re-

ceived using the first set of parity bits and after pemuting the corrected information bits

(using the same permutation used by the encoder), we do another correction using the sec-

ond set of parity bits. This cm be extended to more than two encoderldecoder pairs. To

best use the charactenstics of this coding scheme, decoding can be done using separable

TURBO CODES

symbol-by-symbol Maximum A Posteriori decoding. In other words, we want to assign to

each symbol (bit) received a probability that a ' 1 ' (or a '0') was transmitted (soft-decision

decoding) instead of deciding irnmediately between the two possible outcornes (hard-deci-

sion decoding).

Once the first two iterations are completed we can iterate the overall pro-

cess to refine the probabilities m e r . This has to be done with care as the correlation be-

tween the infornation symbols and tbe redundancy symbol increase with each iteration.

1.4 Decoding of a turbo code

Using the simple turbo coding scheme of Figure 1.1 we can decode using

two decoders DEC 1 and DEC2 corresponding to the two constituent codes. While the sec-

ond decoder uses a11 the parity bits to estimate the message, DECl only uses the redundan-

cy bits of the first code. The reason for that is that DEC2 receives a version of the message

whose estimate was improved by DEC1. It is therefore logicai to use DEC 1 on the estimate

received codeword yielded by DECL. For this to be possible, we need to use a soft-decision

decoder for both DECl and DEC2. Also, while the estimated message given to DEC2 by

DECl is uncomlated with the second set of parity bits, the estimate given to DECl by

DEC2 is correlated to the first set of parity bits that DEC 1 used to correct the message. It

is shown in bodge 11 that the larger IN-kJ is, the less correlated are the values given by

DEC2 about the APP probability of the received information and redundancy symbols.

We know tbat for this scherne to work, we need a soft-decision decoder. It

is clear that a soft-decision demodulator would be a good improvement. The problem now

is to find a decoding algorithm that will yield a sofi-decision on every symbol. Clearly the

traditional decoding method would not work. It is shown in [Lodge 11 that a modified

BAHL algorithm will yield the symbol-by-symbol sofi-output that we need. Before goiog

into the details of how this algorithm works, we look at the general shape of the decoder

(Figure 1.2) that we have discussed so far .

Chapter 1:uiuoduction

Figure 1.2 Simple T h o code decoder

For simplicity, the diagram of Figure 1.2 does not show the demodulator.

Also, the feedback mechanism going into DECl could be going into a copy of DEC 1 so the

whole process could be done serially. This would be more viable since the decoding would

only introduce a delay on the first codeword. The input vectors (in brackets) represent the

soft-decisions made by the dernodulator on the received bits belonging to X, 1, and & of

Figure 1.1. The only thing stopping us from iterating the process by using another DEC2

on the outputs of the second DECl is the correlation between the values representing the

information symbols and those representing the redundancy symbols. To minimize the cor-

relation between successive bits of a codeword, we can do the permutation on more than k

bits. We use l*k bits for the permutation. If l is big enough, there is only negligible corre-

lation between the two sets of redundancy bits. In this case, we cm have Z parallel DEC l y s

whose outputs are regrouped for the permutation and then separated again for the second

decoding using I parallel DEC2's.

1.4.1 The BAH Cs A.P. P. decoding algorithm

The algorithm proposed by BAHL [Bahl] yields a soft-decision on each re-

ceived symbol by evaluating the probabilities of state transitions through a trellis. In order

TUEU30 CODES

to be able to follow this somewhat complex algorithm, it is important to have a good un-

derstmding of the process of trellis decoding. To illustrate how a trellis is constmcted and

used to decode, we look at VITERBI's algorithm. This algorithm is the rnost commonly

used to decode convolutional codes (it can also be used for linear block code). The trellis

is fomed by using the state transition diagram of the encoder. The states are identiQ by

al1 the different possible memory contents of the encoder. The longer the memory is in the

encoder, the more states need to be considered. As an input bit gets into the encoder, the

state will change and an output depending on the input and the previous state will be pro-

duced. This output cm be of more than one bit.

To construct a trellis describing the encoder, we start fiom the state O which

represents the all-zero vector in the encoder memory. We then have two branches corning

out of that state, one for each possible input (O or 1). Each branch goes to a different state.

These new states are determined by the state diagram of the encoder. We repeat this pro-

cess for each state that we add to the trellis. Each new set of transitions represents a change

of state of the encoder and therefore is occuming at different times. They can be conve-

niently represented by spacing them horizontally in the trellis, while the different possible

aates at a given time are spaced vertically. When using a finite trellis to decode, we need

to end the finite message by enough zeros to r e m the encoder in the ail-zero state. For

this reason, the trellis for a convolutional code is as long as the message plus the number

of memory elernents contained in the encoder. To illustrate, we will consider a typical ex-

ample where the encoder has 2 position register memory and the message is composed of

three bits. We will therefore have 5 stages (time) in our trellis fiom the initial to the final

state which are the all-zero state. In the trellis of Figure 1.3, the boxes represent states while

the branches identiQ the input causing this branch as well as the corresponding output of

the encoder (input/output). In the encoder, the boxes represent the content of a 2 bits shift-

register (being in state 10 means that the 2 bits shift-register contains the past and second

past values of the input "1" and "O"):

Such a trellis can be used to encode as well as decode. For example if the

input is 10 1, then we cm see that the first bit to enter the encoder at tirne O will be 1.

Chapter 1 :Introduction

Convolutional Encoder

Trellis time O thne 1 time 2 time 3 time 4 time 5

Figure 1.3 PîterbiS treiIis decoding of a convoiutionai encoder

We will leave state 00 at t h e O to go to state 10 at time 1 creating an output Y,Y2 = 1 1.

Next, a O would be fed into the encoder which would yield a transition to state 0 1 at tirne 2

creating the output Y ,Y2 = 0 1 and so on. M e r the end of the message, two zeros would be

forced into the encoder to retum it to state 00. The transmitted codeword is then:

( 1 1 O 0 0 0 1 1 1 . To decode, we compare the output of each branch with the correspond-

ing received two bits. We note the Hamming distance between those two (number of errors

in the received codeword given this path was used). Once every branch has a weight, we

elirniaate the unlikely branches in the following manner: starting from the lefi, we associ-

ate to eacb state a metric which is the sum of the weight of the branch reaching this state

and the metric of the previous state From which this branch cornes from. If more than one

branch connects to a state, only the one with the lowest sum is kept. Once we have done

this exercise we trace back the path in the trellis From the final state toward the initial state

following the surviving path of lowest weight. This path yields an estirnate of the transmit-

TURBO CODES

ted message. The above example was based on a convolutional code, but it is easy to ex-

tend the trellis decoding to block code.

BAHL's algorithm decodes in a different way than VITERB13s, but it uses

the same trellis smicnire. This algorithm rninimizes the bit error probability and yields the

A Posteriori Probability (APP) for each decoded bit. To illustrate BAHL's algorithm, we

use a (N,k) block code. To stay in general t e m , variables are used to identiQ the difierent

states. Figure 1.4 gives a partial view of a typical trellis. The focus of the following dis-

cussion is a transition that occurs between time t-1 and t.

time O time 1 time t- 1 t h e t tirne N- 1 time N

Every node in the trellis is uniquely identified by a time index (subscript)

and a state (bracket). The maximum number of states at a given time is directly related to

the parity check matrix 0 of the block code used. The elements of the space generated

by the columns of the parity check rnatrix each represent a distinct possible state. Hence,

the state space is r-dimentional where r = N - k is the number of elements of a c o l w of

H. The maximum size of the space generated by the columns of H is therefore 2r. The trel-

Lis starts and ends in state O. The trellis then grows until al1 the possible states are accessible

by a branch. In BAHL's algorithm, we are interested in transition probabilities. A measure

Chapter I :Introduction

of the probability of the transition between state m' at time t-1 and state rn at time t is de-

veloped. This probability is used to give a APP on the bit transmitted at time t- 1. Finally,

we note that the trellis is over a time period of N bits. This is because a codeword is corn-

posed of N bits so we know that the end of a codeword is reach at that t h e .

Between time t-1 and t h e t, there are a maximum of 22 different transi-

tions. Half of those are the result of an input bit equal to 1 and the other half are the result

of an input bit equal to O. Some of the transitions may have zero-probability (in the begin-

ning and at the end of the trellis). The probability that a 1 (or 0) was transmitted at t h e t-

1 is the sum of the probabilities of al1 the transitions between t-1 and t that require a 1 (or

O). We rnust now detemine the probability of every transition between rimes t- 1 and t. To

do so, the following probabilities are defined [Bahl]:

a,(m) = forward probability of reaching state m at time t, it involved al1

the possible paths ftom the initial state to state m;

PXm) = backward probability of reaching state m at time t, it involved

al1 the paths fkom state m at rime t to the final state;

&(m) = probability of reaching state m at time t, &(m) = a,(@ B,(m);

y,(m,m') = probability of the transition between state rn' at time t-1 and

state m at time t. y,(-') = P[(S, = m),Y,I(S,-, = m')];

q(m,m') = refined probability of the transition between state m' at time t-1

and state m at time t.

Al1 those probabilities are conditional to the received codeword. The for-

ward and backward probability are obtained using recursive fonnulas starting with some

known initial conditions. The initial conditions are readily estimated by the nature of trel-

lis-codes. We know that we always start and finish in the state O. That gives us a condition

TURBO CODES

on a and p at those two specific tirne. At time 0, the fonvard probability of having state O

is one and the probability of al1 other states is zero. The same way, we Say that the back-

ward probability of being in state O at time N is 1:

%(O) = 1, ao(rn) = O for m # 0, h ( 0 ) = 1, h(m) = O for m + 0,

With those initial conditions we use the following fomulas to calculate re-

cursively al1 the fonvard and backward probabilities of al1 states of the trellis. Note that the

proof of these formulas is not given here but is readily obtainable Rom [Bahl].

P t @ ) = Pt + 1 (m')yI+, (m, mg), and m'

( 1 -26)

The transition probability (y) used in equation (1.25) and (1.26) is calculated

using the characteristics of the channel and the received bit corresponding to that transition.

Between tirne t-1 and time t, one bit is received (q,). The probability y,(m,m9) is channel

dependent. We will consider a Binary Symmetric Channel (BSC) with cross-over proba-

bility P[O[ 11 = P[110] = E, and a soft output demodulator yielding a probability that a 1 was

received. The trellis tells us what the transmitted Y, should have beeo if we are on that

branch while the demodulator gives us a probability that 9, was a one. For exarnple, if Y,

should be O, we know that two things can occur: either there was no transmission error and

we received a O (probability yield by the demodulator is srnalier then 1/2), or there was a

transmission error and we received a 1 with probability larger than 112. The transition

probability is then computed by summing the probability of those two events:

In equation (1.27) and (1.28), we have neglected the intrinsic probability of

the transmitted bit to be a 1 (or a O). This is not cntical here, since gor a good linear block

code this probability is usually 1/2, and it could be factorized out of the two equations. As

we will see later, this 112 would be carried as a factor al1 the way to the final probability

which has to be nomalized by an arbitrary factor anyway. Therefore, by dropping the 1/2

at this point we simply change the normalizing factor without changing any of the proba-

bilities.

Combining the above three probabilities will yield a refined transition prob-

ability for every transition of the trellis:

The soft-decision on the received bits are then computed by summing a11 the

refined transition probabilities corresponding to a specific time and transmitted bit. We c m

easily normalize the resulting probabilities so that P[Y, = O] + Pv, = 11 = 1 by multiplying

equation (1 -29) by an appropriate constant. This constant can be found with the first set of

probabilities and used to compute al1 the others.

1.4.1.1 Example using a SPC code

Using the very simple Single Parity Check (SPC) code having the following

generator and parity-check matnx:

G = H = [l I l ] ,

G is chosen so the code is systematic. in the case of a linear block code, the

trellis is consmicted in the following way: fi-om a given state Sb1(m'), we add to it the prod-

uct of the corresponding input bit and the comsponding column of H.

In equation (1.3 1) h,, is the (t-1)th column of H and the columns of H are

numbered from O to N-1 in the general case of a block code @ere N = 3). Note that S,,(m')

and h,, have the same length. Also, X,,h,, is not a normal vector product since X,, is only

TURBO CODES

a scalar (O or 1) and h,, is usually a vector. The product is to be interpreted as the addition

of h,, if XI., = 1 and no change of state if X,, = O. In the particular case of the SPC code,

the columns of H are one bit long and therefore equation (1.3 1) c m be used directly. After

we have used al1 the input bits, we r e m to the dl-zero state by changing one bit of the

state at the t h e starting by the most significant one. This is done by adding to a state the

corresponding column of H if the bit to be put to zero is one and by not changing state if it

is not the case.

hthecodeof(1.30)thecolumnofHare:ho= 1,h, = 1.andh2= 1. Wecan

now coastruct the trellis for this SPC code using the same conventions to identiw the tran-

sition and the states as were used in the first Viterbi's example of Figure 1.3. Here. the

states are represented by only one bit because only two states are necessary for this very

simple code.

Figure 1.5 Mlis for the SPC code of equation (1.30)

Now, let's transmit the message X = 10. The codeword is y = 10 1. The de-

modulator gives as its output the probability that a 1 was received. Given that the decoder

received the following vector [0.95 0.6 0.91, and that the channel is a BSC with cross-over

probability E = 0.1, we would compute the probabilities in the trellis as follows:

State

O

O

1

1

Table 1.1 Probabilities in the trellis of the SPC code

t = O

a= 1

Pz.463

a=O

p=O

transition

y,-,=. 14

y0-,=.86

y,,= O

y1. l=O

t = 1

a=. 14

P=.5512

a46

B=.4488

transition

y@,=.42

y,,=.S8

y , p . 5 8

yl-l=.42

t = 2

a=.5576

Pz.18

a4424

P=.82

transition

y,,=. 1 8

YO- [=O

y1.*=.82

yl-l=O

t = 3

a=.463

P=l

a=O

p=O

Chapt er 1 : htroduct ion

Using equation (1.29) we compute the probabilities on tbe received symbol

using a normalization factor 5:

We get the following relations between those probabilities:

P[uo= 11 > PIYo= O],

P[u,= O] > P[Y,=l], and

Ply,= 11 > P[Y2= O].

We could estimate the transmitted codeword to be 10 1. Note tbat if a hard-

decision demoduiator/decoder pair would have been used, an erroneous codeword would

have been decoded. We would have known that there was an error, but no more then that.

With the soft-decision algorithm, we have decoded into the correct transmitted codeword.

Since the code is systematic, we estimate the message as the first two bits of the estimated

codeword: x = 10. -

TURBO CODES

Chapter 2

Bounding the BER of a block code

In this chapter, a bound on the BER of a block code is developed using the

information Redundancy Weight Enumerator Function (IRWEF) of the code. First the case

of a general block code is studied and the results are then extended to a parailel concatena-

tion of block codes in a turbo coding scheme.

2.1 Bound for a block code

2.1.1 General bound for the binary symmetric channel

Consider the situation where an (N,k) linear block code is used to commu-

nicate one of 2k equally likely messages over a Discrete Memoryless Channel (DMC) with

a Maximum Likelihood Decoder (MLD) at the receiver. The BER can be estimated by as-

suming that the all-zero codeword was traosmitted and that x - represents the estimate of

the information k-tuple. Under these conditions, the number of bits in error is the weight

TURBO CODES

of the estimate x - . Therefore, the BER is the expected value of this weight divided by the

number of information bits sent:

The quantity P[W(X) - = il can be bounded using a standard union bound

argument. We are using the code over a DMC, decoding with a MLD and the probability

that the estimate will have a weight different than zero is equivalent to the probability of

error given that the all-zero codeword was transmitted. Therefore, we can use the result of

equation (1 -20) to bound the probability on the weight of the estirnated value. In this case

the exponent in equation (1.20) becomes w(a) since yi = Q (because the all-zeros codeword

was transmitted). We then have that:

In equation (2.2), the weight enwnerator cp, represent the number of code-

words of information weight i and coded weight m. This function can be computed for spe-

cific i and m by using the generator rnatrix of the code. Using the above bound in equation

(2.1) gives us the foliowing bound on the BER of a linear block code:

It will tum out to be more useful, when dealing with turbo codes, to intro-

duce a different (but equivalent) weight enurnerator function cailed the Information Redun-

dancy Weight Enumerator Function @(WEF).

2.1.2 Information Redundancy Weight Enumerator Function (IRWEF)

The IRWEF of a code is a polynomial in two variables whose coefficients

give the weight distribution of the code:

Chapter 2:Bounding the BER of a block code

In equation (2.4), fj is the number of codewords of information weight "i"

and parity weight "j". Let y be a codeword, y = xG, where G is the generator matrîx of a

linear block code and is the information k-tuple. Then, the information weight of y is

w@) and its redundancy weight is w(yJ - wm. Clearly, the number of codewords of coded

weight "m" is:

M i l e B, is fixed for any (Nk) block code, f, depends on the particular en-

coder used to generate the code. The particular relationsbip between the coefficients f,

and <pi, defined in section 2.1.1, is readily seen to be:

1.2.1 Relating BER and IRWEF

Replacing cpi, by f,, in equation (2.3) yields the following:

Sometimes it is preferable to work with polynomials. Using (2.4), it is pos-

sible to find an expression very similar to equation (2.7) by differentiating f(W,Z) and by

replacing W and Z by y:

If W = Z = y , then equation (2.7) and (2.8) differ only by the factor W and

a normalisation factor l/k. Although in equation (2.7) the index "i" runs from zero to k, the

TüRBO CODES

term inside the sum will always be zero when "i" equals zero. Because of this, the double

sum of equation (2.7) and (2.8) nuis over the same range. The bound on the BER using

polynomial notation can be wrîtten as:

1 a BER < - (W - r n f (W, Z))/

W = 2 = 7

Equation (2.9) above is identical to the bound on the BER of a convolutional

code as given in equation (4.4.13) of Viterbi and Omura [Viterbi]. The latter can be inter-

preted as a block code with infinite length and is compatible with (2.9). The form of equa-

tion (2.9) is not practical to use. To simpliq the calculations, we rearrange the sum in

equation (2.8) by setting W = Z = H and we have:

Note that even if W = 2, we still retain al1 the weight infornation in the coefficient fij. Also, m nins from 1 to n since m = i + j and i = O yields a nu11 tem. We now defme another parameter to break down the computation into srnall manageable sums:

We define Dm as: 1 Dm = i; - i fi, , and we now have (2.11)

i + j t m

2.1.2.2 Development for the AWGN channel

In Chapter 1, the value of y for this type of channel was given. Since we are

now cornputing the BER, we are only interested in the information symbols. The energy

per information bit is given by: EN = REb (where R is the rate of the code R = k/N, and Eb

is the energy per bit). Using this in equation (2.12) we obtain, for the AWGN with antip-

odal signaling:

Chapter 2:Bounding the BER of a block code

Like we have done in Chapter 1, it is possible to obtain a tighter bound by

using the explicit formula for the probability of emor applied to the AWGN charnel. It fol-

lows that:

Again, note that (2.14) reduces to (2.13) when we use the standard bound

Q(x) < ëX2". In doing cornputer calculations, we used the cornplernentaty error function

(erfc) since moa of the programming languages have it as an imbedded function. Using

the following relation between the Q function and the erfk we rewrite the bound on the

BER:

2.1.3 Example using the Hamming (7,4) code

The code in this example is the Harnming (7,4) single error correcting code.

As a cyclic code, it is generated by g(x) = 1 + x + x3. We calculate the IRWEF with respect

Chapter 2:Bounding the BER of a block code

In the simple case described in section 1.3, the information bits X are trans-

rnitted over the channel with two sets of par@ bits. The first set is provided by X QI while

the other one is given by X P 42 where P is a k x k permutation matnx. The two constituent

codes are (n,,k) and (nz,k) linear block codes. The scheme generates a total of

(n, - k) + (n2 - k) redundancy bits. The IRWEF of such a code will have the following

fonn:

Here, the fij". "(P) coefficient represents the number of codewords of infor-

mation weight "i" and redundancy weight "j". This quantity depends on the actual pemu-

tation used by the coding scheme as well as on the two constituent codes. This is illustrated

by the argument P for a particular permutation and by the upper index (1,2) denoting the

two constituent codes. These coefficients can be expressed as follows:

The function 6 is the dirac function (takes the value one if its argument is

zero and zero othenvise). In equation (2.20) the argument of the delta function will be zero

if and only if the total redundancy weight is j. Note that if this notation was used for the

IRWEF of section 2.1.2 we would have:

The coefficient of equation (2.20) will in general be hard to evaluate. Fol-

lowing Benedetto and Montorsi [Ben95], we instead average them over al1 permutations to

where #(P) is the total number of possible permutations on k bits. A line

TURBO CODES

above the coefficient is used to identify it as an average. We dropped the argument P since

this averaged coefficient no longer depends on any particular permutation. We now know

how to compute the IRWEF of the constituent codes and will r eamge equation (2.22) to

separate the redundancy weight of each constituent code. First we divide the surn over a11

information vectors of weight i into two sums keeping in the last sum only the codeword of

a specific redundancy weight coming from the first constituent code. To do so we need to

surn over al1 the possible redundancy weights of this code. Doing this, we get:

Since the permutation appears only in the last sum, we change the order of

summation so that the sum over all permutations become as simple as possible:

lüe summation over al1 permutations cm be simplified by noting that as we

the permutations P, the k-tuple @ will assume al1 the k-tuples of weight

w(xJ = i. Moreover, it will assume the value of each k-tuple of weight i exactly

#(P)/(!) thes . Hence (2.24) reduces to:

ii, -k

The last sum in the above expression is the explicit form of the coefficient

of the IRWEF (see equation (2.21)) of the second constituent code ci),,. The middle sum

can be replaced by the coefficient of the first constituent code since for a fixed pair (i, I )

there are c;' k-tuples that will satisfy the two conditions of the sum while the result of the

second sum depends only on i and is invariant with respect to - Therefore, the coefficients

Chapter 2: Bounding the BER of a block code

of the averaged IRWEF of the turbo code cm be expressed as:

The above expression gives us an easy way to compute the averaged IR-

WEF for the turbo coding scheme using the M F ' s of the two constituent codes. Using

the same ugument as in section 2.1.2.1 it is now possible to derive a bound on the BER of

a turbo code averaged over al1 permutations. This bound should give us an idea of the av-

erage performance of the turbo coding scheme.

2.2.2 Relating BER and IRWEF

The above expression for the RWEF averaged over all possible permuta-

tions on k bits will provide a bound on the averaged BER for the concatenation of two

codes. This is a direct generalization of the bound found in section 2.1.2.1. We get the av-

eraged Dm's by substituting the coefficient of equation (2.26) into equation (2.1 1):

Using equation (2.27), we get a bound on the average BER:

This result can be explicitly evaluated for the AWGN as in section 2.1.2.2.

The only problem here is the minimum distance of the code. This quantity will Vary for

different pemutations and therefore we need to define 4, for the average over al1 permu-

tations. Since L,(P) is the minimum distance of the code generated by [ Ik : Q1 : P Q2 1, the average over al1 permutations cannot have a minimum distance smaller then the small-

est of the L,(P). By using the m i o h of these Ln for our average code, we will have

TURBO CODES

a valid bound. Doing that and following the same denvation as was done in section 2.1.2.2

will yield the following bound:

This bound is for the AWGN channel with antipodal signaling.

Example using two Hamming (7,4) codes

Using a parallel concatenation of two identical encoders for the Hamming

(7,4) code, the turbo code will produce 6 redundancy bits for every 4 information bits.

Therefore, the rate of the turbo code is R = 215. For such a simple setup. it is easy to corn-

pute the coefficients of the averaged RWEF using equation (2.26). The two constituent

codes are identical and they both have the IRWEF given in equation (2.16). To emphasize

the fact that we use two identical constituent codes, we use the index (1.1) in the represen-

tation of the IRWEF. Using these in equation (2.26) will give the following averaged

rREWF:

Table 2.2 Coefficients of the IRWF of the turbo code

Using the above table (or the equation) the only non-zero coefficients 6,

Chapter 2:Bounding the BER of a bIock code

obtained from equation (2.27) are:

Since both constituent codes are identical, the minimum distance is the Cn of the Hamming (7,4) which is 3. Now, setting S N R = E,,/n(o it is possible to obtain a bound

on the averaged BER using equation (2.28):

With this simple code, it is possible to compute the RWEF for al1 specific

permutations. Actually, it will tum out that there are only two distinct IRWEF for al1 24

permutations. For the trivial permutation (no permutation), the IRWEF is easily found us-

ing the technique outlined in section 2.1.3. Note that permuting the information bits is

equivalent to pemuting the rows of the generator matrix. With the generator matrix used

in that section, al1 the permutations of the first, second and fourth row yield the sarne IR-

WEF as long as the third row is not rnoved. This is due to the fact that those three rows of

the Q rnatrix are al1 the 3-tuples of weight 2. If we take al1 the Iinear combinations of those

three rows, it is clear that the result does not depend on the order in whicb the rows appear.

Since the third row is the all-ones vector, al1 it does is change the resulting k-tuple into its

"conjugate". Finally, for any position chosen for the third row there are 3! permutations

possible for the other three rows. Knowing that, for a fixed third row, the [RWEF will not

change when we permute the other rows, there are only three other possible IRWEF to com-

pute. Again, using the technique of section 2.1.3, we find that the three other positions for

the third row yields just one new IRWEF.

We have the following IRWEF:

TURBO CODES

To compare the results obtained using those LRWEF's with the averaged IR-

WEF, we can average the two functions of equation (2.32) and (2.33) according to the num-

ber of permutations that produce eacb of them, i.e.:

The lRWEF of equation (2.34) is the same as the one shown in equation

(2.30). Using equation (2.32) and (2.33) we can compute the bound on the BER for those

two specific permutations and compare them with the averaged BER of equation (2.3 1 ) (see

Figure 2.1 ).

For this example, the BER found from the IRWEF averaged over al1 permu-

tations gives a good indication of the tendency of the different codes created from specific

permutations. Also given in Figure 2.1 is the bound on the BER for the constituent code.

This latter shows a better performance than the concatenated scheme. It is not always the

case, and for example (see [Ben95]) using a BCH (63,57,5) a gain of 1 dB is obtained by

using a concatenated scheme. This seems to indicate that the nurnber of bits permuted in-

fluences greatly the performance of the turbo coding scheme.

To get an idea of the general accuracy of our bound on the BER, we have

included on the graph the explicite BER obtained without coding. This expression is only

a function of the Signal to Noise Ratio:

This cuve is represented by the "x"s on the graph. We see that for a S N R

smaller than 4 dB, we are better off without coding. This is a well known fact that for low

signal to noise ratio, the MLD fails and the decoder which decodes into the nearest code-

word actually introduces errors. On the other hand, for moderate to high S N R we see that

Chapter 2:Bounding the BER of a block codz

Figure 2.1 BER for averaged versw specifc permutations

Our bound gives a better BER than the standard c w e obtained without coding which con-

firrns that our calculations are valid.

l'URI30 CODES

Chapter 3

MacWilliams Identity for the IRWEF

The MacWilliams identity WacWilliams] is a relation between the weight

enmerator iünctions of a linear code and of its dual code. The dual of a code is its nul1

space, i.e. the collection of al1 N-tuples which are orthogonal to al1 the codewords. For a

(N, k) linear code, its dual is a (N, N-k) Iinear code [Costello] (section 3.6). This identity

gives us more flexibility in computing the weigbt distribution of a linear code whenever k

is large. Since our evaluation of the BER of a turbo code is based on the IRWEF, it might

prove usehl to develop a similar identity between the IRWEF's of a code and its dual.

3.1 Classical MacWilliams ldentity

MacWilliams identity for bioary linear codes runs as follows: "let V be a bi- N

nary m,k] Iinear code with weight enumarator C A~Z ' and V' its dual with weight enu- S i = O

merator C B,z', then i = O

TURBO CODES

This identity allows us to compute the Bi's in terms of the Ai's (or the re-

verse by interchanging the roles of V and V-' since vu = V). This is useful when one of

the codes is much srnalier then the other, Le. when k « N - k or N - k « k, since then one

of the weight enurnerators rnay be obtained by brute force enurneration of al1 codewords.

3.2 MacWilliams ldentity for the IRWEF

For a linear code V with generator maaix G, the IRWEF can be written as:

Now if H is the generator matrix of the dual code V-' we also have:

In the special case when both G and H are systematic we have the following

result:

We now proceed to prove this theorem. The proof follows along the same

lines as one for the classical MacWiIliams identity. We shall first need a lemma which we

now establish after introducing a few necessary concepts. Define vJ = (-I)'sx"> where

< . , . > denotes the usual dot product. Let V be an (N, k) binary linear code and v - ~ its dual,

the proof of the following identity may be found in [MacWilliams]:

/ 2k;if y ~ ~ i

Chapter 3:MacWiîiiams Identity for the RWEF

We can now proceed with the lemma.

Lemma: Let a E F:, b E F Y - ~ , then - k-w(o) Y-k-w(b) w(b

w""Z"'"[(~, - - b), (x, - - y)]=(l + W) (1 - ~ j ( ( 1 + Z) (1 -2) . (3.6) x E F: y E ~ z - '

where (a, k ~ ) is the concatenation of a and b.

By definition of the operator [,] we can write the following statement:

To simpliQ the notation, we will just use 5 and y as index for the sum but

we Unplicitiy sum over aii vectors in FZk and FZsmk respectively. The first step is to expand

the exponent on the right hand side of (3.7):

It is now possible to regroup al1 the terms that are ooly dependent on y and

to separate the two summations:

In the next two steps, we replace the sum in the exponents by product s u

of the arguments and we change the order of summation to have the product surn before the

sum. By doing this, we sum over al1 possible values of a component xi (or yj) for a given

index i (or j) of the product sum:

TURBO CODES

Since xi,yj,ai and bj can only be either O or 1, the two sumations above cm

be reduced to the simple fom:

therefore:

and the lemma is proved..

We now prove MacWilliams identity for the IRWEF. Let V be the code

generated by G = [I, : Q] and vL its dual generated by H = [Q' : 1,-J. Set:

We will evaluate A in two different ways by changing the order of the sum-

mations, thereby obtaining the required identity.

First, we observe that [x , y + ZJ = [x , y&, zJ. Tae lemma can be applied

to A and yields:

Chapter 3:MacWiUiarns Identity for the IRWEF

If g is fixed, then as runs over FZk, (Z + i) will assume every k-tuple ex-

actiy once. For every - we have the same sum over z E FZk. Since there are 2k jf we ob-

tain:

The SM in the above equation is the IRWEF of the code with z as the mes-

sage vector and where the dummy variable representing the weight of the information is

(1-W)/(l+W) and the dummy variable for the redundancy weight is (1-Z)/(l+Z). We can

therefore rewn-te equation (3.17) as:

Now we reevaluate A with the order of the summations in equation (3.15)

interchanged.

A = CC W * ' " Z " " ' ~ ~ [(x, y), (5 q Q ) + (g, slQ)] y

(3.19) f *

CC[(xl - - y), (2. - - zQ) + (2: - z'Q)] - = CK?, y). (2, ~ Q ) I Z i ( ~ . y). ( 5 5'411 (3.21)

Using equation (3.5) on the summation of equation (3.22) gives us the fol-

lowing:

The summations in the above equation are the IRWEF of the dual

lUiZBO CODES

code VI with the diimmy variables W and Z inversed fiom equation (3.3),

so we have:

A = (2k) 'fH(~y W) (3 .24)

Finally, companng the two expressions for b of equations (3.18) and (3 X),

we get:

By inversing the role of Z and W in equation (3.25) we get the following:

which is the same as equation (3.4) and therefore proves theorem 2.

Example of use of the MacWilliams identity to corn- pute the IRWEF

Let us consider a systematic code of length N = 5 with k = 2 information

bits. With an arbitras, generator matnx, we cm find the corresponding parity check matrix

and using the same method as in section 2.1.3 we get the two IRWEF of the code and its

Now using equation (3.4) on f,(W,Z) we get:

Chapter 3:MacWiUiams Identity for the IRWEF

Expandiog the above (for example using Maple) yields the same IRWEF as

equation (3.28) which confirms theorem 2 and concludes this example..

TüRBO CODES

Chapter 4

n r b o codes versus block codes

It is clear that the ensemble of turbo codes is a subset of the ensemble of al1

linear block codes. In this chapter, we develop random coding type bounds to be used in

cornparing the average performance of turbo codes with the averaged performance of al1

Iinear block codes.

Def inition

In Chapter 2, the simple turbo code descnbed was using a petmutation on k

bits. In practice, we want to permute a larger set of information bits so that we can achieve

a good degree of randomness at the output of the encoder. To do this without enlarging the

constituent codes, we can permute k bits instead of sîmply k. Looking at the systernatic

generator matrix of a linear (n, k) block code we see that it contains IWO submaaices. The

first one is the identity kxk matrix and the other one, usually denoted as the kx(n - k) matrix

Q, generates the redundancy bits. In the case of a turbo code with two constituent codes,

TURBO CODES

there are two such submatrices denoted Q and R that create redundancy bits. The first of

these two submatices works on the infomiation bits directly while the second works on a

permutated version of the information bits.

To work with 1.k bits, the two redundancy submatrices are diagonal matri-

ces with their non-zero elements being the submatrix Q of a (nJc) linear block code. This

gives the option of using the same submatrix in al1 the diagonal elements of a particular sub-

matrix or to use different codes for each of the 1 messages being processed. Depending on

this choice, we cm define two ensembles of turbo codes which wiIl be denoted as Fixed

Turbo Codes and The-Varying Turbo Codes respectively.

4.1.1 Time-Varying Turbo Codes

Consider the ensemble of codes \; ith generator matrix of the form:

Figure 4.1 Generator maair for the Erne- m i n g case

Setting r, = n, - k and r, = n, - k (the amount of redundancy generated by

the encoder 1 and 2 respectively), we have that the Qu's are k x r, matrices, the b ' s are

k x r, matices, P is a k x k permutation matrix, 1 is the fk x ik identity matrix and the 0's

are k x q zero-matrices ( i = 1 in the Qu's submatnx and i = 2 in the Ru's submatrix). If the

matrices

their resoective maces. then the resulting ensemble of codes constitutes the ensemble of

The-var vin^ Turbo Codes.

Chapter 4:Turbo codes versus block codes

4.1.2 Fixed Turbo Codes

The Fixed Turbo Codes are derived from the Tirne-V@ng Turbo Codes bv

choosine the matrices such that: 0, = 0, = ... = Q = O and R = R, = ... = RI-, = R where

O. R and P are chosen inde~endentlv of each other, Le. we use only two constituent codes.

4.1.3 Systematic block codes

A systematic block code has a generator matrix of the form:

Figure 4.2 Generator matrik of a systematic block code

The first matrix is the general case. Ln the second matrix, the added indexes

gives us compatibility of parameters with the two turbo coding schemes. We define the en-

semble of Systematic block codes as the set of al1 codes with the O subrnatrix chosen

eauiorobably from the s~ace of al1 lkxl(r,+r,) - - matrices.

4.2 IRWEF of the average of all codes

In order to study the average behavior of the different ensembles of codes

defined in section 4.1, we now want to find an expression for the IRWEF of an average code

representing each ensemble. To do so, we will average the IRWEF over al1 possible per-

mutations as well as over al1 the possible redundancy subrnatrices.

4.2.1 Time-Varying Turbo Codes

In order to find the average IRWEF of the Time-Varying Turbo Codes, we

first average over al1 permutation matrices P. The IRWEF of a specific code has the fom:

lk

W 0 CODES

where is composed of I information vectors of length k:

x = [ h, s,, ..., 5-,]). To simpliQ the notation, we define [QI and [RI as the two subrnatri- - ces of G containing the Qu's and the R,,'s as their respective diagonal elements. Averaging

over al1 P, (4.1) becomes: Ik

Here, we use the same argument used to develop equation (2.25) to achieve:

l k wi This may be rewritten as: F(w, Z) = i=O

Averaging over al1 Qu, Ru's we obtain:

We now evaluate the expectations in equation (4.5). First, we have:

Since the Qu are kxr, binary matrices, there are 2"" of them. For a fixed g,

i &Qu as a linear transformation from the space of al1 k, binary mamces to

the vector space of a11 binary vectors of length r,. As Qu assumes al1 the possible values of

its space, by linearity of the transformation, &Qu will take on al1 the values of its respective

space the same number of times. Since there are 2" possible %Qu, each one will be

matched to exactly 2kr'/2r1 Qu's. Finally, in the vector space defined above, there are ( y ) vectors of weight i. The case where x, is different fiom the all-zeros vector, equation (4.6)

becomes:

Chapter 4:Turbo codes versus block codes

Sirnpli@ing, we get:

With equation (4.8) we can solve the product sums of equation (4.5):

where p(xJ is the number of non-zero x, in the vector 3. Finally, we define

We can now give a compact f o m for the averaged IRWEF of equation (4.5):

"Averaged IRWEF fb, the Time-Vaying (4- Turbo Codes"

To help in the computation of the function Yi(Z), we can write it as foilows:

where the second sum is over al1 sets of j integers satisfying the two given

conditions. The bound on the BER of Chapter 2 will be applied using this averaged RWEF

in section 4.3 where we compare the behavior of the three ensembles of codes.

TURBO CODES

4.2.1.1 Limitation in computing the IRWEF

The equation for the averaged IRWEF is easy to program in any program-

ming language. In fact, it is a simple sum of polynomials in 2. The problem is computing

the Yi(Z)'s. The main sum of equation (4.12) is not straightforward. In trying to evaluate

it, we have to find and use al1 the sets (i,, ..., ij) such that the s u . of the components is i and

such that al1 the components are positive. For a specific pair (j,i) this is the set of al1 integer

partitions of i which contain j integers. This is an easy problem for very small parameters.

It becomes quite intricate when i gets larger than 6. The program that we have implemented

to compute the Yi(Z) is using a rnatrix of partitions of al1 integers smaller than or equal to

the information weight i. In its construction, the ma& contains an index which gives the

number of integers in each partition. This ma& is generaied using the previous partition

to create the next. Unfominately. for every new integer the program has to be modified.

The rnethod that creates the partitions need a new set of loops have to be implemented each

tirne we add one integer. In each loop, a set of conditions have to be met and the number

of conditions grows linearly with the size of the integer to be panitioned. Also. a new van-

able is needed to index the matrix in each new set of loops. It becomes very time consum-

ing to program for i > 10.

The other problem with this method is that the maaix has to be big enough

to accommodate the largest partition. Because of the way it is constmcted, the size of the

partitions have to be known in advance. This is a problem since there is no known way of

evaluating the number of partitions of a particular integer. (There is an approximation but

it is only good for very large integers and we are not close to that!). To bypass this problem,

we have constmcted the matrix with a lot of overhead. Consequently, we have limited the

size of the maximum integer to 17. Note that it is possible to ver@ experimentally (using

a mesh representation of the partition matrix) that there is overhead left at the end of each

partition. It was found that with an integer of 18 there is no room to spare in some area of

the matrix. It is therefore safe to assume that the matrix is missing some partition of the

integer 1 8.

Cbapter 4:Twbo codes versus block codes

Since i nuis from zero to k in equation (4.1 1), this means that we are limited

to codes with a product lk 5 17. This is very smali considering that turbo codes by their

construction, are suppose to be good only when this product is large! Costello, Perez and

Seghers [CostelloZ] explain that this is due to the generally poor minimum distance of the

turbo codes. By studying the distance spectmm of turbo codes, they illustrate that the use

of large interleavers results in spectral thiming, Le. the number of codewords of minimum

weight is small. The actual size of the interleaver has a great effect on this spectral thinning.

In the examples that they used the efTect of spectral thinning was aot visible with a total

block length N = (l(k+rI+r2)) of 1024. It was very visible for block length of 16384 and

65536. This agrees with the results of Battail [Bat951 for specific turbo codes.

A solution to the problerns created by the partition matrix would be to find

another way to generate the partitions. It wouid be best to generate them only when they

are required in the program. Also, it would be a big improvement if oniy the partitions con-

taining a specific number of parts could be generated. This would reduce enormously the

amount of memory space used for the partition mauix. As far as the computation time is

concerned. the program should be written in a iower levei language to make it more effi-

cient. As wili be discussed later, this approach was taken to improve the performance of

the program used for the IRWEF of the Fixed Turbo Codes with astonishing resuits. To do

so here wouid not be of much help since we still have the problem of finding an aigonthm

to generate the partitions of large integers.

4.2.2 Fixed Turbo Codes

The IRWEF for the ensemble of Fixed Turbo Codes averaged over al1 per-

mutations is found by replacing Qu by Q and R, by R in equation (4.4). We can then aver-

age this function over ali possible Q's and R's. Note that Q and R are still independent.

Recailing that the message has the f o m = &, xi, ..., 5-,), we have:

TüRBO CODES

These expectations are:

Let qti be the i-th column of Q, then

, which reduces to: 90 q r t - 1

An identical denvation yields the expectation for the R submatrix. We

would get the same result as in (4.16) with r2 instead of r, and qti would represent the i-th

colirmn of R. To include both submatrices, we set:

where t is used to denote which subrnatrix is processed. Using (4.17) to

compute the IRWEF of equation (4.13) we get:

"Averaged IRWEF for the Fixed Turbo Codes" (4.18)

i

Equation (4.1 7) and (4.1 8) can be written in a cornputer program and a

bound on the BER of this ensemble of codes can be evaluated using the results of Chapter

2. This bound is computed and compared with the bound for the ensemble of Systematic

block codes in section 4.3. In the next sub-section, we talk about the Limitation encountered

Chapter 4:Turbo codes versus bIock codes

when programming those two equations.

4.2.2.1 Limitation in computing the IRWEF

The equation for the IRWEF is easily evaluated once we have found the nec-

essary Qi(')(Z) functions. The problem was in generating these latter. The programming of

the function was çtraigh~orward. Using matlab it was easy to get results for small code

parameters. Unfortunately, the problem of computing time was encountered. It took mat-

lab 4 days to conclude the computation of a code wîth I = 10 and k = 3. This is due to the

enormous range of the summing indices that are oecessary in the equation. When summing

over al1 codewords of information weight i, there are 2* possible kl-tuples to consider. Ail

of them have to be considered each t h e we change the weight i. This explains the time it

takes since matlab is not very efficient in handling loops.

The program was rewritten in C language which accelerated the process. A

computation that was taking 3 hours was done in about 2 minutes. Unfominately, another

problem came up. The index that was used to go through al1 the kl-tuples had to be an in-

teger for practical reasons. Due to the way an integer is stored in memory, there is a phys-

ical limit to its size. Ln the particular system used this number is MAX-NT = 231. This

is very large, but not enough for our purposes. This means that the product k has to be

smaller than 3 1. In practice however, we found that another limit existed on this product.

When the product k was larger than 20 the program corrupted the cornputer memory and

the computation stopped. The memory comption problem is probably due to inconsistent

memory allocation, i.e. some data are stored where the program's instructions are kept.

That problem can be very difficult to solve and because of the other limit on the index (ik

< 3 1) we did not attempt at lengtb to resolve the problem. Keeping these limitations in

mind, we were able to compute the BER for a small range of the parameters (1, k, r,, r2) in

keeping the product k smaller than or equal to 20.

4.2.3 Systematic block codes

In this case, there is no restriction of the f o m of the submatrk of G except

TURBO CODES

that it be systematic. To keep the same parameters for later comparisons, we set r = r, + - the total number of redundancy bits. The RWEF for a specific code is :

We average over al1 the submatrices Q.

If5 + Q , then as Q assumes al1 k x Ir binary matrices, xQ assumes al1 binary

Zr-tuples exactly #(Q)/2" times, hence:

"IRWEF for the

(4.2 ! )

(4.22)

average over al1 systematic block codes"

(4.23)

This IRWEF was easy to program and no limitations were encountered.

4.3 Cornparison of the BER'S

Io the present section, we assume that the coding schemes discussed so far

are used over the AWGN channel with antipodal signalling. This allows us to compute the

average BER of the different ensemble of codes using the results of Chapter 2 in particular,

equation (2.15). We show graphs of the bound on the average BER corresponding to the

IRWEF found in section 4.2 for the three ensembles. Ail the graphs use semi-logarithmic

scale and present the BER as a function of the Signal to Noise Ratio (EdK) in dB. The

ensemble of Fixed Turbo Codes and Time-Varying Turbo Codes are compared with each

other as well as with the ensemble of al1 Systematic linear block codes. The explicit value

of the BER for the AWGN channel used without coding is also shown on a few graphs in

Chapter 4:Turbo codes versus block codes

order to verify the tightness of our bounds. The effect of varying different parameters

(mainly I and k) will be shown and analyzed. Due to the limitations discussed in sub-sec-

tions 4.2.1.1 and 4.2.2.1, we lirnit our analysis to a product fk smaller than 17 when we

compute the BER of Time-Varying Turbo Codes. At the end of the section, we consider

only Fixed Turbo Codes and Systematic block codes so we are able to use slightly larger

parameters.

The first graph (Figure 4.3) shows the tendency of the scheme with k = 4,

I = 1, r, = 3. r2 = 3, N = 10, R = 2/5 (where N is the total block length of the code and R is

the rate). These values have been chosen to correspond to the parameters of the Turbo Code

coostructed in section 2-23 using two Hamming (7,4) codes. The BER computed for this

Turbo Code is based on the IRWEF of this construction averaged over al1 permutations:

Figure 4.3 CornparLFon of codes with k = 4, ! = I . r, = r, = 3

TtTRBO CODES

The Hamming (7,4) code bas a good minimum distance and is considered a

good linear block code. It is therefore normal that the bound on the construction of section

2.2.3 is tighter that those averaged over a wider range of codes. We note fkom Figure 4.3

that the ensemble of Systematic block codes is better than the other two ensembles. The

worst is the ensemble of Time-Varying Turbo Codes. The most significant result however

is the very poor accuracy of our bounds. We lmow that the averaged Systematic block

codes should improve the BER of the channel used without coding at moderate to high

Sm. We see that this is not what our bound indicates and we have to conclude that our

bounds are very loose. We now Look at the three ensembles but this t h e with 1 = 4 (Figure

4.4). In the Turbo Code of Chapter 2,1= 1 by its construction so we will use it only as a

reference for the othen. It will hopefùlly give us some insight in the effect of increasing

the block length while keeping the same rate.

1 oO + [ \ I I t I I I I I

\

1 O-'

1 o - ~

+ -t

K . -. . + \ O --. - . \ *

; N. + \- - - -+ -.* . N =4O --- -. O

\ - + . \ . \ - R = 2/5 + --- .. \ -

+ .- .-• .- - - 4- '- x - \ - + .. \ O

'. \ - + . - i + '- \ O . O + '. \. 1 o - ~ '- + *

'. -c -

+ '. q . - 0 ' . h

f w ' . -. + -. ' L.

1 o - ~ + ' . \ - \

10°F

Io*-

+ ' . \ -. : + ' . + '-

+ \ - - + \.

+ \. \.

- ...... Tirne-Varying Turbo Codes + \. - - - - - Fixed Turbo Codes + \. - + E +++ + ++ Systematic IOC CL COQS \. + . -.-.-.-.-.-.-.-. Hamming 0-4) of Figure 2.1 (/ = l), N= IO \. + \. :

+ 1 O-' t fi 1 I I I I I I

\.

O 1 2 3 4 5 6 7 8 9 10

Chajxer 4:Twbo codes versus block codes

The bound on the Systematic block codes is a lot better then the other two.

Using the Hamming turbo code as a reference, we see that there is no irnprovement at high

SNR for the other two ensembles. At low S M , we notice that the bounds are very poor

and actually useless for S M < 1 dB. In analyzing the perfomiance of the turbo coding

schemes, we have to keep in mind that they are not expected to be good codes for high S N R .

This is due to their generally poor minimum distance. Also, the precision of our bounds for

very low SNR is not very good as we use the union bound in ail our denvations. The turbo

coding scheme relies on the idea that the output follows a distribution close to a gaussian

randorn variable. To get doser to that ideal distribution, we need a greater number of bits

going through the generator matrix. It is therefore natural to expect those codes to perform

better as we augment the nurnber of messages being permuted (0.

Figure 4.5 Cornparison of codes with 2 = 2 and 8, k = 2, r, = r, 4.

TURBO CODES

On the other hand, this shouId have a limited influence on the bound of the

average over al1 linear codes. To venfj this hypothesis. we look at the bounds on the three

ensembles with I = 2 and 8, k = 2, r, = r, = 1, Le. N = 8 and 32, R = 112 (Figure 4.5). We

observe that contrary to the expected resuits, the three bounds are getting worse at low S N R

as we increase the number of messages. We observe again that the T h e Varying Turbo

Codes ensemble is a lot less affected than the Fixed Turbo Codes ensemble and at Z = 8, it

is, in fact, better. We see again that either our bounds are very poor or turbo codes are not

at al1 efficient in average with the given parameters since the BER of the channel used with-

out coding is much better over the complete range of SM. Although, the ensemble of Sys-

tematic block codes is much better at high SNR, we will compare the bound mainly at low

SNR where the Turbo Codes have been proven to work best [Bat95],[Berrou] and [Lodge

11. The schemes used so far have a moderate to low rate. In an effort to increase the rate,

we next look at the Fixed Turbo Codes and the Systematic block codes ensemble with

k = 9. The results were interesting and are included and shown in the next graph (Figure

4.6).

Figure 4.6 shows that the Turbo scheme seems to improve the performance

of poor constituent codes. It would be interesting to see the effect of augmenting even more

the rate of the code or to look at the effect of varying I on a code of high rate. Since we

cannot increase I with k = 9 (due to the computing limitations), we will look at the general

tendency taken mder variation of k. The behavior of the schemes under different values of

k should tell us a little more about what could be expected for codes of higher rate.

Our next graphs (Figure 4.7) look at the effect of vq ing k. The effect is to

change the rate of the code. What we are looking for is if the Turbo Codes tend to get any

closer to the averaged over al1 systematic block codes when the amount of redundancy is

diminished. The same effect could be obtained by changing the parameter r, and r2. But,

due to the limitations discussed earlier, if we change these two parameters, it would be dif-

ficult to augment the rate of the code, since we would need k to get Iarger then what our

program can support. For this reason, r, and r2 are kept to one or two.

Chapter 4:Turbo codes versus block codes

+ + \

Fixed Turbo Codes + + + + t + + + + Systematic block codes +

f

Figure 4.6 Fixed vs. Systematic; I = 2. k = 9. r, = r, = 1.

Again, in Figure 4.7 and Figure 4.8 we observe that as k gets Iarger and the

rate of the constituent codes is increased, the bound on the Turbo coding schemes don't im-

prove as much as the bound on the Systematic block codes. The behavior of the ensemble

of Systematic Block Codes can be explained by the fact that although we increase the rate

of the code, the block length is also increased and the overall performance at moderate and

high SNI2 is improved. At low SNR, the effect of the increase in rate overwhelms the effect

of the increased length.

To shidy the effect of Z on the scheme using constituent codes of high rate,

we next Vary Z while keeping k = 4, r, = 1, r, = 1. The graph of Figure 4.9 shows the resuits.

For I = 1, the two bounds are indistinguishable while they are quite distinct for 2 = 3 and 5.

This seems to indicate that increasing 1 is not the way to go. But the idea of Turbo codes

relies on the random nature of the output of the encoder and we depend on I to achieve that.

TüRBO CODES

Figure 4.7 1 = 3. r., = r2 = 2. k tukes the values 1. 3 and 5.

It may be that the average over al1 permutations does not yield an equivalent interleaver that

is optimum Perez, Seghers and Coaello [ConelloZ] have shown that the size of the inter-

leaver is not enough to guarantee that there wiil be only a small number of codewords of

minimum weight. Another thing to keep in rnind is that while the Turbo coding scheme

can use the same small constituent codes and therefore the same simple encoderldecoder

pair when i is increased, this is not the case for the Systematic block codes. The bound does

not take into account the complexity of the decoder that would be needed to satisQ the pa-

rameters that we look at. ui the next graph (Figure 4.9), the rate of the codes is fixed to

R = 213 while the block length is increased. For the I values 1,3 and 5, we have the respec-

tive block lengths N = 6, 18 and 30.

In Figure 4.9, the rate is constant (R = 213) and the block length is increased.

The effect on the Turbo code is a general degradation of the performance. For the ensemble

Chapter 4:Turbo codes versus block codes

- Fixed Turbo codes. k = 2.R= V2.N = 8 , 0-4 - - - - Fked Turbo codes. k = 6 3 = 3t4.N = 16

- O O O Systematic block codes, k = 9.R = 9/11.N = 22

O 1 2 3 4 5 6 7 8 9 10 Eb/No in dB

Figure 4.8 1 = 2. r, = r2 = 1. k takes the values 2, 6 and 9.

of Systematic block codes, we see an irnprovement at moderate to bigb SNR. This goes in

accordance with the known fact that block codes are better when either the rate is decreased

or the block length is increased.

To compare encoders of the same cornplexity, we would need to compare

al1 the different bounds of the Turbo coding scheme obtained by changing I with the bound

on the Systematic block codes when 1 = 1 (The line with the "+" on the graph). If we do

this, we see that the Turbo codes are getting doser to the other codes when the SNR is large.

TURBO CODES

Figure 4.9 k = 4. r, = r, = 1. 1 takes the values 1. 3 and 5.

Finaiiy, we confirm again that the bounds on the turbo coding schemes do

not even get close to the BER expected without coding.

Chapter 5

Conclusions

In this chapter, we surnmarize the analysis doue in the previous chapter and

give our opinion on the work accomplished. We also make some suggestions as to the di-

rection that subsequent research on the subject of bounding the BER of Turbo codes should

take.

Bound on the BER of Turbo Codes

In Chapter 2, we were successful in bounding the BER of turbo codes using

the weight distribution of the code. Since a turbo code uses an interleaver between its con-

stituent codes, the RWEF of the overall code is dependent on the permutation chosen. We

have shown that by computing an averaged IRWEF we get a bound on the BER that gives

a good estimate of the bounds obtained for the different codes corresponding to specific

permutations. Using a parallel concatenation of two Hamming (7,4) codes we have noticed

that the constituent codes actually had a better performance than the turbo code (similar re-

TURBO CODES

sults were found by Benedetto and Montorsi Pen951). That fact alone was a good moti-

vation to investigate the performance of turbo codes when their intnnsic characteristics are

changed.

Classification of Turbo Codes

The classification of turbo codes according to their intrinsic charactenstics,

was the main objective of this paper. We wanted to study the effect of varying the intriasic

characteristics of the coding scheme without having to program a different encodeddecoder

pair for simulation purposes for every new set of parameters. This was achieved with a

mal1 degree of success. Theoretical bounds were developed for the ensemble of Time-

Varying Turbo Codes and Fixed Turbo Codes.

Due to the computing limitations discussed in section 4.2.1.1 and 4.2.2.1.

we were ooly able to look at a small range of codes. Also, the bounds appear to be vety

weak at low S N R which is the area that has to be investigated when dealing with turbo

codes. From the few results that could be obtained, we have observed ftom the bounds that

when the rate of the code is increased, the hirbo codes performance approaches that of the

average linear block code. On the other hand we have seen that, with a fixed rate, aug-

menting the block length deteriorates the bounds. This actually contradicts what is to be

expected according to the different analysis found in the literature (see for example @3at95],

[Lodge 11, [Berrou], and [CostelloZ]). Adding to that the fact that the bounds are generally

abover the BER of the charnel used without coding, we cm conclude that the bounds com-

puted in Chapter 4 are not very indicative of the actual performance of turbo codes. This

is explainable by the small size of interleaver used in our computation as well a s by the io-

accuracy of our bounds. This inaccuracy is due in part to the union bound used in the eval-

ution of these bounds which is known to fail at low SNR.

Finaliy, we need to remind the reader that al1 the tendencies observed in

Chapter 4 are for code of total block length smaller than 3 1. This is a direct consequence

of the limitations we had on the size of the parameters k and f. Many turbo codes which

Chapter 5: Conclusions

have been shown to be exceptionally good at low S N R were using a very large block length

(larger than 10000 bits) [Bat95], bodge 11, [Berrou], and [CostelloZ]. Benedetto and Mon-

torsi @3en95], have demonstrated using bounds similar to ours that the performance of par-

allel concatenated convolutionai codes is l e s ~ than that of the constituent codes for small

interleavers. In a simple example using only one memory in each constituent encoder, they

show that the turbo codes stan to gain on the constituent codes when the interleaver per-

mutes over 100 bits.

The best explmation of the observed behavior has been given by Perez, Seg-

hers and Costello [CostelloZ]. In their paper they explain the good performance of turbo

codes by their distance spectm. One of their conclusions is that even if they have a gen-

erally poor minimum distance, the turbo codes have few codewords of minimum weight

when the interleaver is chosen correctly. This rneans that they should not be expected to

be very good at moderate to high SNR unless the constituent codes have a good minimum

distance. It also means that for the performance to be good at low SNR, we need the weight

distribution of the turbo code to be close to a N o m 1 distibution (this was also noted by

Batail [Bat95]). When the block length is small, it is not possible to get vexy close to that

kind of distribution even with the best interleaver. That would explain the general tendency

observed in this thesis since we could not get bounds for turbo codes with block lengths

large enough for this effect to be noticeable

Area of further study

The first thing that should be doue to improve the results observed is to de-

velop a new bound based on the IRWEF but without the use of the Union bound. Hence

we need to bound the probabiiity of the union of error events (see section 1.2.3) in some

other way.

The study of turbo codes is centered around two fundamental questions.

The first question is "why do turbo codes perforrn so welI?". This thesis was centered on

that question. The second question is "does the iterative decoding scheme always converge

TURBO CODES

to the optimum solution?'. In an effort to address the second question, the effect of varying

the oumber of messages being pennuted in the interleaver, i.e. increasing the block length

while keeping the rate constant could be investigated (this could be done in simulation us-

ing specific constituent codes and decoding using the modified Bahl's algorithm). This

would alIow to observe the effect of using different nurnbers of iterations in the decoding.

Since a larger amount of messages perrnuted would decrease the correlation between the

redundancy vectors, it would probably allow the decoder to efficiently use more iterations.

Finally, research of a more efficient way to program the formulas used in

computing the different bounds on the turbo codes could yield interesting results. If we

could use our bounds on codes with large block length (N >1000) we could more readily

compare with the specific codes studied in the literature and validate our results.

References

References

[Ben951

[Viterbi]

[Gahger]

f Wozencraftj

[MacWiUiams]

[Costelio J

[Lodge 11

[Berrou]

G. Battail, "On random-like codesW,Fourth Canadian Workshop on Infor- mation Theory and Application, 1995.

G. Battail, "We can think of good codes and even decode them",Euro- code792, Udine Italy, 26-3- Oct. 1992; P. Camion, pCharpin and S. Harari, Eds, CISM Courses and Lectures No.339, pp. 3 53-368, Wien: Springer, 1993.

Sergio Benedetto and Guido Montorsi. "Unveiling turbo-codes: some results on parallel concatenated coding schemes". IEEE Transactions on informatin Theory, September 1995.

Andrew I. Viterbi and Jim K. Omura, "Principles of Digital Dommunica- tion and Coding", 1979 McGRaw-Hill hc.

Robert G. Gallager, "Information Theory and Reliable Communication", 1 968 , John WILEY and Sons, Inc.

John M. Wozencraft and invin Mark Jacobs, "Principles of Communica- tion Engineering", 1965, John WILEY and Sons. Inc.

F.J. Macwilliams , "A Theorem on the Distribution of Weights in a Sys- tematic Code", Bell Syst.Tech. J., 42, pp.79-94. 1963.

Shu Lin and Daniel J. Costello, Jr., "Enor control Coding: Fundamentals and Applications". 1 983, Prentice-Hall. Inc.

J. Lodge, R. Young, P. Hoeher, J. Hagenauer, "Separable MM "filtes" for the decoding of product and concatenated codes", in Proc. IEEE Globecorn Conf., San Francisco, Calif.. pp. 1298- 1303, December 1994.

C. Berrou, A. Glavieux and P. Thitimajshh, "Near Shannon Lirnit Error- Correcting Coding and Decoding: Turbo-Codes," Proc. of ICC'W, Geneva, Switzerland, pp. lO64-1070,23-26 May 1993.

L. R. Bahl, J. Codke, F. Jeinek and J. Raviv, "Optimal decoding of linear codes for minimizing symbol emor rate", IEEE Trans. Inform. Theory, vol IT-20, pp. 248-287, Much 1974.

G.E. Séguin and S. Fournier, "The Dual of a Turbo Block Code", 18th Biennial Symposium on Communications, Kingston, Ont, 1996

Lance C. Perez, Jan Seghers, and Daniel J. Costello, Jr., "A Distance spec- tnim Interpretation of Turbo Codes", IEEE Trans. Inform. Theory, Vol. 42, No. 6, November 1996.

Thomas M. Cover, and Joy A. Thomas, "Elements of Information The- ory", 1991, John Wiley & Sons, Inc.

Matlab's computation of the averaged BER

I TURBO CODES

dear;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%%%%%%%%%%% % This mfile will corn pute the BER for the three kind of Oh averaged IRWEF obtained from.

% % 1. TlME VARYING TURBO CODES (Mc) % 2. FtXED TURBO CODES (Rc)

% 3. Averaged over al1 Systematic block codes Oh Oh The averaged is over al1 matrices Q as well as over al1 permutations.

%%%%%%%%0h%%0h%%%0h0h%%%%%%%%%%%%%%?4%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Parameter's set up

%%%%oh%0h%%%%%%%%%%%%%%%0h%%%%%%%0h%%0h%%%%%0h0h%0h0h%%%%%%

l = input('Give the value for 1: '); k = input('Give the value for k: '); r l = input('Give the value for r l : '); r2 = input('Give the value for R: ');

% Matrix of the partition of integer of k'l %0h%%%%%%%%%%%0h%%%0h%%%%%%%%%%%%0h%%%%%%%0h%%%0h%%%%%%%0h

0h%%%%%%%%%%%%%%%%%%%%0h%%%%0h%%%%%%%%%%%0h%0h%0h%%%%%%%%% Corn putation of the P~i(w)((l+z)/2)~r), for r l Ohoh ~~~~~~~~h~h~h~h%~~~0h0h%0h0h0h0h0h0h%0h%0h0h%0h%%0h%0h0h0h0h0h%0h Ohohoh OhOhOh%O!h%OhOh

Psi 1 = zeros(kœl+ 1 ,r l *l+ 1 ); Psil(1,l) = 1; Psi2 = zeros(k*l+ 1 ,r2*1+ 1 ); PsiS(1,l) = 1; for w = 1 :l'k

partition = partv2(F.w); totalsum = zeros(1 ,rl'l+ 1); for j = 1:I

poly = zeros(1 ,rl'l+l ); for i = O : r l * j

paly(i+l) = cornb(rl*j,i); end; poly = p0ly./(2~(rl "j));

Oh sum over al1 (il ,i2,..ij) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%

partialsum = 0; for s = l:size(partition,l)

lop = partition(s,sue(partition,2)); if lop == j

productsum = 1; for i = 1:lop

productsum = productsurn * comb(k,partition(s,i)); end;

Annex A: Matlab's computation of the averaged BER

a = 1; n = [l; n(a) = 1; for t = 2:lop

if partition(s,t) == partition(s,t-1) n(a) = n(a) + 1;

a = a + l ; n(a) = 1 ;

end; end; multiset = w; f o r t = l : a

multiset = multisetlfact(n(t)); end; partialsum = partialsum + productsum'multiset;

end; end; totalsum = tatalsum + partialsum'polyacom b(1.j);

end; Psi1 (w+ 1 ,l :size(totalsum .2)) = totalsurn ;

%0h%%%%%%%%%%%%%%%0h%0h%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%0h%% % Cornputation of the P~i(w)((l+z)/2)~r), for 1-2 0 ~ 0 ~ ~ 0 m a ~ 0 ~ 0 ~ ~ 0 ~ ~ 0 ~ 0 ~ % % 0 ~ ~ ~ 0 ~ 0 ~ 0 ~ a r 6 0 m Q m Q ~ ~ ~ 0 m 0 ~ 0 m 0 ~ 0 ~ 0 ~ a m O m a m 0 ~ 0 ~ 0 m ~ 0 m a ~ ~ ~ 0 ~ 0 ~ 0 ~ ~ a ~ 0 m 0 m a ~ 0 ~ 0 ~ 0 ~

totalsum = zeros(1 ,R'I+I); for j = O;I

poly = zeros(1 ,r2'1+ 1); for i = O : r2-j

poly(i+ 1) = com b(r2"j.i); end; poly = p0Iy./(2~(r2*j));

Oh sum over al1 ( i l ,i2,..ij) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Yo%%%%%%%%Yo

partialsum = 0; for s = 1 :size(partition,l )

lop = partition(s,ske(partition,2)); if top == j

productsum = 1 ; for i = 1:lop

produdsum = productsurn * comb(k,partition(s,i)); end; a = 1; n = 0; n(a) = 1 ; for t = 2:lop

if partition(s,t) == partition(s,t-1) n(a) = n(a) + 1 ;

else a = a + l ; n(a) = 1 ;

end; end; multiset = w; f o r t = l : a

multiset = multiset/fact(n(t)); end; partialsum = partialsum + productsum*multiset;

TIiRBO CODES

end; end; totalsum = totalsum + partialsumapoly*cornb(l,j);

end; PsiZ(w+ l ,1 :size(totalsum ,2)) = totalsurn;

end;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%% % Computation of the coefficients of the IRWEF %%%Oh%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%Oh%%%%%%

ftvtc = reros(k*l+l ,(rl +r2)'1+ 1); for w = O:k*l

for z = 0:(rl +rZ)*I for i = O:rlal

if z-i >= O 8 z-i <= QaI ftvtc(w+ 1 ,z+ 1) = fMc(w+ 1 .z+l ) + llcom b(l*k,w)*Psil (w+f ,i+l)*Psi2(w+l ,z-i+ 1);

end; end;

end; end; %%0h0h%%%%%%%%%%%%%%0h%%%%%%0h%%%%0h%0h%%%0h%%%%0h0h0h0h%%%%%0h%% % Computation of the Dm's for the Time-varying Turbo code

Oh%OhOhO~O~ 0 ~ 0 ~ 0 ~ % 0 ~ 0 ~ % 0 ~ 0 m ~ 0 m ~ % 0 m 0 m 0 m 0 m 0 m 0 m 0 ~ 0 ~ 0 m 0 m 0 ~ 0 m 0 ~ 0 ~ 0 ~ ~ ~ 0 m 0 m 0 m 0 m 0 ~ ~ 0 m 0 ~ 0 ~ ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~ 0 ~

for m = O : (k+rl+r2)*l D(rn+l) = 0; for w = O: k*!

z = rn-w; if z >= O & z c= (rl+R)*I

D(m+l) = D(m+l) + w/Wlaftvtc(w+f ,z+l); end;

end; end;

%%%%%%0h%%%%%0h0h%%%0~%%0h%%%%%%%%%%%0h0h%%%%%%%%0h0h0h%0h%0h%%%% % Computation of the BER of the rime-varying Turbo code %%%%%%%%%%0h%%%%%0h%%%%%%%%%%%%%%%%%%%%%%%%%%%0h%%%%0h%0h%

SNR = linspace(0,10,50); R = k/(k+rl+r2);

snr = 10.A(SNRflO); ber1 = 1 /2*D'erfc(sqrt(Ralinspace(O,size(D ,2)-1 ,size(D ,2))"snr));

%%oh%%%%%%%%%%%%%%%%%%%%%%%%%%%%0~%0m%%%%%%%%%%%0h%%%%0h%% % Computation of the phi(w)(rl) %%%%%%%%%%%Oh%%%%%Oh%%%%%%%%%%%%%%%%%%Oh%Oh%%%%%%%%%%%%%% All-kltuples = ktuples(k'1); All-ktuples = ktuples(k); phi 1 = zeros(kal+l ,rl'l+l); phil(1 J) = 1 ; phi2 = zeros(kal+ 1 ,r2*1+ 1 ); phi2(1,1) = 1 ; for w = l:lak

for i = 1 :2*(kaI)

Annex A: Malab's computation of the averaged BER

x = All-kîtuples(i,:); if sum(x) == w

P O ~ Y = O; a = 0; for j = 1:2"k

exponent = 0; a = zeros(1 ,size(poly,2)); q = Ali-ktuples(j,:); for u = O:!-1

exponent = exponent + rem(x(u*k+l:u*k+k)*q',2); end; a(exponent+l) = 1 ; if size(poly.2) c ske(a,2)

poly(expanent+l ) = 0; end; poly = poly + a;

end; powerpoly = polypo~(poiy/(2~(k)),rl); phi1 (w+l , l :size(powerpoly,Z)) = phil(w+l ,l:size(powerpoly.2)) + powerpoly;

end;

end; 0 ~ 0 ~ ~ 0 ~ % ~ ~ ~ ~ a ~ 0 ~ 0 ~ 0 ~ ~ 0 ~ ~ 0 ~ ~ ~ 0 ~ 0 m 0 ~ 0 r 6 0 ~ o r 6 % % 0 r b ~ ~ ~ 0 ~ ~ ~ ~ ~ ~ 0 ~ 0 ~ 0 r b 0 ~ o ~ ~ ~ ~ ~ ~ 0 ~ ~ ~ ~ ~ 0 ~ 0 ~ Oh Phi(w)(r2) %0h0h0h%%%%%%%%%%%%0h%0r6%0h%0h0h%%%%%0h%%0~%%0h%%%%0h%%%%0h%%%%0h%%

for i = 1 :ZA(k*I) x = Ali-kltuples(i,:); if sum (x) == w

poJy = O; a = 0; for j = 1 :2Ak

exponent = 0; a = zeros(1 ,size(poly,2)); q = All-ktuples(j,:); for u = 011-1

exponent = exponent + rem(x(u*k+l:u*k+k)*q',2); end; a(exponent+l) = 1 ; if size(poly,2) c size(a,2)

paly(exponent+ 1) = 0; end; poly = poly + a;

end; powerpaly = polypa~(poly/(2~(k)),rl); phi2(w+ l,l :size(powerpoly,2)) = phi1 (w+l ,i :size(powerpoly.2)) + powerpoly;

end; end;

end;

fitc = zeros(k'l+ 1 ,(rl +R)*t+ 1 ); for w = O:kal

for z = O:(rl+r2)*I for i = O:rl*l

if z-i >= O 8 z-i c= r2*I fftc(w+l ,z+l) = fftc(w+l r+1) + l / w m b(lak,w)*phil(w+l ,i+l)*phi2(w+l ,z-i+l);

TüRBO CODES

end; end;

end; end;

for m = O : (k+rl+r2)*I D(m+l) = O; for w = O:kal

z = m-w; if z ,= O 8 z <= ( r i +r2)*1

D(m+l) = D(m+l) + w/kfl*fftc(w+l,z+l); end;

end; end;

SNR = linspace(0,10.50); R = k/(k+rl+R); snr = 10.A(SNWIO); ber2= 1/2*D*erfc(sqrt(R*1inspilce(O,size(D,2)-1 ,size(D,2))"snr)); %%%0~%%%%%%%0h%%%%0h%%0h%%%%%0h%%%0h%%%%0h%%%%%%%0h%%%%0h%%0h%%

% AVERAGED OVER ALL LINEAR CODES %%%%%%0h%%%%%%%%%%%%0h%%%%%%%%%%%%%%%%%%%0h%%%%%0h%0h%%%%0h

r = r l +r2; f = zeros(Pk+l ,r'l+i); for w = 0:I"k

for z = 0:r"l f(w+ 1 ,z+ 1 ) = corn b(l*k,w)*wrn b(l*r,~)/(2~(l'r));

end; end; f(1,l) = 1,

%%%%%%%%%%0h0h%%%%%%0h%%%0h%%%%%%0h%%%%%%%0h%%%%%%0h%%%0h%%%% % Computation of the Dm's for the "All Iinear codesn %%%%%%0h%0h%0h%%%%0h%%%%%%%%%%%0h%%%0h%0h%%%%%%%%%%%0h%%%0h%%0h

for m = O : (k+rl+R)*l D(m+l) = O; for w = O: k*l

z = m-w; if z >= O 8 z <= (rl+R)'I

D(m+i) = D(rn+l) + wM*f(w+ï ,z+l); end;

end; end;

%%%%%%%%%%%%%%%%%%%%%%%0h%%%%%0h%%%%0h%%0h%0h%oh0h0h0h%0h%0h%0h0hoh Oh Camputation of the BER of the "Al1 linear cades %%%%%%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%Oh%%%%%%%%%%%%%%%

SNR = linspace(0,10,50);

Amex A: Matiab's computation of the averaged BER

R = k/(k+rl+R); snr = 10.YSNW10); ber3= l/2*D*erfc(sqrt(R'Iinspace(OIsize(D,2)-1 ,ske(D,2))'*snr)); semilogy(SNR,berl ,SNR,ber2,'-',SNR,ber3,','); title(*Campanson of bounds on BER'); xlabel('SNR in dB') ylabel('6ER') text(1,0.00055,'- Time varying turbo wdes'); text(1,0.0003,'- Fixed turbo wdes'); text(1,0.0002,' .... Averaged over al1 linear cades'); axis([O rnax(SNR) min(ber3) 11);

TURBO CODES

C program for the BER of Fixed nirbo Codes and Systematic block codes.

TURBO CODES

#indude <stdio.h> #indude cstdlib.h> #indude cmath.h> #define result

P Dedaration des fondions utilisees */

int *dectovec(int dec,int length); int sum(int aU.int n); float *polypow(float *poly,int power,int degre); double camb(int A,int 6);

main() {

FILE *fp; int w,z,m ,exponent,l,k,rl.r2.r,t,u,ternp=O,j,i.ç; int *x,*q,degree=O; fioat *poly,*phil ,*phi2,'fftc,'f,*powerpolyl, *powerpoIy2.'D,*Ber2,*Ber3.snr,SNR,R;

printf("Donnes 1: "); P lecture des caracteristiques */ ~canf("~hi",&l); P du code * l printf("\nDonnes k: "); scanf("%in,&k); printf(%Donnes r i : "); scanf("%in,&rl ); printf("\nDonnes r2: "); scanf("%in,8R);

phi 1 = calloc((k*l+ l)*(r l *I)+k*l+ 1 ,sizeof(float)); phi2 = calloc((kœl+ 1 )'(r2*l)+ka1+ 1 ,sizeof(fioat)); fitc = calloc((k*l+ 1 )"((r2+rl)*I+ 1 ),sizeof(float)); D = calloc((k+rl +R)*l+l ,sizeof(float)); r = r l +r2; phil[O] = 1.; phi2[0] = 1 .; fRc[O] = 1.; R = (float)k/((float) (k+rl +r2));

P program me principale "1 for (W = i ; WC= I*k;w++) {

for (i = 0;i<(pow(2,kal));i++) {

x = dectovec(i,lgk); rprintf("\n i is: %in,i); prinff("\n 2A%i is: %f',k*l,(pow(2,kœl))); prinff("li x is: ... %i %i %in,x[lœk-3],x[Iak-2],x[l*k-11); '1 if (sum(x,l*k) == w) {

poly = caIloc((kœl+l ),sizeof(float)); degree = 0; poly[O] = IIpow(2,k); for (SA ;s<(pow(S,k));s++) {

q = dectovec(s,k);

Annex B: C program for the BER of Fixed Turbo Codes and Systematic block codes.

exponent = 0; for (t = O;t<l;t++) C

temp = 0; for(u=O;u<k;u++)

temp += (x[t*k+uJaq[u]); exponent += temp%2;

3 if (poly[exponent] == 0)

poly[exponent] = V(pow(2,k)); else

poly[exponent] = polyfexponent]+ l/(paw(Z,k)); if (exponenbdegree)

degree = exponent; fiee(s);

3

powerpolyl = polypow(poly.rl.l); powerpoly2 = polypow(poly,r2,1);

for (j=O;jc=l*rl ;j++) phi 1 [w*(l"rl+ l)+d += powerpolyl u;

for Cj=O;jc=l'r2;j++) phi2[wa(l'r2+ 1 )+a += powerpoly2bj;

free(poly); free(x); free(powerpoly1); free(powerpoly2);

1 1

/" Calculation of the IRWEF for fixed turbo codea/

for (j=O;j<=lOrl ;j++) {

for (t=O;t~=l'rZ;t++) fftc[w*(I*(rl +r2)+ l)+j+t] += phi1 [wa(l*rl +l)+jJ*phi2[w'(l*r2+1)+t];

1 for (j=w*(l*(rl +r2)+ l);j<=w"(l"(rl+r2)+1)+l*(rl +r2);j++)

fftcu /=corn b((kal),w);

/* Calculation of the Dm's for the averaged fixed turbo code "1

for (rn=O;rnc= (k+rl+r2)*l;m++) {

for (w=O;wc=kol;w++) {

if((m-w)>=0 && (m-w)c=(rl +r2)'1) D[m] += fftc[w0(i*(rl+r2)+1)+(rn-w)]' ((fioat) w)/((tïoat) (kal));

1 1

TURBO CODES

r Calculation of the Bit Error Rate of the averaged fixed turbo code *

*/

Ber2 = calloc(50,sizeof(float)); #ifdef resuit fp = fopen("ber2.txt","w");

for (j=O;jc=49;j++)

S NR = (float) j 1 ((fioat) 5); for (rn=O;mc=(k+rl +R)*l;rn++) fler2(jl += .5*D[rn]*erfc(sqrt(rn'R'pow(l O.((float) SNR/((float)lO)))));

fpnntf(fp,"%l . lo f ",BeRbj); } fclose(fp); #endif P results */

P Same thing for the average over all linear cades P * - - m + m * . r c w ) ) t v ~

for (W = O;wc=l*k;w++) { for (z = O ;z <= r'l ;z++)

qw*(lœr+l)+z] = (float) (corn b(l*k.w)"wrnb(i'r,z))l((float) pow(2,l'r)); 1 901 = 1.;

for (rn = O ; m <= (k+rl+r2)*l;m++) { D[rn] = O;

for (W = O; WC= kl;w++) {

if (rn-w a= O 88 rn-w c= r*l) D[rn] += ((float) w/((float)

1 1

(k'l)) ) ' f[wo(l*r+l )+m-w];

AM~X B: C program for the BER of Fixed Turbo Codes and Systematic block codes.

1 " Calculation of the Bit Error Rate of the averaged over al1 code * -------al---/

Ber3 = 0lloc(50,skeof(float)); #ifdef result fp = fopen("ber3.txt","w");

for (j = O; j-49; j++) C

SNR = 0.2-j; for (m=O;mc=(k+r)*l;m++)

Ber3[jl += .S'D[m~erf~sqrt(m*R*pow(lO,(SNR/10.))));

printf("\n\n\nin\nw); #endif P result */

/* Liberation de la memoire */

) I" fin du programme main() '/

TURBO CODES

P Fonction qui transforme un nombre decimal en vecteur ' et retourne l'adresse de la premiere valeur du vecteur *

- Give the number and the length of the binary representation -*I

int *dectovec(int dec,int length) C

int t,*ptovec;

ptovec = malloc(süeof(int)*length);

for(t=l ;t<=length;t++) {

ptoveqlength-t] = dec%2; dec = div(dec82).quot;

1 return ptovec; 1

/"" Program qui calcule la sum des composantes d'un vecteur '-1

int sum(int aI].int n) C

int i; int surn=O;

for (i=O;icn;++i) surn +=am;

return sum ; 1

/" Fundion that retum the palynomial given to the power given in the argument: polyA(power) where poly has degree degre. The polynorne m n be of float type with no restn'dional

b a t *polypow(float 'poly,int power,int degre) { int a,b,i,j; float *y,*ytemp;

y = calloc((degreOpower)il ,sizeof(float)); ytemp = calloc((degreupower)+ 1 ,sizeof(fl oat));

for (i=O;i<=degre;i++) YU]= polyfl;

for (i=2;ic=power;i+i) {

for (a=0;a<=degre+degre0(i-I);a++) C

for(b=O; bc=degre;b++) {

if (a-b>=O &8 a-b <= degreO(i-1)) ytemp[aI = yternplal + y[a-blO~ol~lbl;

1 1 for (a=O;a<=degre*i;a+i) C

y[aI = Semp[aI;

Annex B: C program for the BER of Fixed Turbo Codes and Systematic block codes.

r Fundian that cornpute the cornbinatoric A chose 8 */

double cornb(int A,int 8) { double nom=l ,denorn=l .i;

if (Es0 II B>A) return 1;

else for (i=A;i>(A-B);i-)

nom = nom'i; for (i=l ;i<=B;i++)

denorn = denom'i;

return nomldenorn ; 1

TURBO CODES

Annex C

Matlab functions used

TURl3O CODES

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Om%OmOmOmom%am%%%%%%%%o~%~%% % Fundion that generate a % Matrix of the partition in integer of the first few integer % For integer <= 10, "it works for num berofinteger c l 7" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%%%

fundion y = part2(num berofinteger)

Omnurnberofinteger = 8; maxfact = 200; F = zeras(nurn berofinteger, 1 OO*rnaxfact+l ); F(1.1) = 1; for i = 2:nurnberafinteger

F(i,l) = i; for j = 1 maxfad

if F(i-1 ,l+(j-1)*100) > O F(i.1 +lOO*j) = 1; for t= 1:num berofinteger

F(i,t+1+100"j) = F(i-1 ,t+(j-1)"lOO); end; rnarkerl = j;

end; end; i f i > 2

marker2 = markerl + 1 ; for i = 1 :maxfact-markerl

if F(i-2.a-?)'100+1) >O 8 F(i-2,(j-1)'100+1) -= 1 F(i, 1 +1 OO*markerZ) = 2; for t = 1:num berofinteger

F(i,t+ 1 + 100*rnarke1-2) = F(i-2.t+(j-1)'100); end; rnarker2 = marker2 + 1;

end; end; end; if i >3

marker3 = marker2 + 1 ; for j = 1 :rnaxfact-rnarker2 if F(i-3.u-l)*lOO+l) >O 8 F(i-3,(j-1)*100+1) -= 1 ... & F(i-3,U-1)*100+1) -= 2

F(i,l+100emarker3) = 3; for t = 1 :num berofinteger

F(i ,t+ 1 + 1 0O8marker3) = F(i-3,t+(j-1)*100); end; marker3 = marker3+1;

end; end; end; if i =44

marker4 = marker3 + l ; for j = 1:maxfact-marker3 if F(i-4.U-1)'100+1) >O & F(i4,(j-l)*l00+1) -= 1 ... & F(i4,(j-1)*100+1) -= 2 8 F(i-4,Q-1)'100+1) -= 3

F(i, i +100*marker4) = 4; for t = 1 ;num berofinteger

F(i.t+ 1 +l OO"marker4) = F(i-4,t+(j-1)*1OO); end; marker4 = marker4+ 1 ;

Annex C: Malab functions used

end; end; end; ifi >5

market5 = marker4 + 1 ; for j = 1 :maxfact-marker4 if F(i-5,(j-1)*100+1) >O & F(i-5.(j-1)*100+1) -= 1 ... & F(i-S,(j-1)*100+1) -= 2 & F(i-5,G-1)'100+1) -= 3 ... & F(i-5.u-1)*100+1) -= 4

F(i,1+100*marker5) = 5; for t = 1 :numberofinteger

F(i,t+I+lOO*marker5) = F(iSi,t+(j-1)'lOO); end; marker5 = marker5+1;

end; end; end; if i >6

marker6 = rnarkerfi +1; for j = 1 :maxfa&marker5 if F(i-6.0-1)*100+1) >O 8 F(i&,(j-1)*100+1) -= 1 ... & F(i-6,Q-1)*100+1) -= 2 8 F(i-6,Q-1)*100+1) -= 3 ... & F(iô,(j-1)*100+1) -= 4 8 F(i-6,(j-1)*100+1) -= 5

F(i, 1+100arnarker6) = 6; for t = 1:numberofinteger

F (i,t+l+100amarker6) = F(i-G,t+Q-1)'lOO); end. marker6 = marker6+ 1 ;

end; end; end, if i >7

marker7 = rnarker6 + l ; for j = 1:maxfact-marker6 if F(i-7,(j-l)"l00+1) >O & F(i-7,(j-1)a100+l) -= 1 ... & F(i-7,o-1)"100+1) -= 2 & F(i-7,Q-1)*100+1) -= 3 ... & F(i-7,(j-1)"100+1) -= 4 & F(i-7.0-1)*100+1) -= 5 ... & F(i-7,(j-1)'100+1) -= 6

F(i,"i lOO"marker7) = 7; for t = 1:numberafinteger

F(i,t+1+100amarker7) = f(i-7.t+(j-l)"100); end; marker7 = rnarker7+1;

end; end; end;

end; y = F; end;

TURBO CODES

%'?!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Fundion that get the partition of integer with a specific nurnber of Oh partition s in it. It uses the rnatrix generated by part2(numberofinteger) %%'?hl-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

white i c 20001, while f(s,i) -=O,

y(b,a) = f(s,i); a = a+l; i = i+l; if f(s,i) == O

y(b.s+l) = a-1; b = b+1; a = 1;

end; end;

i = i+l; end; end;

Annex C: Matlab fiinctions used

y = fad(A)/fact(B)/fact(A-6); end; end;

TüRBO CODES

%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%Oh%%%%%%%%%%%%%% % F unction that return the polynomial raised to the power % given in the argument: polyA(power) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%%%Oh0h%%%%%%

function y = polypow(poly,power) Y = poly; for i = 2:power

yternp = zeros(1 .size(y,2)+size(poly,2)-1); for a = O:size(y,2)+size(poly,2)-2

for b = O:size(poly,2)-1 if a-b >= O & a-b c= size(y,2)-1

ytemp(a+l) = ytemp(a+l) + y(a-b+l)*poly(b+l); end;

end; end; y = ytemp;

end; end;

Annex C: Matlab fhctions used

%%%%%%%%%%%%%%%%%%%%%%%Oh%%%%%%%%%%%%%%%%% % Fatorial of an integer i %Oh%%%Oh%Oh%%oh%%%%%%%%%Oh%%Oh%%%%%%%%%%%%Qh%%%%

y = 1; for j =f :i

Y = j*y; end;

end;

if i CO y=o ;

end;

TURBO CODES