SOURCE-CHANNEL CODEC FOR A WCDMA BASEDMULTIMEDIA SYSTEM

by

Boris Backovic, B.Eng.Toronto, September 2005

A Project Report

presented to Ryerson University

in partial fulfillment o f the

requirements for the degree

Master of Engineering

in the Program of

Electrical Engineering

Toronto, Ontario, Canada, 2005

© Boris Backovic 2005

PROPrtrTYQF RYER3QW UBRAFY

UMI Number: E C 53005

All rights reserved

INFORMATION TO USERS

The quality of this reproduction is dependent upon the quality of the copy

submitted. Broken or indistinct print, colored or poor quality illustrations and

photographs, print bleed-through, substandard margins, and improper

alignment can adversely affect reproduction.

In the unlikely event that the author did not send a complete manuscript

and there are missing pages, these will be noted. Also, if unauthorized

copyright material had to be removed, a note will indicate the deletion.

UMIUMI Microform EC53005

Copyright 2008 by ProQuest LLC All rights reserved. This microform edition is protected against

unauthorized copying under Title 17, United States Code.

ProQuest LLC 789 East Eisenhower Parkway

P.O. Box 1346 Ann Arbor, Ml 48106-1346

The Journey of Success

W hen choosing the path to follow, I selected the road heading west.It began in the Forest o f Childhood, and ceased at the C ity o f Success.

M y bag w as packed full o f knowledge, but also some fears and some weights.M y m ost precious cargo was a vision o f entering the C ity ’s bright gates.

I reached an im passable river, and feared that my dream had been lost.B ut I found a sharp rock, cut down a tree, and created a bridge, which I crossed.

It started to rain, and I was so cold, I shivered and started to doubt.But I made an um brella out o f some leaves and kept all the cold water out.

The journey took longer that I had planned; I had no food left in my dish. R ather than starve before reaching my dream , I taught m yself how to fish.

I grew aw fully tired as I walked on and on, and I thought o f the weights in my pack.I tossed them aside, and I sped up again. Fear was all that was holding me back.

I could see the C ity o f Success, just beyond a small grove o f trees.At last, I thought, 1 have reached my goal! The whole world will envy me!

I arrived at the city, but the gate was locked. The man at the door frowned and hissed, “Y ou have wasted your tim e. I can’t let you in. Y our name is not on my list.”

I cried and I scream ed and I kicked and 1 shook; 1 felt that my life had ju st ceased. For the first tim e ever, I turned my head, and for once in m y life faced the east.

I saw all the things 1 had done on my way, all the obstacles I ’d overcom e.I couldn’t enter the City, but that d idn’t mean I hadn’t won.

I had taught m yself how to ford rivers, and how to stay dry in the rain.I had learned how to keep my heart open, even if som etim es it lets in some pain.

I learned, facing backwards, that life m eant more than ju st survival.M y success was in m y journey, not in my arrival.

Nancy Hammel

Abstract

The project deals w ith the operation o f a Source-Channel Codec for a W CDM A

Based M ultim edia System. The system is m eant to transfer and receive both digitized speech and still image signals. It uses a part o f the W CDM A technology to mix up the

transm itted signals throughout the im plem entation o f Direct Sequence Spread Spectrum

and C hip Sequencing m ethodologies. The W alsh code algorithm is used to ensure the orthogonality am ong different Chip Sequences.

O n the transm itter side the system first offers the form atting stage where both a speech

and a still image signal are digitized. The follow ing stage in the system exhibits a

significant degree o f data com pression applying appropriate com pression algorithm s: L em pel-Z iv-W elch for the speech signal and Huffman Code Algorithm for the still

image. These com pression algorithm s are im plem ented in the Source Encoder stage o f

the system . The system also provides basic EEC (Forw ard Error Correction) capabilities, using both L inear Block Code and Convolutional Code algorithm s introduced in the

C hannel Encoder stage. The goal o f these EEC algorithm s is to detect and correct errors

during the transm ission o f data due to the channel imperfections. At the W CDM A stage

the tw o signals are added together form ing an aggregated signal that is being transm itted through the channel.

O n the receiver side a digital dem odulator separates the aggregated signal to into two

signals using the feature o f the orthogonality o f vectors. Then the Channel D ecoder stage follow s, w here both signals, which have gotten corrupted during the transm ission through

the channel due to channel imperfections, are recovered. The imperfections in the channel

are sim ulated by random noise that is added to the aggregated signal in the W CDM A

stage o f the system . The last stage in the system, the Source Decoder stage, deals with the conversion o f the received signals from the digital to analog form and reconstruction o f

the signals in the sense that they can again be heard (speech) and seen (still image).

Each stage in the system is sim ulated using M ATLAB program m ing language. The

report is form ed o f three m ajor parts; the theoretical part where the theory behind each

stage in the system is explained, the exam ple part where applicable numerical exam ples are provided and analyzed for better understanding o f both the theory and the M atlab

code, and the result part where the M atlab results for each stage are analyzed.

II

Table of Contents

Abstract I

Chapter 1 : INTRODUCTION 1

Chapter 2: FORMATTING 7

2.1 Speech Form atting 72.1.1 Sam pling g2.1 .2 Q uantizing 92.1.3 Pulse Code M odulation (PCM ) 132.1 .4 M atlab Im plem entation o f Form atting 14

2.2 Im age Form atting 19

Chapter 3 : SOURCE CODEC 23

3.1 H uffm an Coding A lgorithm 233.1.1 H uffm an Encoding 253.1 .2 H uffm an D ecoding 283.1.3 M atlab Im plem entation o f Huffm an Coding A lgorithm 30

3.2 Lem pel-Z iv-W elch A lgorithm 323.2.1 M atlab Im plem entation o f Lem pel-Z iv-W elch A lgorithm . 37

Chapter 4 : CHANNEL CODEC 41

4.1 Channel Codes 414.1.1 Parity Check Codes 414.1.2 L inear Block Codes 434.1.3 L inear Block Code Encoding 454.1 .4 L inear Block Code Decoding 464.1.5 M atlab Im plem entation o f Linear Block Codes 53

4.2 C onvolutional Codes 544.2.1 Convolutional Code Encoding 56

4.2.1.1 Impulse Response o f the Convolutional Encoder 574.2.1.2 Polynom ial Representation o f Convolutional Encoding 58

Ill

4.2.1.3 State D iagram 594 .2 .1 .4 Trellis D iagram 60

4.2.2 C onvolutional C ode D ecoding 614.2.2.1 V iterbi D ecoding A lgorithm 62

4.2.3 M atlab Im plem entation o f the (2,1,4) Convolutional C ode 66

Chapters: WCDMA 67

5.1 W CD M A Technology 675.1.1 D irect Sequence Spread Spectrum 685.1.2 C ode D ivision M ultiple Access 70

5.1.2.1 W alsh O rthogonality 735.2 M atlab Im plem entation o f W CD M A 77

Chapter 6: RESULTS 80

Conclusion 92

References 94

Appendix A: Matlab Simulation Files 96

Appendix B: Matlab Example Files 135

IV

List of Figures

Figure 1 : Source-Channel Codec for a W CDM A Based M ultim edia System . 3Figure 2: Frequency Spectrum o f the Signal ‘speech.w av’. 15Figure 3: Signal ‘speech .w av’ in Time Domain. 16Figure 4; Grey Level Im age o f a Diagonal Black Line. 20Figure 5; PD F function o f the Black Diagonal Line. 22Figure 6; A H uffm an Tree. 26Figure 7; Single Parity Check Code. 42Figure 8: R ectangular Parity Code (I = 4, J = 6). 43Figure 9: D ecoding Table for Linear Block Codes. 48Figure 10; (2 ,1 ,4) Convolutional Encoder. 55Figure 11 : Trellis D iagram Representation o f (2,1,4) Convolutional Encoder. 60Figure 12: The D ecoder Trellis Diagram (with H am m ing D istances). 64Figure 13: The D ecoder Trellis Diagram after t=5 Time U nits. 65Figure 14: Direct Sequence Spread Spectrum. 68Figure 15: R ight Shifted G enerator Polynomial. 69Figure 16: Speech Data. 80Figure 17: Still Im age Data. 80Figure 18: Frequency Spectrum o f Speech Signal. 81Figure 19: PD F o f Q uantization Error. 81Figure 20: Frequency o f U sage o f Q uantizer Levels. 83Figure 21 : PDF o f the Image. 84Figure 22: B ER vs. SN R for (12,8) Linear Block Code. 88Figure 23: Received Speech Data. 91Figure 24: Received Im age. 91

List of Tables

Table 1: G enerator Polynom ials for Convolutional Codes. 56Table 2: Im pulse Response o f the (2,1,4) Encoder. 57Table 3: State D iagram o f the (2,1,4) Convolutional Encoder. 59Table 4: H um m ing D istance Used for D ecoding Convolutional Codes. 61

Chapter 1: INTRODUCTION

Even though nowadays personal com puters are the dom inant Internet access

client, m obile phones and handheld devices will very soon become the m ajor source o f

In ternet connections. Unlike the first-generation (IG ) o f mobile com m unication systems

designed m ostly to carry the voice application traffic, the third-generation (3 0 ) o f

com m unication system s prom ises unparalleled access in ways that have never been

possible before. Internet access, voice com m unication over Internet protocol, and

transm ission o f still and m oving images are just a few o f the com m unication techniques

used in the “alw ays-on” type o f access that 3 0 developers have been working on. These developers envision the picture o f an ordinary user receiving live music, conducting

interactive w eb sessions, and having sim ultaneous voice and data access with multiple parties at the sam e time. U ndoubtedly, this kind o f access requires a special technology

so that com puters, handheld devices, or any other appropriately geared com m unication

device, m ay all be connected anytime, anywhere. W ideband code division m ultiple access (W C D M A ) technology is one o f the main technologies for the im plem entation o f

the th ird-generation o f com m unication system s that allows very high-speed multim edia services to be performed. W CDM A will support high rate high quality data, multimedia,

stream ing audio, stream ing video, and broadcast type services am ong users. In addition, W C D M A designers contem plate that broadcasting, mobile com m erce, games, interactive

videos, and even virtual private networking will be possible in a near future, all from

small portable devices.

These new and exiting possibilities o f connecting people, businesses, and even industries

all over the world by using rapidly developing technologies and standards make im possible to imagine what modern living would be like w ithout access to reliable,

econom ical, and efficient m eans o f com m unication. Consequently, an ordinary person

m ay g e t easily confused and scared away with concepts o f all these com m unication

technologies and their im plem entations. However, the sole purpose o f com m unications has not changed much since the time o f Guglielm o M arconi who, in 1897, had dem onstrated rad io ’s ability to provide continuous contact with ships sailing the English

channel. Even today, a com m unication is still a way o f conveying or transm itting

inform ation from one place to another. The definition sounds sim ple and does not tell us

m uch abou t w hat is involved in information transm ission as a mean o f com m unication

betw een tw o o r m ore parties. The answer would probably turn out to be more com plicated then it m ight first appear and will, in fact, form a basis for this project.

A sim ple answ er would be that the transm ission o f inform ation requires some kind o f

signals in o rder to convey a m essage to the other party. The signal could be a voltage, but

once the voltage is established there is not much availability to convey information unless w e change the value o f the voltage. The next step would be to attach the voltage (a

battery) to a variable resistor creating more variations o f possible voltages that in addition

allow s us to associate more parts o f information with different voltage levels. I f a signal

varies w ith tim e (120V, 6GHz sinusoidal voltage) then variations o f the signal may be created by either changing the am plitude or the phase o f the sinusoidal voltage.

A t this point we will make a rough aberration between two types o f signals that could be

used in a process o f inform ation transmission. If a signal is a continuous electrical signal that varies with tim e, it is considered to be an analog signal. On the other hand, if a signal

is non-continuous, it is said to be a digital signal. Furthermore, analog signals can take on

any value from an infinite set o f values in a specific range while digital signals take on

one o f tw o possible am plitude levels called nodes. Digital signals consist o f pulses or

digits w ith d iscrete levels or values. The value o f the signal is specified as one o f two possib ilities such as either I or 0, high or low, true or false and so on.

In the process o f transm itting information, all signals bearing information are

contam inated by noise. Noise is generated by many natural and man-made events that introduce errors during the transm ission and create serious problems for the party receiv ing the inform ation to be able to properly recover transm itted messages. W hen an analog signal is affected by noise it is much more difficult to regenerate that signal than it

is to recover a signal that is o f the digital origin. W hen affected by noise a pulse

(representation o f a digital signal) degrades as a function o f line length. Before the pulse

is degraded to an am biguous state, it could be am plified by a digital am plifier that recovers the pu lse’s original shape. In the case o f analog signals, once an analog signal is

distorted , the distortion can never be com pletely rem oved by the process o f am plification.

B esides being m ore resentful to noise, there are many other features that make a digital

signal m ore suitable to convey information com pared to an analog signal. Some o f them

are: reliability o f the system , flexibility o f the hardware, and pricing o f the

im plem entation. On the other hand, a digital transm ission would typically require a

g rea ter system bandwidth to com m unicate the same information in a digital format as

com pared to those in an analog format. All this m akes a digital signal to be a signal o f choice for the com m unication when inform ation is to be transmitted in a reliable, robust,

and relatively not expensive manner.

FORMATTING:

Channel A

(Speech)

Sampling ^ Quantizing^ Pulse Code

Modulation (PCM)

SOURCEENCODING:

% Lempel Ziv

Welch Algoiitlun

(LZ\\^

CRANNELENCODING:

WCDM4:

4= LmeaiBlock

(LBC)

m W S N O T T E R

EORE'IATTING:

Channel B(Image)

* PDF

SOURCEENCODING:

HuffmanCoding

Algoiitlun(HCA)

SOURCEDECODER:

Chamiel A

(Speech)

4= Lempel Ziv

Welch Algorithm

(LZW)

RECELER 4 -

SOURCEDECODER:

Cliaimel B( l in a g e )

HuffmanCodhig

Algoridun(HCA)

CHANNELDECODER:

LmeaiBlockCode(LBC)

^ Duect Sequence

Spied Spectimn

(DSSS)

CodeDivisionM ultipleAccess(CDMA.)

Pseudo Random Sequence Geiieiatoi

(PRSGl)

CRANNELENCODING:

WCDMA-

Convolutional

CHANNELDECODER;

Dh ect Sequence

Spied Spectimn

(DSSS)

H' Code Division Midtiple Access (CDMA)

Pseudo Random Sequence Genei-atoi

(PRSG2)

Z

Dii ect Sequence

Spied Spectimn

(DSSS)

T

ViterbiAlgoiitlun

^ P se u d o Random ( Sequence Generator V (PRSGl)

'H Dhect Sequence

Spied Spectimn

(DSSS)

iK CodeDivisionMultipleAccess

(CDMA)

Pseudo Random Sequence Generator]

(PRSG2)

Figure 1: Source-Channel Codec for a W CDM A Based M ultim edia System.

This project sim ulates the basic elem ents o f a digital com m unication system shown in

Figure 1. Keeping in analogy w ith the introduction given in the beginning o f this report, th is system may be considered as a prim itive system o f the third generation (3G) o f com m unication system s. Here are the reasons:

a) It is a digital system ; m eaning that regardless o f the format o f the input

m essage, the system will conventionalize the message into a stream o f binary digits. An exception applies only in the case when the input message is

already in the digital format. This process o f transform ing any form o f input

m essage into a stream o f bits (digital form ) is called form atting and will be discussed in details in C hapter 2;

b) It is a m ultim edia system; m eaning that it processes more than one type o f

m edia at the sam e tim e (speech and still image). The system is tw o channeled,

w here channel A is carrying a speech data while channel B is dealing with a

still image data. In addition, the system is considered to be a simplex transm ission system. In many cases it is desirable to maintain tw o way

com m unications, o r at least be able to send a message back to its origin for possible verification, comparison, or control. To simply implem ent this type

o f system , called full-duplex, another set o f blocks (stages) exactly the same ones as the ones shown in Figure 1 should be incorporated into the system but

in a reverse order; That second set o f blocks would be responsible for the transm ission o f information from the destination back to the source;

c) It uses W CD M A technology in the process o f transm itting data from the

source to the destination. At this point, it would be beneficial to say that W CD M A is more than Just a technology; it is a standard that establishes and defines various com m unication protocols and procedures used during a

com m unication session. In this project m any o f these protocols and

procedures will be ignored for the sake o f simplicity. However, one that is

considered as the main representative o f W CDM A standard, called M edium A ccess Control (M AC), will be fully presented and simulated. Medium access

control defines how digital signals from different sources use the same allocated frequency spectrum to convey different information to different

recipients. M AC is discussed in C hapter 5;

A fter form atting, the next block in our digital com m unication system is the source

encoder stage. The source encoding is the process o f rem oving redundant bits from the

sequence o f bits carrying information. In other words, the source encoding com presses

data in a w ay tha t only necessary hits that make up the original information are processed

fu rther th rough the system. In Chapter 3, two source encoding techniques are

dem onstrated . One for the channel A that deals with the source encoding o f a speech

signal form atted into a stream o f hits in the previous stage (formatting stage). That source

encoding technique is called the Lem pel-Ziv-W elch technique. The other technique

called H uffm an Coding A lgorithm will he used for the channel B to encode the still

im age data that is, at this point in the process, already in the binary form. The chapter will

also introduce the w ays how data encoded by the two encoding techniques can he

decoded on the receiver end. Since a source encoder and source decoder generally

opera te in pairs, this com bination o f a source encoder and decoder is called a source codec (coder-decoder).

The stage fo llow ing the source encoding stage is the channel encoding stage. The purpose

o f the channel encoder is to introduce, in a control m anner, some redundancy in the

b inary inform ation stream that can he used at the receiver to overcom e the effects o f noise and interference encountered in the transm ission o f the bits through the channel. In

o ther w ords, the system will now add some redundant hits to the binary information for the purpose o f detecting and correcting some o f errors occurred during the transmission.

The redundan t bits are formed and organized in such a way that no response hack from

the receiver is needed in order to detect and correct errors. This type o f error control is called forw ard error correction (FEC) and requires a one-w ay link only.

In C hap ter 4, tw o types o f channel encoding techniques are presented. One, for channel

A, called the L inear Block Code technique and the other one, for the channel B, called the C onvolu tional Code technique. Along with both techniques their associated decoding

procedures, such as Viterbi decoding algorithm for Convolutional codes, will be discussed and im plem ented. Similarly as for a source encoder and decoder, a channel encoder and decoder operates in pairs as well, thus form ing what is called a channel

codec.

C hap ter 5 entirely deals with the access control o f the medium and, as it has earlier been

m entioned, presents the way o f m anaging this issue as defined by the W CDM A standard (technology). A digital modulation technique called Direct Sequence Spread Spectrum is

presented along w ith the Code Division M ultiple A ccess technique. The role o f each one

o f th em in the channelization process is discussed.

The last chapter, Chapter 6, provides an overview o f the results obtained from the

sim ulation in such a w ay that the input/output binary sequences to/from each block in the

system s are exam ined and discussed. The chapter also gives a conclusion on the project

as well as recom m endations on how to improve the efficiency and efficacy o f the system presented and sim ulated in this project.

F o r the end o f this introductory section, 1 would also like to mention one important thing

that w ould becom e m ore evident once a reader starts reading through the incoming pages.

N um erical exam ples! Throughout my entire education as an undergraduate as well as a

graduate student I had always felt that any engineering theory or topic introduced in a class w ould have made much more sense to me if it was accom panied by a relevant

exam ple. T hat is why at the end o f each chapter in this report an example associated with

the algorithm or technique presented in that chapter is worked out. I believe that these

exam ples, besides clarifying the theory, will also help a reader to understand how the

presented algorithm or technique is im plem ented in Matlab.

Chapter 2: FORMATTING

2.1 Speech Formatting

In general, signals in the com m unication theory could be considered either as analog o r digital ones. Signals in digital form are also told to be discrete signals.

H ow ever, bo th types o f signals bear information that is conveyed from the source to the

destination through different types o f medium(s) (e.g. air, copper wire, etc.). Besides analog and digital types o f information mentioned above, the data at the source, also

called source inform ation, may be found in a textual form. Such kind o f source data is term ed as tex tual information.

In the digital com m unication systems the first and essential step in the process o f

conveying inform ation from the source to the destination is to format the source inform ation regardless o f the form that information is in. That essentially m eans that the

source inform ation has to be processed in a certain way that will make it suitable for fu rther digital processing. L et’s consider all three possible forms o f source information

that have been m entioned earlier and the way they are formatted in order to be

com patib le w ith the next stages in a digital com m unication system.

The sm allest headache would give us an information source, or data, that is already in the

d igital form . It will sim ply ju st bypass the form atting stage and proceed to the next

processing step. The problem begins with the fact that the most data in com m unications

is e ither in tex tual or in analog format. Data in textual format would usually be encoded w ith one o f several standards such as: ASCII (Am erican Standard Code for Information

In terchange), EB CD IC (Extended Binary Coded Decimal Interchange Code), Baudot, and H ollerith . The aforem entioned standards transform the textual data into a digital

form at.

N ow , w e com e to analog information at the source and we are interested how it can be

m ade su itab le for further digital processing. Simply said, we would like to determ ine the necessary conditions which will allow us to change analog information to digital one w ithou t loss o f inform ation. As a criterion o f how well the process o f converting the

analog inform ation into the digital one can be carried out, we use an important condition

that the orig inal inform ation can be fully reconstructed by using reversible, digital to

analog processing steps.

In essence, analog inform ation is form atted using three separate steps: sampling, quantization, and coding.

2.1.1 Sam pling

The link betw een an analog signal and the corresponding digital signal is given by

w hat is know n as the sam pling theorem . The sam pling theorem states the following:A real valued band limited signal having no spectral com ponents above a frequency o f

fm (H z), know n as the m axim um frequency o f the analog signal, is determ ined uniquely by its values at uniform intervals spaced no grater than Tg seconds apart, where Tg is:

( 0

This statem ent is a sufficient condition that an analog signal can be reconstructed

com pletely from a set o f uniformly spaced discrete samples in time. The output o f the

sam pling process is called the pulse amplitude m odulation (PAM ) because the successive output intervals can be described as a sequence o f pulses with amplitudes derived from

the sam ples o f the analog signal. I f equation stated in (1) is applied, all replicas o f the original spectral density are ju st tangent to each other and an ideal low pass filter can be

used to reconstruct (theoretically) the original analog signal from the sampled version o f

that signal. H ow ever, if the sam pling interval Tg becom es slightly larger than the right side o f the equation (1), then there will be an overlap o f spectral densities and the original signal will not be successfully reconstructed from its sampled version with the help o f an ideal low pass filter. In order to avoid the situation described in previous sentence it is

essential and absolutely necessary in the process o f sampling o f an analog signal that:

' ' ' i t -

The equation given in (2) is a m athem atical interpretation o f the sampling theorem. The

m axim um tim e interval Tg is called the N yquist interval. I f we want to see the equation

(2) as the relationship betw een the maximum frequency o f the analog signal f ^ and the sam pling frequency fg, where fg is reversely proportional to Tg then we arrive to the

fo llow ing equation:

(2)

The equation (3) given in term s o f the sampling and maximum frequencies is called the

N yquist Sam pling Rate. In practice, the full potential o f the sam pling theorem usually

cannot be realized and the equations (2) and (3) serve as upper bounds on actual

perform ance. O ne reality fact that we are faced with in dealing with the sam pling

theorem is that w e cannot build an ideal low pass filter. We can only build a low pass

filter w ith as fast an attenuation rate as possible. One thing that we can do to overcome

the inability to have an ideal low pass filter is to increase the sampling frequency to allow

som e frequency space before the next frequency replica o f the sampled analog signal appears.

A nother reality fact being responsible for the sam pling theory not being used in its full

potential is the fact that a time limited signal is never strictly band limited. When such an

analog signal is sam pled, there will always be some unavoidable overlap o f spectral

com ponents. Furtherm ore, in reconstructing the sampled version o f the signal, frequency

com ponents originally located above one half o f the sampling frequency will appear

below this point and will be passed by the low pass filter. This is known as aliasing and results in a d istortion o f the signal. The effects o f aliasing can be partially elim inated by applying as good as possible low pass filtering before sampling and by sam pling at rates

g rea ter than the N yquist rate.

A n interesting question arises here; if we want to apply the sampling theorem to bandpass

signals, do w e still have to obey the rule given by the equation (2), stating that we have to sam ple bandpass signals at tw ice the highest frequency? The answer would be no, since

the m inim um sam pling rate depends on the bandw idth o f a low pass signal rather then on its h ighest frequency. In the case o f low pass signals these tw o conditions coincide.

H ow ever, w hen sam pling a bandpass signal we should use a minimum sam pling rate in

the range betw een 2 and 4 times the bandwidth o f the signal. This minimum rate

requirem ent for bandpass signals approaches the limit o f tw ice the bandwidth as the

center frequency o f the signal increases.

2.1 .2 Q uantization

A fter the sam pling, there comes the second step in formatting o f an analog signal;

quantization. Q uantization or quantizing is the task o f m apping samples o f an analog

signal, obtained through the process o f sampling, to a finite set o f amplitudes. To make

an analogy w ith the sam pling process, the process o f quantization is related to the y-

ordinate sim ilarly as the process o f sampling is related to the x-axis. The sim plest

quan tizer perform s m apping o f each sample o f the sampled analog signal to one o f the

10

predeterm ined quantizer levels. I f those predeterm ined quantizer levels are equally

spaced then we say that the quantizer is a linear quantizer. Similarly, if the levels are not

equally spaced then we say that the quantizer is a nonlinear quantizer. Since this project deals with the linear quantization only, the further discussion on the process o f the

quantization o f a sam pled analog signal will be strictly limited to a linear quantizer and

its characteristics. H ow ever, it is important to m ake a comm ent that the nonlinear

quantization provides much better the signal to noise ratio (SNR) than the linear

quantization does. This is particularly evident in speech com m unication where very low speech volum es predom inate 70% o f the time.

A linear quantizer is the universal form o f the quantizer in a sense that it makes no

assum ption about the amplitude statistics and correlation properties o f the input analog signal. The only tw o conditions that have to be known in order to implement a linear

quantizer are: the dynam ic range o f the sampled signal (DR) and the number o f bits that

each sam ple is represented. The dynamic range is defined as:

DR = [max_ sig - min_ sig], (4)

w here 'm ax sig ' and 'm in sig ' are the maximum and minimum values o f the sampled analog signal. The second condition, the number o f bits that each sample in the sampled version o f the analog signal will be represented with, is directly related to the num ber o f

levels o f the desired quantizer. The relation between the num ber o f bits for each sample

representation and the num ber o f levels o f the quantizer is given by:

L = 2'^. (5)

In equation (5) L is the num ber o f the levels o f the quantizer and R is the num ber o f the

bits that each sam ple is represented with. Now , by taking a logarithm with the base 2 to

each side o f the equation (5), we get another relation between the num ber o f bits for each

sam ple representation and the num ber o f levels o f the quantizer:

R = log2 L. (6)

At th is point it is essential to reveal a restriction and an observation associated with the

process o f quantization. The restriction is that the equations (5) and (6) are valid only if w e intend to apply the fix length representation o f each sample. In the case o f variable

length representation, the equations (5) and (6) are no longer valid. M ore overview and

d iscussion on fixed and variable length representations o f sam ples will be given later in

11

C hapter 3, when we deal w ith the Hutfrnan code algorithm. The observation to be

revealed is that once quantized, the instantaneous values o f the analog signal can never be exactly reconstructed again.

O nce we establish a desired num ber o f levels for the quantizer, we can easily determine

the size o f each quantizer level, usually called the quantile interval. A rem ainder that a

quantizer has all its quantile intervals o f the same size only if that quantizer is a uniform quantizer. The size o f each quantile interval q is given by the following equation:

q _ (max s ig -m in sig)

A linear quantizer w orks in a very sim ply way. Inputs to the quantizer are samples o f an

analog signal obtain through the process o f sampling, while the outputs from the

quan tizer are predeterm ined values o f the quantizer’s levels that the samples are mapped

to. The difference betw een the input and output o f the quantizer is called the quantization error. M athem atically, the quantization error is represented as:

e ( n ) ^ x ( n ) - x ( n ) , (8)

w here x(n) represents the input vector containing the signal samples, x (n ) is the

quantized output vector, while e(n) is the error vector. The quantization error vector is

also referred to as the quantization noise. Under the restriction that the input signal has asm ooth probability density function (PDF) over the quantization interval, it can be

assum ed that the quantization errors are uniformly distributed over the quantization

interval ranging from -q /2 to q/2. Each probability density function must be greater or

equal to zero and m ust satisfy the following condition:

q / 2

| f ( x ) d x = l. (9)

- q / 2

In o rd er to satisfy the equation (9), the probability density function o f the quantization

erro r p(e) m ust be equal to 1/q in the interval from -q/2 to q/2. Outside that given interval

p(e) m ust be equal to zero. Here is m athem atically interpretation o f what has ju st been

said:

1 2

p(e) =

1 . 1 1— for — < e < —q q q0 otherwise

( 10)

A usefu l figure o f m erit for a uniform quantizer is the error variance, where the error variance is defined as:

q / 2

= | ( e - m x ) ^ p(e)-de,- q / 2

( 1 1 )

w here m ^ is the erro r m ean that is equal to:

m .

q / 2 q / 2

- e p ( e ) d e = e — de = —■ J J a a

- q / 2 - q / 2

q / 2

= 0.

- q / 2

(12)

N ow , by placing the error mean m ^ from the equation (12) into the equation (11) we get the error variance:

q / 2

<3^= J e ^ - p ( e ) d e =

- q / 2

q / 2 2_q _

- q / 2

(13)

The erro r variance is also com m only known as the noise power. Similarly, the signal

variance (signal pow er) is defined as:

(js = j x ^ p ( x ) d x . (14)

and can be substituted with the expression for the peak pow er o f an analog signal

norm alized to 1 Q:

P s = V s I s = V s ^ = Vs2 = f Vp p l JhlS]Rs I 2 J I 2 j (15)

13

The ratio betw een the signal variance (signal power) given in the equation (15) and the

noise pow er given in the equation (13) yields the quantization signal to noise ratio,(SNR)q:

L2-q2

= ^ = (16)noise power q

12

The (SNR)q is usually given in the units o f decibels [dB] and is obtained by applying the fo llow ing conversion formula:

(SNR)q[dB] = 10-logio(SNR)q. (17)

From the equation (16) we conclude that the signal to noise ratio for a uniform quantizer

solely depends on the num ber o f levels L. Since from the equations (5) and (6), the

num ber o f levels L is directly proportional to the num ber o f bits used to represent each sam ple, thus we can also say that the signal to noise ratio depends on the number o f bits used to represent each sample. The more bits are used to represent a sample, the better the

signal to noise ratio. It is easily proven that for each additional bit used to represent a

sam ple (e.g. increase from a 5-bit sample representation to a 6-bit sample representation),

the im provem ent in the (SNR)q is approxim ately about 6 dB. A quick approxim ation o fthe (SNR)q for a quantizer is to multiply the num bers o f bits that each sample isrepresented with by 6 dB. However, the real quantization (SNR)q is much sm aller due to

the im perfections in the quantizer itself (linear vs. nonlinear quantizer). As the number o f

levels L approaches infinite, the signal approaches to its form before the quantization (PA M form at) and the (SNR)q approaches to infinity. In other words, with the infinite

num ber o f quantization levels, ultimately there is no quantization noise.

2.1 .3 Pulse Code M odulation (PCM)

The next step in the process o f converting an analog signal into a digital one is to

assign a digital value for each quantile interval in such a way that each interval has a one to one correspondence with the set o f real integers. This is called the digitization. The

p rocess o f digitization reduces the original analog signal to a set o f digits, at the successive sam ple tim es, m apped into L quantizer’s levels. Each sample o f the original

analog signal is assigned a quantization level (quantile interval) closest to the value o f the

14

actual sam ple. The digits are expressed in a coded form. The most comm on code used for

th is purpose is a binary code where each digit is represented as a com bination o f zeros

and ones. Each binary 1 is further represented b) a pulse and each binary 0 is represented

by the absence o f a pulse. Thus, instead o f transm itting the individual samples, a

com bination o f zeros and ones, binary pulse code, is sent at each sample tim e carrying

the intended inform ation in digitized form. Com m unications systems m aking use o f this

kind o f data representation during the transmission are commonly called pulse code m odulation system s; PCM systems.

W aveform carrying inform ation can be transmitted even more efficiently if we represent them as sequences o f transitions between upper and lower voltage levels. When the

w aveform is at the upper voltage level it represents a 1. Similarly, w hen the waveform is at the low er voltage level it conveys a 0. There are many PCM waveform types classified

in m any groups. Here only a few will be mentioned: N on Return to Zero (NRZ), Return

to Zero (RZ), Phase Encoded, and M ultilevel binary. The most comm only used are NRZ

PC M w aveform s. The reason why there are so many different types o f PCM waveform s lies in d ifferences in perform ances for different waveform coding schemes. Some schem es are better in perform ing error detections, some are better in correcting data

errors, some schem es again are better in increasing the efficiency o f bandwidth

utilization. Certain types o f PCM waveform s are more immune than others to noise. All this contributes that so many o f PCM waveform schemes are used, but the decision which

one and w hen to be used greatly depends on required perform ances and characteristics o f

the digital system used.

2 .1 .4 M a tla b Im p lem en ta tio n o f F o rm a ttin g

The form atting process explained in preceding sections is simulated in M atlab

w ith three M files: ‘Sam pling.in ' and ‘Q uantization.m ’ files that describe the sampling and quantization processes and ‘PC M .m ’ file that deals with the process o f digitization,

w here each quantized sam ple is coded to a corresponding binary word. In the ‘S am pling .m ’ file an analog signal in the form o f a sound file ( ‘speech.w av’) is loaded in

the M atlab environm ent. M atlab provides the command ‘w avread’ which performs sam pling autom atically w hen the sound file is loaded into the environm ent. The standard

sam pling rates for PC based audio hardware are 8,000, 11,025. 22,050, and 44,100 sam ples per second. M ono signals are returned as one column matrix, while stereo

signals are returned as two column matrices. The first colum n o f a stereo audio matrix

corresponds to the left input channel, while the second column corresponds to the right

input channel. The ‘w avread’ com m and, after being executed, returns two output

15

variables, the sam pled data and the sample rate. The sample rate used for the speech file

used in th is project was 22,050Hz. Let s see how the sam pling frequency is obtained.

I f w e plot the frequency spectrum o f the speech signal as shown in Figure 2 we see that

the m axim um frequency o f the signal is 11,025 Hz. Now , if we recall the equation (3) stating tha t the N yquist Sam pling Rate must be equal or greater than the maximum

frequency o f the signal, it is clear why the sample rate that our signal is sampled with is 22 ,050 Hz.

1800

1600

1400

1200

1000

BOG

600

400

200

S p ec tru m of th e S p e e c h Signal

1 ^1 . ;..6 8

F requ en cy [Hz]10 12 14

X 10

Figure 2: Frequency Spectrum o f the Signal ‘speech.w av’.

The sam pled data contains 1 10,033 samples. That num ber can be obtained through a sim ple calculation where the num ber o f samples is equal to the product o f the duration o f

the signal (in seconds) and the number o f sam ples for 1 second. W e already know that the num ber o f sam ples for 1 second is equal to the sam pling frequency, which is 22,050, and

w e can conclude from the Figure 3, which shows the ASCII representation o f the speech signal, that the duration o f the speech signal is 4.9902 seconds. A simple calculation gives us the num ber o f samples that the original speech signal is represented after the

sam pling process;

4.9902 X 22,050 = 110,033 samples.

16

ASCII Version of The Speech Signal

2 2.5 3time [sec.]

Figure 3; Signal ‘speech .w av’ in Tim e Dom ain.

In the ‘Q unatiz ing .m ’ file, as per earlier discussion (the equations (4), (5), and (6)), we

firstly have to specify the dynam ic range o f the sam pled signal as well as the num ber o f

levels th a t ou r quantizer will have. The M atlab com m ands, ‘m ax’ and ‘m in’, allow us to obtain the dynam ic range o f the sam pled signal (see the equation (4)). The num ber o f

levels L is chosen to be 256 since we deal w ith a speech signal. Speech signals in general requ ire 8 bits per sam ple since it is the m inim um num ber o f bits that can allow us to hear

o rig inal speeches in via their quantized versions. A pplying the equation (5), we see how

the num ber o f quantization levels is obtained.

N o w w hen w e know the dynam ic range o f the sam pled signal and the num ber o f levels

fo r the quantizer, we can calculate the quantization step q that is also the size o f the

quan tile interval. O nce again, since w e deal w ith a uniform quantizer each one o f 256

quan tile in tervals w ill have the sam e size. To obtain the value o f q we sim ply apply the

equation (7).

U pon its execution , the M atlab com m and ‘quantiz’ produces as the output tw o variables

fo r each sam ple o f the sam pled signal; the quantization index ‘indexl and quantized

ou tp u t value ‘q u a n ts l ’:

[indexl, quant; si ] — quant, iz ( sig, part it ionl, codebookl ) ,

17

w here ‘p a r t i t io n !’ is a real vector w hose elem ents are values o f 256 quan tizer’s levels

assigned to each sam ple. The elem ents o f the ‘partition! ’ vector m ust be given in strictly

ascend ing order. The input variable ‘codebookl ’ is a vector codebook that prescribes a

value fo r each partition in the quantization and its length exceeds the length o f the

p a rtitio n ! vecto r by one. As m entioned earlier, the output variables are; ‘ index 1 ’ and ‘q u a n ts ! ’. I f the ‘partition! ’ vector has the length o f n, then the ‘index! ’ vector is a co lum n vec to r w hose k^^ entry is given by:

index (k ) =0 if sig(k) < partition( 1)

m if partition( m) < sig(k) < partition( m + 1 ).

n if partition( n) < sig (k)

T he ou tpu t variable ‘quants 1 ’ is a row vector whose length is the sam e as the length o f the input sam pled signal. The row vector ‘quants 1’ contains the quantization o f the

sam pled signal based on the quantization levels and prescribed values. The ‘quants! ’ is re la ted to the ‘codebook l ’ and ‘index! ’ variables by the follow ing equation:

quants(k) = codebook (index(k) +1).

In th is equation , k takes on integer values between 1 and the length o f the sam pled signal.

T he variab le ‘index l ’ contains elem ents (decim al num bers) that represent the m em bersh ip o f each sam ple to one o f 256 quantization levels. In other words, those

decim al num bers represent which level L each sam ple has been assigned to. Any num ber fro m 0 to 255 is a valid num ber that a sample can be assigned to. The task o f ‘PC M .m ’

M at lab file is to convert the decim al index num ber, indexl (k), o f each sam ple to a related

co d e w ord . Since code w ords are supposed to be in the binary form, thus each code w ord

is c rea ted as a com bination o f binary ones and zeros.

T he conversion from decim al to binary is perform ed by the ‘convert2binary .m ’ M at lab

file. T he code does the following: It takes each elem ent o f the ‘index! ’ vector and keeps

d iv id in g the value by 2 until the result is 1, keeping the track o f rem ainders for each d iv ision . W hen 1 is reached as a result o f dividing, the rem ainders are appended to the

resu lting 1, form ing the binary representation o f the decim al value. Here is a quick

exam ple:

L e t’s co n v e rt the num ber 109 to its binary representation:

18

1 0 9 :2 = 5 4 (1 ),

54 : 2 = 2 7 (0 ) ,

27 : 2 = 13(1),

13 : 2 = 6 ( 1 ),6 : 2 = 3 (0),3 : 2 = 1 ( 1 ).

T he resu lting b inary representation o f the num ber 109 is: I 1 0 1 1 0 1 . Here is also the

resu lt obtained by the M atlab file ‘d 2b .m ’ file, w here the ‘d2b .m ’ file is the m odified

version o f the ‘convert2b inary ’ and can be used separately from the main ‘PC M .m ’ file tha t perfo rm s the process o f d ig itization ( ‘d2b .m ’ is listed in the A ppendix B section o f th is report):

ans = 1 1 0 1 1 0 1 .

T here is now one m ore th ing that has to be taken care of. In the equations (5) and(6 ) w e have given the relation betw een the num ber o f levels o f the quantizer and the num ber o f bits that each code w ord should be com posed of. Since we use a 256 level

quan tizer in this project, by apply ing the equation ( 6 ) we obtain that each code word

should consist o f 8 bits. H ow ever, we know that only the num bers from 128 to 255 need 8 bits fo r their b inary representation. So, what happens with the num bers from 0 to 127

th a t do not need all 8 bit for their binary representations (e.g. the num ber 109 needs 7 bits, see the exam ple above). W ell, to have each num ber in the range from 0 to 255

rep resen ted by 8 bits, w e have to add the so-called ‘m issing ' bits. For instance, thenum ber I0 9 ’s b inary representation is a 7 bit com bination o f ones and zeros: 1 1 0 1 1 0

1. To obtain a 8 bit b inary representation we have to concatenate one 0 at the beginning,

so th a t the resulting 8 bit representation is: 0 1 1 0 1 1 0 1. The file ‘convert2binary .m ’

takes care that every num ber in the [0,255] interval is represented with 8 bits creating so- called a fixed length representation o f code words.

A t th is po in t w e also have to create a stream o f bits m ade up o f the binary representation

(code w ord) for each sym bol in the inform ation that is to be transm itted. The code words

are p lace in the stream in the FIFO m anner, m eaning that the code word for a sym bol that

is to be sen t ou t first is placed at the beginning o f the stream , the second code word is

appended on to the first one and so on until all sym bols from the information are

p rocessed .

19

2.2 Image Formatting

M atlab sto res a gray level image in the form o f a tw o-dim ensional matrix, where each elem ent o f the m atrix corresponds to a single pixel in the displayed image. For

exam ple, an im age com posed o f 200 rows and 300 colum ns o f d ifferently colored dots

w ould be sto red in M atlab as a 200 by 300 m atrix. Som e images, such as RGB (Red-

G reen-B lue) im ages, require a three-dim ensional m atrix to be stored in, where the first

plane in the th ree d im ensions represents the red p ixels’ intensities, the second plane

represen ts the green p ixels’ intensities, and the third plane represents the blue p ixels’ in tensities.

I f there is a need to convert a gray level image into a color level image, a straight forward

conversion is used. T hat is exactly what has been done in this project. A color image is

converted into gray level im age in order to save com putational tim e as well as m em ory

space. T ha t m eans th a t instead o f processing three tw o-dim ensional m atrices for a color im age, w e w ill be processing only one 2 -dim ensional m atrix for a gray level image.

M em ory w ise tha t m eans that instead o f using 3 bytes o f m em ory storage for each

co lo red pixel, w e w ill need only 1 b>te to store the num ber representing one pixel. The

conversion w ill be carried by using the following formula:

A = 0 .2989 i(: ,: , l )+ 0.5870 i(:,:,2) + 0.1140-i(:,:,3). (18)

In the equation (18), A is a 2-dim entional m atrix representing a gray level image, I(:,:,l)

is the ‘R ed In ten sity ’ part o f the color image, 2) is the ‘G reen Intensity’ part o f the co lo r im age, and ï(:,:,3 ) is the ‘Blue Intensity’ part o f the co lo r image. The coefficients

th a t the in tensities are m ultiplied with in the equation ( 18): 0.2989, 0.5870, and 0.1140

are related to the eye's sensitivity to a Red, Green, and Blue co lo r according to the N TSC (N ational T e lev ision System (s) Com m ittee) standard and are obtained throughout an

experim en tal way.

A s m entioned earlie r each pixel in a gray level representation (2 -dim entional m atrix) will

be rep resen ted by 8 bits. T his m eans that intensity o f each pixel will be in the range o f 0

to 255 , w here 0 represents the pure black color, while 255 represents the solely w hite co lo r. T his can be show n by a sim ple M atlab program that displays a 7 by 7 gray level

im age rep resen ting a black diagonal line as pictured in Figure 4:

20

2 3 4 5 6 7

Figure 4: Grey Level Im age o f a D iagonal B lack Line.

H ere is the sim ple M atlab code that displays the image show n in Figure 4:

» A = [ 2 5 5 2 5 5 2 5 5 2 5 5 2 5 5 2 5 5 0;

2 5 5 2 5 5 2 5 5 2 5 5 2 5 5 0 2 5 5 ,

2 5 5 2 5 5 2 5 5 2 5 5 0 2 5 5 2 5 5

2 5 5 2 5 5 2 5 5 0 2 5 5 2 5 5 2 5 5

2 5 5 2 5 5 0 2 5 5 2 5 5 2 5 5 2 5 5

2 5 5 0 2 5 5 2 5 5 2 5 5 2 5 5 2 5 5

0 2 5 5 2 5 5 2 5 5 2 5 5 2 5 5 2 5 5 ] ;

» im a g e (A ) ;

» c o l o r m a p ( g r a y ( 2 5 6 ) ) ;

» im w r i te ( A , ' d i a g j i n e ' , 'J P G ') ;

» A = im r e a d ( 'd ia g _ l in e ' , 'J P G ');

A t the first double-prom pt line a 2-dim ensional m atrix A o f the size 7 x 7 (49

pixels) is crea ted w here the all 255 num bers represent pure white intensity o f the pixels,

w hile the all 0 num bers represent pure black intensity o f the pixels. N ext, the m atrix is

d isp layed using M atlab ‘im age’ com m and. The ‘colorm ap(gray(256))’ com m and sets the

co lo r m ap o f the im age to black and white (gray level image). The ‘im w rite’ com m and saves the im age as ‘d iag_ line .jpg’ file. I f we now want to import the file ‘d iag_line.jpg’

into the M atlab environm ent for any further processing and m anipulation, we would use

21

th e ^ m re a d com m and and the black diagonal line image will be im ported back into

the A m atrix . The above dem onstrated procedure is exactly how the image in th is project is fo rm atted by using M atlab environm ent.

A natural question that a curious person m ight ask at this point is: how do we convert the

A SC II represen tation o f the image (a m atrix contain ing the num bers in the [0-255] range)

to its equ ivalen t b inary interpretation? Recall that we have earlier said that in order to

p rocess any inform ation through a digital com m unication system , we have to have the inform ation being represented (so is our image) by a com bination o f zeros and ones.

The answ er is that w e will produce a normalized probability density function (PD F) o f

o u r im age, calcu late the probability o f occurrence o f each num ber in the [0-255] range and th en use the H uffm an coding algorithm to obtain the binary representation o f the

im age. In the process o f calculating the probability o f occurrence for each num ber in the

[0-255] range, any num ber in the range that does not appear in the image m atrix is

considered as irrelevant as if it carries no inform ation relevant to our image. All those ‘ irre levan t’ num bers w ill be rem oved from the list o f leaves that are supplied to the H uffm an code algorithm for data compression. O nce the frequency o f occurrence o f each num ber in the m atrix is calculated, it is divided w ith the total num ber o f elem ents in the

m atrix fo rm ing the probabilities o f occurrence for each num ber in the [0-255] range that

appears in the m atrix.

F o r exam ple, if w e consider the matrix A containing pixels o f the 7 x 7 black diagonal line im age from F igure 4, we see that the num ber 255 occurs 42 tim es, while the num ber

0 o ccu rs on ly 7 tim es. D ividing their frequencies o f occurrences w ith the total num ber o f elem ents in the m atrix, 49, we obtain the corresponding probabilities o f occurrence for

th e num bers 0 and 255:

P robability o f occurrence for 0: 7/49 = 0.1428 = 14.28 (%),P robab ility o f occurrence for 255: 42/49 - 0.8571 = 85.71 (%).

T he correspond ing PD F function o f the black diagonal line would look as shown in

F igure 5. In the project, the M atlab file, ‘F o rm atting .n r, perform s the form atting o f a co lo r im age ‘w o rld .jp g ’. The file is written in exactly the sam e m anner as being explained above: it perform s the RGB to gray conversion (equation (18)), calculates PDF

o f the im age, and gives, fo r the next stage in the process, tw o vectors, one that contains

all num bers p resen t in the image, and the other one that contains the respective p robab ilities o f occurrence for all num bers in the range [0-255] present in the image.

11

0:9

0.8

0.7

0.6

0.5

0.4

0.3

0:2

0.1

0

N orm alized PD F of th e B lack D iagonal Line

...- 1

V= 0.85714

: i x = 0-1

1I Y= 0/I428E 1............

> I-50 0 50 100 150 200 250

Figure 5: PDF function o f the Black Diagonal Line.

3 0 0

23

Chapter 3: SOURCE CODEC

3.1 Huffman Coding Algorithm

B efore starting a discussion on the Huffm an coding algorithm , som e basic term s

and defin itions needed for understanding the principles o f the Huffman algorithm will be first p resented . W e will start with the definition o f a finite discrete source, w here a finite

d iscre te source is a d iscrete source that is em itting, at a constant rate, a finite num ber o f

sym bols from the source alphabet called a stream o f symbols. Each sym bol in an infinite long (theo retica lly ) stream em itted by the source is assigned its own probability o f

occurrence P. O nce we know the probability o f a sym bol, we can introduce the concept

o f the inform ation , w here the information I is defined as a value inversely proportional to the p robability o f occurrence o f the letter P:

l o c i . (19)P

M ore precisely , the inform ation content o f a sym bol Xj is defined as a base 2 logarithm tak en from the probability o f occurrence o f the sym bol x ; . M athem atically:

I(xi) = log2— l - = - log2P(Xi) . (20)P(Xi)

S ince the source alphabet contains a finite num ber o f sym bols N , we would be m ore in terested in find ing a m easure o f the inform ation content for all the sym bols in the source a lphabet, rather than dealing with the inform ation content o f each sym bol independen tly . Such a m easure o f the information content for all the sym bols in the

source a lphabet is called the entropy o f the source and is defined as the average am ount

o f in form ation em itted by the source:

N NH (X ) = P ( x j ) • l ( x j ) = P ( X j )• log 2 P ( x i ). (21)

i = l i = ]

T he unit fo r the en tropy is a bit per event and should not be confused with a binary digit

called a bit. I f all sym bols in the source alphabet are equally probable (have the sam e

p robab ilitie s o f occurrence), then the entropy o f such a source is given by the follow ing

equation :

24

’ H (X ) = log2 N , (22)

w here N is a fin ite num ber o f sym bols in the source alphabet.

N o w , w e w ill consider the process o f encoding, o r representing each sym bol in the source a lphabet by a sequence o f binary digits; ones and zeros. Since there are N possible

sym bols in the source alphabet, the num ber o f b inary digits R per sym bol required to uniquely encode each sym bol is given by;

R = log 2 N . (23)

T he equation (23) holds only w hen the num ber o f sym bols in the source alphabet is a p o w er o f 2. I f it is not the case, the equation (23) requires a m inor adjustm ent and is re­w ritten as;

R = Llog2 N j + l , (24)

w here the sym bol, [ J , identifies the largest integer less than MogiN’. The efficiency o f

the encod ing is defined as the ratio between the entropy H(X) and the num ber o f binary

d ig its R needed to represent any o f N possible sym bols in the source alphabet. The binary

rep resen ta tions o f N equally probable sym bols are called the hxed-length code words.

O n the o ther side, when the source sym bols do not have the sam e probability o f

occurrence , as it is the case in the real world, a different encoding m ethod is required. An exam ple o f such an encoding schem e is the H uffm an coding algorithm . H uffm an coding

a lgorithm is a variable-length encoding algorithm based on the probabilities o f the source

sym bols. The a lgorithm is considered as an optim um algorithm in the sense that the average num ber o f bits required to represent the source sym bols is minimum . In other

w ords, the objective o f the algorithm is to provide a system atic procedure for constructing uniquely decodable variable-length code w ords that are efficient in the sense

th a t the average num ber o f bits per source, defined as the follow ing quantity;

Nn = ^ P ( X j ) n i , (25)

i=l

is m in im ized . It is abso lu tely necessary that these variable-length code w ords arc

uniquely and instan taneously decodable. They also have to satisfy the prefix condition th a t requires that fo r a given code word C o f length k, no other code word o f length m, w here m > k, contains the code word C as its prefix.

T he m easu re o f efficacy o f the variable-length encoding is the ratio between the source

en tropy H (X ) and the average length o f the code word n , and is represented by the G reek le tter r\:

r| = i l 2 ^ (o /o )^ 0 < T |< 1 . (26)

T he equation (26) te lls us that the entropy o f the source can not be greater than the average length o f the code word. Based on this, w e can present another equation that g ives the relation betw een the source entropy and the average length o f the code word:

H ( X ) < n < H ( X ) + l . (27)

3.1.1 H u ffm a n E n co d in g

L e t’s now take a look how the coding algorithm w orks. The H uffm an coding a lgo rithm that generates variable-length code w ords is considered to be a tree form ing

p rocess. T he process starts by listing all letters in the source alphabet, along with their p robab ilities in descend ing o rder o f occurrence. These entries correspond to the leaves

(ends) o f the tree. Each leaf is assigned a leaf w eight equal to the probability o f occurrence o f th a t leaf. The tw o leaves with lowest probabilities o f occurrence are m erged together form ing a branch whose w eight will be equal to the com posite

p robab ilities o f the m erged leaves. A fter each m erging, the new branch and the rem aining

b ranches (or leaves) are reordered to assure that the reduced tree preserves the descend ing o rder o f the probability o f occurrences for all current braches and rem aining

leaves.

H ere we d istinguished tw o different ways o f reordering the rem aining branches: the top

and the bo ttom approach. To dem onstrate these tw o approaches, let’s assum e that a new ly fo rm ing branch has the com posite probability equals to 0.25, and that during the p rocess o f reo rdering the branches in the tree we find that another two branches have the

sam e p robab ilities o f occurrence o f 0.25 as the new ly form ed branch does. The top

app roach pu ts the new ly form ed branch on the top o f the branches with the same

26

probability . O n the o ther side, the bottom approach places the newly branch at the bottom

o f th e 0.25 probability group. The process o f m erging the tw o branches with the lowest

p robabilities and then reordering all branches in the deseending order according to their

p robab ilities o f occurrence continues until last tw o branches are m erged together p roducing a root o f the tree w hose probability o f occurrence is equal to 1 .

O nce w e end up w ith the com plete tree (obtaining a root with the probability o f

occurrence equals to 1 ), all the upper branches o f the tree are labeled with a 0 and all the

low er branches are labeled w ith a 1. A Huffm an tree in Figure 6 dem onstrates the p rocedure ju s t explained:

I0.6 0

P(S1)=0.4

P(S2)=0.3

P(S3)=0.2

P(S4)=0.1

RO O T

Figure 6 : A Huffrnan Tree.

The code w ord fo r each sym bol in the source alphabet (leaf) is a sequence o f zeros and

ones obtained by traveling the tree in the direction starting from the root and ending at the

le a f w hom the code w ord is being formed for. Here are the code words for the Huffm an

tree show n in F igure 6 :

S , = 1Sz = 0 0

53 = 0 1 05 4 = 0 1 1

T he best w ay to understand the Huffm an coding algorithm is to go throughout an entire

exam ple tha t will em phasize all steps o f the algorithm : the merging, the top approach

reordering , the entropy, the average length o f code w ords, and the encoding efficiency.

EXAMPLE (Huffman Coding Algorithm):

A d i s c r e t e m e m o r y l e s s s o u r c e (D M S ) h a s a n a lp h a b e t o f e i g h t s y m b o ls w i th th e f o l l o w i n g

p r o b a b i l i t i e s : 0 .2 5 , 0 .2 , 0 .1 5 , 0 .1 2 , 0 .1 . 0 .0 8 , 0 .0 5 , a n d 0 . 0 5

27

a ) D e t e r m i n e b in a r y c o d e w o r d s f o r th e s o u r c e o u tp u t ;

b ) D e t e r m i n e th e e n t r o p y o f th e s o u r c e ;

c ) F i n d th e a v e r a g e c o d e le n g th ;

d ) D e t e r m in e th e e f f i c i e n c y o f th e e n c o d in g ;

S O L U T I O N :

P(S1)=0.25

P(S2)=0.2 -

P(S3)=0.15

P(S4)=0.12

P(S5)=0.1 -

P(S6)=0.08

P(S7)=0.05

P(S8)=0.05

0.25

0.2

0.15

0.12

0.1

0.25

0.1 0

0.08

0.18

0.15

. 0.12 0

. 0.1

0.25

0.22

0.33

0.2

0.18 0

0.15

0.25

\0 .2 2 n

0.2

0.42

0.33 0

0.25

0.58 0

0.421.0

S j = 0 1

82 = 11 S 3 = 0 0 1

8 4 = 1 0 0

8 5 = 0 0 0 0

86 = 0 0 0 1 8j = 1 0 1 0 8 8 = 1 0 1 1

h )

8

Ii=l

H ( X ) = ^ P ( x i ) - I ( x j ) = 0 . 2 5 - 2 + 0 .2 - 2 . 3 2 + 0 . 1 5 - 2 . 7 4 + 0 . 1 2 - 3 . 0 6 + 0 .1 - 3 .3 2 +

+ 0 .0 8 - 3 .6 4 + 0 .0 5 ' 4 .3 2 + 0 .0 5 - 4 .3 2 = 2 .7 9 7 9

28

c) J 0 - .

8

' ^ ( x i ) = 2 - 0 . 2 5 + 2 - 0 . 2 + 3 - 0 . 1 5 + 3 - 0 .1 2 + 4 - 0 . 1 + 4 - 0 ,0 8 + 4 - 0 .0 5 + 4 - 0 . 0 5i=l

= 2 .8 3

d )

7? = = ^ 7 ^ = 0 .9 8 8 6 = 9 8 .8 6 ( % )— 2.0 3n

3 .1 .2 H u ffm a n D ecod ing

Earlier, it w as m entioned that it was absolutely necessary for variable-length code w o rd s to be un iquely and instantaneously decodable. W e will use an exam ple to

dem onstra te w hat it m eans. Below are show n three code assignm ents for representing the sym bols 8 2 , 8 2 , 8 3 , and S 4 :

8 vm bols C ode I Code 2 C ode 3

S] 1 0 0

8 2 0 0 1 1 0

S3 0 0 0 1 1 1 1 0

8 4 1 1 0 1 1 1 1 1 1

L e t’s now determ ine w hich codes are uniquely and instantaneously decodable and w hich

are no t. 8 uppose we are presented with the follow ing sequence to be decoded:

0 1 0 0 0 1 1 1 1 0 0 1 . . .

U sing C ode 1 it is clear that the first tw o sym bols correspond to 8 3 8 ,. The next tw o bits are not un iquely decodable. They can be decoded either as 8 3 o r 8 3 . O bviously this code

is n o t unique and therefore is not useful for the purpose o f decoding. The second code,

C ode 2, w ill decode the first four bits as S 2 S ] S |. The problem begins with the bits 5, 6 , 7, and 8 . W ould the bits 5 and 6 correspond to 8 2 , or we have to wait for the bit 7 and

decode the sequence (bits 5, 6 , and 7) as 8 3 , or we have to wait even more for the bit 8 to

d eco d e the new sequence (bits 5, 6 , 7, and 8 ) as 8 4 ? Code 2 is not instantaneously

decodab le and can not be used for decoding. N ow , C ode 3 will decode the given sequence as 8 , 8 2 8 2 8 2 8 4 8 2 and so on. It is uniquely and instantaneously decodable. N ote

29

th a t no code w ord in th is code is a prefix o f any o ther code w ord. This is exactly w hat helps us decode a stream o f bits at the receiver side.

T here are several m ethods to decode sequences previously encoded by the H uffm an cod ing algorithm . For all o f these m ethods one th ing is in com m on; the decoder must

have som e kind o f know ledge about the encoding procedure in order to w ork properly.

O ne w ay is to send the list o f all sym bols and their counts to the receiver so the receiver

can recreate the H uffm an tree on its end. Then when a sequence to be decoded arrives,

one bit in the sequence is read at a tim e and the H uffm an tree is traveled from the root

d ow n until a lea f is reached. Let s apply this m ethod using the tree from Figure 6 . W e assum e th a t the tree is reconstructed at the decoder’s end and that the follow ing sequence has arrived:

0 1 0 0 0 0 1 1 1 0 1 0 . . .

T he first bit, 0, is read and the tree will be traveled in the upward direction. Since the node w e have arrived at is not a leaf node, we go on and take the next bit, 1. We now

travel the tree in the dow nw ard direction. O nce again, we check if the reached node is a le a f node, ft is not, so we take the next bit, 0, and travel upwards. This tim e we have reached a lea f node, the leaf S3 . We decode the first three bits as the sym bol S 3 and go

back to the root o f the tree to start decoding the next symbol. Sim ilarly, we take the next 2 bits (one at a tim e) and reach the leaf S 2 . The process continues until we run out o f bits

in the a rriv ing sequence.

A n o th er m ethod is to store the code words for each sym bol in different structures (tables) acco rd in g to the length o f the code words. In other words, all code words that are 5 bits

long w ill be stored in the table 5, for example. Those with the length o f 7 will be stored in

the tab le 7 and so on. We also create a so-called general table which contains the code w o rd s w ith the length o f 1, 2, 3, and 4 bits (since there are not too many o f them ). All

these tab les are then sent to the decoder. Now, when a stream o f bits arrives at the decoder, each bit in the stream is read at a tim e and the cum ulative subsequence (could be

form ed from 1 o r m ore bits) is search for in the appropriate table. I f a corresponding

sym bol is found, we have the decoded symbol. I f not the next bit is appended to the

subsequence . This m ethod is used in this project.

T he m ost in teresting decoding m ethod (but also the m ost tim e consum ing m ethod) is so-

called th e adaptive H uffm an tree decoding. In this m ethod we do not have to have the

know ledge abou t the probabilities o f occurrence o f the sym bols. All we have to know is

30

how m any sym bols w e could have in the source alphabet. That information is shared w ith

the decoder. T ha t is all that has to be transm itted to the decoder. Thus, instead o f sending

the list o f all sym bols and their counts or the tables with the code words, all we send is a

sim ple num ber that represents how many sym bols the source has in the alphabet. The

next s tep is tha t both the transm itter and the receiver assign all the sym bols’ counts to 1 . That resu lts in having the same probabilities o f occurrence for all the sym bols.

F or exam ple, if w e w ant to com press a hum an speech we will need an 8 -bit quantizer

genera ting 256 possib le output levels. So, the num ber o f symbols in the source alphabet

is 256. T his is sent to the receiver and both the transm itter and receiver set the counts for each o f 256 sym bols to 1 that generates the same probability o f occurrence for each

sym bol (1 /256 = 0 .003906). N ow , the code w ords are generated using the Huffman a lgo rithm on both ends. For exam ple, when a sam ple that is quantized to the level 56 is to

be transm itted , the transm itter firstly, sends the code word that corresponds to the symbol

56 to the receiver, secondly, increm ents by one the count o f the symbol 56 which

au tom atically increases the probability o f occurrence o f the symbol 56, and thirdly, re­genera te the new code w ords using H uffm an algorithm for all the symbols according to their new p robab ilities o f occurrence. At this point the transm itter is ready to process a new quantized sam ple.

The receiver, on the o ther end, firstly, decodes the received code word (to 56) by com paring it w ith the existing code words at the receiver, secondly, increm ents by one the coun t o f the sym bol 56, and thirdly, re-generates the new code words using H uffm an a lgorithm for all the sym bols. A fter this the receiver is ready to accept the next code

w ord .

W e see th a t during the procedure o f encoding and decoding the code words are constantly chang ing and tha t is w hy th is m ethod is called the adaptive o r dynamic H uffm an coding. The o rd inary H uffm an discussed in the m ost o f this chapter is in some books called the

static H uffm an coding.

3 .1 .3 M atlab Im plem entation o f H uffm an C oding Algorithm

T hree M atlab files im plem ent the H uffm an Coding Algorithm. The M atlab file

"Im age H uffm an .m " is the main program that contains calls for the routines called at

various tim es during the execution o f the program . It perform s the m erging o f tw o leaves w ith low est probabilities o f occurrences form ing a new branch whose w eight is equal to

31

the com posite probabilities o f the leaves. It also calculates the entropy, the average length code, and efficiency o f the encoding.

T he file “M akeS tructH uff.m ’ is the M atlab function that creates a cell ‘H u ff w ith the fo llow ing structu re elem ents; probability that keeps the values o f probability for each

source sym bol, ‘co d ew ’ which keeps the binary ones and zeros obtained at various stages

o f the H uffm an tree process. A t the end o f the process the structure ‘codew ’ contains the

code w ords for all the source symbols, 'sym bo ls’ containing the rearranged original position o f each letter assigned to each source letter at the beginning o f the Huffm an tree p rocedure;

T he file “ S orting H uff.m ” is the M atlab function that keeps the order in the Huffman tree . It m akes sure that at the beginning o f the execution o f the algorithm , the

p robab ilities o f the source sym bols are arranged in a descending order. Also, this routine

is called every tim e w hen a compound branch is form ed in the tree (from the 2 branches tha t have the low est probabilities o f occurrence);

T he last file “ D ecoder_lm age_H uffm an.m ” is the M atlab file that decodes a sequence o f code w ords using the tables containing all possible code words. These tables are created

acco rd ing to the lengths o f the code words. They m ust be sent to the receiver before the p rocess o f decod ing may begin.

H ere is how the code works: A t the beginning, the “M akeStructH uff.m ” routine is called to c rea te a cell (structure) in order to allocate the necessary m em ory space for holding the

p robab ilities o f all the source symbols, their original positions in the source alphabet, and a 2 -d im entional m atrix that keeps the binary ones and zeros assigned to each source sym bol w hile the code travels throughout the Huffm an tree. O nce the cell is created, the m ain p rog ram calls the “Sorting_H uff.m ” routine that check the order o f the probabilities all th e e lem en ts in the ‘H uff.probability’ structure. I f necessary, it rearranges the existing

o rder o f the probabilities to obey the descending order requirem ent. N ow , the main p rogram takes from the ‘H uff.probability’ the tw o sym bols w ith the lowest probabilities

and c rea tes their com pound probability. The process repeats itself until only one

com pound probability is left (equal to 1). The decoding procedure is straight forward and

is exp la ined into details in the previous section o f this chapter.

A t the end o f the H uffm an code algorithm discussion, let’s take a look at the M atlab print

ou t to the H uffm an algorithm program . The same input sequence o f the source sym bols

32

used in the earlier exam ple w ill be used as the input array: [0.2500 0 . 2 0 0 0 0.1500 0 1 2 0 0

0 .1000 0 .0800 0.0500 0.0500]

» H u f f C o d e A I g o r i th m

S I = 0 1 8 2 = 1 1 S 3 = 0 0 1 S 4 = 1 0 0 8 5 = 0 0 0 0 86=0001 8 7 = 1 0 1 0 88=1011

A v e r a g e C o d e L e n g th ( u s in g m y c o d e ) : 2 .8 3 E n t r o p y o f th e S o u r c e H (x ) = 2 .7 9 7 9 E f f i c i e n c y = 9 8 .8 6 5 %

H u f f =

p r o b a b i l i t y : [ 0 .2 5 0 0 0 .2 0 0 0 0 .1 5 0 0 0 .1 2 0 0 0 .1 0 0 0 0 .0 8 0 0 0 .0 5 0 0 0 .0 5 0 0 ]c o d e w : [ 8x 8 d o u b le ] s y m b o l s : [ 8x 8 d o u b le ]

s o r t e d H u f f =

p r o b a b i l i t y : 1.0000 s y m b o l s : [ 8x 1 d o u b le ]

It can be easily observed that the M atlab code has produced the same output o f

code w ords as the exam ple dem onstrated earlier. The entropy o f the source, the average code length, as w ell as the efficiency m atch those obtained in the exam ple.

3.2 Lempel-Ziv-Welch Algorithm

U nlike the static H uffm an coding where we must know the probabilities o f

occurrence for all source sym bols, the Lem pel-Z iv-W elch com pression algorithm does no t require any p rior know ledge o f occurrence o f source symbols. This is sim ilar to what

goes on in the real w orld, where the statistics o f sources are very often unknown.

D oes th is m ean that we can not design a H uffm an code fo r a source for which we do not k now the statistics o f occurrence o f the source sym bols? Theoretically the answ er is a yes

33

but p ractically the answ er w ould be a no (adaptive H uffm an coding is extrem ely tim e

consum ing). Furtherm ore, theoretically it can be done by estim ating the probability o f

source sym bols th rough observing a long inform ation sequence em itted by the source and then em pirica lly obtain ing the probabilities o f all the source symbols. How ever, this

w ould be very im practical since the process o f em pirically obtaining those probabilities w ould be com putationally very com plex. Not to even m ention the com plexity o f

H uffm an cod ing algorithm being applied onto sources w ith m em ories. Thus, the

app lication o f the H uffm an code algorithm for many real sources is generally considered

im practica l and a different m ethod o f source encoding is needed. This is w here the L em pel-Z iv -W elch algorithm com es into the picture.

T he L em pel-Z iv-W elch algorithm for the source encoding is designed to be com pletely

independen t o f the probabilities o f occurrence o f source symbols. Due to this fact the a lgo rithm is placed in the class o f algorithm s called the universal source coding

algorithm s. The algorithm w orks in a following manner; the output sequence from the

source is parsed into variable-length blocks called subsequences. A new subsequence is c rea ted every tim e w hen a sequence o f bits comes out o f the sources and has not been in troduced in any previously defined subsequence. The subsequences are then listed in a L em pel-Z iv -W elch tab le in the order o f appearance. Besides having a colum n in which all subsequences are listed, the subsequence colum n, the Lem pel-Z iv-W elch table con ta ins th ree o ther colum ns: the mem ory location colum n, the m em ory representation

co lum n, and the code w ord column.

T he m em ory location colum n contains entries that are binary representation o f the

decim al num bers listed in ascending order starting from 1 and representing the appearance o f the d ifferent subsequences. As we have earlier said the subsequence co lum n con ta ins the subsequences where the later subsequences take lower positions in the subsequence colum n. The fields (positions) in the m em ory location colum n and

subsequence colum n are m utually related in a sense that every new subsequence in troduced has the m em ory location num ber associated with it being greater by 1 than the

m em ory location num ber o f the last known subsequence. The very first obtained subsequence occupies the third position in the subsequence colum n (having the m em ory

location o f 3) since the positions (m em ory locations) I and 2 are reserved for subsequences 0 and I that are usually assum ed even before the source starts em itting bits.

T he last bit o f each subsequence entered in the subsequence colum n is called the innovation bit and is im portant for creating the code w ords that will be placed in the code

w ord co lum n. B efore considering how the code word colum n is filled, let s first consider

how w e ob tain the entries for the mem ory representation. In order to do that, we will

34

assum e that the source has ju st started em itting a stream o f binary ones and zeros. The

assum ption goes on assum ing that the very first tw o bits emitted are a I and 0, creating a new subsequence (recall that a 0 and 1 are considered as the first two subsequences

occupying the first tw o positions in the subsequence table). O ur new subsequence 10 is

assigned the m em ory location 0 1 1 (binary representation for the decim al num ber 3 ) and

respective ly the position 3 in the subsequence colum n. The last bit o f the subsequence 10

is the bit 0 and is considered, as we have previously said, to be the innovation bit.

To obtain the m em ory representation for the subsequence 10, we divide the subsequence

in tw o groups. The first group contains n - 1 bits (where n is the total num ber o f bits in the subsequence), w hile the second group contains only one bit, the innovation bit. In our

case the first g roup form s bit 1 (since n = 2 ), while the innovation bit, 0 , forms the second

group. A t this po in t w e will also m ention that the first tw o subsequences 0 and 1 do not usually have their respective m em ory representations in the m em ory representation

colum n. N o w , w e take the first group, the bit 1 and look for its exact replica am ong

p rev iously in troduced subsequences. This means that we would look for subsequence 1 am ong the m - 1 previous entries in the subsequence colum n, where m is the position in the m em ory representation table that the current subsequence holds (in our case m = 3). So, w e will look for the subsequence 1 in the positions 1 and 2 in the subsequence

colum n. W e find it at the position 2 and that is the first num ber o f the mem ory

represen tation o f the subsequence 10. We repeat the same procedure for the innovation bit and find it at the position 1. T hat represents the second num ber o f the m em ory represen ta tion for the subsequence 10. N ow , those tw o num bers are concatenated fo rm ing the num ber 2 1 and this is the num ber being the m em ory representation o f the

subsequence 1 0 .

T he last colum n in the Lem pel-Z iv-W elch table is the code word column. A code word rep resen ta tion for a sequence is obtained from its m em ory representation along with the value o f its innovation bit. To dem onstrate this, we will still stick with our subsequence

10 w hose innovation bit is 0 and its m em ory representation is 21. The code word is ob tained on the follow ing way; the m em ory representation entry 2 1 is divided into tw o

groups. The first g roup is form ed from k-1 num bers, where k is the total num ber o f

num bers in the m em ory representation entry. In our case k = 2 and the first group con ta ins the num ber 2. The second group would contain the last number o f the m em ory

rep resen ta tion en try w hich w ould be I . N ow , the code word for the subsequence 10 w ould consist from the binary representation o f the num ber(s) in the first group, in this

case 2 , and the innovation bit that would be appended to it. It is here im portant to m ention tha t w e have to know in advance how many bits the binary representation o f the

35

first g roup w ould occupy. W e w ill assum e that in our exam ple to represent the binary

rep resen ta tion o f the first group, we would use 3 bits. So, the num ber 2 w ould be

converted to 010. A ppending the innovation bit to il we would obtain the code word 0100 fo r the sequence 10. W e see that our code word is consisted from 4 bits.

E ven though we have ju s t presented how the m em ory location, subsequence, m em ory

representation , and code w ord entries are obtained for one subsequence, the 1 0

subsequence , a com plete exam ple o f Lem pel-Ziv-W elch algorithm follows as it would be beneficia l to a reader to fully understand the process o f obtaining subsequences from a stream o f ones and zeros em itted by the source. It would nicely round up the discussion

o n the L em pel-Z iv-W elch algorithm and provide us w ith the firm foundation for the M atlab im plem entation o f the algorithm.

EXAMPLE (Lempel-Ziv-Welch Algorithm):

O b ta in c o d e w o r d s f o r th e f o l l o w i n g s tr e a m o f b i n a i y n u m b e r s e m i t t e d b y th e s o u r c e :

1 0 1 0 1 1 0 1 0 0 1 0 0 1 1 1 0 1 0 1 0 0 0 0 1 1 0 0 1 . E a c h c o d e w o r d s h o u ld b e g iv e n in a 5 - b i t fo r m a t .

S O L U T I O N :

S t e p l :

S u b s e q u e n c e s S t o r e d (S S ):

0, 1D a t a to b e P a r s e d (D P ):

10101101001001110101000011001

S u b s e q u e n c e s S t o r e d (S S ):

0, 1, 10D a t a to b e P a r s e d (D P ):101101001001110101000011001

S u b s e q u e n c e s S t o r e d (S S ):

0, 1, 10 , 101 D a t a to b e P a r s e d (D P ):101001001110101000011001

S u b s e q u e n c e s S t o r e d (S S ):

0, 1, 10 , 101 , 1010 D a t a to b e P a r s e d (D P ) :01001110101000011001

S t e p _ 2 :

S t e p _ 3 :

S t e p _ 4 :

36

S te p _ 5 :

S t e p ô :

S t e p ? :

S t e p 8:

S u b s e q u e n c e s S t o r e d (S S ):0, 1, 10 , I C I , lO IO , 01 D a t a to b e P a r s e d (D P ):

001110101000011001

S u b s e q u e n c e s S t o r e d (S S ):0. 1, 10 , 101 , 1010 , 01, 00 D a t a to b e P a r s e d (D P ):1110101000011001

S u b s e q u e n c e s S t o r e d (S S ):0 , 1, 10 , 101 , 1010 , 01, 00 , 11 D a t a to b e P a r s e d (D P ) :10101000011001

S u b s e q u e n c e s S t o r e d (S S ):0, 1, 10 , 101 , 1010 , 01, 00 , 11, 10101 D a t a to b e P a r s e d (D P ):000011001

S te p _ 9 :S u b s e q u e n c e s S to r e d (S S ):0, 1, 10 , 101 , 1010 , 01 , 00 , 11 , 10101 , 000 D a t a to b e P a r s e d (D P ):011001

S te p 1 0 :S u b s e q u e n c e s S t o r e d (S S ):0, 1, 10 , 1 0 1 , 1 0 1 0 , 01, 00 , 11. 1 0 1 0 1 , 0 0 0 , O i l D a t a to b e P a r s e d (D P ):001

S t e p _ l l :S u b s e q u e n c e s S to r e d (S S ):0, 1, 1 0 , 1 0 1 , 1 0 1 0 , 01, 00, 11, 1 0 1 0 1 , 0 0 0 , O i l , 0 0 1 D a t a to b e P a r s e d (D P ):

M E M O R Y M E M O R Y C O D EL O C A T I O N S U B S E Q U E N C E S R E P R E S E N T A T I O N W O R D

1 0 - 000002 1 - 000013 10 21 001004 101 3 2 001115 1010 4 1 010006 01 12 000117 00 11 00010

37

8 n 22 001019 10101 5 2 0101110 000 71 O H I O11 O i l 6 2 0110112 001 72 01111

It should be noticed that the Lem pel-Ziv-W elch table show n above has encoded

the orig inal 29 bit sequence into 12 code words o f 5 bits each, resulting in 60 coded bits.

O ne m igh t ask w hat is the point in sending 60 bits through the channel instead o f sending only 29 b its? Is it not the purpose o f the source coding to remove any redundant bit from

the sequence that is to be transm itted? Should not the source encoding provide som e kind o f com pression , and in this exam ple we have just seen that the algorithm provided no

d a ta com pression at all? As a m atter o f fact in the latest exam ple the algorithm has

in troduced an expansion o f the data. The answer to all these questions is very sim ple; the

ineffic iency o f the Lem pel-Z iv-W elch algorithm is due to the fact that the sequence o f b its w e have used in ou r exam ple is very short. As the sequence is containing m ore bits to

be tran sm itted , the Lem pel-Z iv-W elch encoding procedure becomes more efficient and resu lts in a com pressed sequence that is much shorter than the original sequence, p rovid ing a sign ifican t data compression.

The L em pel-Z iv-W elch algorithm is widely used in the com pression o f the com puter files (e.g . Z IP application). M any so-called ‘com press’ and ‘uncom press’ utilities under the

U N IX and M icrosoft operating systems are nothing m ore than the im plem entations o f

various versions o f this algorithm .

D ecoding o f code w ords that have been encoded by using Lem pel-Z iv-W elch algorithm

is stra igh t forw ard . The transm itter must provide the table to the receiver and then each b it in the stream is read at a tim e and the cum ulative subsequence (could be form ed from 1 o r m ore bits) is search for in the table. I f a corresponding sym bol is found, we have the

decoded sym bol. I f no t the next bit is appended to the subsequence.

3.2 .1 M atlab Im plem entation o f Lem pel-Z iv-W elch Algorithm

Six M atlab files im plem ent the Lem pel-Z iv-W elch algorithm . The M atlab file

“ L Z W _E ncode .m ” is the main program that contains calls for the routines called at

various tim es during the execution o f the program. The file Subsequence.m is the

M atlab function tha t is doing the parsing o f the “D ata-to-be-Parsed sequences into variab le-leng th subsequences. It creates the first tw o colum ns in the Lem pel-Z iv-W elch

table: the m em ory location colum n and the subsequence column;

n'rRYEBSGN ü6WimiTY UBRAHY

38

The file Check_up.m is the Matlab function that checks if a chosen subsequence parsedfrom the D ata-to-be-Parsed sequence has already been created in the Lem pel-Ziv- W elch table. I f it has, the subsequence is rejected from being entered into the table.

S ubsequence .m file will chose a new subsequence that will be formed from the rejected

subsequence plus the next allow able bit from the "‘D ata-to-be-Parsed” sequence. The new

subsequence w ill be checked against the existing subsequences by the “Check up.m ” routine;

T he file “T ab le .m ” is the M atlab function that com pletes the rest o f the Lem pel-Ziv- W elch tab le creating the m em ory representation colum n and the code word column. The

function “C onvert2b inary .m ’ converts the m em ory representation entry (m inus the innovation part) into its binary representation.

T he M atlab function “T ailing .m ” checks if the tail o f the original sequence contains the

bits th a t are left ou t and can not form a new subsequence (those bits are already part o f the prev iously form ed subsequence). The routine also creates an array o f com pressed data

that is to be transm itted . The decoding process is the part o f digital to analog conversion perfo rm ed in the “D toA .m ” M atlab file.

In genera l the code w orks in the follow ing manner; the main program calls the

“ S ubsequen t.m ” routine that is responsible for parsing the “D ata-to-be-Parsed” sequences into variab le-leng th subsequences. The first two colum ns in the Lem pel-Ziv-W elch table

are crea ted as well. The m ain program then calls the “Table.m ” routine that com pletes the

rest o f the Lem pel-Z iv-W elch table. At this point the Lem pel-Ziv-W elch table is com ple te contain ing all four colum ns: the m em ory location column, the subsequence co lum n, the m em ory representation colum n, and the code word column. At the end, the

“T a ilin g .m ” routine creates an array o f com pressed data containing the binary rep resen ta tion o f the subsequences obtained through the process o f parsing.

A t the end o f this section, let’s take a look at the output o f the M atlab code that im p lem en ts the Lem pel-Z iv-W elch algorithm . The sam e input sequence used in the

exam ple above w ill be used as the input array in the code.

> > L e m p e l Z i v W e l c h

39

/ 1] [ 0] 0 [ 1 x 5 d o u b le ][ 2] [ 1] [ ] [ 1 x 5 d o u b le ][ 3 ] [ 1x 2 d o u b le ] [2] [ 1 x 5 d o u b le ][ 4 ] [ 1 x 3 d o u b le ] [ 3 ] [ 1 x 5 d o u b le ][ 5 ] [ 1 x 4 d o u b le ] [ 4 ] [ 1 x 5 d o u b le ][ 6] [ 1x 2 d o u b le ] [ 1] [ 1 x 5 d o u b le ][ 7 ] [ 1x 2 d o u b le ] [ 1] [ 1 x 5 d o u b le ][ 8] [ 1x 2 d o u b le ] [ 2] [ 1 x 5 d o u b le ][ 9 ] [ 1 x 5 d o u b le ] [ 5 ] [ 1 x 5 d o u b le ][ 10] [ 1 x 3 d o u b le ] [ 7 ] [ 1 x 5 d o u b le ]

[ 11] [ 1 x 3 d o u b le ] [ 6] [ 1 x 5 d o u b le ][ 12] [ 1 x 3 d o u b le ] [ 7 ] [ 1 x 5 d o u b le ]

» c { : ,2]

a n s = 0I

» c { : ,4 }

a n s

1 01 0 11 0 1 00 10 01 11 0 1 0 10 0 00 1 10 0 1

A]

0 0 0 0 00 0 0 0 10 0 1 0 00 0 1 1 10 1 0 0 00 0 0 1 10 0 0 1 00 0 1 0 10 1 0 1 10 1 1 1 00 1 1 0 10 1 1 1 1

It can be seen that thec are identical to their respective columns obtained in the earlier example . The third

colum n in the cell c contains only the first number found in the memory representation

40

colum n in the exam ple; the innovation bit part is om itted. The reason for that is that the

innovation bit is no t needed to be in the third colum n in order to create the fourth colum n.

It is faster to convert the th ird colum n to its binary representation w ithout considering the

innovation bit and then ju s t add the innovation bit when form ing the code word. The

sam e M atlab file as being used for the m ain sim ulation is used. The only change is that

for th e exam ple the input sequence is 29 bit stream given in the example.

41

Chapter 4: CHANNEL CODEC

4.1 Channel Codes

4.1 .1 P a r i ty C h e c k C odes

T he d iscussion on channel codes w ill begin with the introduction o f parity check codes. O nly the b inary code w ords o f information will be considered m eaning that any

code w ord consists o f only tw o digits as the elem ents o f the binary alphabet: a 1 and a 0 . T hose code w ords are called binary code w ords. Parity check codes involve the

arithm etic operations o f both addition and m ultiplication perform ed on binary code w ords. Since the binary alphabet is used to represent any binary code word, the

operations o f addition and m ultiplication perform ed on binary code w ords are also called the b inary add ition and binary m ultiplication. These operations are perform ed according

to the conven tions o f the algebraic set S that contains the alphabet’s sym bols as its elem ents. T he set satisfies the following properties associated with the binary addition:

1. if a, b are binary code words and a ,b e S, then a + b e S ;2. if a, b, and c e S, then a + (b + c) = (a + b) + c;

3. if a, b e S, then a + b = b + a;4. the set S contains a zero elem ent that satisfies the condition: a + 0 = a ;5. every elem ent in the set S, except zero, has its own negative elem ent and their

add ition yields the zero elem ent. Hence, if a € S, its negative elem ent is —a

and a + ( - a ) = 0 ;6 . the operation, designated with the symbol © is defined as:

0 0 0 = 0

0 0 1 = 1

1 0 0 = 1 101 = 0

and the fo llow ing ones for the binary m ultiplication:

1. i f a ,b are binary code words and a ,b e S, then a • b e S;

2. i f a , b, and c e S, then a ■ (b • c) = (a • b) ■ c;

3. i f a ,b e S, then a • b = b - a;4. i f a , b, and c e S, then (a + b) • c = a • c + b • c;

42

5. every elem ent in the set has its ow n ‘identity’ elem ent ‘1 where a l = a ;

6 . every elem ent in the set, except zero, has its ‘ inverse’, where a a~* = 1 ;7. the operation is defined as:

0 x 0 = 0

0 x 1 = 0

1 x 0 = 0

1 x 1 = 1

P arity check codes are types o f codes that use the sum o f all bits present in information, o r in a b lock o f inform ation, to detect errors inflicted in the information as a result o f

channel im perfections. W e differentiate tw o types o f parity check codes: the single parity check code and the rectangular code.

In the single parity check code all bits in a codew ord are summed up and a resulting bit, the parity bit, is concatenated at the beginning o f the code word, as shown in Figure 7.

The parity bit could be either a 1, sym bolizing that the sum m ation o f all the bits in the code w ord yields an odd result, or a 0 , sym bolizing that the sum m ation o f all the bits in the code w ord yields an even result. I f the added parity bit is designed to yield an even

result, the m ethod is called the even parity m ethod. Similarly, if it yields an odd result, it is called the odd parity m ethod.

C ode word: 0 1 0 1 0 1 0 1;

Parity bit: 0 + 1 + 0 4 - 1 + 0 + 1 + 0 + 1 = 0 ;N ew code word: 0 0 1 0 1 0 1 0 1;

Figure 7: Single Parity Check Code.

In the rec tangu lar code, a rectangle o f code w ords is formed containing I rows and J colum ns. N ow , for each o f 1 rows, a horizontal parity bit is appended at the beginning o f

each row . A lso, for each o f J colum ns, including the very first column representing the

horizon tal parity bits, a vertical parity bit is appended at the beginning o f each colum n,

creating a new block o f the (1 + 1) x (J +1) dim ensions.

L e t’s now take a look at the error detection capabilities o f these two codes. The single

parity check code can only detect errors if the odd num ber o f bits in a code word is w rongly received. I f the num ber o f corrupted bits in a code word is even, the code will

no t be able to detect it since the parity bit will rem ain the same. Even though the single

43

parity code can detect the odd num ber o f corrupted bits, it cannot correct them since the

code has no know ledge w here in the code word the eorrupted bit(s) is(are) located. On

the o ther hand, unlike the single parity code the rectangular code can correct a single

e rro r anyw here in the block (rectangular) o f code words since every bit in the I x J block is un iquely located at the intersection o f the horizontal and vertical parity bits.

Hoiizoutal Paiitv Bits

m

IP4

I>

0 0 0 0 1 i 0i 0 0 1 1 0 10 1 0 0 0 1 0

1 1 1 0 0 0£ 0 1 0 0 0 1

Figure 8 : Rectangular Parity C ode (1 = 4, J = 6 ).

F o r exam ple, if in the block shown in Figure 8 , the bit in the second row and third

co lum n w ere received as a 1 , then the horizontal parity bit for the second row would be 1

instead o f 0, w hile the vertical parity bit for the third row would change from 0 to I. This

w ay the location o f the corrupted bit is easily identified according to the locations o f h o rizo n ta l and vertical parity bits. Rectangular codes are defined by tw o param eters: k

and n, w here k = I • J and n = (I + 1) • (J +1) . It is a com m on practice that rectangular codes are depicted w ith the (n, k) notation. For the case in Figure 8 , we say that the

show n rectangu lar code is o f type (35, 24).

4 .1 .2 L in e a r B lock C odes

L inear block codes are, as mentioned earlier, characterized by the (n, k) notation.

T he pu rpose o f the code is to encode a block o f k-bit code words into a bigger block o f

code w ords w here each code w ord contains n bits. In o ther words, the encoding p rocedu re assigns to each o f the 2 possible k-bit code w ords a unique n-bit code word chosen from the set o f 2" possible code words. A block code represents a one to one

a ssig n m en t w here 2^ k-bit code words are mapped to 2 n-bit code words. The only

p rob lem here is how to select a subset o f 2 n-bit code w ords from the set o f 2

p ossib le n-bit code words. To answer this question we first introduce two term s, a vector

space and vector subspace. A vector space Vp is the set o f all n-bit code words defined

44

over the binary field o f two elements: 0 and 1. A subset o f the vector space V„ iscalled a subspace if the following two conditions are met:

1 . the subset contains the all zero vector, (28a)

2 . i f c o d e w ords C i ,C j € V n then Ck e V n , where Ck = C j ® C j . (28b)

The second condition means that the sum o f any two code words that belong to the vector

space V] must also belong to the vector space . This condition is fundamental for the ex istence o f linear block codes. Once we have defined the vector space and vector

subspace, let 's now establish the goal that a chosen block code has to achieve. The

selected block code should have as large as possible vector space (containing as m any code w ords as possible in it). Then, we should select code words that will form the

subspace S^, in the way that the selected code words are as far apart from each other in the subspace as possible. Now, we have to define the distance from two code words in a

subspace in o rder to select those that are as far apart as possible. The Humming distance is the m easure o f distance between two code words. The Humming distance is defined to be the num ber o f elements in which two code words differ. Why is it so important that tw o w ords are as far apart as possible?

L e t ’s assum e that on the receiver side a received code word has 2 corrupted bits. I f the corrupted received code word is the same as another code word from the subspace, there

would be no w ay that the corrupted bits in the received word are detected as erroneous

and possib ly corrected. The decoder would simply consider the received code word as a valid code word from the designated subspace set and no correction will be performed. Every th ing looks O.K. but the problem is that the decoded symbol is not the one that had

been transm itted . This is exactly why it is desirable that code words in a subspace have as m any different bits among themselves as possible in order to prevent the above situation.

H ow ever, choosing code words from a subspace that are as far apart as possible is not an

easy jo b by any means.

A lot o f researches have been done on the issue o f finding code words from a subspace that are as far apart as possible but so far no acceptable algorithm has been developed that would help us select the best set o f code words for a subspace. In this project we will follow the recom mendation from the B. Sclar’s book “Digital Communications , where it

is suggested that for a particular subspace to be considered as valid one, it is sufficient

that the code w ords from the subspace satisfy the conditions given by the equations (28a)

and (28b). In o ther words, the subspace contains the all zero element and the sum o f any

tw o code w ords in the subspace is another code word that also belongs to the subspace.

45

Let s now consider a block code with n = 7 and k = 4 , a (7,4) block code. The code has

the vecto r space Vy containing 2 = 1 2 8 distinct code words, and the vector subspace S 4

that need to have 2 = 1 6 code words. We can choose any 16 code words out o f 128

possible code words from the vector space as long as they satisfy the conditions (28a) and (28b). Here are, for instance, four code words that satisfy both (28a) and (28b) conditions and can be a part o f any S 4 subspace:

0 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1

1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 + 0 1 0 1 0 1 0 + 1 1 1 1 1 1 1 + 1 0 1 0 1 0 1 + 0 1 0 1 0 1 0

1 1 1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

W e see that the selected four code words contain the all zero code word and that the sum

o f any tw o code words, among those four code words, produces a code word that is one o f those four code words. Addition between the all zero code word with the remaining three code w ords is omitted as obvious. One o f the ways to encode a message with a (7,4) b lock code is to use a look up implementation and for each message look up in the table an associate code word. But would that work if we, for example, had k = 92 , as it is the case in the real world. Certainly not, since we would need a table with 4.95 • 1 0 ^

entrances. Time wise, it would be totally unacceptable, or to be more realistic, it would

be impossible.

4.1 .3 L inear B lock C ode Encoding

Luckily, it is possible to obtain a set o f n-bit code words that will generate all 2*

vectors o f the subspace. In other words, any o f 2^ possible n-bit code words can be

genera ted by using the following matrix equation:

U = m • G = mj • Gi + my • G 2 H rm j^ -G k- (29)

w here m = [m, m y • • • ] is a vector that contains all message symbols that are to be

transm itted, and G is a ‘generator matrix’ with the dimensions (k x n ) .

46

G i ' g n 8 1 2 ” • 8ln

G = G z- 8 2 1 8 2 2 •” 8 2 n

Gk _8kl Sk2 ••• 8kn .

(30)

In (30) the vectors G i, G i , G k ^re linearly independent vectors that can generate all code w ords. I f the generated code words contain the original messages that are parts o f

the c o d e w ords, then w e say that the code is ‘system atic’ and the generator matrix is o f the fo llow ing form:

G = Ik (31)

w here the m atr ix P is a parity matrix o f the (k x (n - k)) dimensions, while Ik is an

identity m atrix, the dim ensions (k x k ) , with ones on the main diagonal and zeros

e lsew here . This simplifies the process o f encoding significantly since now we only have to find a m atr ix P in order to obtain a generator matrix G . Once the generator matrix G is k n o w n w e can easily encode any message by using the equation (29).

T o sum up the process o f encoding o f a message with a (n, k) block code: we first create a subspace o f 2^ n-bit code words. Among those code words we select k o f them in

order to genera te a generator matrix G , in such a way that our generator matrix is created from a parity m atrix P and an identity matrix I k . Now, once we obtain the generator

m atr ix w e encode any m essage symbol by simply applying the equation (29). The process is i llustrated in a simplified flow chart shown below:

(32)

4.1 .4 L in ear B lock C ode Decoding

W hen it com es to decoding o f a received code word, one should think that the

process o f decoding would ju s t be an opposite process from the process o f encoding. By looking at the linear block code encoding flow chart in (32), it would be natural to think

that by m ultip ly ing the received code word with the inverse generator matrix, we would

be able to obtain the transmitted message. The idea sounds O.K. but unfortunately it does

not w o rk since an inverse generator matrix G does not exist in the sense that it would help us recover the encoded message symbol. To overcome this problem we will keep the

47

initial decod ing idea presented e ^ l i e r but instead o f an inverse generator matrix we will

in troduce a parity check matrix H that indirectly enables us to decode the received code

w ord , w here the parity check matrix H with its dimensions ((n — k) x n) is defined as:

H = In-k PT (33)

W e k n o w that a received code word, encoded at the transiver side, is an n-bit vector with d im ensions ( 1 x n) and that the matrix multiplication between the received code word and

the parity check m atrix H , with dimensions ((n - k ) x n ) , is mathematically impossible. So, ho w do w e then decode the received code word?

I f w e take a careful look at the matrix H we can notice that if w e transpose the matrix we get the d im ensions that allow us to perform the matrix multiplication between the

transposed parity check matrix and the received code word. The transposed parity check m atr ix is defines as:

In-k(34)

C an w e consider the result o f the matrix multiplication between the transposed parity

check m atrix and the received code word to be the recovered original message symbol?

N o , at least no t yet. With the matrix multiplication betw een the transposed parity check m atr ix and the received code word we obtain a (I x n) vector S called a syndrome:

TS = r H , (35)

w here the vecto r r is a received code word. The purpose o f the syndrome vector is to de te rm ine if the received code word has been corrupted during the transmission. Here is

h o w the syndrom e vector S achieves it. We will start with the assumption that the received code w ord has been corrupted during the transmission and that it is defined as.

r = U + e. (36)

w here the vecto r U is one o f 2^ n-bit code words from the subspace, while the vector e

is the e rro r vector introduced by the channel. From the equation (35) we obtain the synd rom e S o f the received code word r . I f the syndrom e S is an all zero vector, then

48

the received code w ord has not been corrupted by the channel. Conversely, if the

syndrom e S is not an all zero vector, then the received code word has been corrupted by

the channel and the decoder has to make an effort to correct the corrupted code word.

B efore w e get to that point, let s first rearrange the equation (35) by replacing the vector r w ith the value from the equation (36):

S = ( U + e ) H = U H + e - H ^ = e H ^ , (37a)

since U ■ = m ■ G • = 0 . (37b)

The equality G • H — 0 used in (37b) can easily be proven from the equations (31) and (34). T he equations (37a) and (37b) tell us that the syndrome vector S has a one to one

co rrespondence betw een the error vector e and the corrupted code word r . This is a very

pow erfu l observation that enables us not only to detect the error but also to correct it.

L e t ’s conside r all 2" possible n-bit code words that can be received at the receiver. Recall tha t the num ber o f valid n-bit code words belonging to the subspace is 2^ , and that the rest 2 " - 2 ^ n-bit code words represent all possible combinations o f the

corrup ted valid code w ords having the w number o f corrupted bits, where w = l,2 , . . . , n . I f w e put w ha t has just been said into a two dimensional array, we will end up with a two

d im ensiona l array as shown in Figure 9.

U i U2 UI U2"

62 U2 + 6 2 U; + 6 2 U2" + 6 2

63 U 2 + 6 3 Ui + 6 3 U2" + 6 3

Gj U 2 + 6 j U i + e j U2" + 6 j

62"-" U 2 + 6 2 " “ U i + 62"“ U 2 ‘ + 6 2 '“

F igure 9; Decoding Table for Linear Block Codes.

T he decod ing table for linear block codes must satisfy the following conditions.

1 . the first row contains 2 ^ valid code words;

49

2. the first column, starting from the second element, contains correctable error

patterns. The first element in the first column contains the first valid code word from the subspace which is an all zero code word;

3. the rest o f the array is filled in such a way that each element in the array is

obtained as the sum o f the first element o f that column (valid code word) and the first element in that row (error pattern);

A t th is point we have all necessary information in place to detect and correct errors during a transmission, except one; what error patterns are going to be considered

detec tab le and correctable? To determine that, we have to recall the definition o f the H u m m in g distance which states that the Humming distance is the number o f elements in

w h ich tw o code w ords differ. Another interesting term here is the Humming weight o f a

code w ord that defines the number o f non-zero elements in a code word (e.g. for the code w ord : 1 0 0 1 1 0 1, the Hum m ing weight o f the code word is: 4). Both the Humming

d istance (directly) and the Hum m ing weight (indirectly) are tied to the code’s detection and correction capabilities through the following equations:

e - d m i n - l (38a)

t -

‘ min

e2

(38b)

w here d^^j^ is the m inim um Hum m ing distance o f the code, e is defined as the error

de tec ting capability o f the code, while t is the error correcting capability o f the code. At

th is po in t w e may notice that in order to obtain the error detecting and correcting capabilit ies w e have to calculate the Huffman distance between any two code words in the subspace in o rder to find the minimum Hum m ing distance o f the code. It is easy to realize tha t it would take a long time, especially for the codes with the large k (e.g. for a

(127, 92) code w here k = 92 , the subspace contains 5 x 1 0 ^ ^ code words). However, as w e have discussed earlier, we know that the sum o f any two code words in the subspace

y ields ano ther code word from the subspace. Thus, the Hum m ing distance between any tw o code w ords in the subspace is in fact the H um m ing weight o f the resulting code w ord . So, instead o f calculating the Humming distance for any pair o f code words in the

subspace , we simply ju st calculate the Humming weight o f the code word obtained by their addition, which theoretically could be any code word in the subspace. It is now easy

to conclude that the smallest Humming weight o f the code is in fact the minimum

H u m m in g distance o f the code.

50

To sum up the decoding procedure for the systematic block codes: upon the arrival o f a

code word, w e first calculate the syndrome o f the received word using the equation (35), then w e locate the error pattern in the table that gives a one to one correspondence

be tw een syndrom es and error patterns. Once the error pattern is identified the received code w ord is b inary added with the identified error pattern and the original message

sym bol is obtained. W e will demonstrate the encoding and decoding procedures with an

exam ple that will provide a comprehensive overview o f these two procedures. In the exam ple w e will use a (5,3) code to encode and decode 8 message symbols.

E X A M P L E ( L i n e a r B l o c k C o d e s ) :

a ) F i n d a l l th e c o d e w o r d s p r o d u c e d b y a (5 ,3 ) e n c o d e r ;

b ) F i n d a c o m b i n a t i o n o f e r r o r p a t t e r n s th a t a (5 ,3 ) d e c o d e r u s e s to d e t e c t a n d

c o r r e c t r e c e i v e d c o d e w o r d s th a t a r e e r r a n t;

c ) D e c o d e th e f o l l o w i n g in p u t s e q u e n c e to th e d e c o d e r : 1 0 1 0 0 1 1 1 0 1 1 0 0 0 0;

S O L U T I O N :

a ) n = 5, k = 3. T h e v e c t o r s p a c e c o n ta in s 2 ^ = 3 2 p o s s i b l e 5 - b i t c o d e w o r d s . T h e

s u b s p a c e S j c o n t a i n s 2 ^ = 8 c o d e w o r d s . T h e n u m b e r o f a l l p o s s i b l e 3 - b i t m e s s a g e

s y m b o l s i s 2 ^ = 8 . T h e m e s s a g e s y m b o ls a r e : 0 0 0 , 001 , 01 0 , O i l , 1 0 0 , 1 0 1 , 1 1 0 , a n d

1 1 1 . O n e p o s s i b l e c o m b in a t io n o f 5 - b i t c o d e w o r d s th a t m a k e u p th e s u b s y s te m

c o u l d h e :

0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 0 0 1 0 1 1 0 0 1 0 0 1 1 0 0 1 1 0

T h e s h o w n s u b s p a c e s a t i s f i e s th e c o n d i t io n s (2 8 a ) a n d ( 2 8 b ) ; it c o n ta in s a n a l l z e r o

v e c to r , a n d t h e s u m o f a n y tw o c o d e w o r d s y i e l d s a c o d e w o r d f r o m th e .su b sp a c e .

H o w e v e r , w e n e e d to c h o o s e o n ly th r e e 5 - b i t c o d e w o r d s f r o m th e s u b s p a c e to o b m i n

a l l 8 p o s s i b l e c o d e w o r d s . I n o th e r w o r d s w e n e e d to b u i l d th e g e n e r a to r m a tr ix G o f

t h e d i m e n s i o n s ( k x n ) t h a t s a t i s f ie s th e c o n d i t io n f r o m th e e q u a t io n (3 1 ):

51

G =1 1 0 0 1 0 1 0 1 0 0 1

'0 f f 0 O 'w h e r e : P - 0 1 a n d 13 = 0 1 0

1 1 0 0 1

N o w , b y u s i n g th e e q u a t io n (3 ) w e c a n g e t a l l 8 c o d e w o r d s . T h e y a r e l i s t e d b e lo w :

M E S S A G E :0 0 00 0 10 1 0O i l1 0 01 0 11 1 01 1 1

C O D E W O R D S : 0 0 0 0 0 1 1 0 0 1 0 1 0 1 0 1 0 0 1 1 . 0 1 1 0 0 1 0 1 0 1 0 0 1 1 0 1 1 1 1 1

H U M M I N G W E IG H T : 0 3 2 3 2 3 2 5

b ) dfjj ifj = 2 , s o w e c a n d e t e c t a l l 1- b i t e r r o r s in th e r e c e iv e d c o d e w o r d s , a n d

c o r r e c t 2 '^ — 2 ^ - 1 = 3 e r r o r p a t te r n s . W e w i l l c h o o s e t h a t o u r e r r o r p a t t e r n s a r e :

e -

0 0 1 0 0 1 0 0 0 0 0 0 0 0 1

c ) F ir s t , w e r e a r r a n g e th e in p u t s e q u e n c e in to 5 - b i t r e c e i v e d c o d e w o r d s :

0 o] 0 i\

r 3 = [ l 0 0 0 O]

r / = [] 0 1

r 2 = \ l 1 1

— TN e x t , w e f i n d t r a n s p o s e d p a r i t y c h e c k m a tr ix H w i th th e ( (n — k) x n ) d im e n s io n s

u s i n g th e e q u a t i o n (3 4 ):

52

1 00 I

0 I

0 I

1 1

w h e r e : l 2 =o '

'0 I '

0 1a n d P = 0

1I

1

N o w , w e c a l c u la t e th e s y n d r o m e m a tr ix f r o m th e e q u a t io n (3 7 a ):

— — J'S = e - H =

0 1 1 0 1 1

T h e n , w e f o r m a o n e to o n e c o r r e s p o n d e n c e b e tw e e n th e e r r o r p a t t e r n s e a n d th e

s y n d r o m e S :

'0 0 1 0 o' '0 f1 0 0 0 0 o 1 00 0 0 0 1 1 1

T h e f o l l o w i n g t h i n g i s to c a l c u la te th e in d iv id u a l s y n d r o m e s f o r th e e a c h r e c e i v e d

c o d e w o r d a s p e r e q u a t io n (3 5 ):

S i = { l l ] S 2 = [ 0 l ] S 3 = [ l O]

U s in g th e o b t a i n e d in d iv id u a l s y n d r o m e s w e f i n d th e i r c o r r e s p o n d in g e r r o r p a t t e r s

f r o m e :

~ei = [0 0 0 0 l ]

6 2 = [ 0 0 1 0 O]

e j = [/ 0 0 0 d \

T o o b t a i n th e c o r r e c t c o d e w o r d s w e p e r fo r m th e f o l l o w i n g b in a r y a d d i t io n :

Ü - r - v e

53

T h a t y i e l d s :

U l = [ l 0 1 0f / 2 = [ 7 1 0 0

f 7 i = [ o 0 0 0 0]

F r o m h e r e i t is e a s y to d e t e r m in e w h ic h m e s s a g e s y m b o l s h a v e b e e n tr a n s m i t te d .

B y u s i n g th e ta b le f r o m th e p a r t a ) w e c a n e a s i l y f i n d th e c o r r e s p o n d in g m e s s a g e

s y m b o l s :

m i = { l 0 l \ n i2 =[0 0 l ] n i3=[ 0 0 O]

4 .1 .5 M atlab Im plem entation o f L inear Block Codes

T he following Matlab files simulate the linear block code implementation:

The M atlab file “Generate M atrices.m” that creates the (12, 8) linear block code that is used for the project simulation. The file also creates the following matrices needed for the

encod ing and decoding procedures: Ik, P ,G , H ^ ,e , and S . The “LBC_Encode.m ” and

“L B C D ecode .m ” M atlab files that perform the encoding and decoding procedures im plem ented in an exact fashion as presented in the discussion and example earlier.

To conclude the discussion on linear block codes, let’s take a look at the printout o f the M atlab code that w orks out the example above (the code for “ LBC Exam ple” file is

p rovided in the A ppendix B o f this report):

> > L B C _ E x a m p l e

G = 0 1 1 0 0

0 1 0 1 0 1 1 0 0 1

111

m e s s a g e { 1} = 000 m e s s a g e {2} = 001 m e s s a g e { 3 } = 010 m e s s a g e { 4 } = O i l

m e s s a g e { 5 } = 1 0 0

m e s s a g e {6} = 101

U { 1 } = 0 0 0 0 0

U { 2 } = 1 1 0 0 1

U { 3 } = 0 1 0 1 0

U { 4 } = 1 0 0 1 1

U { 5 } = 0 1 1 0 0

U {6} = 1 0 1 0 1

m e s s a g e { 7 } = 1 1 0

m e s s a g e {8} = 111U { 7 } = 0 0 1 1 0

U {8} = 11111

r e c e i v e d c o r r u p t e d s e q u e n c e = 101001110110000

54

e = 0 0 1 0 01 0 0 0 00 0 0 0 1

H t = 100001

01I

111

S = 0 11 0 1 1

d e c o d e d _ m e s s a g e _ s y m b o l s = 101 001 000

W e see tha t in both cases, the example and the Matlab code, w e get the same results ob ta in ing the same decoded message symbols: 101 001 000 as expected.

4.2 Convolutional Codes

Convolutional codes are generated by passing a specified number o f bits, o r a single bit itself, through a linear finite state register. The linear finite state register is also

co m m o n ly called a convolutional encoder. Convolutional codes are usually specified by th ree param eters: n, k, and m, where k is a number o f bits in a codeword coming in the

e ncoder (input sequence), n is a number o f bits in a codeword going out o f the encoder (o u tpu t codew ord), and m is a number o f stages o f m emory registers. From the definition o f the index m we note an important characteristic o f Convolutional codes; convolutional encoders contain memory. The encoder in Figure 9 is classified as a (2,1,4) convolutional

encoder due to the following reasons: the output o f the encoder contains a 2-bit codew ord , the input to the encoder contains a I-bit input sequence that is shifted through

4 stages o f the m em ory register (encoder).

It is assum ed that each bit in the input sequence (in the case in Figure 9 only one bit

m akes the input sequence) is equally likely to be either 1 or 0 and is independent from any o th e r bit in the sequence. The mutual bit independency is exactly a main reason why

w e are able to perform the channel coding operation before the data is transmitted out. F o r exam ple, if during a transmission one or more bits o f information are incorrectly

received ; from the receiver’s prospective what can be done to correct those corrupted

bits?

55

Input OutputSwitchBoxSequence Codewoi-d

(nl +u2)

Figure 10: (2,1,4) Convolutional Encoder.

First o f all, there would be no way we could determine that the transmission has caused a corruption o f a bit, o r stream o f bits, unless we have a previous knowledge o f the data

that had been sent out. Secondly, the concepts mentioned in the previous sentence pretty m uch con trad ic t each other; if we know in advance what is to be transmitted, then why do

w e need to transm it it at all? This is where convolutional codes come into the picture by

adding a previously determ ined number o f redundant bits that carry the information about the o rder and structure o f bits that the data is formed of. Convolutional codes help control the o rder and structure o f the data bits at the receiver side where some bits might be

received in error. In general, it is the function o f the encoder to introduce redundant bits

in the transm itted information sequence which then can be used at the receiver side to

overcom e the effects o f noise and interference encountered during the transmission o f the signal th rough the channel. The amount o f redundancy introduced during the encoding

p rocedure is measured by the ratio n over k.

O nce the convolutional code indexes (n, k, and in) are determined (the responsibility o f the designer), the convolutional encoder is easy to implement. First, m boxes representing

the m m em o ry register stages are drawn, then n modulo-2 adders are placed, and finally

the m em ory register stages are connected to the adders using the generator polynomial vector. The genera tor polynomial vector g determines a way in which the stages o f the

m em ory register are going to be connected to the modulo-2 adders. The generator po lynom ials , as the elements o f the generator polynomial vector (also called connection

vectors), g ive the convolutional code its unique error protection quality. 1 here arc many

choices for a selection o f generator polynomials for an m order code. However, they all

do not result in ou tpu t codeword sequences that have good error protection properties.

T hose genera to r polynomials that do provide good error protection properties are usually obtained by com puter simulations. The generator polynomials used for the simulation o f

a convolu tiona l code in this project are taken f r o m Peterson and Weldon s book Error

C orrec ting C odes” and they are presented in Table 1 ;

56

N m n b e i of Registei-s F i r s t Polynom ial Second Polynom ia ls

3 1 1 14 1 1 0 1

10 1 1 1 1 0

5 1 1 0 1 0 1 1 1 0 16 1 1 0 1 0 1 1 1 1 0 1 17 1 1 0 1 0 1 18 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 0 1

10

1 1 0 1 0 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 0 1

1 1 0 1 1 1 0 0 1 0 1 1 1 0 0 1 1 0 0 1

Table 1; Generator Polynomials for Convolutional Codes.

F ro m T ab le 1 we conclude that we must have the same number o f generator polynomials

as w e have output bits from the encoder. Also, each generator polynomial would contain

m e lem en ts that can be represented by a 1 or a 0, where 1 represents a connection and 0

the absence o f the connection between an adder and a memory stage. In the case o f the(2 ,1 ,4) encoder shown in Figure 10, we have two output bits (n = 2) and four stages o f a reg is ter (m = 4). Thus, w e would need two generator polynomials, each o f whom would have 4 elem ents.

4.2 .1 C onvolutional Code Encoding

Several m ethods are used to represent the process o f convolutional encoding. The

m ost popular ones are: impulse response o f the encoder, polynomial representation, state representa tion and the state diagram, and the trellis diagram. Each o f these methods will

be briefly discussed here in this chapter while the impulse response and the polynomial

representa tion m ethods will be implemented in the Matlab simulation.

L e t ’s now go back to our (2,1,4) convolutional encoder from Figure 10. At each input

t im e, one bit from the input sequence is shifted into the leftmost stage (box 1) o f the e ncoder and the bits in the other stages o f the encoder (boxes 2, 3, and 4) are shifted one

position to the right. The switch box at the output samples the outputs from the adders,

in terlaces them , and forms an output codeword (ni & n o ) . The sampling process keeps on

repea ting until no more bits are available at the input o f the encoder. The connections betw een the m odulo-2 adders and the memory stages o f the (2,1,4) encoder are given by

the fo llow ing tw o generator polynomials:

g l = [1 I 0 I] and g2 = [I 1 I 0],

57

4.2 .1 .1 Im pulse R esponse o f the C onvolutional Encoder

A fter having described the structure o f the convolutional encoder in Figure 10, we will now take a look at the functionality o f the encoder and how it responses to a single

input bit 1 that m oves throughout the register's stages from the left to the right. This is called the impulse response o f the encoder and creates the so-called a walking one effect

that sim ulates the impulse excitation o f the encoder. At the certain time t only one stage

o f the register (encoder) contains a 1 while the rest o f the stages are populated with zeros.

The corresponding outputs at U] and ni are obtained and saved in two separate vectors. At the t im e t -t-1, the 1 moves into the next right stage o f the register while the previously

occupied stage is populated with a 0. The output bits U| and ni are again obtained and added to the existing values in the vectors. The process is repeated until the 1 is walked th rough all the stages o f the register:

Time Register Contents Output bits: ny n?1 1 0 0 0 1 1

2 0 1 0 0 1 1

3 0 0 1 0 0 14 0 0 0 1 1 0

Table 2: Impulse Response o f the (2,1,4) Encoder.

F rom T ab le 2 w e see that for each time t corresponding outputs from n, and n; are

sam pled and stored creating vectors: n, and no. The impulse response o f the encoder is

form ed by alternately tak ing bits from vectors ni and n? at times t = 1 ,2 ,3 , and 4:

Impulse Response o f the Encoder = 1 1 I 1 0 1 I 0.

N o w w hen w e know the impulse response o f the encoder, we can easily find the response

o f the encoder to any type o f input. All we have to do is to apply the principle o f the superposition , o r linear addition o f the time shifted impulses. For instance if we want to

obtain the codew ord for the input sequence [ 1 0 1 1], all we have to do is that for each 1

in the input sequence we assum e that the encoder produces the impulse response at the output, and for each 0 in the input sequence the encoder produces a zero response at the

output. A t each tim e t we have two outputs o f the encoder, n; and n;, and at the time t +

1 the ou tpu t response o f the encoder will be shifted two bits to the right compared to the

ou tpu t response at tim e t:

58

[t] [Input] [Output]

1 1 1 1 0 1 1 0 (impulse response)^ ® 0 0 0 0 0 0 0 0 (zero response)^ 1 1 1 1 0 1 1 0 (impulse response)^ 1 1 1 1 0 1 1 0 (impulse response)

T he resulting codeword: 1 1 1 1 1 0 1 0 1 0 1 1 1 0 (input response)

4 .2 .1 .2 Polynom ial R epresentation o f Convolutional Encoding

A n o th e r method that a convolutional encoder can be represented with is the

po lynom ia l representation. In the polynomial representation each generator polynomial is rep resen ted by a real polynomial. Since in the binary world addition and subtraction give the sam e results, the sign o f each term in the polynomial representation is o f no interest to

us. A s a convention, w e will assume that each polynomial factor contains a positive sign.

The tim e will be represented by the degree o f each polynomial factor. An input bit that

gets into the encoder at time t = 1 will be multiplied by the polynomial factor on the t - 1degree , w hich m eans if a 1 is a first input bit entering the encoder it would be assigned the X° polynom ial factor. I f we now go back to the encoder in Figure 10, then the genera to r polynom ials in the polynomial representation will be declared as:

g l (X ) = l - X ° + l - x ’ + 0 - X ^ +1-X ^ =1 + X + X ^

g 2( X) = l - X ^ + l - X ^ + 1 X ^ + 0 X^ =1 + X + X^;

I f w e w an t to encode the input sequence m = [1 0 1 1] that we have used in the previous section, then that input sequence in the polynomial representation would look like:

m (X ) = l - X ° + 0 - X ’ + 1 -X ^ + 1 -X ^ =1 + X ^ + X ^

T he ou tpu ts o f the adders (n, and n?) are found by multiplying both generator po lynom ials , g i(X ) and g 2 (X) individually with the input polynomial, m(X):

n i ( X ) = m (X )-g i (X ) = l -X ® + 1-X' +1 • X^ + 3 -X ^ + 1 • X'^ + 1 -X^ +1 • X^

= 1 + X + X ^ + 3 X ^ + X “ + X ^ + X ^ ;

In the result for nj from above, we can notice that the polynomial factor X has the prefix 3 as a result o f the multiplication. However, we know that in a binary system we

can have only tw o numbers a 0 and a 1. The num ber 3 in binary, is in fact, 1 + 1 + 1, and

is equal to 1 applying the method o f binary addition. N ow we have.

59

n](X) = l + X+X^+X^+X"*+X^+X^;

w hich is basically equal to n, = [l 1 1 1 1 1 1]. Similarly, we get the following result for the second outpu t ni:

n 2 ( X ) = m ( X ) g 2 ( X ) = IX ® +1-X^ + 2 -X ^ + 2 -X ^ +2-X^^ + 1-X ^ + 0 -X ^

= l + X + 2- X^ + 2 - X^ + 2-X"^ + X^

= 1 + X + X ^

Since 2 = 1 + 1 = 0 in the binary system, the polynomial factors X ^ X \ and X" will dim inish . This is equal to the vector where ni = [1 1 0 0 0 1 0]. The final codeword is ob tained by alternately taking one bit from the vectors m and n :

Codeword; 1 1 1 1 1 0 1 0 1 0 1 1 1 0 .

The fo llow ing tw o m ethods o f the convolutional encoder representation; the state d iagram and the trellis diagram will not be discussed in details. Rather, they will be

in troduced in such a m anner and quantity that a reader can follow the rest o f the report

and be able to understand the decoding techniques based on the state and trellis representa tions o f a convolutional code.

4 .2 .1 .3 S tate D iagram

The state diagram represents the possible contents (states) o f the rightmost m -1 reg is te r s tages (m is the number o f the memory register stages), and the transitions

b e tw een states as a result o f the arrival o f a new input bit to the leftmost register stage.

Table 3 provides the state diagram for the convolutional encoder shown in Figure 10.

CiirTEiif S tate Next Slate OutDUt0 1 0 1

------ ------ n l n2 n l n2o tn 000 100 0 0 ^ 1 1001 000 100 1 0 0 1010 001 101 0 1 1 0O il 001 101 1 1 0 0100 010 110 1 1 0 0101 010 110 0 1 1 0110 o i l 111 1 0 0 1111 o i l 111 0 0 1 1

Table 3: State Diagram o f the (2,1,4) Convolutional Encoder.

60

Since the encoder in Figure 10 has 4 stages o f the mem ory register, its last 3 stages will

create the states o f the encoder. The last 3 stages can generate 2^ = 8 possible states. The

next state that the current state moves to every time a new bit enters the encoder depends

on the fact if the new bit is either a 0 or a 1. To sum up. each current state can transit to

one o f tw o possible next states depending weather the first stage o f the memory register

contains a 0 or a 1. From Table 3 we see that, for example, if the current state is 100, it

m eans that the three rightmost stages o f the memoiy' register are occupied with 1, 0, and 0

respectively. N o w , the outputs ni and m o f the encoder depend if the very first stage o f

the m em o ry register is occupied by a 0 or 1. If the first stage contains a 0, then the output

m = 1 and n? = 1. Similarly, if the first stage contains a 1, then the output n, = 0 and n j =

0. The next state depends on the occupancy o f the first stage as well. In this case, if the

first s tage has a 0 in it, the next state is 010 (after shifting all bits to the right by one), and i f the first stage contains a 1, the next state is 101.

4.2 .1 .4 Trellis D iagram

The trellis diagram is characterized with row states as well as with column states. The trellis d iag ram requires 2 '^ ” ' nodes to represent 2^^” ' possible encoder states, where m

is, once again, the num ber o f stages o f the memory register that determines the state o f

the encoder.

000 ■

100 ■

010 I

110 I

1 0 1 1

001 I

o i l I

111,

2 t = 3 t = 4 t = 5 t = 6 t = 7 t = 8

(00) (00) (00) (00) _ (00) (00)

■ / ^

Figure 11: Trellis Diagram Representation o f (2,1,4) Convolutional Encoder.

61

Figure 11 shows the resulting trellis diagram for the encoder in Figure 10 w hen the input

sequence is 1011. In the trellis diagram the transition from one state to another state upon

the arrival o f a 1 is represented by a dashed line. The transition cased by a 0 at the input

o f the encoder is pictured by a solid line. From Figure 11 we see that if the current state is

000 and a bit 1 com es in, the encoder reaches the state 100. The output values for m and t \2 are 1 and 1 respectively.

4 .2 .2 C o n v o lu t io n a l C o d e Decoding

I f we assum e that 3 bits are encoded using (2,1,4) convolutional encoder in Figure 10, then the output code sequence from the encoder will contain 6 bits, since each input

bit is represented by 2 bits at the output o f the encoder. These 6 bits may o r may not have

errors. W e know from the encoding procedure that these bits are uniquely mapped. That m eans that a 3 bit sequence will have a unique 6 bit output. But due to errors, we can

receive any o f all possible combinations o f 6 bits. The permutation o f 3 input bits results

in eight possible input sequences. Each o f these input sequences has a unique mapping to a six bit code sequence by the encoder. These form a set o f permissible sequences and the

d e c o d e r ’s task is to determine which code word has been sent. Table 4 summarizes what has ju s t been explained:

Input Valid Code Sequence Receii'ed Sequence Humming Distance

000 000000 111100 4001 000011 111100 6010 001111 111100 4o i l 001100 111100 2100 111110 111100 1101 111101 111100 1110 110001 111100 3

111 110010 111100 3

Table 4: Hum m ing Distance Used for Decoding Convolutional Codes.

L e t ’s say w e have received the following sequence: 111100. As it can be seen from Table

4 the received sequence does not match to any o f the 8 valid code sequences. How do we d ecode the received sequence? There are two approaches: we can do a correlation

be tw een each valid code sequence and the received sequence and pick the sequence with

the best correlation. This approach is known as the soft decision decoding. The second

app roach deals with the term the Hamming Distance introduced earlier when we

62

discussed decoding o f linear block codes. As we have defined it, the Hamming distance

betw een tw o code w ords is defined to be the number o f elements in which those two code

w ords differ. In our case we compute the Humming distance between the received word

and all possible valid code words and choose the code word with the smallest Hamming

distance. This is basically what is behind the so-called hard decision decoding. As the

num ber o f bits to be transmitted increases, the number o f calculations required to be done

during the process o f decoding (Humming distance) significantly increases such that it is no longer practical to do decoding this way. This brings a need to find a more efficient

m ethod that does not examine all options and has a way o f resolving ambiguity such as in

cases w hen w e have tw o or more code words have the same minimal Hamming distance (in Table 4, both 111 110 and 11 1101 valid code sequences have the Hamming distance

o f 1) Then, w hich one to choose? To conclude; a method o f decoding is needed in which for a m essage containing in bits we do not have to check each o f 2"^ possible code words

to determ ine w hich valid code word has been transmitted. This is where Viterbi decoding a lgorithm com es along.

4.2 .2 .1 V iterb i D ecoding Algorithm

Viterbi decoding is the best known implementation o f the maximum likelihood

decoding. The m axim um likelihood concept is a com m on sense way to make decisions w hen the statistical knowledge o f the probabilities is available. In other words, the decoder chooses a valid code word c, as the transmitted sequence, if the likelihood P(r|c) is g rea ter than the likelihood o f all the other valid code words. The Viterbi algorithm had

been designed with the following two assumptions kept in mind:

1. the probability o f occurrence o f a single error is very small;2. the probability o f occurrence o f two errors in a row is even much smaller than

the probability o f occurrence o f a single error;

W hat these tw o assumptions mean is that in comm unication systems errors occur

infrequently and that they are randomly distributed. The Viterbi decoder examines an entire received sequence o f a given length and then the decoder computes a metric for

each path and m akes a decision based on this metric. The most common metric used is the H am m in g distance metric. All paths are followed until two paths converge on one

node. Then the path with the lower metric is kept and the one with higher metric is

d iscarded . The paths selected are called the survivors. The early rejection o f the unlikely

paths greatly reduces the decoding complexity. Even though the Viterbi decoding

a lgorithm solves the problems o f unnecessary checking all o f the 2 possible code

63

w ords, as well as the ambiguity when two code words have the same Hamming distance,

it can no t be consider as the ideal. It can be only considered as the best decoding

algorithm am ong the existing ones. In some aspects o f decoding it does overpow er other decoding algorithm s but it self has severe constrains imposed on by hardware complexity .

T he Viterbi decod ing algorithm uses the trellis diagram to perform the decoding

p rocedure. H ow ever, the d e c o d e rs trellis diagram is a bit different than the encoder’s

trellis diagram . For the d e c o d e r s trellis it is much more convenient to label each trellis

pa th at tim e t (path betw een any two states in the trellis) with the Hamming distance

betw een sequence received at the time t and the corresponding code word. It is important to realize that the encoder’s trellis is known a priory to both the encoder and the decoder.

T o fully understand the operation o f the Viterbi algorithm, let’s assume that the following

stream o f bits have been received at the receiver side and is to be decoded by the Viterbi algorithm:

received sequence; 1 1 1 1 0 1 1 1 0 1 0 1 1 1 .

Since the decoder has the knowledge about the encoder, it knows that the encoder is a(2,1,4) convolutional encoder. That gives a clue to the decoder that the received sequence o f bits has to be partitioned into subsequences where each subsequence will contain 2 bits

(since each input bit at the transmitter side is encoded into a code word o f 2 bits). So, the

rearranged received sequence would now look like this: 11 11 01 11 01 01 11. The decoder trellis d iagram along with the respectful metric values is shown in Figure 12.

Looking at Figure 12, we see that all output pairs (bits ni ni) have been replaced by the metric values (H am m ing distance values). For instance, the metric value for the path that

represents the transition from the state 000 to 100 is equal 0. That means that the output o f the encoder and the received sequence at time t = 1 are the same (Hamming distance is

equal to zero if all bits on a received word and the corresponding code word are the

same). Similarly, for the transition from 000 to 000 (when 0 arrives at the input o f the encoder) the metric is equal to 2. The meaning is that the received sequence and the code

sequence at that particular state differ in 2 bits. Going back to the state diagram shown in

Table 3 w e can see that for the state 000 the code sequence (output bits), when a 0 arrives, is 00. F rom the Figure 12, we see that the received sequence at time t — 1 is 11. Thus, w ords 00 and 11 differ in both their bits, thus the corresponding metric is therefore

equal to 2.

64

received sequence:t =

000 t = 6 t =11

7 t=8

100

010

110

101

001

o i l

111

Figure 12: The Decoder Trellis Diagram (with Hamming Distances).

A t t im e t = 5 there are path crossing in every possible state (column nodes). These is

w here the Viterbi algorithm will get rid o f some paths and reduce the complexity o f the

decod ing w hich , as seen from Figure 12, already has become quite complex even though only 5 tim e units have elapsed.

L e t ’s start with the state 000 (first node in the t = 5 column). Two paths merge in this node. The first path is 000, 000, 000, 000, and 000 with the metric equals to 7 (2+2+1+2). The second path is 000, 100, 010, 001, and 000 having the metric value o f 1 (0+0+0+1 ). The a lgorithm will keep the second path since its cumulative metric is smaller than the

cum ula tive metric o f the first path. The second node (state 100) has the following two paths that get merged in the node. The first one is: 000, 000, 000, 000, 001 with the

m etric equals to 5 (2+2+1+0), and the second one: 000, 100, 010, 001, 100 with the m etric value o f 1 (0+0+0+1). The algorithm will select the second path and discard the first one due to the fact that the first path’s metric is higher than the second path’s.

Similarly, the algorithm will choose only 8 survivors and the trellis tree will have the

sam e num ber o f branches as it did at the t - 1 unit o f time. The resulting trellis diagram is

show n in Figure 13. There we see that the path that has been running through the 000 states (first row) has completely disappeared and that for the time t = 1 only one state

65

transition rem ains; from 000 to 100 with metric 0. I fw e now go back to the slate diagram

in T ab le 3, w e can conclude that transition from 000 to 100 happens when a 1 is at the

input o f the encoder (1 is encoded). Therefore, the decoder decodes the first code word(sequence) to be a 1.

received sequence: 11 H qi

t = 2 t = 3 t = 4 t = 5000 "

100

010

110

101

001

O il

111

01 01 11 t = 6 t = 7 t = 8

Figure 13: The Decoder Trellis Diagram after t=5 Time Units.

It is in teresting to notice that it has taken 5 time units before the decoder could pull the

decision tha t a 1 has been received at the time t = 1. These 5 time units represent a decod ing de lay w h ich can be as much as five times the constraint length ( if m = 4, the delay is 20 tim e units). This decoding process will continue until the last transmitted code

w ord is decoded . In general, when a binary convolutional code with the constraint length K is decoded by Viterbi algorithm, there are 2* * states as well as 2* * surviving paths at each stage. Furthermore, a convolutional code with k bits being shifted in the encoder

at a t im e will generate a trellis with 2 k - ( K - l ) states. At each stage o f the trellis, there are

2 paths tha t get merged at each node. Only one path survives and this is the most

p robable (m in im u m distance) path. Thus, the number o f computations in decoding

perfo rm ed at each stage increases exponentially with k and K. This is a main reason why

the use o f V iterbi algorithm in decoding convolutional codes is troubled. Another

d isadvan tage o f the Viterbi decoding is that the decoding delay is usually too long for

m ost practical applications, which often is not acceptable. Also, the memory required to

66

store the surviving sequences wculd be o f a humongous size and can be very expensive.

The solution to this problem would be to modify the Viterbi algorithm in a w ay that

results in a fixed decoding delay without significantly affecting the optimal performance

o f the algorithm. Viterbi decoding is quite important since it also applies to decoding o f block codes. This form o f trellis decoding is also used for Trellis-Coded Modulation (TCM ).

4.2.3 M a t l a b Im p le m e n ta t io n of th e (2,1,4) Convolu tional C ode

Tw o M atlab files simulate the processes o f convolutional encoding and decoding. The M atlab file “ConvolEncoder’" performs the encoding procedure implementing a

(2,1,4) convolutional encoder. The file does all three types o f encoding which have been

discussed in sections above; Impulse Response, Polynomial, and Trellis. The file “V iterb i_D ecoder.m ” performs the decoding algorithm as being discusses in previous

section. The decoding delay has been set to 20 time units.

At the end o f this section, let 's show the print out as the result o f running an example code im plem enting the (2,1,4) convolutional code in Matlab (the file

“C onvo lE xam ple .m ” is listed in Appendix B o f the report):

> > C o n v o lE x a m p le

D a t a to b e e n c o d e d :

1 0 1 1

C o n n e c t io n R e p r e s e n ta t io n

C o d e d : 1 1 1 1 1 0 1 0 1 0 1 1 1 0

P o l y n o m i a l R e p r e s e n ta t io n

C o d e w o r d : 1 1 1 1 1 0 1 0 1 0 1 1 1 0

T r e l l i s R e p r e s e n ta t io n

E n c o d e d S t r e a m : 1 1 1 1 1 0 1 0 1 0 1 1 1 0

D e c o d e d S e q u e n c e : 1 0 1 1

D i f f e r e n c e b e tw e e n D e c o d e d a n d E n c o d e d S e q u e n c e s ( U s in g V ite r b i D e c o d in g

A l g o r i t h m ) : 0

67

Chapter 5: WCDMA

5.1 WCDMA Technology

The first generation o f mobile communication systems is represented by the

analog system s designed to carry the voice application traffic. Those systems are mostly based on the analog frequency modulation technique and all systems employ the

F requency Division M ultiple Access (FDMA) channelization technique to share the

m edium for the transmission o f information. In FDMA, the transmission medium is divided am ong all stations which share the same allocated frequency bandwidth called

the spectrum. The spectrum is divided into N channels where N represents the number o f

stations sharing the medium. Each o f N stations transmits its information continuously on

an assigned channel and during the period o f the transmission no two stations share the

sam e channel. Bandpass filters are used to contain the transmitted energies within assigned channels. For the reasons mentioned above, FDMA is suitable for stream traffics and finds its use in connection oriented systems. On the other hand, FDMA is extremely

inefficient for bursty traffics since during the periods o f no traffic the channel will sit idle, essentia lly a wasted resource.

The second generation o f mobile communication systems relies exclusively on digital

m odula tion techniques and two channelization techniques: Time Division Multiple A ccess (T D M A ) and C ode Division Multiple Access (CDMA). In TDMA, each station transm its during its assigned time slot and uses the entire spectrum during its

transm ission . In the basic form o f TDMA, each station is assigned the same size time

slot, 1/N for N stations, so each station could approximately have the same transmission bit rate. H ow ever, the stations’ bit rate can be made variable by allocating several slots to

the sam e channel o r by changing the duration o f time slots. Nevertheless, in TD M A the bit rate allocated to a station is static and this medium access approach, like FDM A, is

not very desirable for bursty traffic.

B oth F D M A and T D M A can be used to connect to a base station or a controller in two

ways; using frequency division duplexing (FDD) and time division duplexing (TDD).

F D D provides tw o distinct bands o f frequencies for each station (two channels), the

forw ard band (forward channels) that provides traffic from the base station to other

stations, and the reverse band (reverse channels) that provides traffic from the stations to

the base station. The frequency separation between each forward and reverse channel is

constan t th roughou t the system regardless o f the particular channel being used (usually

68

45 M H z). T D D uses time instead o f frequency to provide a forward and a reverse link. In

T D D , each station has both a forward and a reverse time slot to facilitate bidirectional

com m unication . I f the time separation between the forward and reverse time slot is small, then the com m unication between the base station and other stations seems to be

sim ultaneous. The FD M A channelization technique is used with FDD, while the TDM A

channelization technique can be used with either FDD or TDD. CDM A provides another

type o f channelization technique. In TD M A and FDM A the transmission from different

s tations is clearly separated either in time or in frequency. The CD M A technique

com bines the T D M A and FD M A techniques and the transmissions from different stations occupy the entire frequency (like TDMA) at the same time (like FDMA).

5.1.1 D irect Sequence Spread Spectrum

Transm issions from different stations are separated with the use o f the Direct

Sequence Spread Spectrum (DSSS) method where a baseband data, before being transm itted , is modulated into a signal that occupies a much larger bandwidth than the original data. The technique is conceptually quite simple; the data is modulated via the

X O R operation with a much higher data rate spreading code. Figure 14 shows an

exam ple o f the D SSS method:

1 0 1 1 0 1 0 0^ Data inpul A | |_______ | |_______ | |________________

\ 1~5 0 ^ 0 1 1 0 1 0 0 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 1 0

)/ 0 1 1 0 0 1 1 0 0 1 10 1 0 1 1 10 1 0 0 0 1 1 1 0 1 101 1 0

_ n _ n r i n r n n i u i t l

0 1 1 0 0 1 1 0 0 1 1 0 1 0 1 1 1 0 1 0 0 0 1 1 1 0 1 1 0 1 1 0

j T j ^ “ i n ^ i n _ r T n r L

1 0 0 1 0 1 1 0 1 0 01 0 10 01 0 1 0 w o 0 10 w o w o

r L n n ^ j i ^ ^ i r ^ L T i r x

1 0 1 1 0 ^ 0 0

I I I

Received signal C

Locally generated PN b it stream identical to B

above

D ata ou tp u t A = C ® B

Figure 14: Direct Sequence Spread Spectrum.

69

The spreading code is generated by a pseudo-random number generator. The spreading

code only appears to be randomly generated but is in fact known to both the transiver and receiver. It can be very long, even infinitely long and must satisfy the following

im portant properties, it has to be uniformly distributed, independent, and to have the correla tion and cross-correlation property. The spread signal can not be detected by an eavesd ropper unless they can generate the exact same spreading code and can

synchronize its generation in time. The energy from the spread wideband signal within a

na rrow band is very low, and even if detected will appear to be a random noise. As more

D S S S signals occupy a wideband channel the noise baseline will eventually increase until com m unica tions is no longer possible.

In general, the result o f the DSSS process produces a very robust signal for transmission. W e will go back to the CD M A technique once we introduce the third generation o f m obile com m unica tions systems which use a new technology called Wideband C D M A (W C D M A ).

T he preceding discussion about DSSS requires that both the transmitter and receiver select very long random spreading codes ahead o f time before they even begin to com m unica te . This is very impractical and another method that will provide an

au tom ated m eans o f generating pseudo random spreading codes is needed. A simple genera to r po lynom ial (a shift register) used for the cyclic block codes can be modified to produce these spreading codes. Figure 15 shows a right shifted generator polynomial:

11-1

Figure 15: Right Shifted Generator Polynomial.

I f the initial s tate o f the registers R ^ .R ^ - i , - appropriate feedbackconnections are selected, then the contents o f the registers will cycle through all possible

2 " — 1 nonzero states before repeating. The resulting sequences are o f the maximum length possib le and can be considered to approximate random binary sequences in the

sense that shifted version o f a sequence are approximately uncorrelated. The length o f the sequences depends on the number o f the registers in the shifter and it is left to a designer

to decide how m any elem ents the sequences will have. Let’s now take a look at the

70

process o f generating approxim ately random sequences by a right shifter that has 4

reg isters and w hose initial state is: 1 0 0 0. The same shifter configuration shown in

F igure 15 is used having 4 registers, from Rg to R g . By having the same hardw are (right sh ifter) installed at both ends, the transm itter and receiver can generate the same

sequences and synchronize them by im plem enting a constant delay. This is crucial to reco v er the received signals and extract them for the right channels:

Time R^ Ri R. Rn

0 1 0 0 0 4- Initial State1 0 1 0 0

2 0 0 1 03 1 0 0 1

4 1 1 0 0

5 0 1 1 0

6 1 0 1 1

7 0 1 0 1

8 1 0 1 09 1 1 0 1

1 0 1 1 1 0

11 1 1 1 1

1 2 0 1 1 1

13 0 0 1 1

14 0 0 0 115 1 0 0 0

W e can see tha t 15 different sequences are generated before the sequence 1 0 0 0 repeats

itself.

5 .1 .2 C o d e D iv ision M ultip le Access

The th ird generation o f mobile systems is designed to support wideband services

such as high speed Internet access, video and high quality image transm ission along with

the voice service. The W CD M A technology is developed in order to create a global

standard for real tim e m ultim edia services. With the support o f International T elecom m unication U nion (ITU ) a specific spectrum was allocated (2GHz) used for

sharing inform ation am ong the third generation system s. The W CDM A system s use the C D M A m edium access technique but a bit different than we have introduced in the

71

earlier paragraphs. It goes a step further in the CDM A technology by using a so-called

ch ip sequences instead o f pseudo-random generated spreading code sequences.

L et s suppose that a data is generated at R bits/seconds and that each bit in the data is transfo rm ed into G bits by m ultiplying each bit (represented by either a +1 representing a

1 o r a -1 represen ting a 0) by G chip elements. All ch ip 's elem ents are also represented

by e ither a +1 o r a -1. The ehip sequence is produced by a special code and appears to be

random excep t that it repeats after a very long period. The spreading factor G is chosen in such a w ay th a t the transm itted signal occupies the entire speetrum. The chip sequences

th a t are used for different channels must be selected so that any pair o f chip sequences w ill have low cross-eorrelation. Otherwise, the reeeiver would not be able to separate the

tran sm issio n s com ing from stations using these chip sequences. One way to ensure that the cross-eo rrela tion betw een any pair o f chip sequences will have low cross-correlation is to use the o rthogonality property.

I f w e define tw o chip sequences, A and B, both w ith the same num ber o f elem ents n as:

A = ( a ] ,a 2 ,a 3 , . . . ,a p ), and

B = (b i ,b 2 ,b 3 ,. . . ,b p ) .

T hese tw o chip sequences are said to be orthogonal if their inner product, also called the

d o t p ro d u ct, is zero:

A B — ^ ^ a ; - bj = a j - b j + a 2 'b 2 -i t&n ’^n (^9)i = l

Since each ch ip sequence consists ft"om +1 and -1 elem ents, we can define that:

nA • A = ^ ^ a ;^ = a[^ + a 2 ■!---------------= " , (40a)

1=1

g . B = ^ ^ b j^ = bj^ + b 2 ^ H 1 bj = n. (40b)

i = i

I f w e now assum e that w e w ere sending a binary 0 in the channel that uses the chip

sequence A , and a b inary I in the channel that uses the chip sequence B. The binary 0

spread o u t by the chip sequence A will be:

72

0 = -A = (-a,,-a2 , .„ -a J ,

w hile the b inary 1 spread ou t by the chip sequence B will be:

1 = B = ( b i ,b 2 , . . . ,b „ ) .

The aggregate signal that will appear in the spectrum as the result o f adding the spread versions o f b inary 0 and 1 is:

-5 11 ) “ ( “ ^ 1 + 1>],—32 + b 2 , . . .,-3^1 + b^ ) = —A + B. (41)

W hen a receiver tries to recover the data sent through the channel w ith the chip sequence

A, it w ill m ultip ly the received signal T with the chip sequence A and then integrate (add) a ll th e e lem ents o f the resulting sequence:

n

^ ^ t j 3; = t] -3] + t 2 32 H l-tn ‘^n = T A = ( - A + B )-Ai = l

= - A A + B A - - n + 0 = - n .

S im ilarly , the data in the channel w ith the chip sequence B will be decoded as:

n

^ tj bj = t] b] + 12 b 2 4 = T B = (—A + B) Bi= ]

= —A •B + B ‘B = 0 + n = n.

Sum m ariz ing the results o f the m ultiplying the received signal with the appropriate chip sequences, the receiver will determ ine that a binary 0 has been sent through the channel w ith the chip sequence A (-n) and a binary 1 through the channel with the chip sequence

B (n ) .

L e t’s now take a look how we can choose chip sequences to use them within the same spectrum but for different channels. As we know from the discussion above, those chip

sequences m ust be orthogonal among them selves. That means that the dot product o f any

ch ip sequence w ith itse lf will yield a positive num ber n, where n is the num ber o f e lem ents in the chip sequence. Also, the dot product o f any chip sequence with its

com plem en t w ould yields a negative num ber n. And finally the dot product o f any

sequence w ith any other sequence or their com plem ents will yield a zero number.

73

5.1 .2 .1 W alsh O rthogonality

n = 2

O ne o f the m ethods to obtain orthogonal chip sequences o f length n, where

is to use the W alsh matrix. The Walsh m atrix is build from binary coefficientsand is defined recursively as:

W2n =

W i = [ + l l

Wn W„

W„ Wn

(42a)

(42b)

w here W n is the com plem ent o f the element W^ ..In some literature the W alsh m atrix

W% is defined as having its only elem ent equal to - I. Regardless o f the type o f

the W | W alsh m atrix being used, higher order W elsh m atrices (n = 2, 4, 8 , . . .) provide a set o f o rthogonal sequences. It is straight forward to show the construction o f the W alsh m atrix fo r n = 2, 4, 8 o r any higher order W alsh matrix. The W 2 is formed by having

m atrices W j and W 1 as its elements:

W o =+ 1 +1

+ 1 -1

T hen, the W 4 m atrix is form ed from W 2 according to (42b):

W4 =

+ 1 +1 +1 +1

+ 1 —1 +1 —1

+ 1 +1 - 1 - 1

+ 1 - 1 - 1 +1

A nd finally , the W g m atrix is form ed from the W4 m atrix and its com plem ent W 4 ;

74

Wg =

+1 4-1 + l + l 4-1 4-1 4-1 + \4-1 - 1 + l - 1 + l - 1 + 1 - 14-1 4-1 - 1 - 1 4-1 + \ -1 - 14-1 - 1 - 1 4-1 4-1 - 1 -1 4-14-1 4-1 4-1 4-1 - 1 -1 -1 -14-1 - 1 4-1 -1 -1 + ] -1 4-14-1 4-1 - 1 -1 -1 -1 4-1 4-14-1 - 1 -1 4-1 -1 + 1 + 1 -1

G enerally , som e o f the n orthogonal sequences in the do not alternate quickly

b etw een +1 and -1 and as the result these sequences w ill not produce a transmitted signal

that is spread over the w hole available spectrum. Due to this fact, the W alsh matrices are

prim arily used to provide channelization while in practice the matrices are com bined with

other additional spreading such as D SSS.

A ll the tim e in our d iscussion about CDM A technology, w e assumed that at the receiver

end each received signal has the same power level. This is very important assum ption and

in order to w ork properly the C D M A technique requires all signals at a given receiver to

have approxim ately the sam e power. If this is not the case, then transm issions from

nearby stations could overw helm transm issions from distant stations. This problem is

called the near-far problem. The goal is that the receiver receives the same power level

from all stations regardless o f distance from the receiver (within a cell). I f the pow er level

from on e station is higher than needed, the quality w ill be excessive thus taking a

disproportionate share o f the resources and generating unnecessary interference with

signa ls from other stations. On the other hand, if the power level from a station is too

low , it w ill result in poor quality o f the received signal. In order to keep the received

p ow er at a suitable level, the W C D M A standard defines a fast power control that updates

pow er lev e ls from each station 1500 tim es every second.

L e t’s n ow take an exam ple that will summarize everything that has been said regarding

the C D M A technology:

EXAMPLE (CDMA Channelization):

A tw o c h a n n e l s y s te m u s e s th e f o l l o w i n g tw o o r th o g o n a l c h ip s e q u e n c e s ( ta k e n a s th e

3'^‘ a n d r o w f r o m th e W e ls h m a tr ix W g.'

C _ A = ( + 1 , + 1, - L - I , + L + L - L -V>

75

- i . -1 , -1 , -1 . + 1, + 1 ).

T r a n s m i t b i ts . 1 0 0 o n th e c h a n n e l A a n d b its : 0 1 0 o n th e c h a n n e l B . R e c o v e r th e

s i g n a l o n th e c h a n n e l A .

S O L U T I O N :

T o t r a n s m i t a 0 t h r o u g h th e c h a n n e l A , th e s y s te m w i l l a p p ly th e c o m p le m e n t o f th e

c h i p s e q u e n c e C _ A a n d to tr a n s m i t a 1, it w i l l t r a n s m i t th e c h ip s e q u e n c e C _ A its e lf .

S im i la r ly , to t r a n s m i t a 0 th r o u g h th e c h a n n e l B , th e s y s te m w i l l s e n d o u t th e

c o m p l e m e n t o f th e c h ip s e q u e n c e C _ B a n d to tr a n s m it a 1, it w i l l s e n d o u t th e c h ip

s e q u e n c e C _ B .

F o r t h e c h a n n e l A , b i t s 1 0 0 a r e s p r e a d b e fo r e b e in g t r a n s m i t te d a s

s h o w n b e lo w :

0

Chaimel A11

0 0

1 14-

0 0 : 0 04-

1 1

0 0

1 1

0

0 0

1 1

0 0

1 1

O n th e c h a n n e l B , b i t s 0 1 0 a r e s p r e a d o v e r th e s p e c tr u m in th e f o l l o w i n g m a n n e r :

Cliaimel B0 1 0 ;

-------------------------- 1

11 11 11 11 11 11 :

0 0 0 0 0 0 0 0 0 0 0 0111

S o , t h e d a t a b e i n g s p r e a d a t th e c h a n n e l A is:

A = 1, + 1, - 1, - 1, + 1 ,+ 1 ,-1 ,- 1 ,- 1 ,- 1 ,+ 1 ,+ 1 ,- 1 , - 1 ,+ l . + l r L - l , + l , + l . - l - L ' ^ l . + l ) .

w h i l e t h e s p r e a d d a ta a t c h a n n e l B is:

76

T h e a g g r e g a te s i g n a l (A + B ) m a d e u p b y a d d in g th e s ig n a l s o n th e c h a n n e ls A a n d B lo o k

l i k e th i s :

SigiialA+

Signal B

1 1

1 1

0 0

0 0

0 0

0 0

1 1

1 1

0 0

0 0

11

1 1

A + B = (+ 2 , +2,-2, -2 ,0 ,0 .0 ,0 , -2,-2, +2, + 2 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,- 2 ,- 2 , +2, + 2).

A t t h e r e c e i v e r s id e , to r e c o v e r th e s ig n a l m e a n t f o r th e c h a n n e l A th e a g g r e g a te s ig n a l

i s m u l t i p l i e d w i th th e c h ip s e q u e n c e A a t th e c o r r e la to r :

CIlipSequence

A

1 1

0 0

1 1

0 0

1 1

0 0

1 1

0 0

1 1

0 0

1 1

0 0

C _ A = ( + I , + I , - l , - l , + l , + l , - l , - l , + 1 .+ 1 , + 1,+ 1 ,-1 ,- I , + l , + l , - l , - l , + l , + l , - l , - l ) ;

Outjiut of

1 1 1 1

1 1 1 1

CoiTelator 00 00 00 00

00 00 00 00

C o r r O u t = (+ 2 ,+ 2 ,+ 2 , + 2 ,0 ,0 ,0 ,0 ,-2,-2, -2, -2,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,- 2 ,- 2 , -2 , -2 ),

N o w , a t t h e i n t e g r a to r e a c h s e q u e n c e o f 8 b its c o m in g o u t o f th e c o r r e la to r w i l l b e a d d e d

d u e to th e f a c t t h a t e a c h b i t is r e p r e s e n te d ( s p r e a d ) b y 8 b i ts ( c h ip s e q u e n c e ) .

O u t p u to f

ü i t e g i ' a t o r

1 1

1 1

1 1

1 1

0 0

0 0

0 0

0 0

77

0 0

0 0

0 0

0 0

I n t O u t = (8, -8 , -8 ):

I t i s n o w e a s y to d e t e r m in e th a t th e r e c o v e r e d s ig n a l o n c h a n n e l A is:

RecoveredSignal

oilCliaimel A

0 0

5.2 Matlab Implementation of WCDMA

S ix M atlab files im plem ent the process o f W C DM A. The main file

“A g g r eg _ o n e .m ” that contains calls for the routines called at various times during the

ex ecu tio n o f the program. It first calls the routine that create the W elsh matrix, W g. N ext,

it perform s the D S S S modulation. Then, the chip sequences for the channels are extracted

from the W alsh matrix (row s 3 and 7). An aggregate signal is created next by performing

the A + B matrix operation. The noise is created and added to the aggregate signal that is

at th is point ready to be transmitted through the air. At the receiver end the received

sign a l is m ultiplied with both the chip sequence A and chip sequence B. The all elem ents

o f th e resulting sequences are added up and a received bit is recovered (extracted). The

M atlab function “W elsh_M atrix.m ’' that creates a W elsh matrix with the size 8 x 8 . The

M atlab function "Comp le ment, m" that creates the com plem ents o f the chip sequences A

and B . T he M atlab function “D SSS.m ” that performs the Direct Sequence Spread

Spectrum on both channels. The Matlab function “C D M A _H ncode.m ’ in which each bit

o f data is represented (spread) by the chip sequence. A 1 is replaced by the chip sequence

w h ile a 0 is replaced by the com plem ent o f the chip sequence. And the last one,

“C D M A D eco d e .m ” Matlab function in which the received signal is multiplied with the

78

chip sequences and the resulting sequences are added up. The decision whether a I or a 0

is received is m ade on the follow ing way: if the value obtained from the integrator is

g rea ter o r equal to 4, a bit I is recovered. Similarly, if the value is less or equal to -4, a bit

0 is recovered . The values obtained between -4 and 4 give us the meaning that that part o f the received signal is no t m eant for that particular channel;

A t the end o f the d iscussion about W CDMA let's take a look how the M atlab code works

w ith the sam e input conditions as in the example above; bits on the channel A; 1 0 0, bits and the channel B: 0 1 0. N ote that the M atlab code implements the DSSS m odulation w hile the exam ple above did not.

> > a g g r e g _ o m

O u tp u t f r o m C o r r e la t o r ( c h a n n e l A ) : 0 0 - 1 0 - 2 - 2 - 2 -2

O u tp u t f r o m I n t e g r a t e d o u tp u t: -9

O u tp u t f r o m C o r r e la to r ( c h a n n e l A ) : 2 2 2 2 0 0 0 0

O u tp u t f r o m I n t e g r a t e d o u tp u t: 8

O u tp u t f r o m C o r r e la to r ( c h a n n e l A ) : - 2 - 1 - 2 - 2 0 0 0 0

O u tp u t f r o m I n t e g r a t e d o u tp u t: - 7

O u tp u t f r o m C o r r e la t o r ( c h a n n e l B ) : 0 0 -1 0 2 2 2 2

O u tp u t f r o m I n t e g r a t e d o u tp u t: 7

O u t p u t f r o m C o r r e la t o r ( c h a n n e l B ): 2 2 2 2 0 0 0 0

O u tp u t f r o m I r i t e g r a te d o u tp u t: 8

O u t p u t f r o m C o r r e la t o r ( c h a n n e l B ) : - 2 - 1 - 2 - 2 0 0 0 0

O u tp u t f r o m I n t e g r a t e d o u tp u t: - 7

S e n t o n c h a n n e l A : 1 0 0 R e c o v e r e d o n c h a n n e l A : 1 0 0

S e n t o n c h a n n e l B : 0 1 0 R e c o v e r e d o n c h a n n e l B : 0 1 0

C o m p a r e d a t a s e n t & r e c o v e r e d o n c h a n n e l A : 0 C o m p a r e d a t a s e n t & r e c o v e r e d o n c h a n n e l B : 0

W e see tha t at the receiver the data for both channels A and B is recovered properly . O ne m ight notice that in the example above the values out o f the integrator are

79

alw ays either 8 or -8, w hile from the M atlab printout we can see that the values out o f the

in teg rato r are not alw ays either 8 or -8. This is due to the fact that the M atlab sim ulation

m ixes the aggregate signal w ith random noise before the corrupted signal is decoded at

the receiver end. In the exam ple above we assum ed that the transm ission m edium was noiseless. T he same M atlab file “aggreg one.m ” used for the main sim ulation was used

fo r the exam ple.

80

Chapter 6: RESULTS

In o rder to sim ulate the operation o f the source-channel codec, two signals are transm itted through the system in Figure 1. On the channel A, a 4.99 second long speech file, speech .w av , saying: “The D iscrete Fourier Transform o f a real valued signal is con jugate sym m etric” , is applied. On the channel B, the image file, ‘w orld.jpg ',

represen ting a g ray level still image with the resolution 102 x 132 is applied. Both signals are show n bellow:

speech.wav woild jpq

2 3time (sec.]

4Ü GÜ BIJresolution [pixelf

Figure 16: Speech Data. Figure 17: Still Image Data.

A t the Form atting stage, speech signal on the Channel A is first sampled at the sampling frequency o f 22050 H z w ith the help o f ‘w avread ' M atlab comm and. The sampled version o f the speech signal contains 1 10,033 sample points. In order to explain how these num bers are obtained, we first plot the frequency spectrum o f the speech signal as

show n in Figure 18. From Figure 18 we see that the m axim um frequency oi the signal is 11,025 Hz. A pplying the N yquist Sampling Rate criteria w hich states that the sampling

frequency o f an analog signal m ust be equal or greater than tw ice the maximum frequency o f the signal, it is clear w hy the sample rate o f the speech signal is 22,050 Hz. S ince the num ber o f sam pled points in one second is equal to the sampling frequency, we conclude that the sam pled version o f the speech signal would have 22,050 sampled points

fo r one second. W hen that num ber is multiplied with 4.99, which is the total duration o f

the speech signal, w e get 110,033 sampled points for the entire speech signal. In further

d iscussion the sam pled version o f the speech signal w ill sim ple be referred to as sampled

signal.

81

18001600

1100

1200

1000

800

600

100

200

0

. i l l

1

1

6 BFrequency [Hz]

10 12X 10

Figure 18: Frequency Spectrum o f Speech Signal.

N ex t, the sam pled signal is quantized to 256 levels since a speech signal requires 8 bits to rep resen t each sam ple. The m aximum value o f the sampled signal is 0.9921, while its

m in im um value is -0.7268. Consequently, the quantile interval q is 0.0067. Since the 8 bit quan tizer used in th is project is a linear quantizer, every level in the quantizer (e.g. 256 levels) w ill be o f the same size, the size o f the quantile interval q. At this point we obtain

the quantized signal as a result o f the quantization.

15000PDF of 8 bit quan tiza tion error!

„ 10000

-4 -3 -2error [e]

N orm alized PDF of 8 -b it Q u a n tize rX 10'

m 0.15

error [ej X 10

Figure 19: PDF o f Q uantization Error.

82

T o observe the quality o f the quantization which is measured through the quantization

signal to noise ratio (SN R )q, we first have to obtain the error caused by the quantization.

The erro r is calculated as the difference between the sampled signal and the quantized

signal. T he probability density functions (PDF) o f the quantization error as well as its

norm alized version are shown in Figure 19. The quantization signal to noise ratio is the

ratio betw een the signal pow er and the noise pow er where, in this case, the noise pow er is

the quan tization error. The quantization error power can be found in two ways. The first

one is to in tegrate the product o f the squared difference o f the quantization error and the its m ean, w ith the probability density function o f the quantization error (e.g. see the

equation (11) in C hap ter 2). The second way, which is used in this project, is to calculate the quan tization error pow er as the mean o f the squared error vector:

n

error power =

w here e is the quantization error vector and n is the size o f the quantization error vector. T o get the value in [dB]s:

error pow er [dB] = 10 • log| q (e rro r_ power).

T he signal pow er is obtained exactly the same way but instead o f the quantization error vector, the signal vector, containing the values o f the samples signal, is used. Using M atlab , the quantization signal to noise ratio (SNR)q is then obtained to be:

SNR = 30.9077 [dB],

I f w e recall from C hapter 2; a quick approxim ation o f the (SNR)q for a quantizer could be m ade by m ultip ly ing the num ber o f bits that each sample is represented with by 6 dB.

A pp ly ing that, the theoretical value o f the (SNR)q in this project would be 48 dB. The

reason w hy w e obtain much lower the (SNR)q is due to the fact that we have used a linear quantizer. In general, linear quantizers are wasteful for speech signals since m any o f the quantization levels are rarely used. To support this claim a histogram given in

F igure 20 show s the frequency o f usage for each o f 256 levels for our 8-bit uniform quantizer. It can be seen that the m ajority o f the quantization levels used in the process

falls in the fo llow ing range [96, 125] o f quantization levels. We also see that the quan tization level 111 contains 9,164 sampled points which makes alm ost 10 percents o f

83

all sam pled points in the sam pled signal. A non-uniform quantizer would provide a better

quan tization and the quantization error would be made proportional to the signal size.

T he overall (SN R )q would improve by reducing the error for the predom inant parts o f the signal, a t the expense o f an increase in error for the rarely occurring parts o f the signal.

Usage of Quantizer Levels10000

9000X=111 Y= 91648000

7000

e o o o

5000

I 4000

3000

2000

1000

-50 250200 300

Figure 20: Frequency o f Usage o f Q uantizer Levels.

The last part o f form atting the speech signal is the Pulse Code M odulation, where the quan tization level for each elem ent o f the sampled signal veetor is eonverted to its binary represen ta tion . H ere a sim ple decimal to binary conversion is used where the quantization level (decim al num ber) for each signal elem ent is eonverted to 8 bit binary number. The

ob tained b inary values are appended into a binary sequence that represents the digitized speech data. The data on the Channel A consists o f 880,264 binary digits (8 x110 ,033).

O n the C hannel B, the form atting stage for the image is much sim pler com pared to that o f

the speech signal. W e only have to calculate the PDF o f the image. The image is read into the M a tlab ’s environm ent in the ASCII format. The PDF o f the image, shown in Figure

21 , w ill g ive us the frequency o f occurrence for each o f 256 num bers that the intensities o f the im age’s pixels are represented with. In the process o f calculating the probability o f occurrence fo r each num ber in the [0, 255] range, any num ber in the [0, 255] range that

does not appear in the image matrix is considered as irrelevant since it carries no inform ation relevant to our image. Onee the frequency o f occurrence o f each num ber in

the im age m atrix is calculated, it is divided by the total num ber o f elem ents in the image m atrix , fo rm ing the probability o f occurrence for each num ber in the [0, 255] range that

84

appears in the m atrix. The x-axis lists num bers in the [0, 255] range showing all 256

possib le in tensities o f the image. The y-ordinate lists the probabilities o f occurrence for the intensity o f each pixel that is present in the image.

0.03

0.025

0.02

0.015 -

0.01

0.005 -

Norm alized P D F of th e Im age

50 100 150 200pixel in tensity

250 300

Figure 21; PDF o f the Image.

T he next stage in the system for both Channels is the source encoder stage. Channel A

uses the L em pel-Z iv-W elch algorithm to perform the source encoding; to compress the d ig itized speech signal as much as possible. The principles o f the Lem pel-Ziv-W elch

a lgo rithm are explained in the Chapter 3 and here only the Lem pel-Ziv-W elch table is presented as the result o f the M atlab simulation. Since the table has 45705 entries, it w ould be im possible to list all o f them here in this report. However, the first 50 entries

w ill be listed, thus a reader can get a feeling how the algorithm has performed the operation . D uring the im plem entation o f this algorithm , Fve taken a liberty to change the w ay th a t the code word column is implemented. Instead o f having a fixed-length im plem entation o f the code word (12 bits per code word), I ’ve used a variable-length

im plem entation . In the real world this change would require a bit more complex decoder. The softw are im plem entation o f the decoder only requires knowledge o f how m any bits

are used to encode every single code word. This is solved by adding another colum n in the L em pel-Z iv-W elch table called code word length that provides the information to the

d ecoder about how m any bits are used to encode each code word. Let s take a look at the first 50 entries o f the Lem pel-Ziv-W elch table obtained through the M atlab sim ulation.

85

M E M O R Y MEMORYL O C A T IO N SU B SEO U EN C ES REPRESENT

CO DEW ORD

W ORDLENGTH

I 0 0 .2 1 13 01 1 11 24 10 2 100 35 11 2 101 36 010 3 110 37 110 5 1010 48 1101 7 1111 49 011 3 111 310 0110 9 10010 5II 101 4 1001 412 1011 11 10111 513 0101 6 1101 414 10110 12 11000 515 101101 14 11101 516 1010 11 10110 517 11011 8 10001 518 01011 13 11011 519 01101 10 10101 520 011011 19 100111 621 010110 18 100100 622 11010 8 10000 523 110110 17 100010 624 1011011 15 11111 525 0101101 21 101011 626 10101 16 100001 627 1011010 15 11110 528 1101101 23 101111 629 0110110 20 101000 630 10110111 24 110001 631 00 1 10 232 n o m 17 100011 6

33 101101111 30 111101 6

34 0110111 20 101001 6

35 1011011110 33 1000010 7

36 1101111 32 1000001 7

37 0111 9 10011 5

38 000 31 111110 6

39 001 31 111111 6

40 10111 12 11001 5

41 10110111101 35 1000111 7

42 101110 40 1010000 7

43 01101110 34 1000100 7

86

44 011011100 43 101011045 1101110 32 100000046 0110111001 44 101100147 1011100 42 101010048 11011100 45 101101049 110111001 48 110000150 101111 40 1010001

O nce the tab le is com pleted, the code words from the fourth column are appended to each o ther creating a binary sequence; a new com pressed binary representation o f the speech

data. T he new binary sequence contains 684,407 binary digits, which m akes the com pression ratio o f 22% for the speech signal. W hat is interesting for the Lem pel-Ziv-

W elch a lgo rithm is that when the input sequence is longer, the com pression ratio is better (h igher).

O n the channel B, the image data is source encoded with Huffman Coding Algorithm. As

it has been m entioned in C hapter 3, Huffman Coding Algorithm provides a system atic p rocedure for constructing uniquely decodabie variable-length code words that are e fficien t in the sense that the average num ber o f bits per source is minimized. Since we have 256 possib le intensities for each pixel, it means that we have 256 (from 0 to 255)

possib le sym bols in the source alphabet. Here is the result o f the M atlab sim ulation which assigns a variable-length code word to each o f these 256 symbols as per discussion in C hap ter 3. The alphabet sym bols and their respective binary representations are listed in

the left-to-righ t up-to-dow n manner.

80=1001111011

814=1001011001

819= 10001110100 823= 10001110110

827= 10100101 831= 11110001 835= 1001011100

839= 10100111 843=10001101 847= 10111001

851= 11110010

855= 10111011 859= 100000110

863= 10001000

83=10001110001815=1001011010820=100111110

824=1011001828=110110010

832=110110011836= 10001111000

840=1010110844=100000101848=10101000

852=11110011856=10111100860=10000110

864=1111101

87=10001110010817=10001110011521=101100011525=100111111529=10001010533=10001011537=10100110541=100000100

545=10000101549=10111010553=10000000

557=1101110561=10000111

565=10111101

513=1001011000 518=1001011011 522=10001110101 826=100000010 830=10001110111

834=10111000 838=100000011

842=10001100

546=1111111 550=1101101 854=1011111 858=10101001 862=11110100

866=111001000

87

867=10001001

871= 10101010

875=111001100

880=101000001

884=11110110

888= 10001111010 894=10001111101

898=111010001

8105=111010011 8110=1001100100

8114=111010101

8119=1001100110

8123=10010000011

8 131=1001101000 8 135= 111011000

8140=10010001001 8 144=10010001100

8149=10010001110 8155= 1001101110

8160=111011011

8166=10010010010

8173=10010010100 8177=10010010111

8183=1001110011

8190=10010011101

8197=10010011111

8202=111011101

8209=111011111 8215=10010100101

8222=10010100111

8227=1001110111

8236=10010101010

8 240=10010101110

8244=1010111 8 2 48= 111000 8253=1001111010

868=111001001

872=1001011101

876=1001011110 881=10001111001

885=111001110

889=10001111011

895=10001111110

8100=1001100010

8107=10010000000 8111=1001100101

8115=100001000 8120=11111000

8124=101000100 8132=10010000101

8136=10010001000 8141=1001101010 8145=111011010

8150=1001101100

8157=101000110 8161=10010001111 8169=10010010011 8174=1001110010 8180=10010011000

8185=1001110100 8191=1001110101 8198=10010100000

8204=111011110 8210=10010100011

8217=11111001 8223=101100010

8228=111100000 8237=10010101011

5241=111100001

8245=10101011 8249=10110000

8254=1100

869=111001010873=11110101 877=11011000

882=1001100000 886=111001 111

890=1 11 010000 896=100000111 8101=10001111111

8108=10010000001 8112=111010100

SI 16=101000011 8121=111010110

8126=10010000100 8133=10010000110

8137=1001101001 8142=10010001010

8147=1001101011 8151=1001101101

8158=1001101111 8164=10010010000

8170=111011100 8175=10010010101 8181=10010011001 8186=10010011011 8194=101000111 8199=10010100001 8205=10010100010

8211=100001001 8218=10010100110 8224=10010101000

8233=1001111000 8238=10010101100

8242=10010101111

8246=1101111

8251=]1010 8255=0

870=101000000 874=111001011 879=1001011111

883=111001101 887=101000010

892=10001111100 897=1001100001 8103=111010010

8109=1001100011 8113=10010000010 8117=11110111

8122=1001100111 8130=111010111 8134=10010000111 8138=111011001 8143=10010001011 8148=10010001101 8154=101000101

8159=1001110000 8165=10010010001 8171=1001110001 8176=10010010110 8182=10010011010

8187=10010011100 8196=10010011110 8201=101001000 8208=101001001 8213=10010100100

8219=1001110110 8226=10010101001 8234=1001111001 8239=10010101101

8243=10001110000 8247=1111110 8252=101101

88

A verage Code Length: 3.482

Entropy o f the Source H(x)= 3.42933 Efficiency = 98.4872%

A t th is point in the system , the image data is represented by 8,705 binary digits. The

e ffic iency as the ratio between the entropy and the average code length is 98.48% , which is considered to be an alm ost m axim um com pression.

The fo llow ing stage in the source-channel codec system is the channel encoding. As

d iscussed in C hap ter 4, the purpose o f the channel encoder is to introduce, in a control m anner, som e redundancy in the data binary sequence that can be used at the receiver to

overcom e the effects o f noise and interference encountered in the transm ission o f the signal th rough the channel. In other words, redundancy in the data binary sequence aids

the receiver in decoding the desired data sequence. The Channel A uses the Linear Block C ode algorithm to add redundant bits to the data binary sequence obtained from the

source encoder stage. The (12, 8) type o f Linear Block Code is used, meaning that every 8 b its from the data binary sequence will be transform ed into a 12 bit sequence, thus

add ing 4 redundant bits. To show the ability o f the code to detect and correct errors, as per the d iscussion in C hapter 4, a Bit E rror Rate (BER) versus Signal to Noise Ratio (SN R ) p lo t is obtained and shown in Figure 22.

B E R vs. SNR

m 10'

SNR[dB]

Figure 22: B ER vs. SNR for (12,8) Linear Block Code.

89

The B it E rro r Rate is defined as the num ber o f errors not being corrected by the decoder

d iv ided by the total num ber o f bits in the input sequence to the Linear Block Code

system . The Signal to N oise Ratio is, as before, defined as the ratio between signal pow er

and the noise pow er. Four points are plotted on the graph obtained for four dilTerent

values o f the noise. From the plot we see that the first uncorrected errors occur while the

S N R is still relatively high (~33 dB). This would lead us that the code is not sufficiently

capab le o f perform ing erro r corrections since the BER vs. SNR curve from the reference [2, pp 267] starts having first undetected errors around 10 dB.

T he reason for the early appearance o f uncorrected errors lies in the fact that the Linearn-k - 1 = 15B lock C ode ( 1 2 x 8 ) used in the simulation can detect and correct only 2

e rro r patterns (see C hapter 4). That means that only 12 single bit error patterns plus three

2 b it e rro r patterns can be detected (in the project a single bit error patterns are detectable and correc tab le only). To im prove the performance o f the code, more error patterns must

be detec ted and corrected which requires the longer tim e execution o f the code (it takes 5

hou rs for M atlab to generate ( 1 2 x 8 ) code 's B ER vs. SNR). However, in the real world o f com m unications errors happen approxim ately 1 in 10^ transm itted bits and the possib ility o f having 2 errors in a row are very rare. Here are the m ain m atrices obtain in

the M atlab sim ulation for the (12, 8) Linear Block Code:

P = [ 1 1 1 1 1 0 0 0

010

Ht = [ 1 0

1 0 0 01 101011

0 1

0 0 0 0

0 1 0

000

10110 ];

0 0 0 0101

010

0 01 101011

10110 ];

G = [ 1 0 1111111000

01 0 0 0 1 1

0111011

0

1 0 0 1 0 0

00

0 0 0 0 1 0 1 0 0 0

0 0 0 0

0

0 0 0000

00

0

0 0 0 0 0

0 0 0 0

0 0 0 0 1 1 1 0

0 0

0 0 0 000 0

0 0

0 0

00 00 0

00 0

I0 1

0 0 0 0

00

0 001 ];

90

[ 0 1G 1 G G

G G G G G G G GG G G G G G G G

1G G G 1G G G GG G G GG G G GG GG GG GG G

GGG

G G G G G GG G

G G G G G

G GG G

]G G G G G 1G G G G G G

GG

o g g g g g g g g g o g g g g g g g g

G G G G G G G G 1 G G G G G G GG 1 G G G G G GG G 1 G G G G G

G G G G G 1 G G G Go g g g g i g g g

G G 1 G

Se = [ G G G 1 G G

1 ];

G 1 G G G G

G 1 G 1 1

G 1 G 1 G 1

G G

G G 1 G

1 G

G 1G G 1 1G1G1

1G11

0 ];

The C hannel B applies a different algorithm for error detection o f the (2,1,4)

C onvolu tional C ode. It is called the Viterbi algorithm. For this algorithm the m easure o f

effic iency o f the code cannot be established through the BER vs. SNR since the Viterbi

a lgo rithm guaranties that any input sequence will be correctly decoded as long as the decod ing delay is sufficiently long. The decoding delay used in this project is 2G. The

p rincip les o f the C onvolutional Code algorithm as well as those o f the Viterbi algorithm are fu lly presented in C hapter 4.

The last stage on the transm itter side is the W CDM A stage that consists from Direct Sequence Spread Spectrum m odulation and CDM A channelization (M edium Access

C ontro l). F irst, each signal is XO R-ed with different pseudo-random generated sequences and th a t each bit o f these new obtained signals is represented by the chip sequences. Two chip sequences obtained from the W elsh matrix are used. For Channel A:

C _ A = [ + 1 +1 + 1 - 1 - 1 ];

and fo r the C hannel B:

c _ B = [ + i +1 -1 -1 -1 -1 +1 + 1 ];

and then both signals are added. One aggregate signal is transm itted through the air

m edium . To sim ulate the noise and interference during the transm ission through the airy channel a random sequence o f Is and Gs is produced and added to the transm itted signal.

The noise is generated by using the following M atlab comm and:

noise = randsrc(1,len_trans,[0,1;p_0,p_l]);

91

w here p_0 is the probability o f occurrence o f 0 while p i is the probability o f occurrence

o f 1. The variable le n tr a n s is the length o f the transm itting aggregated signal. In the

sim ulation , p_0 and p i are set to 98% and 2% respectively. The 2% occurrence o f ones in the noise g ives us approxim ately 60,000 errors that are to be corrected.

A t tbe receiver side, the received corrupted signal is first m ultiplied with both chip

sequences, C A and C B, in order to extract the signal meant for the Channel A as well

as fo r the C hannel B. The error correction is performed as well as per discussion given in

C hap ter 5. I f not all errors are detected and corrected during this stage, the remaining

ones w ill be corrected during the channel decoding stage (Linear Block Code decoder for the C hannel A and Convolutional Code decoder for the Channel B). After separating the

received signal to tw o Channels A and Channel B signal, each o f these signals is XOR-ed w ith the pseudo generated noise used at the transm itter end (both pseudo generated

sequences are known in advance to the receiver). At this point, we have two received

signals that represent the received speech and still image data. The decoding principles and capab ilities o f L inear Block Code and Convolutional Code algorithm are presented and d iscussed in Chapter 4 and will not be discussed here again. A reader is encouraged to read C hap ter 4, in order to understand the decoding principles o f the both channel decoders. At the source decoding stage, the speech and image data are reconstructed

u sing Lem pel-Z iv-W elch and Huffman Code tables to their analog equivalents and the bo th signals are displayed. Once again, the coding tables are known to the decoder in

advance for both techniques. The exception is the adaptive Huffman tree decoding, where the encoder and decoder only share the knowledge about the num ber o f symbols in the

a lphabet.R eceived Speech S ignal

-i------ 1------~rReceived Image

time s e c .

20 40 60 100 120

F igure 23; Received Speech Data. Figure 24: Received Image.

92

A s the very last step on Channel A, the received digital speech signal is converted back to its analog representation and is played in M atlab using the ‘w avplay’ command. The

received speech signal plotted in the time domain is shown in Figure 23. The received im age is show n in Figure 24.

Conclusion

The project presents a source-channel codec for a W CDM A based multim edia system. The system consists from two Channels, A and B, where the Channel A is reserved for transm itting /receiv ing a speech signal while the Channel B transm its/receives a still im age data. O n the transm itter end, the Channel A has the following stages: formatting (sam pling , quantizing, and PCM ), source encoding (Lem pel-Ziv-W elch Encoder), channel encoding (L inear Block Code Encoder), and W CDM A medium access control (D SSS plus Chip Sequencing) stage. On the other side, the Channel B has these stages at the transm itter: form atting (ASCII), source encoding (Huffman Code Encoder), channel encod ing (C onvolutional Code Encoder), and W CDMA medium access control (DSSS plus C hip Sequencing) stage.

A t the receiver end, firstly, the received aggregate signal is divided into two signals; the signal m eant for the Channel A and the signal meant for the Channel B. The signal meant for the C hannel A (speech signal) goes through the following stages; channel decoder (L inear B lock C ode Decoder), source decoder (Lempel-Ziv-W elch Decoder), and digital to analog conversion. The signal that is selected for the Channel B goes through; channel decoder (C onvolutional Code Decoder) and source decoder (Huffman Code Decoder).

T hroughou t the project 1 have faced with quite a few challenges in implementing the codec. The biggest one was the time o f the execution. M atlab as a language o f choice for

th is p ro jec t has not had the needed speed capabilities. Some implementations o f the algorithm s such as Lem pel-Ziv-W elch encoding and Convolutional Code decoding have

taken w ay too m uch tim e, approxim ately 2 days for each simulation! An interested student w ho would have patience and willingness to update this project should choose a p rogram m ing language that is based on the concurrent type o f code execution instead o f

using a language based on sequential type o f code execution. The sequential languages such as M atlab execute a code in a line by line fashion, while the concurrent languages such as V H D L execute several lines o f code at the same tim e (the analogy would be as if

a com puter had several processors that are capable o f executing the lines at the sam e tim e). T h a t’s w hy a concurrent language would be more appropriate for this type o f

p ro ject. The ideal solution would be to implement this project in hardware.

93

A nother recom m endation to im prove this project is not to use Lem pel-Ziv-W elch a lgorithm for a speech signal. Besides the fact that the execution o f its simulation is too

slow , another problem is that a Lem pel-Ziv-W elch code word can contain 12 bits only. I f a b inary data sequence is too long, as they usually are, there is a need to create more than one tab le w hich takes additional space in the memory. The better solution would be to use adaptive H uffm an C ode algorithm to compress a speech signal.

T he error detecting and correcting capabilities are another thing that could be improved. A L inear B lock C ode that would detect and correct as many bit error patterns as possible is needed (in the real world (127, 92) linear block code is used). As stated earlier in this concluding section, a program m ing language that can execute the code faster must be used (e.g. C ++) to improve the speed o f detecting and correcting processes.

T o sum up, the project shows all necessary stages that are needed for transm itting and receiv ing m ultim edia signals through a digital system. Com plex m ultim edia systems may have m ore stages than the system show n in this project does, but the stages presented and im plem ented in th is project make a foundation for all digital systems and absolutely can

no t be excluded as a part o f their implem entations.

94

References:

1] F. G. Strem ler, I n t r o d u c t io n to C o m m u n ic a t io n S y s te m s . , Add id son - Wesley Series in E lectrical Engineering, 1997.

2] B. Sklar, D i g i ta l C o m m u n ic a t io n s : F u n d a m e n ta ls a n d A p p l ic a t io n s . , PTR Prentice Hall, 1998.

3] J. G. Proakis, D ig i ta l C o m m u n ic a t io n s . , M cGraw Hill, 2001.

4] T. S. R appaport, W ir e le s s C o m m u n ic a t io n s : P r in c ip le s a n d P r a c t ic e . , Prentice Hall, 2002.

5] A. L. G arcia, I. W idjaja, C o m m u n ic a t io n N e tw o r k s : F u n d a m e n ta l C o n c e p ts a n d K e y A r c h i t e c tu r e s . , M cGraw Hill, 2003.

6] M . A bram ovici, M. Bauer, D ig i ta l S y s te m s T e s t in g a n d T e s ta b le D e s ig n ., Wiley In ter Science, 1990

7] W . W . W u, E le m e n t s o f D ig i ta l S a te l l i te C o m m u n ic a t io n . , Com puter Science Press, 1995.

8] R. J. S luyter, “D igitalization o f Speech” in P h i l ip s T ec. R e v ., vol. 41. pp. 201-221.

9] N . S. Jayant, D ig i ta l C o d in g o f W a v e fo r m s ., Prentice Hall, 2000.

10] T. J. Lynch, “D ata Com pression Techniques and Applications”, in L i fe t im e L e a r n in g P u b l ic a t io n s , Belmont, Calif., 1985.

11] J. M . T ribolet, “ Speech Coding”, in I E E E T r a n s a c t io n s o n C o m m u n ic a t io n s , vol. 27, no. 4, April 1979, pp. 710-737.

12] W. W. Peterson, E r r o r C o r r e c t in g C o d e s . , M IT Press, 1972.

13] G. C. C lark, E r r o r C o r r e c t in g C o d in g f o r D ig i ta l C o m m u n ic a t io n s . , Plenum Press, 1981.

14] R. G. G allager, I n fo r m a t io n T h e o r y a n d R e l ia b le C o m m u n ic a t io n . , John W iley & Sons, 1968.

15] The Student Edition o f M ATLAB, T h e U s e r ’s G u id e . , The Math W orks Inc., 1995

16] A . J. Viterbi, C D M A : P r in c ip le s o f S p r e a d S p e c tr u m C o m m u n ic a tio n . ,

A ddison - W esley Readings.

95

[17] “C oding T echniques for Digital Com m unications,” class notes for EE8110,

D epartm ent o f Electrical and Com puter Engineering, Ryerson University,

W inter 2005.

96

Appendix A:

Matlab Simulation Files

97

%%%%%%%%%%%%%%%%%%%%%%% M-File : "Main.m" ^% AUTHOR: Boris Backovic %% page: l/l %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

clear all;close all;% FORMATTING STAGE %

%Channel A Sampling;Quantizing;PCM;

%Channel BFormatting;

% SOURCE ENCODING %%Channel A

LZW_Encode;%Channel B

Image_Huf fman;% CHANNEL ENCODING %

%Channel AGenerate_Matrices ;LBC_Encode;

%Channel BConvolEncoder;

% WCDMA Medium Access Control % aggreg_one;

% CHANNEL DECODING %%Channel A

LBC_Decode ;%Channel B

Viterbi_Decoder;% SOURCE DECODING %

%Channel A DtoA;

%Channel BDecoder_Image_Huf fman;

98

;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% M-File : "Sampling. m" ^% AUTHOR: Boris Backovic %% page: 1/1 %e o o o o o o o o o o o o o o o o o o o o ^ "b "o ^ "6 "

clear all; close all;

display('IN_Sampling.m' )[sig, f s] =wavread{ ' f ilel.waV ) ; wavplay(sig,fs);t =(1 :length(sig))/fs ;plot(t,sig);grid;title('ASCII Version of The Speech Signal'); xlabel('time [sec.]'); ylabel('voltage [V] ') ; drawnow;F_sig=fft(sig,num_fft_pnt) ; f igure;plot (abs (F_sig ( 1 :num_f f t_pnt/2 ) ) ) ; grid;title('Spectrum of the Speech Signal'); xlabel('Frequency [Hz] ' ) ; ylabel('|sig|'); drawnow;figure ;semi logy ( abs (F_s ig ( 1 : num_f f t_pnt / 2 ) ) ) ; grid;title('Log-magnitude Spectrum of the Speech Signal'); xlabel('Frequency [Hz] ' ) ; ylabel('log_l_0|sig|'); drawnow;save('Sampling.mat', 'sig', 'fs') ; display('OUT_Sampling.m ');

99

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%^^^^^^^^^^^^^^^% M-File: "Quantizing.m" ^% AUTHOR: Boris Backovic ÿ% page: 1/2 %

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all; close all;display('IN_Quantizing.m' ) load('Sampling.mat', 'sig', 'fs ' )levels = 2 5 6 ;sigmax = max(sig);sigmin = min(sig);q = (sigmax-sigmin)/levels ;parti = sigmax-q;partitionl= [sigmin:q:partl] ;codebookl = [sigmin:q:sigmax] ;[indexl, quant si] =quantiz (sig,partitionl, codebookl) ; sigl = sig';errorl = sigl - quantsi;f igure;subplot(211);hist(errorl);grid;title('PDF of 8 bit quantization error!'); xlabel('error [e]'); ylabel ( ' [pdf (e) ] ' ) ;[yl, xl] = hist(errorl) ; yl = yl ./length(sig); subplot(212); bar(xl,yl,1); grid;title{'Normalized PDF of 8-bit Quantizer');xlabel('error [e]');ylabel ( ' [pdf(e)/length(sig) ] ' ) ;drawnow;wavplay(quantsi,fs);sig_power = io*loglO(mean(sigl. 2)); noise power = 10*log10(mean(errorl. 2) ) ;SNR_q = sig_power - noise_power; f printf ( ' \nSNR = %g [dB] \n\n ' , SNR_q) ;

figure ;x3 = (0 :1 :2 5 6 ) ;[yy, XX] = hist (indexl, x3 ); bar(xx, yy, 1); grid;

100

;%%%%%%%% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

M-File: "Quantizing.m" %AUTHOR: Boris Backovic %page: 2/2 %

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

title('Usage of Quantizer Levels'); xlabel('qunatization levels'); ylabel('frequency of occurrence');save { ' Quantizing .mat ' , ' indexl ' , ' quants 1 ' ) display ( ' OUT_Quant izing.m ' )

101

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File: "PCM.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% Containing M-Functions : %% 1. convert2binary .tn %' o ' O O O ' o ' O O O O O O O O O O O O O O O O O O O O ^ ' o ' O '

clear all; close all;display('IN_PCM.m' )load ( ' Quantizing .mat ' , ' indexl ' , ' quant si ' )clear cnt ; k=l ;for(cnt=l: 1 : size(indexl))

c {cnt,1} - cnt; c{cnt,2} = quantsl(cnt); c{cnt,3 } = indexl(cnt); c = convert2binary(c,cnt);

endtran = cat(2 , c{:,4 }); c_PCM = C;save ( ' PCM. mat ' , ' tran ' ) ; display('O U T P C M . m !) ;

102

i % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "convert2binary.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [c] = convert2binary(c, cnt)clear remndr;clear fixed_code_length;clear added_bits;clear to_add;remndr = 0 ;tmp_dec = c{cnt, 3 } ;if(tmp_dec == 0)

c{cnt,4 } = zeros(1 ,8 ); else

if(tmp_dec == 2 5 6 )tmp_dec = tmp_dec - 1 ;

endwhile (tmp_dec ~= 1)

remndr = [mod(tmp_dec, 2) remndr); tmp_dec = tmp_dec/2; tmp_dec = fix(tmp_dec);

endremndr = remndr(1 : (length(remndr)-1) ) ; var_code_length = [1 remndr];if (length(var_code_length) < 8)

to_add = 8 - length(var_code_length); added_bits = zeros(1, to_add);fixed_code_length = [added_bits var_code_length];

else fixed_code_length = var_code_length;endc{cnt,4 } = f ixed_code_length;

end

103

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File: "Formatting.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all; close all;display('IN_Formatting.m') ;I = imread('a r r o w j p g ');A = 0 .2 9 8 9 *I(: , : ,1 )+0 .5 8 7 0 *I(: , :,2 )+0 .1 1 4 0 *I(:, :,3 ) ; f igure;colormap(gray(2 5 6 )); image (A) ;title('Transmitted Image'); drawnow;[m,n] = size (A) ; clear i; for (i=l:l:m)

if (i == 1)B = A ( 1 , : ) ;

elseB = IB A ( i , : ) ] ;

endend[j,k] = size(B);X = [0 :1 :2 5 5 ] ;[s,n]=hist(B,x); if (sum(s)~=(j *k))

display('ERROR: mismatched number of pixels\n') ;endss=s./k; figure ;bar(n, ss, 1) ;title('Normalized PDF of the Image');grid;drawnow;z=l ;clear i ;for (i=l:1 : length(ss))

if (ss (i)~=0)p (z) = ss (i) ; index(z) = i - 1; z = z + 1 ;

endend .save ( ' Image_Format ting .mat index ) ,display('OUT_Forma1 1 ing.m ');

104

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% M-File: "LZW_Encode.m" %% AUTHOR: Boris Backovic %% page: l/l %% Containing M-Functions : %% 1, subsequence.m %% 2 . Table.m %% 3 . tailing.m %%%%%%%%%%%%%%%%%%%%%%%%%%%«-5-5-5-°-9-

clear all;display('IN_LZW_Encode.m') load('PCM.mat','tran');sig = tran; c{l,l} = [1] c{l,2} = [0] c{2,l} = [2] c{2,2} = [1]c = subsequence(c, sig); c = Table(c);Itransmition, decode_c, last_bits] = tailing(sig, c) ;save ( ' LempelZivWelch.mat ' , ' c ' , 'decode_c ' , ' transmition' , 'last_bitsI 1 .

display('OUT_LZW_Encode.m ' ) ;

105

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "subsequence.m" %% AUTHOR: Boris Backovic %% page: l/l %% Containing M-Functions : %% 1. check_up.m %' o ' o ' O ' O ' o o ' o ' b ' o ' 6 o o ' o ' o ' b o © ‘S ' o ' o ' o ' b ' b ' b ' b ' o ' o ' b ' B ' Ç ' b ' © ' ^

function [c] = subsequence (c, sig)pnt = 1;k = 1;curr = 3 ;set_over = 'N' ;while (pnt <= length(sig))

if (pnt+k <= length(sig)) temp = sig(pnt:pnt+k) ;

elsetemp = sig (pnt :length(sig) ) ;

endcheck_res = check_up(c, temp); if (check_res == 'Y ')

while (check_res ~= ' N ')if((pnt+k+l) <= length(sig))

k = k + 1;temp = sig(pnt:pnt+k); check_res = check_up(c, temp);

elsecheck_res = ' N '; set_over = ' Y ';

endend

endif(set_over ~= 'Y')

c{curr,l} = curr; c{curr,2} = temp; curr = curr + 1; pnt = pnt+k+1; clear temp; k = 1 ;

elsebreak;

endend

106

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function : "check_up.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [check_res] = check_up (c, temp)clear check_res; clear cnt;for(cnt=1:1 : length(c))

if(length(c{cnt,2}) == length(temp)) if(c{cnt,2} == temp)

check_res = 'Y';%fprintf('found at k=%d \n',cnt); break;

endendcheck_res = ' N ' ;

end

107

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "Table.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% Containing M-Functions : %% 1. convert2binaryl.in %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function[c]= Table(c)c{l,4 } = [0] ; c{2,4 } = [1] ;clear cnt;for{cnt = length(c) : -1: 3 )

%fprintf('%d\n' ,cnt) ; tempi = c{cnt,2}; len = length(tempi) ; temp2 = tempi(1 :len-1) ; for(cntl=cnt-l:-1; 1)

if(length(c{cntl,2} ) == length(temp2 ) ) if(c{cntl,2} == temp2)

c{cnt,3 } = cntl; c = convert2binaryl(c, cnt); inovat ionbit = tempi(length(c{cnt,2})); c{cnt,4 } = [c{cnt,4 } inovation_bit]; c{cnt,5 } = length(c{cnt,4 }); break;

endend

endend

108

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "converting2binaryl .m" %% AUTHOR; Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [c] = convert2binaryl(c, cnt)clear remndr; clear fixed_code_length; clear added_bits; clear to_add; remndr = 0 ; tmp_dec = c{cnt, 3 }; if(tmp_dec == 0)

c{cnt,4 } = 0 ;else

while (tmp_dec ~= 1)remndr = [mod(tmp_dec,2) remndr]; tmp_dec : tmp_dec/2;tmp_dec = fix(tmp_dec);

endremndr = remndr(1:(length(remndr)-1)); var_code_length = [1 remndr];c{cnt,4 } = var_code_length;

end

109

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %%%%% M-Function : "tailing.m" %% AUTHOR: Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [transmition, decode_c, last_bits] = tailing (sig, c)variable_rate = 0; for(k=3 :1 : length(c))

variable_rate = variable_rate + length (c{k, 2}) ;enddodati = length (sig) - variable__rate; last_bits = sig(variable_rate+l: length(sig));tail_flag = 'N'; clear cnt;for(cnt=l: 1 :length(c) )

if (length(c{cnt,2}) == length(last_bits) ) if(c{cnt,2} == last_bits)

tail_transmit = c{cnt,4 ); len_tail = length(tail_transmit); tail_flag = 'Y ';

endend

endtransmition = cat(2 , c { (3 : end) , 4 } ) ; decode_c = cat(2 , c{ (3 :end),5 } ) ;if(tail_flag == 'Y')

transmition = [transmition tail_transmit] ;decode_c (length (decode_c)+1) = length (tail_transmit) ;

end

110

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% M-File: "Image_Huffman.m" %% AUTHOR: Boris Backovic %% page: 1 / 3 %% Containing M-Functions: %% 1- MakeStructHuff .m %% 2 . sorting_Huff-m %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all; close all;display ( ' IN_Image_Huffman.m' ) ; load('Image_Formatting.mat')H = 0 ; clear i;for {i=l:l:length{p))

X = log(p(i));X = x/0 -6 9 3 1 5 ; y = - p ( i ) * x ;H = H + y;

endsymbols = [1:1:length(p)]; m = length (p) ; code_dim = length(p);Huff = MakeStructHuff (symbols,p) ; sorted_Huff = sorting_Huff (Huff, m, code_dim) ; ushile (m > 1)

last_s = sorted_Huf f . symbols ( : ,m) ; for (u=l:l : length (last_s) )

if(last_s(u,1)~= 0)for(g=l:1:codedim)

if (Huff.codew(g,last_S(u,1 ))-= 0 ) else

Huff.codew(g,last_s(u,1 ))= 1 ; break,-

endend

ende n d

eec_last_s = sorted_Huff . symbols ( : , (m-1 ) ) ;for {u=l :1: length ( sec last_s ) )

if(sec_last_s(u,1)-= 0) for(g=l:1:codedim)

if (Huff.codew(g,sec_last_s(u,l) )-= 0 ) elseHuff.codew(g,sec_last_s(u,X))= -1 ;

break;end

endend

endc l e a r t e n ç ) _ l a s t ;

I l l

%% <%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% %% M-File: "Image_Huffman.m" %% AUTHOR: Boris Backovic %% page: 2 / 3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

clear temp_sec_last;last_prob = sorted_Huf f .probability (m) ,• second_last_prob = sorted Huff. probability (m-1) ; new_prob = last_prob + second_last_prob ; new_Huff.probability(m-1 ) = new_prob; for (s = l-.l-.TO-2)

new__Huff .probability (s) = sorted_Huff .probability (s) ;endnew_Huff-probability = new_Huff .probability (1 :m-l) ; for (t = l; 1 ;code_dim)

if (sorted_Huff.symbols(t,m)~= 0 )temp_last(t)=sorted_Huff.symbols(t,m);

endif(sorted_Huff.symbols(t,m-l)~= 0 )

temp_sec_last(t)=sorted_Huff.symbols(t,(m-1 ));end

endnew_temp= [temp_sec_last temp_last];sorted_Huff . syTObols ( 1 : length (new_temp) , (m-l))= new_temp; new_Huff . symbols = sorted_Huff . symbols ( 1 : code_dim, 1 :m-l) ; m = m -1 ;sorted_Huff = sorting_Huff (new_Huff, m, code_dim); sorted_Huff.symbols ;

endclear cnt;for(cntl=code_dim;-1:1)

for(cnt2=l: 1 :code_dim)source(code_dim-cnt1+1 ,cnt2 )=Huff.codew(cnt1 ,cnt2 );

endendfor(cnt=l: 1 :code_dim)

S(cnt).source=source(;,cnt);S (cnt).source=nonzeros(S(cnt).source); for (h=l : 1 : length (S (cnt) . source) )

if(S(cnt).source(h)==-1)S(cnt) .source(h)=0 ;

endendS (cnt) .source = (S (cnt ).source) ’;

endaverage_length = 0 ; for (cntl=l: 1:code_dim)

Ik = length(S(cntl)-source);pk = p (cntl) ; 1 V \average_length = (average_length + (Ik p '

end

112

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File: "Image_Huffman.m" %% AUTHOR: Boris Backovic %% page: 3 / 3 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear i; clear k;for (k=l : 1 : length(B) )

for(i=l: 1: length(index) ) if(index(i) == B(k))

if(k == 1)bit_stream = S(i).source;length_of_each_letter = length(S(i) .source);

elsebit_stream = [bit_stream S(i).source]; length_of_each_letter = [length_of_each_letter

length(S(i).source)];end

endend

endsave ('Image_Huff man. mat ' , 'bit_stream' , ' length_of_each_letter ' , ' S '

,'index','A');di splay('OUT_Image_Huf fman.m ' ) ;

113

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "MakeStructHuff.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [Huff] = MakeStructHuf f (symbols,p)m = length(p);Huff.probability = p; f or ( j = 1 :1 ; m)

Huff.num_bits(j) = 0 ; for(k=l: 1 :ra)

Huff.codew(k,j) = 0 ;Huff.symbols(k,j)= 0 ;

endendfor (m=l:l:m)

Huff.symbols (l,m)= symbols(m);end

114

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% M-Function: "sorting_Huff.m" %% AUTHOR: Boris Backovic %% page: 1/2 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function [new_Huff] = sorting_Huff (Huff, m, code_dim)X = m; t = m;if(m == code_dim)

while(x >= 1)Temp_probab = Huf f .probability (m) ,•Temp_Symbol = Huff.symbols(:,m); for (w=l:l:TO-l)

if(Temp_probab >= Huff.probability(w)) for(y=m-l:-1:w)

Huff.probability(y+1 ) = Huff.probability(y); Huf f . symbols {: ,y+l) = Huf f . symbols (:, y) ,*

endHuff.probability(w) = Temp_probab;Huff . symbols ( , w) = Temp_Symbol ; break;

endendif(w == m-1)

m = m - 1 ; endX = X - 1;

endelse

Temp^robab = Huf f .probability (m) ;Temp_Symbol = Huff.symbols(: , m) ; for (r=m-l:-l:l)

if(Temp_probab < Huff.probability(r)) for(q=m-l; -1;r+1)

Huff.probability(q+l) = Huff.probability(q);Huff.symbols(;,q+1 ) = Huff.symbols(:,q);

endHuff-probability(r+1 ) = Temp_probab;Huff . symbols ( : , r+1 ) = Temp_SyTnbol ; break;

endif (r == 1)

for(y=m-l;-1:1)Huff-probability(y+1 ) = Huff-probability(y);Huff.symbols(:,y+ 1 ) = Huff - symbols(:,y) ;

endHuff-probability(1 ) = Temp_probab;Huff-symbols(:,1 ) = Temp_Symbol;break

115

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

M-Function: "sorting_Huff.m" %AUTHOR: Boris Backovic %page; 2/2 %

endend

endnew Huff = Huff;

116

; % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File: "Generate_Matrices.m" %% AUTHOR: Boris Backovic %% page: 1/2 %% Containing M-Functions: %% 1. convert2dec.m %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;display { ' IN_Generate_Matrices . m ' ) ;n = 12 ;k = 8 ;flag = 0 ;while(flag == 0)

Ik = eye(k);P = zeros(k, n-k) ; w_P = n - k;P{1 ,:) = randint(1 ,w_P);while ((sum(P(1,:))==0) || (sum(P(1,;))==1) )

P (1 , :) = randint(l,w_P);endclear cnt; for (cnt=2:l:k)

temp = randint(1 ,w_P) ;while ((sum(temp)==0) || (sum(temp)==1))

temp = randint(1,w_P);endfor (cntl=cnt-l:-1:1)

P_flag = 0 ; while (P_flag == 0)

if (temp == P(cntl^:))P_flag = 0 ;temp = randint(1 ,w_P);while ((sum(temp)==0) || (sum(temp)==1))

temp = randint(1 ,w_P);end

elseP_flag = 1 ;P (cnt,:)=temp;

endend

endendG = [P Ik] ;Ink = eye(n-k);Ht = cat (1 , Ink, P) ; all zero = zeros(1, n) ;

117

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File : "Generate_Matrices.m" %% AUTHOR: Boris Backovic %% page: 2/2 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

e = eye(n) ;e = cab(1, all_zero, e);Se = mod(e*Ht,2 ); s2_Se = size(Se);row_Se = s z_Se(1 ,1 ) ; for(i=l: 1 :row_Se)

Se_dec(i)=convert2dec(Se(i, : ) ) ;end Se_dec clear cnt; clear cntl; flag_ok : 1;for(cnt = 6 :1 : length(Se_dec) )

for(cntl=cnt-1 :-1:1)if(Se_dec(cnt) == Se_dec(cntl) )

flag = 0; break;

elseflag = 1;

endendif(flag ==0)

break;end

endendsave ( 'Generate_Matrices.mat' , 'n' , ' k ' , 'r o w Se' , ' I k ' , ' I n k ' , ' P ' , ' G ' ,

' H t ' , ' e ' , ' S e ' ) ;display ( 'OUT_Generate_Matrices .m' ) ;

118

;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

M-Function: "convert2dec.m" %AUTHOR: Boris Backovic %page: 1/1 %

;% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function[d]=convert2dec (b)

d = 0 ;k = length(b); for(cnt=l: 1 :k)

d = (b (cnt) * (2 (k-cnt) ) ) + d;end

119

%%%%%%%%%%%%%%%%%%%%%%%%%%% M-File: "LBC_EnGode.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;display('IN_LBC_Encode.m' )load{'Generate_Matri ce s.mat ' , ' n ' , ' k ' ,'G ') ; load { ' LempelZivWelch.mat ' , ' transmit ion ' ) ;m = transmition; cc = 0 ;while(rem(length(m),8) ~= 0)

m = [m 0] ; cc = cc + 1;

endpnt = 1 ; indx = 1;while(pnt < length(m))

message{indx,l} = m (pnt:pnt+k-l) ;U{indx,l} = mod(message{indx,l}*G,2 ) ; pnt = pnt + k ; indx = indx + 1 ;

endtransmit = cat(2 , U{;,l});save('BLC_Encode.mat', ’transmit' , 'U', 'message', 'm' , 'cc'); display('OUT_LBC_Encode.m ' )

120

% M-File: "ConvolEncoder.m" % AUTHOR: Boris Backovic % page; 1 / 3% Containing M-Functions :% 1. d2o.m

; % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% % % % %

% % % % % % % % % % % : cSrSrSr%SrS'2'2'9'S'S'2-&% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;display('IN_ConvolEncoder.m ' ) ;load('Image_Huffman.mat' , 'bit_stream' ) ;num_output_per_bit = generator = {[1 1 1;

2 ;1 0 1 ],[1 1 0 1 ; 1 1 1 0 ] ,[1 1 0 1 0 ; 1 1 1 0 1] ,

[1 1 0 1 0 1 ; 1 1 1 0 1 1] ,

[1 1 0 1 0 1 ; 1 1 0 1 0 1] ,

[1 1 0 1 1 1 0 ; 1 1 1 0 0 1 1] ,[1 1 0 1 1 1 0 0 1 ; 1 1 1 0 0 1 1 0[1 1 0 1 1 1 0 0 1 0 ; 1 1 1 0 0 1 1

1] , 0 0 1 ] } ;

mM

= 2 ; mM = m + 2 ;

look_up = generator{m, ;gen_vl = look_up(1, : ) ; gen_v2 = look_up(2, :) ;clear i; for (i=l:l:M)

impuls_res = zeros(M); imp_in = impuls_res(1,:); imp_in(i) = 1; output1 = gen_vl .* imp_in; output2 = gen_v2 .* imp_in; outl(i) = sum(output1); out2(i) = sum(output2);if (i==l)

seq_outl = out1(i); seq_out2 = out2(i);

elseseq_outl = [seq_outl outl(i)]; S0q out2 = [seg out2 out2(i)];

endendclear k; for (k=l:l:M)

121

: % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File; "ConvolEncoder.m" %% AUTHOR: Boris Backovic %% page: 2 / 3 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

if (k==l)imp_codeword = [seq_outl(k) seq_ouf2 (k)];

elseimp_codeword = [imp_codeword seq_outl(k) seq_out2( k ) ];

endendimp_codewords_len = length (imp_codeword) ;data = bit_stream; d_size = length(data) ;coded = zeros (1, (imp_codewords_len + (num output per bit * d_size - num_output_per_bit) ) ) ; coded_length = length (coded) ;clear t;for (t=l: 1 :d_size)

parse = data(t);if (parse == 1)

suma = imp_codeword;else

suma = zeros(imp_codewords_len); suma = suma(1,:);

endclear i;for (i=(2*t-l) : 1 ;(imp_codewords_len + (2*t-l)-l))

if((coded(i) + suma( i - (2*t-2))) == 2)coded(i) = 0;

elsecoded(i) = coded(i) + suma(i-(2*t-2));

endend

endcode_check = coded; polyl = conv(data, gen_vl) ; poly2 = conv(data, gen_v2) ;polyl len = length (data) + length (gen._vl) - 1; poly2~len = length (data) + length (gen_v2) - 1;

clear k;for (k=l: 1 :polyl_len)

if(rem(polyl(k),2) ~= 0)polyl(k) = 1;

elsepolyl(k) = 0;

endend

122

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%! %! ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^% M-File: "ConvolEncoder.m" ^% AUTHOR: Boris Backovic ^% page: 3 / 3 ^%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%^^^^^^%^^^^^^

clear k;for (k=l: 1 :poly2_len)

if(rem(poly2(k),2) ~= 0) poly2(k) = 1;

elsepoly2(k) = 0;

endendclear k;for (k=l:1:polyl_len)

if (k == 1)codeword = [polyl(k) poly2(k)];

elsecodeword = [codeword polyl(k) poly2(k)];

endendd_gen_vl = convert2dec(gen_vl) ; o_gen_vl = d2o(d_gen_vl) ; d_gen_v2 = convert2dec(gen_v2); o_gen_v2 = d2o(d_gen_v2);flush = zeros(1 , M-1 ) ; %flushing M - 1 zeros for M=4 memoryregistersmessage = [data flush];trel = poly2 trellis(M, [o_gen_vl o_gen_v2 ] ); encoded_stream = convene (message, trel) ;if(sum(xor(coded,encoded_stream) ) ~= 0)

display('WARNNING: Convolutional Encoding Performed with Errors ! 1 ! ! ' \n') ; endsave('Convoi Encoder.mat','coded','code_check' , 'codeword','trel ' ,

'encoded_stream','data'); display('OUT_ConvolEncoder.m ' ) ;

123

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "d2o.in" %% AUTHOR: Boris Backovic %% page: l/l %

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %'b'o'o'b'o'S'o'S'b'ô o ô o o o o o o o o'b'o'o'S'o'b'S'b'o'o'o'b

function [sol] = convert2binary (d)if(d == 0)

b = 0;else

clear ostatak; ostatak = 0; tmp_dec = d; while (tmp_dec >= 8)

ostatak = [mod{tmp_dec,8) ostatak]; tmp_dec = tmp_dec/8; tmp_dec = fix{tmp_dec) ;

endostatak = ostatak(1:(length(ostatak)-1)); b = [tmp_dec ostatak] ;

endsol = 0 ; •len = length(b); for ( cnt = l ; 1 : len)

sol = (b (cnt)*(10^(len-cnt)))+ sol;end

124

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % %%% ^% ^ ^ ^ ^ ^ ^ ^ ^ ^ ^% M-File: "aggreg_one. tn" ^% AUTHOR: Boris Backovic ^% page: 1/2 ^% Containing M-Functions : %% 1. Welsh_Matrix.m %% 2. complement .m %% 3 . DSSS.m %% 4 . CDMA_encode.m %% 5 . CDMA_decode.m %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;display('IN_aggreg_one.m' ) ;load('BLC_Encode.mat', 'transmit') ;load {'Convol_Encoder.mat', 'coded');1 8 = Welsh_Matrix(8);A = transmit ;B = coded;A 1 = A;B 1 = B;pseudo_A = randsrc (1 , 2 0 , [0 , 1 ; 0 . 5 , 0 . 5 ] ) ;A = DSSS(A, pseudo_A);pseudo_B = randsrc(1 , 2 0 , [0 ,1 ; 0 .5 ,0 .5 ] ) ;B = DSSS(B, pseudo_B);C_A = 1 8 (3 ,:);comp_C_A = complement (C_A) ;seqA = CDMA_encode (A, C_A, comp_C_A) ;C_B = 1 8 (7 , : ) ;comp_C_B = complement(C_B) ;seqB = CDMA_encode (B, C_B, comp_C_B) ;chip_len = length(C_A);if(length(seqA)>length(seqB) )

add_bit = length(seqA) - length(seqB); seqB = [seqB zeros(1 , add_bit)];

el se if ( length ( seqB) > length (seqA) )add_bit = length(seqB) - length(seqA); seqA = [seqA zeros(1 , addbit)];

endsend_out = seqA + seqB;noise = randsrc(1,length(send_out) , [0,1; 0 .9 8 ,0 .0 2 ]),

125

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-File; "aggreg_one.m" %% AUTHOR: Boris Backovic %% page: 2/2 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

received = send_out + noise;[received_A, tmpA] = CDMA_decode ( ' A', received, chip_len, C_A ) ; [received_B, tmpB] = CDMA_decode{'B ', received, chip_len, C _ B );received_A = DSSS{received_A, pseudo_A) ; received_B = DSSS (received_B, pseudo_B) ,-save { ' WCDMA. mat ' , ' received_A' , ' received_B ' ) ;display('OUT_aggreg_one.m' ) ;

126

% % % % % % % % % % % % % % % • & % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "Welsh_Matrix.m" %% AUTHOR: Boris Backovic %% page: l/l %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function [A] = Welsh_Matrix (n)result = n; flag = 0; k = 0;while(result ~= 1)

if(mod(result, 2) ~= 0) flag = 1; break;

endresult = result / 2 ; k = k + 1 ;

endif(flag == 0)

if (n ~= 2)A = [Welsh_Matrix(2 (k-1 ) ) Welsh_Matrix(2 " (k-1 ) ) ;

Welsh_Matrix(2 (k-1 ) ) -Welsh_Matrix(2 "" (k-l) ) ] ;else

A = [1 1 ;1 - 1 ] ;

endelse

display('INVALID ENTRY!!!!');fprint f ( ' %d is not = 2 m, where in = 1,2,3,4,... \n , n) ,

end

127

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "complement.m" %% AUTHOR: Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [B] = complement (A)for(cnt=l: 1 : length(A) )

if(A(cnt) == 1 ) if(cnt == 1)

B = -1 ; else

B = [B -1 ] ;end

elseif(cnt == 1)

B = 1 ; else

B = [B 1 ] ; end

endend

128

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function: "DSSS.m" %% AUTHOR: Boris Backovic %% page: 1/1 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [DS_A] = DSSS (A, pseudo_A)DS_A = [0 ] ;cntl = 1;for(cnt=l: 1 : length(A))

C = xor(A(cnt) ,pseudo_A(cntl) ) ;DS_A = [DS_A C]; cnt1 = cnt1 + 1; if(cntl > length(pseudo_A) )

cntl = 1;end

endDS A = DS A (2 :end) ;

129

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

% M-Function : "CDMA_encode.m" %% AUTHOR: Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [seqA] = CDMA_encode (A, C_A, comp_C_A)seqA = [0 ] ; for (i = l : 1 :length(A))

if(A(i) == 1 )seqA = [seqA C_A] ;

elseseqA = [seqA comp_C_A] ;

endendseqA = seqA(2 :end);

130

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % M-Function: "CDMA_decode.m" %% AUTHOR: Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

function [received_A, tmpA] = CDMA_decode (A, send, chip_len, C_A)

pnt = 1 ;tmpA = [0] ;received_A = [ 0 ] ;while (pnt < length(send))

tempA = send (pnt :pnt+chip_len-l) .* C_A; tempA = sum(tempA); tmpA = [tmpA tempA] ; if(tempA >= 4 )

received_A = [received_A 1 ]; elseif(tempA <= -4 )

received_A = [received_A 0 ] ;endpnt = pnt + chip_len;

endreceived_A = reçeived_A(2 : end) ; tmpA = tmpA ( 2 : end) ;

131

% M-File: "LBC_Decode.m" %% AUTHOR: Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;display ( ' IN_LBC_Decode .tn' )

load ( ' BLC_Encode.mat ' , 'U' , 'message ' , 'm' ) ;load ( ' Generate_Matrices .mat ' , 'n' , ' row_Se ' , 'Ht ' , ' Se' , ' e ' ) ; load('WCDMA.mat', 'received_A' ) ;corrupted = received_A; sz_e = size(e); row_e = sz_e(l,l); for(cnt=l: 1 :row_e)

el{cnt,l} = e(cnt,:);endfor(cnt=l: 1 :row_Se)

SI{cnt, l}=Se(cnt,:);endpnt = 1 ;while(pnt < length(corrupted) )

received = corrupted(pnt:pnt+n-l);Sr = mod(received*Ht, 2 ) ; for (cnt=l: 1 :row_Se)

if(Sr == SI{cnt})decoded = mod(received+el{cnt,;},2); for(cntl=l:1:length(U))

if(decoded == u{cntl,l})sent_mess = message{cnt1,l};

endend

endendif (pnt == 1)

dec_mess = sentîmess ;else

dec mess = [dec mess sent mess];endpnt = pnt + n;

endsave('BLC_Decode', 'dec_mess ' ) ; display ( ' OUT_LBC_Decode.m ' )

132

% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % :

% M-File: "Viterbi_Decoder.m" %% AUTHOR: Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;display('IN_Viterbi_Decoder.m');load('Convol_Encoder.mat'); load('WCDMA.mat', 'rece ived_B') ;coded = cat(2 , received_B, zeros( 1 ,3 4 )); decoded_bits = vitdec(coded, trel, 20, 'cont', 'hard');decoded_bits = decoded_bits (21 : length (decoded_bits) ) ;if(sum(xor(data,decoded_bits))~= 0)

display('WARNNING: Convolutional Encoding Performed with Errors!!!!');

fprintf('\n');endsave ( ' Viterbi_Decoder.mat', 'decoded bits'); display ( ' OUT_Viterbi_Decoder.m ' ) ;

133

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % M-File: "DtoA.m" %% AUTHOR; Boris Backovic %% page: l/l %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

clear all;

display ( ' IN_Digital_to_Analog.m' )load('Sampling.mat', 'fs ' ) load Quantizing.mat ; load PCM.mat ; load LempelZivWelch.mat;LZW_received = cat(2 , c{ (3 : end),2 ) ) ;LZW_received = [LZW_received last_bits] ;pnt = 1;D2 A = 0 ; clear cntl; clear cnt;while (pnt < length (LZW_received) )

temp = LZW_received(pnt :pnt+7 ) ; tempi = convert2dec(temp) ; if(tempi == 2 5 5 )

tempi = tempi + 1;endfor (cnt1=1:1 : length(indexl) )

if (tempi == indexl(cntl))D2A = [D2A quantsl(cntl)]; break;

endendpnt = pnt + 8 ;

endD 2A = D 2A(2 :end);wavplay(D2A,fs)t= ( 1 : length(D2A))/fs ;plot(t,D 2A)gridtitle('Received Speech Signal') xlabel('time [sec.]') ylabel('voltage [V]') drawnowdisplay ( 'OUT_Digital_to_Analog.m' )

134

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%^^^^^^^^^%^%%%%%^^ % M-File; "Decoder_lTTiage_Hufftnan.m" %I AUTHOR: Boris Backovic %% page: 1/2 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

close all; clear all;

display ( ' IN_Decoder_Image_Huffman.m' ) ;load ('Image_Huff man. mat ' , ' length_of_each_letter ' , 'S', 'index', 'A') load ( ' Viterbi__Decoder. mat ' , ' decoded_bits ' ) ; clear i;for {i = l : 1 : length (length_of_each_letter) )

if(i==l)offset = length_of_each_letter(i)-1 ; bit_number = decoded_bits(1:1+offset) ; next_start = length_of_each_letter(i)+1 ; clear j;for(j=l: 1 : length(S))

if(length(bit_number)==length(S(j).source)) if(bit_number == S(j).source)

decoded number = index(j);end

endend

elseoffset = length_of_each_letter(i)-1;bit_number = decoded_bits(next_start:next_start+offset) ; next_start = next_start+offset+1 ; clear j; for(j =1:1 : length(S))

if(length(bit_number)==length(S (j).source)) if(bit_number == S(j).source)

decoded_number = [decoded number index(j)1 ;end

endend

endendmatrix_size = size(A); row_number = matrix_size (1,1); column number = matrix_size (1,2);

135

r%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% M-File: "Decoder_Image_Huffman.m" %% AUTHOR: Boris Backovic %% page: 2/2 %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

R = zeros (row_nuniber, column_number) ;clear m;for(m=l: 1 :row_number)

R(m, :) = decoded_number((m- 1) *column_number+l :m*column_nuttiber) ; endf igurecolormap(gray(2 5 6 ) ) ; image(R)title('Received Image');RESULT = sum (sum (xor (R, A) ) )display ( ' OUT_Decoder_Image_Huf fman.m'} ;

136

Appendix B:

Matlab Example Files

137

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%^^^^^^^^^^^^^^^^^^ % M-File: "HuffmanCodeAlgorithm.m" %% AUTHOR: Boris Backovic %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

SS = [0 . 2 5 0 . 2 0 . 1 5 0 . 1 2 0 . 1 0 . 0 8 0 . 0 5 0 .0 5 ];

z=l;clear i ;for (i = l ; 1 :length(ss) )

if (ss(i)~=0)p (z) = SS (i) ;index ( z ) = i z = z + 1 ;

endend H = 0 ; clear i;for (i=l:1 :length(p) )

X = log(p(i));X = -x/0 .6 9 3 1 5 ; y = p (i) *x;H = H + y;

endsymbols = [1:1:length(p)]; m = length(p); code_dim = length(p) ;Huff = MakeStructHuff (symbols,p);sorted Huff = sorting_Huff (Huff, m, code dim);sorted_Huff.probability; sorted_Huff.symbols ; while(m > 1)

last_s = sorted_Huff.symbols(:,m); for (u=l:l:length(last_s))

if(last_s(u,1)~= 0)for(g=l:1:code_dim)

if (Huff.codew(g,last_s(u,l))~= 0 ) else

138

Huff.codew(g,last_s(u,l))= 1; break;

endend

endendsec_last_s = sorted_Huff.symbols(: , (m-1)); for (u=l: 1 : length(sec_last_s) )

if(sec_last_s(u,1)~= o) for(g=l: 1:code_dim)

if (Huff.codew(g,sec_last_s(u,1))~= 0)else

Huff.codew(g,sec_last_s(u,l))= -1; break;

endend

endendclear temp_last clear temp_sec_lastlast_prob = sorted_Huff .probability (m) ; second_last_prob = sorted_Huff .probability (m-1 ) ; new prob = last_prob + second_last__prob; new_Huff .probability (m-1 ) = new__prob; for (s = l:l:m-2)

new^Huff.probability(s) = sorted_Huff.probability(s);endnew_Huff .probability = new_Huf f.probability ( 1 :m-l) ; for (t=l:1 :code_dim)

if(sorted_Huff.symbols(t,m)-= 0 )temp_last(t)=sorted_Huff.symbols(t,m);

endif(sorted_Huff.symbols(t,m-1 )~= 0 )

temp_sec_last (t) =sorted__Huff . symbols (t, (m-1 )) ;end

endnew_temp= [temp_sec_last terap__last] ;sorted Huff.symbols(1: length(new_temp) , (m-1 ))= new_tenp;

139

new_Huff.symbols - sorted_Huff.symbols(l:code_dim,; m = m -1 ;sorted_Huff = sorting_Huff {new_Huff, m, codedim); sorted_Huff.symbols ;

endclear cnt;for(cnt1=code_dim: -1:1 )

for(cnt2=l: 1:code_dim)source(code_dim-cntl+1 ,cnt2 )=Huff.codew(cntl,cnt2 );

endendfor(cnt = l: 1 ;code_dim)

S(cnt) .source=source(:,cnt) ;S(cnt) .source=nonzeros(S(cnt).source) ; for (h=l : 1 : length(S (cnt) . source) )

if(S(cnt).source(h)==-1)S(cnt) .source(h)=0 ;

endendS(cnt).source = (S(cnt).source)'; fprintf('\nS%d=%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d

\n' , index(cnt),S(cnt).source) endfprintf('\n' ) ; average_length = 0 ; for (cnt1 = 1 :1:code_dim)

Ik = length(S(cntl).source) ; pk = p(cntl);average_length = (average_length + (lk*pk));

endfprintf ('\nAverage Code Length (using my code) :%g\n',average_length) ;fprintf('Entropy of the Source H(x)= %3 .5g\n' , H) ; fprintf ('Efficiency = %g%%\n\n', H/average_length*1 0 0 ) ;

140

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%^^^%^^^^^^^^^^^^^^^^^^^^ % M-File: "LBC_Example.m" ^% AUTHOR: Boris Backovic, %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%^%%^%^^^^^^^^^^^^^^

n = 5 ; k = 3;Ik = eye(k);P = [0 1;0 1;1 1];G = [P Ik] ;Ink = eye{n-k);Ht = c a t (1, Ink, P ) ;e = [0 0 1 0 0;1 0 0 0 0;0 0 0 0 1 ];sz_e = size (e) ; row_e = sz_e(1,1); for(cnt = l ; 1 :row_e)

el{cnt,l} = e(cnt,:);endS = mod(e*Ht,2); sz_S = size(S); row_S = sz_S(l,l); column = sz_S(l,2) ; f o r (cnt = l : 1 :row_S)

Si{cnt, l}=S(cnt,;);endm = [0 0 0 0 0 1 0 1 0 0 1 1 1 0 0 1 0 1 1 1 0 1 1 11 ; pnt = 1; indx = 1;while(pnt < length(m))

message{indx, l} = m(pnt:pnt+k-l);U{indx, 1} = mod(message{indx, l}*G,2); pnt = pnt + k; indx = indx + 1;

endcorrupted = [1 0 1 0 0 1 1 1 0 1 1 0 0 0 0];pnt = 1 ;while(pnt < length(corrupted) )

received = corrupted(pnt:pnt+n-l) ;

141

Sr = mod(received*Ht,2) ; for (cnt=l; 1;row_S)

if(Sr == Sl{cnt})decoded = m o d (rece ived+el{cnt,:},2); for(cnt1 = 1 :1 :length ( U ) )

if(decoded == U{cntl,l})sent_mess = message{cnt1,1};

endend

endendif(pnt == 1)

dec_mess = s e n t m e s s ;else

dec_mess = [dec_mess sent_mess];endpnt = pnt + n;

endfor(cnt=l: 1 : length(message) )

fprintf('\nmessage{%d}= %d',cnt) ; fprintf('% d ',message{cnt,1} ) ;

endfprintf('\ n ' ) ;f o r (cnt = l : 1; length(U) )

fprintf('\nU(%d}= %d',cnt); fprintf('%d',U{cnt,1});

endfprintf('\n\nreceived corrupted sequence = ') ;fprintf ( ' %d ' , corrupted) ; fprintf('\n' ) ;

142

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % M-File : "ConvolExample.m" %% AUTHOR: Boris Backovic, %% % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %

num_output_per_bit = 2 ;generator = {[1 1 1; 1 0 1],

[1 1 0 1; 1 1 1 0] ,[1 1 0 10; 1 1 1 0 1] ,

[1 1 0 1 0 1; 1 1 1 0 1 1] ,

[1 1 0 1 0 1; 1 1 0 1 0 1] ,

[1 1 0 1 1 1 0 ; 1 1 1 0 0 1 1],[1 1 0 1 1 1 0 0 1; 1 1 1 0 0 1[1 1 0 1 1 1 0 0 1 0; 1 1 1 0 0

m = menu('Enter the Number of MemoryRegisters : ' , '3 ' , 4 ' ,■5 ' , '6', ’7 ' ,'8' 19. , '' 10 ' )M = m + 2 ;look_up = generator{m, :} ; gen_vl = look_up(l, : ) ; gen_v2 = look_up(2,:); clear i ; for (1=1 :1 :M)

impuls_res = zeros(M); imp_in = impuls_res{1,:); imp_in(i) = 1;f p r i n t f {'\nv(%d) = [%d %d %d %d] ',i ,imp_in);output1 = gen_vl .* imp_in;output2 = gen_v2 .* imp_in;outl(i) = sum(output1) ;out2(i) = sum(output2) ;%fprintf('\nFirst Adder = %d %d %d % d ',output1); %fprintf('\nFirst Sum = %d \ n ',out1(1))«%fprintf ( ' \nSecond Adder = %d %d %d %d ' , output2) ; %fprintf('\nSecond Sum = %d \ n ',out2(i));if (i==l)

seq out1 - outl(i); seq out2 = out2 (i);

else

143

seq_outl = [seq^outl outl(i)]; seq_out2 = [seq_out2 out2(i)];

endendfprintf \nFirst Out_Sequence = %d %d %d %d \n', seq_outl) ; fprintf <'Second Out_Sequence = %d %d %d %d \ n ' , seq_out2) ; clear k; for (k=l:l:M)

if (k==l)imp_codeword = [seq_outl(k) seq_out2(k) ] ;%fprintf<'\nlmpuls Codewordlll = %d %d %d %d %d %d %d %d

\ n ',imp_codeword) ; else

imp_codeword = [imp_codeword seq_outl(k) seq_out2(k) ] ;end

endfprintf \nlmpulse Codeword = %d %d %d %d %d %d %d %d \ n ',imp_codeword) ;imp_codewords_len = length {imp_codeword) ;data = input('Input a Stream of Bits to be Decoded (e.g. [ 1 0 1 01 0 0 0 1] ) :\n' ) ; d_size = length(data) ;coded = zeros (imp__codewords_len + (num output per bit * d_size -num_output_per_bit) ) ;coded = coded(1,;),•coded_length = length(coded) ;clear t ;for (t = l : 1:d_size)

parse = da t a (t) ;if (parse == 1)

suma = imp_codeword;else

suma = zeros(imp_codewords_len); suma = suma(1,:);

endclear i;for (i= (2 *t-l) :1 : (imp_codewords_len + (2*t-l)-D)

i f ((coded(i) + suma(i-(2*t-2))) == 2)

144

coded(i) = o ;else

coded(i) = coded(i) + suma ( i - ( 2 * t - 2 ) );end

endendfprintf('\n');d i s p l a y ('Connection Representation') ;fprintf('Coded: ') ;fprintf('%d ',coded);polyl = conv(data, gen_vl);poly2 = conv(data, gen_v2);polyl_len = length (data) + length (gen_vl) - 1; poly2_len = length (data) + length (gen_v2) - 1; clear k;for (k=l:1:polyl_len)

if(rem(polyl(k),2) ~= 0)polyl(k) = 1;

elsepolyl(k) = 0;

endendclear k;for (k=l: 1 :poly2_len)

if(rem(poly2(k),2) ~= 0)poly2(k) = 1;

elsepoly2(k) = 0;

endendclear k;for (k=l:1:polyl_len)

if (k == 1)codeword = [polyl(k) poly2 ( k ) ] ;

elsecodeword = [codeword polyl(k) poly2 ( k ) ] ;

endend

145

fprintf('\n\n');display ( ' Polynomial Representation' )fprintf('Codeword: ' ) ;fprintf('%d ',codeword);flush = [0 0 0];message = [data flush];trel = poly2trellis(4, [15 16]);encoded_stream = convene(message, trel);if(sum(xor(coded,encoded_stream) ) ~= 0)

display('WARNNING: Convolutional Encoding Performed with Errors 11 ! ! '\n') ; enddecoded_bits = vitdec(coded, trel, 1,'cont', 'hard'); decoded bits = decoded_bits(2 : length(decoded_bits)) ; decoded bits = [decoded_bits 0 ] ;decoded bits = decoded_bits(1: length(decoded_bits)-3) ;fprintf('\n\n');fprintf('Decoded bits : ');fprintf('%d ',decoded_bits);fprintf('\ n ');if (sum (xor (data, decoded_bits) ) ~= 0)

display('WARNNING: Convolutional Encoding Performed with Errors 1 ! 1 ! '\ n ') ; end