IEEE Circuits MAG SystemsSystems and AZINE · Indonesia (Bali) on October 28–31, 2002 to...

CircuitsCircuitsCircuitsSystemsSystemsSystemsandandand

Does Chaos Work BetterThan Noise?

4

Does Chaos Work BetterThan Noise?

4

Successive ApproximationQuantization for Image

Compression20


Compression20

IEEEMAGAZINE

Volume 2, Number 3, Third Quarter 2002 ISSN 1531-636X

Indonesia (Bali) on October 28–31, 2002to

Singapore on December 16–18, 2002

Calls for Papers and ParticipationCalls for Papers and ParticipationCalls for Papers and ParticipationThe APCCAS 2002 will be rescheduled inSingapore on December 16, 17, 18–2002

As you may be informed, APCCAS2002 was canceled with a great regret.However, IEEE CAS Society considered the importance of continuity of theAPCCAS series by recommending an alternate site for APCCAS2002 to theOrganizing Committee through the International Steering Committee. It maybring more difficulty than we expect, but it may bring a new chance to gettogether and exchange our technical achievements at a real international con-ference. Therefore, our Organizing Committee decided to change the site from:

Indonesia (Bali) on October 28–31, 2002to

Singapore on December 16–18, 2002in strong cooperation with Singapore people.

For further information contact:

Prof. Soegijardjo Soegijoko the APCCAS2002 General Chair at theConference Secretariat, Electronics Laboratory, Department of Elec-trical Engineering, Institut Teknologi Bandung, Jalan Ganesha 10Bandung 40132 INDONESIA; Tel: +62–22–253 4117, +62–22–2539172 (ext. 3227 or 8895); Fax: +62–22–250 1895; E-mail:[email protected]; [email protected]; [email protected].

http://apccas2002.itb.ac.idhttp://www.vlsi.ss.titech.ac.jp/apccas2002/

The International Conference on VLSI Design was started in 1985, as a small work-shop at IIT Madras, under the visionary guidance of Dr. Vishwani Agrawal of BellLaboratories, and Prof. H.N. Mahabala of IIT Madras. From this small start, it has growninto a leading international conference on VLSI design, which draws around 700 at-tendees every year from India and abroad. The proceedings are printed by the IEEEComputer Society Press, USA. The conference is dedicated to all aspects of integratedcircuit design, technology, and related computer-aided design (CAD).

This year, tutorials will be held on January 4th and 5th. Technical program paperpresentations and Exhibits are scheduled for January 6th, 7th and 8th.

GENERAL CHAIR N. Ranganathan, IEEE LiaisonJaswinder Ahuja Dept. CSE/CMRCadence Design Systems University of South Florida, ENB 118Noida, India Tampa, FL [email protected] Phone: 813 974–4760 (fax: 3610)

Email: [email protected]

http://vlsi.nj.nec.com/~chak/vlsi2003/index.html

2003 Southwest Symposiumon Mixed-Signal Design

February 23–25, 2003Harrah’s Hotel and CasinoLas Vegas, Nevada, U.S.A.

The 2003 SSMSD will bring together researchers in academia, industry, and gov-ernment from the areas of mixed-signal circuit design and CAD tool development.This will allow not only the presentation of the latest developments in each field,but also interaction between the areas in order to expedite the development of im-proved integrated systems and “systems on chip”.

Jeffrey J. Rodriguez, General Chair Conference ManagementElectrical & Computer Engineering Conference Management ServicesThe University of Arizona 3109 Westchester Ave.Tucson, AZ 85721 U.S.A. College Station, TX 77845-7919 U.S.A.Tel: 520–621–8732 Fax: 979-693-6600Fax: 520–621–8076 [email protected]@ece.arizona.edu

For further details, see the conference web page:http://www.ece.arizona.edu/conferences/ssmsd

The 3rd IEEE Pacific-Rim Conference on Multimedia

December 16–18, 2002

National Tsing-Hua University, Hsinchu, Taiwan

The 2002 IEEE Pacific-Rim Conference on Multimedia (PCM 2002) is the third annualconference on cutting-edge multimedia technologies and will be held at Tsing-HuaUniversity, Hsinchu, Taiwan. The conference complements this wonderful setting by pro-viding a forum for presenting and exploring technological and artistic advancements inmultimedia. Technical issues, theory and practice, artistic and consumer innovations willbring together researchers, artists, developers, educators, performers, and practitionersof multimedia from Pacific-Rim and the world. Present your work at PCM2002 and de-fine the future of multimedia in the next millennium!

Conference chair Program chairYung-Chang Chen Long-Wen ChangDepartment of Electrical Engineering Department of Computer ScienceNational Tsing-Hua University National Tsing-Hua UniversityHsin-Chu, Taiwan,300 Hsin-Chu, Taiwan,[email protected] [email protected]

http://www.ee.nthu.edu.tw/~PCM2002

Call for Papers[PDF Version available at http://www.icme2003.com/ICME_CFP.pdf]

IEEE International Conference on Multimedia & Expo (ICME), scheduled for July6–9, 2003, is a major annual international conference organized with the objective ofbringing together researchers, developers, and practitioners from academia and industryworking in all areas in multimedia. ICME serves as a forum for the dissemination ofstate-of-the-art research, development, and implementations of multimedia systems,technologies and applications. ICME is co-sponsored by four IEEE societies (theComputer Society, the Signal Processing Society, the Circuits and Systems Society,and the IEEE Communications Society).

Only electronic submission will be accepted. Visit theICME 2003 website, www.icme2003.com, to submit papers.

Regular Paper Submission: Dec 15, 2002Notification of Acceptance: March 1, 2003Camera-Ready Paper Due: March 31, 2003

Min Wu, Assistant ProfessorDept. of Electrical & Computer Engineering

2457 A.V.Williams Bldg.University of Maryland

College Park, MD 20742Tel: (301)405–0401Fax: (301)314–9281

Email: [email protected]: www.ece.umd.edu/~minwu/

1

4

20

IEEE Circuits and Systems Magazine (ISSN 1531–636X) is published quarterly by the Institute of Electrical and Elec-tronics Engineers, Inc. Headquarters: 3 Park Avenue, 17th Floor, New York, NY, 10016–5997. Responsibility for the con-tents rests upon the authors and not upon the IEEE, the Society, or its members. IEEE Service Center (for orders, subscrip-tions, address changes): 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855–1331. Telephone: +1 732 981 0060, +1 800678 4333. Individual copies: IEEE members $10.00 (first copy only), nonmembers $20.00 per copy; $7.00 per member peryear (included in Society fee) for each member of the IEEE Circuits and Systems Society. Subscription rates availableupon request. Copyright and Reprint Permissions: Abstracting is permitted with credit to the source. Libraries are per-mitted to photocopy beyond the limits of the U.S. Copyright law for private use of patrons: 1) those post-1977 articles thatcarry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through the CopyrightClearance Center, 222 Rosewood Drive, Danvers, MA 01923; and 2) pre-1978 articles without fee. For other copying,reprint, or republication permission, write to: Copyrights and Permissions Department, IEEE Service Center, 445 HoesLane, Piscataway, NJ 08855–1331. Copyright © 2002 by the Institute of Electrical and Electronics Engineers, Inc. All rightsreserved. Periodicals postage paid at New York, NY, and at additional mailing offices. Postmaster: Send address changesto IEEE Circuits and Systems Magazine, IEEE Operations Center, 445 Hoes Lane, Piscataway, NJ, 08855.

Printed in U.S.A.

Volume 2, Number 3, Third Quarter 2002

Does Chaos WorkBetter Than Noise?by Maide Bucolo, Riccardo Caponetto, Luigi Fortuna, Mattia Frasca,and Alessandro Rizzo

Chaos and random signals share the property of long term unpredictable irregularbehavior and broad band spectrum. The aim of this paper is not to distinguishbetween random and chaotic dynamics, nor to show the use of chaos, but to focusattention on how chaos and noise help order to arise from disorder.

Successive ApproximationQuantization for ImageCompressionby Eduardo A. B. da Silva, Décio A. Fonini Jr., and Marcos Craizer

Images may be worth a thousand words, but they generally occupy much more spacein a hard disk, or bandwidth in a transmission system, than their proverbial coun-terpart. So, in the broad field of signal processing, a very high-activity area is theresearch for efficient signal representations.

Circuitsand

SystemsCircuits

Systemsand

Circuitsand

SystemsIEEE

MAGAZINE

A

2

Editor-in-ChiefMichael K. Sain

Electrical Engineering DepartmentUniversity of Notre Dame

Notre Dame, IN, USA 46556–5637Phone: (219) 631–6538

Fax: (219) 631–4393E-mail: [email protected]

Publication CoordinatorEric L. Kuehner

E-mail:[email protected]

Editorial BoardMagdy Bayoumi

University of Louisiana

Paulo DinizFederal University of Rio de Janeiro

Orla FeelyUniversity College Dublin

Luigi FortunaUniversity of Catania

S.Y. (Ron) HuiCity University of Hong Kong

Steve KangUniversity of California, Santa Cruz

Chung-Sheng LiIBM T. J. Watson Research Lab

Ravi P. RamachandranRowan University

Mehmet Ali TanApplied Micro Circuits Corporation

Ljiljana TrajkovicSimon Fraser University

Chung-Yu WuNational Chiao Tung University

Editorial Advisory BoardRui J. P. de Figueiredo–Chair, University of California,

Irvine; Guanrong Chen, City University of Hong Kong; Wai-Kai Chen, International Technological University; Leon Chua,University of California, Berkeley; Giovanni De Micheli, Stanford

University; George S. Moschytz, Swiss Federal Institute of

Technology; Mona Zaghloul, George Washington University

IEEE Publishing ServicesRobert Smrek

Production DirectorIEEE Service Center

445 Hoes LaneP.O. Box 1331

Piscataway, NJ 08855–1331, USAPhone: (732) 562–3944

Frequency of PublicationQuarterly

Magazine DeadlinesFinal materials for the CAS Magazine must be received

by the Editor on the following dates:

Issue Due DateFirst Quarter February 1

Second Quarter May 1Third Quarter August 1Fourth Quarter November 1

Columns and Departments3 From the Editor

by Michael K. Sain

3 The ’Umble Ohmby Shlomo Karni

46 TransactionsCSVT Transactions Best Paper Award: 2000–2001A Unified Rate-Distortion Analysis Frameworkfor Transform Coding: A Summaryby Zhihai He and Sanjit K. Mitra

Best Paper Awards Nominations: 2001–2002

50 SocietyAwards Nominations: 2003

Van ValkenburgTechnical AchievementSociety EducationIndustrial PioneerChapter-of-the-Year

52 RecognitionsIEEE Technical Field Awards

Emanuel Piore AwardKiyo Tomiyasu AwardLeon K. Kirchmayer Award

ISI Citation Leader

2002 CASS Fellows

56 Calls for Papers and ParticipationSee also inside front cover and back cover.

Scope: A publication of the IEEE Circuits and Systems Society, IEEE Circuits and Systems Magazine publishes articles presenting novelresults, special insights, or survey and tutorial treatments on all topics having to do with the theory, analysis, design (computer aided design),and practical implementation of circuits, and the application of circuit theoretic techniques to systems and to signal processing. The cover-age of this field includes the spectrum of activities from, and including, basic scientific theory to industrial applications. Submission ofManuscripts: Manuscripts should be in English and may be submitted electronically in pdf format to the publications coordinator [email protected]. They should be double-spaced, 12 point font, and not exceed 20 pages including figures. An IEEE copyrightform should be included with the submission. Style Considerations: 1) articles should be readable by the entire CAS membership;2) average articles will be about ten published pages in length; 3) articles should include efforts to communicate by graphs, diagrams, andpictures—many authors have begun to make effective use of color, as may be seen in back issues of the Newsletter, available at www.nd.edu/~stjoseph/newscas/; 4) equations should be used sparingly.

Circuitsand

SystemsCircuits

Systemsand

Circuitsand

SystemsIEEE

MAGAZINE

3

From the EditorMichael K. SainEditor-in-Chief, IEEE Circuits and Systems Magazine

THE ADVENTURES OF ……THE 'UMBLE OHM

…Shlomo Karni

Ohm with an Ohmlette

A Transition May Undergo a Transition!

More than a decade ago, in 1990, we began the de-sign and layout of the modern incarnation of IEEE

Circuits and Systems Society Newsletter, in a desktoppublishing environment, here in our office at the Univer-sity of Notre Dame.

As time passed, that publication increased its num-ber of pages, added color, and moved to include a steadilyincreasing amount of technical information, all addressedto the entire membership of the Circuits and SystemsSociety. Yet, it all continued to take place right here onthe desktop.

Then, in 2001, we made a transition, and began to dothe same tasks for the IEEE Circuits and Systems SocietyMagazine. This resulted in some changes in appearance,so as to be consistent with typical IEEE magazines. Butthe big change was in the contents, which now began tofeature longer and more detailed technical articles.

While the total number of pages did not immediatelyjump in a big way, and while the layouts—though dif-ferent—did not represent a quantum jump either, it iscertainly true that the task of the editor did increase bothin time required and in complexity; and this is due pri-marily to the necessity of the editor being much moreproactive in the area of acquisition of materials to bepublished.

Thus, when the scheduled end-of-December-2002completion of our time as editor approached, we experi-enced a sort of mixed emotion—some sadness at settingaside an activity to which we had devoted many yearsand yet some relief at being able to put down the ongo-ing load of publication which had spanned almost one-third of our professional career.

We thought that we would build up the supply pipe-line for our successor, arrange for transition of design andlayout to IEEE Publishing in Piscataway, and hang up asign: “Gone Fishin’”!

But these plans for transition were not to be. Welearned two months ago that the Society would like usto stay for an additional year, see the design and layoutof IEEE CAS Magazine safely transferred to Piscataway,and then pass the reigns.

It was a surprise, and one for which we had not pre-pared. However, after consulting with our EditorialBoard, and with others in our Society upon whom we

depend in various ways formaterials, we determined thatwe would be able to performthe requested extra year ofservice. In this way, our plansfor transition underwent atransition… .

One result: A great dealof time and effort went intoour study of whether wewould be able to offer theextra year. About two weekswere needed, during which time the editor did not workon anything else. Consequently, we are running later thanusual on this issue. Moreover, we will have to have ourmaterials to Piscataway by February 1 for the first quar-ter issue of 2003. This means that we have to preparetwo issues in the next three months. The good news isthat the fourth quarter issue of 2002 will likely go to pressmuch earlier than in previous years, so that the delay inthe current issue will be offset by an “advance” in the nextissue.

We wish to thank all those whose help has made itpossible for us to complete these first two years of CASMagazine, and we hope that they will continue to lend ahelping hand for one more year.

4

20

40

60

80

100

120

500 1000 1500 2000 2500 3000 3500

?Coin 1 Yes

p

p p

1-p

1-p 1-pwin

win win

lose

lose lose

1 1

2 2 3 3

Game BGame A

Is X(f) a multiple of 3?

Coin 2 Coin 3

No

Does Chaos WorkBetter Than Noise?

1531-636X/10/$10.00©2002IEEE

Does Chaos WorkBetter Than Noise?by Maide Bucolo, Riccardo Caponetto,

Luigi Fortuna, Mattia Frasca, Alessandro Rizzo

Abstract—Chaos and random signals share the property of long term unpredictable irregular be-havior and broad band spectrum. The aim of this paper is not to distinguish between random

and chaotic dynamics, nor to show the use of chaos, but to focus attention on how chaos and noisehelp order to arise from disorder. This means to investigate the effect of the introduction of either de-terministic chaotic or random sequences in different types of phenomena. In particular the results re-lated to different applications, self-organization in arrays of locally coupled systems in which a cha-otic dissymmetry is present, chaos driven optimization strategies, pattern formation in Drosophilaembryos and some new topics on game theory are treated with the aim to investigate the subject: “doeschaos work better than noise?”.

Figure 1. Chaos, depictedwith its typical structure

in phase plane, worksbetter than noise in a lotof examples presented in

this paper.

5

Introduction

A classical topic in studying realworld phenomena is to distinguish be-tween chaotic and random dynamics.Characterizing the irregular behaviorthat can be caused either by determin-istic chaos or by stochastic processesis not an easy task to perform. More-over, it is still an open problem to dis-tinguish among these two types of phe-nomena. Several time series analysismethods have been proposed to inves-tigate the presence of determinism ina set of data [1]. The main difficulty iscaused by the surprising similarity thatdeterministic chaotic and random sig-nals often show, as for example thecharacteristics of broad band spectra ofboth of them.

From another point of view, theactive use of chaos has been recentlywidely investigated in the literature.Particularly interesting results havearisen in the area of secure communi-cations [2]. Other topics of great inter-est are those concerning chaos controland chaotic circuit design. The inter-est in studying the use of chaotic sys-tems instead of random ones ariseswhen the theme of chaos reaches ahigh interdisciplinary level involvingnot only mathematicians, physiciansand engineers but also biologists,economists and scientists from differ-ent areas. Moreover, several studiesshowed that order could arise from dis-order in various fields (from biologi-cal systems to condensed matter, fromneuroscience to artificial neural net-works [3]). In these cases disorder of-ten indicates both non-organized pat-

Maide Bucolo, Riccardo Caponetto, LuigiFortuna, Mattia Frasca, and Alessandro Rizzo arewith the Dipartimento Elettrico Elettronico eSistemistico, Università degli Studi di Catania,Viale A. Doria 6 – I–95125 Catania, Italy. E-mail:[email protected]

terns and irregular behavior, whereasorder is the result of self-organizationand evolution and often arises from adisorder condition or from the pres-ence of dissymmetries. The origin ofself-organization is faced in [4], wherestarting from evolutionary theory anddiscussing various key points in biol-ogy, the idea that life exists at the edgeof chaos has been emphasized. Otherexamples in which the concept of sto-chastic driven procedures leads to “or-dered” results are Monte Carlo andgenetic algorithms for optimizationprocedures, as well as stochastic reso-nance in which the presence of noiseimproves the transmission of the infor-mation [5].

The aim of this paper is not to dis-tinguish between random and chaoticdynamics, nor to show the use ofchaos, but to focus attention on howchaos and noise help the birth of orderfrom disorder. This means to investi-gate the effect of the introduction ofeither deterministic chaotic or randomsequences in different types of sys-tems: such as complex systems, opti-mization procedures, and biologicalsystems. Therefore, the question thatthis paper tries to discuss is: “doeschaos work better than noise?”. Inother words, which side of the balanceof Fig. 1 will prevail?

This paper addresses the questionby means of numerous multidis-ciplinary examples. In the section Self-Organization in Arrays of DynamicalSystems the result of the presence ofdeterministic dissymmetry is treated intwo cases: Josephson junctions andChua’s circuits array. In the next sec-tion a comparison between the perfor-mance of genetic algorithms that runusing chaotic signals and that of tradi-tional ones is presented. Then the ef-fect of using chaos in an example of arandom-based optimization algorithm,

6

Ant Colony Optimization, is investi-gated. Parrondo’s Paradox is consid-ered in the next section in order to fur-ther investigate the problem. PatternFormation in Drosophila Embryos isperhaps the most fascinating exampleof self-organization, and this sectionpresents new results showing howchaos can be useful to find in a moreeffective way the parameters for pat-tern formation.

Self-Organization in Arrays ofDynamical Systems

In this section topics related to therole of a chaotic induced spatial diver-sity versus a random one are intro-duced in regard to the self-organizingbehavior of arrays of dynamical sys-tems. In particular, recent results re-lated to some particular nonlinear cir-cuits like the Josephson junctions andthe Chua’s circuits are reported.

Arrays of Josephson Junctions

As a starting point, this section willdeal with a valuable example concern-ing arrays of Josephson junctions. Re-search on synchronization of arrays ofJosephson junctions has been moti-vated by the fact that Josephson junc-tions are effective components capable

of generating extremely fast voltageoscillations (typically at terahertz fre-quency); nevertheless their outputpower is extremely low (typically10 nW), making the single junction al-most useless for most electronics ap-plications, if it is not part of a long,synchronized array. It has already beenshown [6] that spatial diversity helpsthe tendency for self-synchronizationof the junctions: if an array of identi-cal junctions is considered, in-phaseperiodic orbits in fact exist for a cer-tain range of the parameter set, butthey are not asymptotically stable. Onthe other hand, Braiman et al. [7] alsoshowed that a moderate increase in thespatial disorder of the array can leadto significant improvements in the syn-chronization process. The results ob-tained parallel those obtained by thesame authors for mechanical systems[8], such as arrays of damped, forcedpendula, in which a moderate, random,spatial diversity introduced in thelength of the pendula enhances thesynchronization capability. In [9–11],several experiments show that gener-ating the spatial diversity by using achaotic law enhances the regulariza-tion and the formation of patterns inmany spatially extended systems. Inparticular, the same array of pendulaconsidered in [8] has been synchro-nized by introducing a moderateamount of diversity generated by cha-otic systems.

In this section an array of non-identical, locally connected chaotic Jo-sephson junctions is taken into ac-count. It is known that, under particu-lar conditions, a single junction canexhibit chaotic behavior. In [12] chaosis achieved by introducing a nonlineareffect in the junction, whereas in thiswork the chaotic behavior of the singlejunction is achieved by feeding the

Figure 2a. Chaotic behavior of identicalJosephson junctions.

20

40

60

80

100

120

500 1000 1500 2000 2500 3000

Time (s)

Cel

l Ind

ex

7

junction with a periodic driving bias.The global behavior of the array isinvestigated in the case of identicaljunctions, where spatio-temporalchaos emerges, and in presence ofspatial diversity, generated by eitherrandom or chaotic law. In the lattercase, it has been confirmed that acertain amount of disorder enhancesthe self-organization capability. Inparticular, in agreement with the re-sults presented in [9–11], the intro-duction of diversity generated by achaotic law leads to a further im-provement in synchronization.

From a mathematical point of viewthe array of Josephson junctions [6] canbe described by the following formula:

˙φ j + aj φ j + sinφj

= I0+ k(φj + 1 – 2φj + φj–1)+ I · sin(ωt)

j = 1, …, N (1)

where φ is the phase difference be-tween the two quantum mechanicalwave functions of the two layers of thejunction (see Appendix), aj is a param-eter related to the physical propertiesof the junction, the term k(φj + 1 – 2φj +φj–1) represents the coupling with theneighboring junctions and I0 + I ·sin(ωt) is the current bias.

Under these assumptions, whenthe array consists of identical junctions(aj = a ∀j), as time elapses, more andmore disordered spatio-temporal pat-terns emerge, denoting a chaotic be-havior. This can be noticed in Fig. 2(a),where a color map codes the variablecharacterizing the behavior of eachjunction in terms of the variable φ ver-sus time. The prevalence of blue in thecolor map is due to the d.c. term in theforcing torque. The evolution appearsdisordered, and is in particular chaotic.By introducing a random, symmetricaldisorder in the aj (in a range of 10–20%of the nominal value), periodic spatio-temporal patterns can be observed(Fig. 2(b)). Our analysis deals nowwith the effects induced by the intro-duction of deterministic, non-orga-nized dissymmetry, like the ones gen-erated by a chaotic attractor. In thisexperiment, a portion of a Chua’sattractor is sampled and adequatelyscaled in order to superimpose a deter-ministic disorder in the junction pa-rameters. The result of simulationshows (Fig. 2(c)) that a chaotic varia-tion on junction parameters leads thearray towards a collective organiza-tion. Junctions in the central region ofthe array are synchronized both in

Figure 2b. Self-synchronization byrandom dissymmetry.

Figure 2c. Self-synchronization by non-organizeddeterministic dissymmetry.

20

40

60

80

100

120

200 400 600 800 1000 1200

Time (s)

Cel

l Ind

ex

1400 1600

20

40

60

80

100

120

500 1000 1500 2000 2500 3000

Time (s)

Cel

l Ind

ex

3500

8


space and time, oscillating with thesame frequency as the forcing current,while in the external bands a regularspatial wave propagates. Therefore,spatio-temporal chaos disappeared,leading to a periodic behavior by sim-ply perturbing the symmetry of thesystem.

Array of Chua’s Circuits

The second example reported inthis work concerns arrays of Chua’scircuits. Interest arises on arrays ofChua’s circuits to investigate spatio-temporal chaos, [13, 14], propagationof impulsive information [15], and for-mation of spiral waves in a two-dimen-sional circuit matrix [16].

All the works performed agree onthe fact that, for arrays constituted byidentical circuits, global behavior isstrongly affected by changes in theconnection coefficient, denoting in aspatio-temporal context all the phe-nomena that can be observed in thesingle circuit: equilibrium states, limitcycles, and, obviously, spatio-tempo-ral chaos. Taking inspiration from theprevious example, an array of Chua’scircuits [17] has been considered. Thefollowing equation describes the k-thcell of the array in the well known di-mensionless form:

xk = αk[yk – m1xk

– 0.5(m0 – m1)[ |xk + 1| – |xk – 1| ]+ D(xk – yk + 1)]

yk = xk – yk + zk + D(yk – xk + 1) (2)

zk = –βyk

A mono-dimensional array of 128adjacent units coupled through linearFigure 3. Chua’s circuits array experiment.


(a) Chaotic behavior of identical circuits.

(b) Self-synchronization by random dissymmetry.

(c) Self-synchronization by non-organizeddeterministic dissymmetry.

20

40

60

80

100

120

20 40 60 80 100 120

Time (s)

Cel

l Ind

ex

140 160 180 200

20

40

60

80

100

120

20 40 60 80 100 120

Time (s)

Cel

l Ind

ex

140 160 1800

20

40

60

80

100

120

20 40 60 80 100 120

Time (s)

Cel

l Ind

ex

140 160 180 200

9

resistors (each circuit is characterizedby parameters chosen according to[17] to generate the Double ScrollChua Attractor) is taken into account,and both a random and a determinis-tic dissymmetry on a circuit parameter(the αk parameter is allowed to vary ina range of 10–20% of its nominalvalue) are introduced. The parametersk and D represent the introduced dif-fusive coupling. As can be observed inFig. 3, very regular spatio-temporalpatterns emerge only when the im-posed perturbation is chaotic, confirm-ing the conjecture that chaos can helpsystems to achieve order and synchro-nization.

A General Remark

Even if only some specific cases ofinduced spatial disorder leading toregular patterns have been presented,various experiments have been carriedout by the authors where the same sce-nario has been observed. Two-dimen-sional arrays like Cellular NonlinearNetworks [10], arrays of Hindmarsh-Rose neurons [18] and distributed net-works of fuzzy dynamical systems[19] have been also considered. Ineach case a weak spatial dissymmetryleads to self-organization, but a strongimprovement has been emphasizedwhen chaos is used instead of noise.

Improving Performance ofGenetic Algorithms

In this section and in the next one,the role played by nonlinear chaoticdynamics versus random processes inoptimization algorithms is examined.

The convergence properties ofGenetic Algorithms (GAs) [20] areclosely connected to the random se-quence applied on genetic operatorsduring a run. In particular, when start-ing some genetic optimizations withdifferent random sequences, experi-ence shows that the final results maybe very close but not equal, and requirealso different numbers of generationsto reach the same optimal value. Therandom sequence generation algo-rithms, on which most used GA toolsrely, usually satisfy on their own somestatistical tests like chi-square or nor-mality. However, there are no analyti-cal results that guarantee an improve-ment of the performance indexes ofGA algorithms depending on thechoice of a particular random numbergenerator.

Chaotic systems have already beenexploited to define new operators to beapplied during genetic optimization, inorder to improve the performance ofGAs. In particular, in [21] a specialmutation operator, applied during generecombination and based on the logis-tic function, is introduced showing in-teresting results in exploration andexploitation of GA capabilities. Also in[22] chaotic time series are used inDNA computing procedures. More re-cently, in [23] chaotic sequences have

The random sequence generation algo-rithms, on which most used GA tools rely,usually satisfy on their own some statisticaltests like chi-square or normality. However,there are no analytical results that guaranteean improvement of the performance indexesof GA algorithms depending on the choiceof a particular random number generator.

10

been used to increase population sizedynamically in order to avoid GAs’premature convergence.

According to the conjecture intro-duced in this work, all the random se-quence generators in a GA are replacedby chaotic generators, without affect-ing the original operator definitions.Therefore, chaotic sequences influencethe behavior of all genetic operators.In particular, a GA uses random se-quences for the following purposes:

• during the creation of the initial popu-lation it is necessary to generate therequired number of individuals usinga random number generator;

• the selection algorithm is based onthe probabilistic choice of individu-als according to their fitness; randomgenerators are used also in this op-eration;

• crossover algorithms are based onthe random choice of points insidethe chromosomes or on the randomgeneration of bit masks;

• the mutation operator is based on therandom change of bits in chromo-somes.

Four different types of test prob-lems have been considered: De Joungfunctions [24], an eigenvalues LinearMatrix Inequalities problem [25], theIterated Prisoner Dilemma (IPD), andthe Traveling Salesman Problem(TSP) [26].

Figure 4 shows comparisons madeon a specific performance index namedoff-line performance [24], evaluatedby using different random or chaoticsequence generators on the De Joungfunction and TSP test problems. As itcan be clearly seen, best performanceis always obtained by using chaoticgenerators.

An improvement has also beenobtained in terms of speed of conver-gence of the algorithm, as illustratedin Fig. 5.

Ant Colony OptimizationAlgorithm

Recent research on ethologicalsystems has emphasized self-organiza-tion in animal colonies as a crucialpoint for the accomplishment of thosetasks which require a high degree ofco-ordination among workers. Antcolonies, for example, can build nests,feed broods, forage for food, and so on[27]. Beyond biological interest, the

Figure 4. Comparison of performance for different random andchaotic generators, made on (a) De Joung function f6, (b) TSP.

De Joung f6

3.50E+10

3.55E+10

3.60E+10

3.70E+10

3.75E+10

3.80E+10

3.85E+10

3.90E+10

3.95E+10

4.00E+10

Rand1Rand2Rand3LogisticChuaSinLoziGauss3.65E+10

TSP

0.00E+00

1.00E+10

2.00E+10

3.00E+10

4.00E+10

5.00E+10

6.00E+10

7.00E+10

8.00E+10

9.00E+10

Rand1

Rand2

Rand3

Logistic

Tent

Sin

Lozi

Gauss

11

0 10 20 30 40 50 60

Generations

300

400

500

600

700

800

Off

Lin

e M

axim

um P

erfo

rman

ce

1

2

34

5

6 6 Chua5 Lozi4 Gauss3 Sinusoidal2 Logistic1 Random

computer science community has en-visioned in these studies a powerfulsource of inspiration [28] to developtechniques to solve complex problems,exploiting a branch of Artificial Intel-ligence called Swarm Intelligence [29].In this context, classical optimizationproblems like the Traveling SalesmanProblem (TSP) have been faced bytaking as an underlying intelligencemodel the collective intelligence ofsocial insect colonies like ants. In na-ture, ant colonies find shortest routesfrom nest to food and vice-versa bylaying and following pheromone trails.TSP, which consists of finding theshortest tour between n cities, visitingeach one only once and ending at thestarting point, is solved by an algo-rithm which parallels the collective antbehavior by exploiting an artificialpheromone. The artificial pheromoneis a paradigm to take into account themost tracked paths; its strength is en-forced by further visits of the route andweakened as time elapses through anevaporation rate. The features of thealgorithm offer the possibility ofimplementation in an agent-based, dis-tributed environment [30].

All the implementations of AntColony Optimization (ACO) algo-rithms rely on a guided random searchprocedure. Instead of considering ran-dom variables to make decisions, achaotic law is adopted. Performancehas been evaluated on different TSPbenchmarks by adopting several cha-otic laws and different, well knownversions of the algorithm (Ant System,Ant System with Elitist Strategy,Rank-Based Ant System, Max-MinAnt System, Ant Colony System). Ifthe average length of the best solutionover several trials is considered as acomparison term, chaos and randomperform quite similarly. However, iffor each algorithm run, the best pathlengths obtained with both the algo-rithms (chaos based and random

Figure 5. Convergence curves of several GAs with differentrandom and chaotic sequence generators.

Maide Bucolo received the Computer ScienceEngineering degree from the University of Cataniain 1997 and the Ph.D. degree in electronic and con-trol engineering in 2001. She is currently visitingprofessor of system theory at the University ofMessina and involved in the DICTAM Project(“Dynamic Image Computing Using Tera-SpeedAnalogic Visual Microprocessors”) supported byEU. Her research activity mainly regards cellularneural networks and nonlinear time series analysisfor biomedical applications.

12

based) are compared, the results,shown in Fig. 6, outline that in 48.8%of the runs the logistic map works bet-ter than the random based algorithm,while in 22.7% of the runs the resultsobtained with the two algorithms arethe same and only in 28.5% of the runsthe random based algorithm worksbetter than the logistic map. This re-sult obtained on the standard bench-mark EIL51, taken from TSPLIB [26]and the Rank-Based Ant System, is

still valid also when other chaoticmaps are applied.

Figure 6 illustrates the perfor-mance obtained by a dedicated visualsoftware developed, whose interface isillustrated in Fig. 7.

Parrondo’s Paradox

Should chaos play a central rolealso in game theory? This question isinvestigated by considering Parrondo’sParadox. This paradox has been intro-duced by Parrondo as a pedagogicalillustration of the Brownian ratchet. Itstates that a resulting winning gamecan be obtained by playing in a randomor periodic fashion two games whichare separately losing [31, 32].

The two losing games are calledGame A and Game B. Game A consistsof a coin having p as winning probabil-ity (p < 0.5 is chosen to obtain a los-ing game). Game B consists of two al-ternatively played coins: if the presentplayed capital is multiple of a giveninteger M then the coin to be playedwill have a winning probability p1, oth-erwise (the present capital is not amultiple of M) the coin to be playedhas a winning probability p2. The two

Figure 6. Comparison of performance on the TSPby Ant Colony System Algorithm.

48.80%

22.70%

28.50%

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

45.00%

50.00%

Logistic Same Random

RBAS: Logistic vs Random

Figure 7. Visual software tool for the chaos-based ant optimization algorithm.

(a) Map of the cities. (b) Evolution of the colony.

13

games, when played independentlyfrom each other, are losing games if thefollowing conditions hold:

1 − p

p > 1 (3)

for Game A, and

(1 − p1)(1 − p2 )M −1

p1p2M −1 > 1 (4)

for Game B. The paradox consists inconstructing a game in which Game Aand Game B are alternatively playedwith a probability γ : this compositegame is winning.

Parrondo showed that the paradoxoccurs if the following condition holds:

(1 − q1)(1 − q2 )M −1

q1q2M −1 > 1 (5)

where q1 = γ p + (1 – γ )p1 and q2 =γ p + (1 – γ )p2.

While in the original Parrondo’sParadox the strategy is based on a ran-dom choice of one of the two gameswith the probability γ , in this work thischoice is based on the value given bya chaotic sequence. Also in this case,many chaotic sequence generatorshave been considered. Figure 8 illus-trates the trend of the capital gained by

playing the two games separately, al-ternated by a random law, alternated bychaotic laws. It is evident that perfor-mance is much increased when a cha-otic law is adopted.

However, one of the limitations ofapplying the original paradox in realworld problems, such as genetics, evo-lution, and economics [32], is that it in-volves only two games and one ofthem is a capital dependent game. Theidea underlying the Parrondo’s Para-

Figure 7. Visual software tool for the chaos-based ant optimization algorithm.

Figure 8. Trend of capital gain in Parrondo’s Paradox (the gameparameters are: M=3; p=1/2–0.005; p1=1/10–0.005; p2=3/4–0.005).

0 10 20 30 40 50 60 70 80 90 1002

1

0

1

2

3

4

5

6

7

Games Played

Gai

n

Game A

Game B

random

logistic

sinusoidal

henon

(c) Best path found after 2 iterations. (d) Optimal path (found after 4 iterations).

14

dox may be extended to n differentgames. While for n = 3 it is still pos-sible to find analytically the parameterscharacterizing the various games (asfor example the losing probabilities ofeach game when independently

played) and leading to the paradox, formore than three games, an optimiza-tion strategy should be used. Even inthe generalized Parrondo’s Paradox thegames are played according to a ran-dom choice. When a chaotic choice isperformed instead of a random one, thegain in the capital is increased. Thecomparison of the results obtainedwith a random choice and those ob-tained with chaotic maps for a six gamesparadox is shown in Fig. 9.

Pattern Formation inDrosophila Embryos

In this section our study focuses ona parameter identification procedurewhere a complex nonlinear model isinvolved. The traditional procedure isbased on a random sequence, while inthis section the effect of using chaosinstead of noise is examined.

The phenomenon of pattern forma-tion in Drosophila embryos has been re-cently studied by carrying out an analy-sis of a mathematical model realizedby Von Dassow and colleagues [33].

Figure 9. Trend of capital gain in a six games Parrondo’s Paradox whendifferent strategies are adopted to choose the game to be played. The sixgames are labeled as A1, A2, B1, B2, C1, C2. The capital reductions for

each game when independently played are also shown.

Figure 10. (a) Pattern of gene expressions in 8 x 2 network of cells. The brightness ofthe color, arbitrarily chosen, reflects the concentration of the gene product in the cell.(b) Graphic representation of a solution. Each parameter value is reported in a spoke

representing the logarithm scale range of the parameter.

0 5 10 15 20 25 30 35 40 45 503

2

1

0

1

2

3

4

5

GAIN AFTER 100 STEPSRANDOM: 5.1772LOGISTIC: 9.6972TENT: 7.2760Henon: 5.8788

A1

B1

C1A2

B2

C2

Random

Logistic (a=4)

Tent

Henon

(a) (b)

15

The model takes into account the bio-logical elements involved in the pat-tern formation, these elements areproducts of genes (both mRNAs andproteins). For all cells a regular hex-agonal form is assumed. The modelconsists of a set of nonlinear ordinarydifferential equations describing theinteractions among the products ofgenes of the Drosophila in terms ofconcentrations of the components in anindexed cell or cell face. In all 136variables and 50 free parameters aretaken into account for each segment,that consists of four cells. The param-eters involved in the model have un-known values, that can vary in a largerange (several orders of magnitude) ofbiologically plausible values.

In order to deal with the high num-ber of unknown parameters of themodel, an iterative procedure is usu-ally set-up by randomly choosing acandidate set of possible parametersand performing a numerical simulationof the model through the Java tool In-genue [34]. This tool is built specifi-cally to model networks of genes in-teracting in a field of cells and providesa user-friendly interface. Ingenue pro-vides a procedure, called Iterator, toexplore the space of possible solutionsby comparing the patterns obtainedwith those actually observed in devel-oping Drosophila embryos.

Figure 10(a) shows the pattern ofgene expression in the cells as obtainedwith the model, while Fig. 10(b) is agraphic representation of the solutions.This pattern closely matches the pat-tern of the Drosophila embryo.

Usually the iterative procedure ofIngenue makes use of random rou-tines. A chaos guided generation of theparameters of the model is here pro-posed. This new approach reveals bet-ter results than a random based choiceof the possible solutions. In a represen-

tative case after 1000 iterations the tra-ditional random based procedurefound 3 feasible solutions, while afterthe same number of iterations thechaos guided algorithm, which ex-ploits a tent map, found 15 feasiblesolutions. Moreover, the first solutionoccurs after 84 iterations with the ran-dom based procedure, while it occursafter 51 iterations when chaos guidesthe choice of parameters.

The results obtained with differentchaotic maps (Tent map, Lozi map,Logistic map, Lorenz peak-to-peakdynamics) are compared as shown inFig. 11 on the basis of several itera-tions starting from different initial con-ditions. Figure 11 reports the averagenumber of feasible solutions found by

Figure 11. The average number of feasible solutions and the number ofsolutions found in the best case by the algorithm based either on the random

generator, traditionally used in Ingenue, or on a chaotic map. The resultsobtained with several chaotic maps (Tent map, Lozi map, Logistic map,

Lorenz peak-to-peak dynamics) are compared.

Riccardo Caponetto was born in 1966. He received the electronic engineeringdegree from the University of Catania in 1991 and the Ph.D. in electrical engineeringin 1995. Starting from 1994 he has been working as researcher at STMicroelectronics.From 1995 to 1998 he was professor of industrial robotics at the University of Messina,Italy. Since 2001 he is assistant professor at the DEES of the Engineering Faculty andhis interests include Soft Computing techniques, modeling and control of complex sys-tems, and robotics.

3 3

28

6.98

4.5 5

1 1

24

0

5

10

15

20

25

30

StandardRandom

Tent 0.5 Tent 0.7 Lozi Logistic Lorenz PPD

Generators

Number of Solutions

Best Mean

2.571428571

20.1

16


the algorithm as well as the number ofsolutions found in the best case.

Conclusions

In this work the topic of using de-terministic chaotic signals instead ofrandom signals has been explored. Of-ten the question “does chaos work bet-ter than noise?” arises. In our experi-ments we are encouraged to assert thatthe benefits of chaos are often evidenteven if a general answer cannot be for-mulated.

Several studies have been per-formed and only a few results havebeen summarized in the paper. Our at-tention has been devoted to two classesof problems: the first one regards therole of spatial diversity to control com-plex systems in order to improve self-organization and synchronization incircuits and systems organization; thesecond one regards in general the classof random based optimization algo-rithms, where random number genera-tors are usually introduced.

In both cases an improvement hasgenerally been noticed when chaos

Figure 12. (a) The Lorenz peak-to-peak dynamics. (b) The Double Scroll Chua Attractor(marks indicate the samples of the chaotic sequence generated from the chaotic attractor).

instead of random number generatorshas been introduced both to generatespatial diversities and to introducenon-organized patterns into the imple-mentation of numerical procedures.Moreover, the same encouraging resultshave been obtained when ParrondoGames have been considered. A furtheroptimization example refers to classi-cal pattern formation in biology, wherethe introduction of a chaotic drivensearch algorithm of parameter identi-fication has been performed.

The question introduced in thispaper is still open and we hope that thiscontribution will stimulate more ex-amples either to reinforce our feelingor to deny it.

In any case we hope to encouragethe debate in new areas of researchwhere, so far, only experiments andsimulations are useful to understandcomplex phenomena.

Appendix

In the appendix both the equationsof the chaotic dynamical systems re-ported in the experiments and the model


(a) (b)

30 35 40 45 5025zn

z n+

1

28

30

32

34

36

38

40

42

44

26

46

-2 -1 0 1 2-3x1

3

x 2

-0.6

-0.4

-0.2

0

0.2

0.4

-0.8

0.8

0.6

17

of Josephson junctions are illustrated.

Chaotic Maps

Determinism, long term unpredict-ability and high sensitivity to initialconditions are the peculiarity of chaos.Systems showing chaos can be bothcontinuous-time systems and discrete-time maps. The chaotic sequences usedin most of our experiments have beengenerated by using different well-known chaotic maps [35], reported inthe following:

• Logistic Map

xn + 1 = 4xn(1– xn) (6)

• Tent Map

xn + 1 =

xn

0.7xn ≤ 0.7

1 − xn

0.3otherwise

(7)

• Sinusoidal Map

xn + 1 = ax2n sinxn (8)

with a = 2.3

• Gaussian Map

xn + 1 = 0 xn = 0

1xn

mod1 xn ≠ 0

(9)

• Lozi Map

xn + 1 = yn + 1 – a|xk| (10)

yn + 1 = bxk

with a = 1.7; b = 0.5.Moreover, two other maps have

been used to generate chaotic se-quences; these maps are obtained start-ing from continuous-time chaotic sys-tems such as the Lorenz system andChua’s circuit. The equations of both

chaotic oscillators are:

Lorenz system

x = σ (y – x)

y = rx – y – xz (11)

z = xy – bz

Chua’s circuit

x = α (y – m1x– 0.5(m0 – m1)[|x + 1| – |x – 1|])

y = x – y + z (12)

z = –βy

with the following parameters:

σ = 10; ρ = 28; b = 8/3; α = 9; β =14.286; m0 = –1/7; m1 = –2/7.

The peak-to-peak dynamics of theLorenz system has been taken into ac-count [35], while as regards Chua’scircuit a chaotic sequence is obtainedby the sampling of a variable of theDouble Scroll Chua Attractor [17].Figure 12 illustrates the Lorenz peak-to-peak dynamics and the phase planeprojection of the Double Scroll ChuaAttractor (marks indicate samples ofthe chaotic sequence).

In this work the topic of using determinis-tic chaotic signals instead of random signalshas been explored. Often the question “doeschaos work better than noise?” arises. In ourexperiments we are encouraged to assert thatthe benefits of chaos are often evident evenif a general answer cannot be formulated.

Luigi Fortuna is professor of system theoryat the University of Catania since 1994. He haspublished more than 250 technical papers and is co-author of six books, among which is Cellular Neu-ral Networks (Springer 1999). He holds several USpatents. His scientific interests include nonlinearscience and complexity, chaos, and cellular neuralnetworks with applications in bioengineering. Heis chair of IEEE Technical Committee on CellularNeural Networks and Array Computing.

18

The Josephson Junction Model

The physics underlying the Jo-sephson effect is regulated by quantummechanics. Nevertheless, the dynam-ics of a Josephson junction, constitutedby two closely spaced semiconductorsseparated by a weak connection, isusually described in classical terms.The current flowing in a Josephsonjunction consists of three main con-tributors: the supercurrent, due to theactual Josephson effect, the displace-ment current, which can be modeled bythe contribution of a capacitor, and theordinary current, modeled by the contri-bution of a resistor. Based on this con-sideration, the junction model adopted inthis work is illustrated in Fig. 13.

Applying Kirchoff’s current andvoltage laws to the circuit in Fig. 13,and exploiting the Josephson current-phase and voltage-phase relations [35],the following dynamical model can bewritten:

layers of the junction. h represents thePlanck constant, e the charge of the elec-tron, and Ic is a critical current, typicalof the junction considered. Moreover,following the normalization reportedin [35], Eq. 13 can be rewritten in di-mensionless form as follows:

d 2φdt2 + a

dφdt

+ sin φ = I (14)

The forcing signal has been chosento be a signal consisting of a sine cur-rent plus a constant bias (i.e. Ib = I0 +Isin ω t), and the following parametervalues have been assumed: I0 = 0.7155,I = 0.4, ω = 0.25, a = 0.75.

Acknowledgement

This work has been partially sup-ported by the Italian “Ministerodell’Istruzione, dell’Università e dellaRicerca” (MIUR) under the Firbproject RBNE01CW3M.

References

[1] D. T. Kaplan, “Exceptional Events as Evi-dence for Determinism”, Physica D, vol.73, pp. 38–48, 1994.

[2] R. Caponetto, M. Criscione, L. Fortuna, D.Occhipinti, L. Occhipinti, “Synthesis ofa Programmable Chaos Generator, Basedon CNN Architectures, with Applicationsin Chaotic Communication” , CNNA ’98,London, UK, April 14–17, 1998.

[3] S. Kauffman, The Origin of Order, Self-Organization, and Selection in Evolution.Oxford University Press, 1993.

[4] R. Lewin, Complexity: Life at the Edge ofChaos. University of Chicago Press, 1999.

[5] B. Andò, S. Graziani, Stochastic Reso-nance: Theory and Applications. KluwerAcademic Publishers, 2000.

[6] K. Y. Tsang, R. R. Mirollo, S. H. Strogatz,K. Wiesenfeld, “Dynamics of a GloballyCoupled Oscillator Array”, Physica D,vol. 48, pp.102–112, 1991.

[7] Y. Braiman, W. L. Ditto, K. Wiesenfeld, M.L. Spano, “Disorder-Enhanced Synchro-nization”, Physics Letters A, vol. 206, pp.54–60, 1995.

[8] Y. Braiman, J. F. Lidner, and W. L. Ditto,“Taming Spatiotemporal Chaos with Dis-

Figure 13. (a) The Josephson junction and (b) the equivalent electrical circuit.

Mattia Frasca was born in Siracusa, Italy, in1976. He graduated in electronics engineering in2000 from the University of Catania, Italy. Currentlyhe is a Ph.D. student in electronics and automationengineering at the University of Catania. His scien-tific interests include nonlinear systems and chaos,cellular neural networks and bio-inspired robotics.

hC

4πe

d 2φdt2 +

h

4πeR

dφdt

+ Ic sin φ = Ib (13)

Equation (13) describes the dy-namics of the single junction only interms of variable φ , which is the phasedifference between the two quantummechanical wave functions of the two

Superconductor

Weak coupling

Superconductor

IcsinφI R C

(a) (b)

19

order”, Nature, vol. 378, pp. 465–467,November 30, 1995.

[9] P. Arena, L. Fortuna, D.Porto, A. Rizzo,“Self-Organization in Arrays of Nonlin-ear Systems Induced by Chaotic Pertur-bation: An Experimental Approach”,ISCAS 99 Conference, Orlando, Florida,vol. 5, pp. 479–482, 1999.

[10] P. Arena, R. Caponetto, L. Fortuna, A.Rizzo, “Nonorganized Deterministic Dis-symmetries Induce Regularity in Spa-tiotemporal Dynamics”, InternationalJournal of Bifurcation and Chaos, vol. 10,no. 1, pp. 73–85, 2000.

[11] P. Arena, R. Caponetto, L. Fortuna, M. LaRosa, A.Rizzo, “Self-Organization inNon-Recurrent Complex Systems”, Inter-national Journal on Bifurcation andChaos, World Scientific, Singapore, vol.10, No. 5, 2000.

[12] S. K. Dana, D. C. Sengupta, K. D. Edoh,“Chaotic Dynamics in Josephson Junc-tion”, IEEE Transactions on Circuits andSystems—I, vol. 48, no. 8, pp. 990–996,2001.

[13] A. M. Dabrowski, W. R. Dabrowski, andM. J. Ogorzalek, “Dynamic Phenomena inChain Interconnections of Chua’s Circuits”,IEEE Transactions on Circuits and Sys-tems—I, vol. 40, no. 11, pp. 868–871, 1993.

[14] M. Gilli, “Investigation of Chaos in LargeArrays of Chua’s Circuits via SpectralTechnique”, IEEE Transactions on Cir-cuits and Systems—I, vol. 42, no. 10, pp.802–806, 1995.

[15] V. Perez-Munuzuri, V. Perez-Villar, andL. O. Chua, “Propagation Failure in Lin-ear Arrays of Chua’s Circuits”, Interna-tional Journal of Bifurcation and Chaos,vol. 2, no. 2, pp. 403–406, 1992.

[16] A. Perez-Munuzuri, V. Perez-Munuzuri, V.Perez-Villar, and L. O. Chua, “Spiral Waveson a 2-D Array of Nonlinear Circuits”,IEEE Transactions on Circuits and Sys-tems—I, vol. 40, no. 11, pp. 872–877, 1993.

[17] R. N. Madan, “Chua’s Circuit: A Paradigmfor Chaos”, World Scientific Series on Non-linear Sciences, Series B, vol. 1, 1993.

[18] M. La Rosa, M. I. Rabinovich, R. Huerta,H. D. I. Abarbanel, L. Fortuna, “SlowRegularization through Chaotic Oscilla-tion Transfer in a Unidirectional Chain ofHindmarsh-Rose Models”, Physics Let-ters A, 266 (1), pp. 88–93, 2000.

[19] M. Bucolo, L. Fortuna, “Spatial Disorderin Complex Neuro-Fuzzy Dynamics”,Progress of Theoretical Physics Supple-ment, pp. 445–452, 2000.

[20] J.H. Holland, Adaptation in Natural andArtificial Systems. Ann Arbor, Michigan:University of Michigan Press, 1975.

[21] J. Determan and J. A. Foster, “Using Chaosin Genetic Algorithms”, Proceedings 1999Congress on Evolutionary Computation,vol. 3, pp. 2094–2101, 1999.

[22] G. Manganaro and J. Pineda de Gyvez,“DNA Computing Based on Chaos”, IEEEInternational Conference on EvolutionaryComputation, pp. 255–260, 1997.

[23] L. Juan, C. Zixing and L. Jianqin, “Pre-mature Convergence in Genetic Algo-rithms: Analysis and Prevention Based onChaos Operator”, Proceedings ThirdWorld Congress on Intelligent Controland Automation, China, 2000.

[24] D.E. Goldberg, Genetic Algorithm inSearch Optimization and Machine Learn-ing. Addison Wesley, 1989.

[25] S. Boyd, L. Ghaoui, E. Feron, and V.Balakrishnan, Linear Matrix Inequalitiesin System and Control Theory. Philadel-phia, SIAM Books, 1994.

[26] http://softlib.rice.edu/softlib/tsplib/[27] J.-L. Deneubourg and S. Goss, “Collec-

tive Patterns and Decision Making”,Ethology, Ecology, and Evolution, vol. 1,pp. 295–311, 1989.

[28] E. Bonabeau, M. Dorigo and G.Theraulaz, “Inspiration for Optimizationfrom Social Insect Behavior”, Nature, vol.406, pp. 39–43, 2000.

[29] E. Bonabeau, M. Dorigo and G.Theraulaz, Swarm Intelligence: FromNatural to Artificial Systems. New York,Oxford University Press, 1999.

[30] M. Dorigo, V. Maniezzo, and A. Colorni,“The Ant System: Optimization by aColony of Cooperating Agents”, IEEETransactions on Systems, Man, and Cy-bernetics B, vol. 6, pp. 29–41, 1996.

[31] G. P. Harmer and D. Abbott, “Parrondo’sParadox”, Statistical Science, vol. 14, pp.206–213, 1999.

[32] G. P. Harmer and D. Abbott, “LosingStrategies Can Win by Parrondo’s Para-dox”, Nature, vol. 402, p. 864, 1999.

[33] G. Von Dassow, E. Meir, E. M. Munro andG. M. Odell, “The Segment Polarity Net-work is a Robust Developmental Mod-ule”, Nature, vol. 406, no. 13, pp. 188–192, 2000.

[34] www.ingenue.org[35] S. H. Strogatz, Nonlinear Dynamics and

Chaos. Cambridge, Massachussetts, Per-seus Publishing, 2000.

Alessandro Rizzo was bornin Catania, Italy, in 1971. Hegraduated in computer engi-neering in 1996 and receivedthe Ph.D. degree in 2000 inelectronics and automation en-gineering from the Universityof Catania. In 1998 he workedat JET Joint Undertaking in aEU project concerning sensorvalidation in tokamak ma-chines. He is currently profes-sor of robotics at the Univer-sity of Messina, Italy, and col-laborates with ENEA, Frascati.His research interests includenonlinear and complex sys-tems, fault detection and iso-lation in industrial plants, andnuclear fusion engineering. Heis chair of the Nuclear andPlasma Science Society of theCentral and South Italy Sec-tion of IEEE.

20


Compression


Compressionby Eduardo A. B. da Silva, Décio A. Fonini Jr.,

and Marcos Craizer

Abstract—Successive approximation (SA) quantization is part of many ofthe state-of-the-art image and video compression methods. In this article

we first make a review of it, starting from the classical optimality considerationsof Equitz and Cover and then proceeding to Mallat and Falzon results concern-ing low bit-rate transform coding. We then develop a general theory of SA quan-tization which we refer to as α-expansions. This theory explains the publishedresults obtained by both scalar and vector SA quantization methods, and indi-cates how further performance improvements can be obtained.

1531-636X/10/$10.00©2002IEEE

21

Eduardo A. B. da Silva and Décio A. Foniniare with PEE/DEL/COPPE/EE, UniversidadeFederal do Rio de Janeiro, C.P. 68504, Rio deJaneiro, RJ, 21945-970, Brazil, e-mail: {eduardo,fonini}@lps.ufrj.br.

Marcos Craizer is with DMAT, PontifíciaUniversidade Católica do Rio de Janeiro, RuaMarquês de S. Vicente, 225, Rio de Janeiro, RJ,22453-900, Brazil, e-mail: [email protected].

Introduction

Images may be worth a thousandwords, but they generally occupymuch more space in a hard disk, orbandwidth in a transmission system,than their proverbial counterpart. So,in the broad field of signal processing,a very high-activity area is the researchfor efficient signal representations.Efficiency, in this context, generallymeans to have a representation fromwhich we can recover some approxi-mation of the original signal, but whichdoesn’t occupy a lot of space. Unfor-tunately, these are contradictory re-quirements; in order to have better pic-tures, we usually need more bits.

The signals which we want to storeor transmit are normally physicalthings like sounds or images, whichare really continuous functions of timeor space. Of course, in order to usedigital computers to work on them, wemust digitize those signals. This is nor-mally accomplished by sampling(measuring its instantaneous valuefrom time to time) and finely quantiz-ing the signal (assigning a discretevalue to the measurement) [1]. Thisprocedure will produce long series ofnumbers. For all purposes of this ar-ticle, from here on we will proceed asif these sequences were the originalsignals which need to be stored ortransmitted, and the ones we will even-tually want to recover. After all, we canconsider that from this digitized rep-resentation we can recover the true(physical) signal, as long as human

eyes or ears are concerned. This iswhat happens, for example, when weplay an audio CD. In our case, we willfocus mainly on image representa-tions, so the corresponding examplewould be the display of a picture in acomputer monitor. However, the dis-cussion in this paper, and especially thetheory developed here, apply equallywell to a more general class of signals.

Several state-of-the-art image andvideo compression methods make useof successive approximation quantiza-tion, in one form or another [2–6]. It’sfound in the literature under severalnames, as successive refinements, em-bedded coding, or progressive quanti-

Images may be worth a thousand words,but they generally occupy much more spacein a hard disk, or bandwidth in a transmis-sion system, than their proverbial counter-part. So, in the broad field of signal process-ing, a very high-activity area is the researchfor efficient signal representations.

zation [4, 7–9]. However, all these re-fer to the same basic idea, namely thatof achieving a representation for a sig-nal made up of several segments; fromthe first, we can recover a coarse ap-proximation of the original, and thenthe next segments give us finer andfiner detail. Thus, by carefully control-ling how many segments we store ortransmit, we may control either theamount of data involved or the qual-ity of the recovered approximation.

This is in fact a very simple ideaand is implemented, for example, inthe most common form of number rep-resentation, which is the decimal ex-pansion. For example, when we wantto refer to the value of π, we may use

22

the string “3.14”; if we need more pre-cision (and we can afford the cost), wemay then use “3.14159”. This secondstring contains the first at its beginning,and then some more data which doesnot mean anything by itself, but doesgive us more information about thevalue of π when used together with thefirst part.

Sadly, the decimal expansion isgood only for human consumption;besides, it’s not optimal for represent-ing signals. Its digital cousin, the bi-nary expansion, is better suited forcomputer use, but is equally non-opti-mal. Suffice it to show that the optimal(least error) 5-digit representation forπ is 3.1416; to reach the optimal6-digit representation, we must get ridof the final 6, substituting 59 for it. So,this representation does not make op-timal use of the allowed space.

Compression

Image compression is usually ob-tained by leaving out “unnecessary”detail. For example, suppose our im-

age is composed of 8-bit pixels, wherethe values from 0 to 255 represent thebrightness of the corresponding pixel.We could retain only the most signifi-cant 4 bits of each pixel, which wouldresult in an image with only 16 graylevels. We would have thus obtained2:1 compression, although at the costof a very degraded image. But what weare really looking for is an acceptablecompromise between compressionlevels and image quality. In general,acceptable means a quality loss notreadily apparent to a casual viewer. Inparticular, coarser quantization ofbrightness levels is usually unaccept-able, because our visual system is veryaware of the resulting difference [10].Moreover, this difference is usuallyperceived as somewhat aggressive tothe eye. For comparison, take a low-pass filtered image: although the dif-ference from the original is still quitevisible, the result has an agreeable,soothing effect.

Transforms

Invertible transforms are widelyused in image processing. Their pur-pose is to represent an image as aweighted sum of elementary images.Mathematically, this is obtained bymultiplying the number sequenceswhich represent the signal (taken as avector) by a matrix [10]. The elementsof the resulting vector—the trans-formed image—have now a totally dif-ferent meaning, not directly correlated

But what we are really looking for is anacceptable compromise between compres-sion levels and image quality. In general, ac-ceptable means a quality loss not readily ap-parent to a casual viewer. In particular,coarser quantization of brightness levels isusually unacceptable, because our visual sys-tem is very aware of the resulting difference[10]. Moreover, this difference is usuallyperceived as somewhat aggressive to the eye.For comparison, take a low-pass filtered im-age: although the difference to the originalis still quite visible, the result has an agree-able, soothing effect.

Invertible transformsare widely used in imageprocessing. Their pur-pose is to represent animage as a weighted sumof elementary images.

23

with the brightness value of any singlepixel. On the contrary, each coefficientof the transformed signal is a functionof several pixels of the original image.Conversely, any alteration in a trans-form coefficient will translate to amodification in most of the pixels ofthe recovered image, when the inversetransform is applied. This is wheremost of the compression is obtained inmodern methods: by carefully quantiz-ing the different coefficients of thetransformed image [11]. Depending onthe transform used, the loss of preci-sion in certain coefficients is less vis-ible than in others, when consideringthe effects of the resulting recoveredimages in the human visual system[12–13].

Of course some transforms are bet-ter suited than others for this purpose.The JPEG standard [14] for compres-sion of static images uses the DiscreteCosine Transform, and so do the sev-eral variants of the MPEG standard[15–17] for compression of movingimages. The modern JPEG2000 stan-dard [6], based on the EBCOT algo-rithm [4], uses the Wavelet Transform.MPEG-4 also allows (optionally) theuse of wavelets.

Image Quality, and Lack Thereof

The ultimate assessment of thequality of a compression method is thesubjective impression caused by therecovered images on suitably trainedhuman individuals. There’s hardly anyneed to say that this is all but impos-sible to determine a priori algorithm-ically, so any objective methods forquality assessment fall very short ofthis ideal case [13]. Nonetheless, someobjective methods must be used, atleast as a first-order approximation forthe quality of a method. Usually, whatis used is some function of the bright-ness differences of the pixels in the

original and the recovered images. Themost common objective measure ofthe quality of an image (in relation toan original) is the MSE—MeanSquared Error, defined [10] as

MSE ≡ 1N

∑(xi – xi)2

where xi is the value of an originalpixel, and xi is the corresponding ap-proximation. The square function isused to make the definition more eas-ily tractable in mathematical terms.

The careful reader will have no-ticed that this is just the Euclidean dis-tance from the original image to its

The elements of the resulting vector—thetransformed image—have now a totally dif-ferent meaning, not directly correlated withthe brightness value of any single pixel. Onthe contrary, each coefficient of the trans-formed signal is a function of several pixelsof the original image. Conversely, any alter-ation in a transform coefficient will translateto a modification in most of the pixels of therecovered image, when the inverse transformis applied. This is where most of the com-pression is obtained in modern methods: bycarefully quantizing the different coefficientsof the transformed image [11].

approximation: if the approximation isidentical to the original, the MSE iszero, and it grows as the approxima-tion departs from the original. This isa measure of distortion. For those whothink that more is better, there is aninverse measure derived from theMSE, which is the PSNR—Peak Sig-nal to Noise Ratio. For 256-gray levelimages, this is defined as

24

PSNR ≡ 10 log102552

MSE

[dB]

Given the simplistic definition ofdistortion used here, we should keepin mind that those measures have asomewhat restricted use. The compari-son of MSE or PSNR values is mean-ingful only when comparing similar al-gorithms applied to a common origi-nal image, and even so these resultsmust be taken with a grain of salt. Thatsaid, the MSE is by far the most usedform of comparing the image qualityobtained from competing algorithms.Computational cost is often also takeninto account.

Data Rates

The data rate is the other factor inthe compression equation. We can de-fine the data rate as the average num-ber of bits per pixel needed to repre-sent an image, and it has direct conse-quences on file sizes and bandwidthneeds. For any given algorithm, betterquality will imply larger data rates.The set of distortion versus rate mea-surements for an algorithm is referredto as its rate-distortion performance.

Information theory provides uswith a theoretical upper bound on theperformance of any coding algorithm,given the statistical behavior of thesource of the coefficients [18]. A cer-tain representation will be said to beoptimal when this upper bound isachieved.

Successive ApproximationQuantization

Although being part of many state-of-the-art image compression meth-ods, successive approximation quanti-zation is not generally optimal from arate-distortion perspective. In this sec-tion, we will first describe the EZW al-gorithm [2], a wavelet-based image

encoder that was the first one to effi-ciently use the concept of successiveapproximation quantization in imageprocessing. We will then present amore formal definition of successiveapproximation, and then proceed toshow why it’s not always optimal. Thatsaid, we will present the reasons whyit’s used in so many algorithms none-theless.

The EZW Algorithm

When the EZW—EmbeddedZerotree Wavelet—algorithm for im-age compression was introduced [19],it represented a breakthrough in imagecompression methods. It is capable ofcompressing an image with excellentrate x distortion performance; besides,the generated bit-stream is such thatthe encoding rate can be precisely con-trolled, with optimal performance forall rates. Several other algorithms havesince been introduced, all based on thesame principle [3–5]. The key to EZWefficiency lies in a clever combinationof wavelet transforms and successiveapproximation quantization.

The 3-stage wavelet transform ofa natural image [20], as can be seen inFigs.1 and 2, has the following fea-tures:

• Its coefficients can be grouped inseveral frequency bands, obtainedfrom an octave band decomposi-tion. Band 0 is a low-pass band;bands 1, 4, and 7 have mainly ver-tical high frequency detail; bands 2,5, and 8 have mainly horizontal de-tail, and bands 3, 6, and 9, diagonaldetail. In each group, frequency in-creases as the band indices increase.

• It has a few high-energy coefficientsand a large number of low-energycoefficients.

• The low-energy coefficients tend toappear in clusters.

25

Fig

ure

1. 3

-sta

ge w

avel

et im

age

tran

sfor

m.

Fig

ure

2. W

avel

et s

ub-b

ands

.

5 6

7

8 9

0 1

2 34

26

• The coefficients in lower frequencybands tend to have higher energiesthan those in higher frequencybands.

• The bands of same orientation (1, 4,and 7; 3, 5, and 8; 3, 6, and 9) tendto have their low-energy coeffi-cients in the same correspondingspatial locations.

The EZW algorithm works byquantizing all coefficients using suc-cessive approximation quantization. Itdoes so by first computing the magni-tude of the highest energy coefficient,and then setting a threshold T equal tohalf this value. It then transmits thesign of all the coefficients with mag-nitude higher than T. It is important to

note that it uses a very efficient methodto encode the location of these coeffi-cients. It considers coefficients withmagnitude lower than T as non-signifi-cant; if a coefficient in a band is non-significant, and all spatially corre-sponding coefficients in the higherbands of the same group are also non-significant, this whole set is then en-coded as a zerotree. Otherwise, thatnon-significant coefficient is simplyencoded as a zero. Note that with azerotree a single symbol is used to rep-resent a large number of coefficients;therefore, the zerotree concept is veryefficient. Since the bands are self-simi-lar, it is likely that a large number ofzerotrees will occur, and thus just asmall number of bits will be spent toencode all non-significant coefficients.

After the sign of the non-signifi-cant coefficients is transmitted, thethreshold is halved. Then, the coeffi-cients that are already significant are

refined (by adding or subtracting T

2

from its reconstruction value), and thesigns of the coefficients that just be-came significant are then transmitted.The whole process is repeated until atarget distortion or bit-rate is achieved.All symbols are encoded using anarithmetic encoder [21]; details can befound in [2]. The rate x distortion char-acteristics of this encoder are verygood; several variations of it have beenproposed [3–5], based on the samebasic principles. Such algorithms rep-resented a breakthrough in image com-pression, and today they represent thestate-of-the art in the field [4].

Figure 3. Coding a Gaussian random number.

- 3 - 2 0 2 31- 1

0 1

- 3 - 2 0 2 30.98- 0.98

10 1100 01

- 3 - 2 - 1 0 1 2 3

000 010 011 100 101 110001 111

(a) 1-bit coding.

(b) 2-bit coding.

(c) 3-bit coding.

27

Successive Approximation Quantizationfor Image Compression

Successive Approximation Quantizationfor Image Compression

The successive approximationquantization lends an important char-acteristic to those algorithms: they areall embedded, that is, the bit-streamthey generate for one rate contains thebit-streams for all achievable lowerrates. This is detailed in the next section.

Embedded Coders

Shapiro [2] characterizes an em-bedded code as possessing two defin-ing properties:

• When coding the same data withtwo different rates (and, implicitly,with two different distortions), thetwo resulting representations mustmatch exactly for the extent of thesmaller one. In other words, thesmaller one must be reproduced ex-actly in the beginning of the largerone. This way, the coded represen-tation for a given data rate containsall the representations for smallerdata rates. In short, the representa-tions get more precise as we addmore symbols to them.

• It should obtain a good representa-tion (low distortion) for a given datarate.

The former conveys the main ideaof embedding, while the latter guardsagainst excessive generalization1. Forexample, binary expansions can beconsidered embedded, if we take someprecautions on reconstructing values,

such as adding 1

2n to a truncated num-

ber. On the other hand, representationsgenerated by vector quantization [22]are usually not embedded: a high-reso-lution data stream cannot always be trun-cated to provide a low-resolution one.

Not Always Optimal

Equitz and Cover (who use theterms Successive Refinements, in [7])have shown that “optimal descriptionsare not always refinements of one an-other”; they establish the conditionsunder which optimal successive refine-ments can occur. A simple examplewhere this does not happen is alsogiven, which goes as follows.

Take a sequence of numbers froma standard normal distributionX ~ N(0, 1). The optimal 1-bit descrip-tion for these numbers is obviouslythat which uses this bit to specify thesign of the number (see Fig. 3a). If wenow decide to use 2 bits, we can have4 quantizer bins, as in Fig. 3b. Noticethat the new bins are subsets of the oldbins. The placement of the bins’ fron-tiers are defined by the Lloyd-Maxconditions [22], in order to minimizethe MSE of the reconstructed values.So far, we still have an embeddedcode; all numbers which were codedas 0 are now coded with a 0 as the firstsymbol. But if we proceed to 3-bitcodes, the new bins (Fig. 3c) are notsubsets of the previous bins anymore;so, we do not have embedded codingif we decide to keep the codes optimal.

It should be noted that optimalitywith respect to rate x distortion de-pends on how we decide to measuredistortion. Although the MSE is the

1 Just appending a whole high-rate description to alow-rate one, for example, should not qualify as em-bedding.

28

most common form, nothing preventsus from choosing another metric as, forexample, the average absolute value ofthe error.

In short, given a random sourcewith known properties, and a distortionfunction, there is a lower bound on theaverage rate necessary to representmessages (symbol sequences) fromthat source with a given distortion [18,23]. Optimal representations would bethose which reach this lower bound, andthey usually would not be embedded.

The Case for SuccessiveApproximations

State-of-the-art algorithms likeEZW, described above, can presentlyrepresent most images with less than1 bit per pixel (bpp), with no discern-ible loss of quality [2– 5]. Most ofthose algorithms do use embeddedcoding of transform coefficients.

This is in apparent contradictionwith the discussion shown in the previ-ous section, Not Always Optimal. How-ever, it turns out that no practical im-age coder is optimal, and embedded cod-ers can be best, in certain cases, amongstthe practical, non-optimal coders.

Low bit-rate quantization—Mallat &Falzon [24] have shown a mathemati-cal analysis of the improvements ob-tained by embedded coding of trans-form coefficients when working withlow bit rates, as follows.

A quantizer is considered to havehigh-resolution when the probabilitydensity function p(x) of the quantizedrandom variable X is approximately

constant inside each quantizer bin. Therate x distortion performance of high-resolution quantizers for Gaussian pro-cesses is well known; the MSE willvary as C(2–2R), with C depending onthe bit allocation. In this case, the uni-form quantizer has been shown to beoptimal [22].

Current algorithms, however, oper-ate on the R < 1 bpp region; therefore,the high-resolution hypothesis does notapply. In this case, the rate x distortionperformance behaves differently.

The wavelet or block cosine trans-form of most images has histogramsresembling that of Fig. 4, which de-picts the histogram of the DCT of theLena image. The clustering of coeffi-cients near the zero region, togetherwith the coarse quantization, results ina large number of coefficients beingquantized as zero. Thus, the zero sym-bol deserves special treatment, andmost algorithms treat it differentlyfrom the other values. For example, theEZW algorithm, as seen before, useszerotrees in order to efficiently encodethe zero elements. Mallat & Falzon, in[24], show that when there is a knowntransform behavior as to the usual lo-cation of large, average and small co-efficients—as is the case with wavelettransforms in which small or zero-val-ued coefficients tend to be clustered—embedded coders can present an ad-vantage over classical coders. The netresult is to have the MSE varying ap-proximately as R–1. This explains thegood performance of embedded en-coders such as EZW and the like[2–5].

Figure 4. Histogram for DCT of the Lena image.

0

29

Applications of embedded codings—Besides presenting improved effi-ciency, embedded coders are bettersuited for several applications wherethe possibility of dynamically choos-ing rate or distortion is a plus.

Consider, for example, the multi-casting of a movie, i.e., the transmis-sion, through a network, from onesource to several destinations. The net-work may form a tree, with the sourceas root and the destinations as leaves.Each branch may have a differentbandwidth. Now, if the source limitsthe transmission to the rate allowed bythe narrower branch, the otherbranches will not get all the qualitythey could. On the other hand, if thesource transmits at full rate, somebranches may not be able to cope withthe large data rate. However, if thetransmission uses an embedded code,special hardware at each branchingpoint may dynamically select from thereceived bitstream just the amount ofdata which can be sent toward each ofits outgoing branches. This way, alldestinations get the largest possiblerate—and thus image quality—al-lowed by the channel extending fromthem to the source.

As another example, imagine ahigh-resolution image database set upfor remote browsing and downloading.We want to be able to make a fastbrowsing to select the images beforeactually downloading them. For thiswe must have a low-quality version ofeach image available, but the best di-mensioning of these previews dependson the bandwidth of our connection to

the server as well as the time availableto complete the job, and may varyfrom user to user.

If the image files are produced witha non-embedded coding, the onlychoice is to have separate files for thepreviews, which is inefficient; besides,there will be a fixed set of previewresolutions. Using embedded coding,the viewer can select2 the data rate whichbest suits her as a compromise betweenimage quality and download time.

A Real-World Example

Another nice example of a real-world method which uses successiveapproximations is Taubman’s EBCOT[4]—Embedded Block Coding withOptimized Truncation, which is thebasis for the JPEG2000 standard [6].

2 We suppose the server can generate the previewsat the selected data rate by selectively transmittingthe proper subsets of the embedded coding.

Consider, for example, the multi-casting ofa movie, i.e., the transmission, through a net-work, from one source to several destinations.The network may form a tree, with the sourceas root and the destinations as leaves. Eachbranch may have a different bandwidth. Now,if the source limits the transmission to the rateallowed by the narrower branch, the otherbranches will not get all the quality theycould. On the other hand, if the source trans-mits at full rate, some branches may not beable to cope with the large data rate. However,if the transmission uses an embedded code,special hardware at each branching point maydynamically select from the receivedbitstream just the amount of data which canbe sent toward each of its outgoing branches.

30

The successive approximations inEBCOT appear at the bit-plane codingof the quantized values of the wavelettransform coefficients. It uses uniform,scalar quantization, except for thenear-zero region. In bit-plane coding,the most significant bits of a set of val-ues are sent first, then the next mostsignificant, and so on, until all bit-planes have been sent. Of course, likeother modern algorithms, the bit-planes are also further entropy codedto take advantage of the redundancypresent in the transform coefficients.

Successive Approximationswith ααααα-Expansions

We will now develop a theory ofsuccessive approximations based on anembedded coding referred to as α-ex-pansions. They can be seen as a gen-eralization of the successive approxi-mation quantization used in the state-of-the-art image transform coding al-gorithms. With it, we develop a gen-eral framework for analyzing embed-ded representations. We will also showthat α-expansions are capable of improv-ing the coding of wavelet coefficientswhen compared to the usual successiveapproximation scalar quantization.

Our first goal is to develop a rep-resentation for generic sequences of

real numbers, through the use of α-ex-pansions. Later, we will apply it to thecoding of wavelet coefficients.

Definition of α-Expansions

Let x be a vector in RRN, and let V ={vk ∈ RRN , k ∈ {1, 2, … M}}, where xis the vector to be coded and V is theDICTIONARY or CODEBOOK.

The SUCCESSIVE APPROXIMATIONS

(SA) representation of the vector x is

x = α ivkii=0

∞

∑ (1)

where 0 < α < 1 and ki ∈ {1, 2, … M}.This is what we will call an α-expan-sion over the codebook V.

This representation can be codedas the sequence (k0, k1, …) of integers(which we will call a k-sequence).Geometrically, the relation expressedby Eq. 1 could be shown as in Fig. 5.

As the figure suggests, given onlythe first L elements (k0, k1, … kL–1 ) ofthe k-sequence, we can find an ap-proximation xL to the vector x:

xL = α ivkii=0

L−1

∑ (2)

To measure the error incurred by thisapproximation we define the residue

rL = x – xL (3)

For this truncated representation tobe useful, we want |rL | to have asmoothly decreasing upper bound, sothat rL tends to zero as L grows. To becomparable to other quantized signalrepresentations, we need |rL | to bebounded (at least) by Cβ L, where C issome positive real constant and0 < β < 1. Observe that this only meansthat as we proceed with the coding, themaximum error in the reconstructedvector will decrease exponentially; |rL |itself may vary otherwise, as long asit stays below this upper bound.

The successive approximationscoding of a vector satisfies the first re-Figure 5. First Approximations.

αvk1

vk0

α2vk 2

x

.

.

31

quirement for an embedded coder:higher rates are obtained by addingelements to the sequence (k0, k1, …).So, it contains all achievable lower-rate representations. The second re-quirement may also be met, dependingon the algorithm chosen to produce thesequence.

The Search for Convergence

In [25], α-expansions were used toquantize wavelet coefficients. Themethod used was to group the coeffi-cients into N-dimensional vectors andthen find k-sequences to representthem. The method was restricted tocodebooks of unitary vectors.

A greedy algorithm was used tofind the k-sequence corresponding toa vector x, as follows:

1. Take r = x.

2. Find vk, the code-vector nearest tor, i.e., the one which minimizes|r – vj |. As all code-vectors are ofunit length, this is equivalent tofinding the code-vector which mini-mizes ang (r, vj ), the angle betweenr and vj .

3. Take r – vk as the next value of r.

4. Multiply all code-vectors by α.

5. If |r | is small enough, stop; else, goback to step 2.

The algorithm is said to convergewhen |r | decreases exponentially, i.e.,it is bounded by Cα n after the n-thstep, where C is some real constant. Inthis case, the α-expansion is defined bythe sequence of numbers k found ateach iteration of step 2. The fact thatwe minimize the approximation errorat each step justifies the classificationof the algorithm as a greedy one.

Now, take an x formed by a given,random k-sequence. Observe that thereis no guarantee that the algorithm willfind the same k-sequence. In fact, it can

even fail to find any k-sequence at all,i.e., it may diverge. Of course, we needto find a codebook V and a value αsuch that the algorithm converges forany x ∈ RRN [26].

The first experiments [25] con-ducted to study the convergence of thealgorithm suggested a connection be-tween α and a certain geometric prop-erty of the codebook, an angle whichwe will call θmax. Simply put, θmax is themaximum angle between any vector xand any codebook vector vk. So, for agiven number of vectors in a code-book, θmax can be minimized by hav-ing the codebook vectors as uniformlyspaced as possible.

Later [27], a more precise evalua-tion of the exact relationship betweenα and θmax needed for convergence wasobtained. The main results are:

1. Convergence can be guaranteed forany codebook, provided that(i) θmax < π /2 and (ii) |x | ≤ β, whereβ is defined in Eq. 4;

2. Under the above conditions, thereexists a special value α min such thatconvergence can be obtained withany α ≥ α min.

3. α min is given by the following

α ≥ 1

2 cos θ max

β = 2cosθmax

, if θmax ≤ π 4

α ≥ sinθmax

β = 1

cos θ max

, if θmax ≥ π 4

(4)

We will now develop a theory of succes-sive approximations based on an embeddedcoding referred to as α-expansions. They canbe seen as a generalization of the successiveapproximation quantization used in the state-of-the-art image transform coding algorithms.

32

The relationship between α min and θmax

is shown graphically in Fig. 6.Thus, “good” codebooks have

small θmax, so they converge with asmall α, which in turn gives us fasterdecreasing errors in the approxima-tions. Codebooks with more vectorswill generally have smaller θmax, butthe corresponding k-sequences willhave more entropy, so we will need alarger data rate to code them. Note thatthe conditions given here are suffi-

the three vectors are of unit length andevenly spaced on the circle. Let α =0.5. The possible values of x1 are thecodebook elements themselves:

x1 =v1

v2

v3

On the next iteration, we have ninepossible values for x2:

x2 =v1

v2

v3

+αv1

αv2

αv3

which can be seen in Fig. 8a. As Lgrows, we can see that the set of pos-sible points xL is as depicted in Fig. 8b.

Now, this last figure looks a lot likea fully fledged fractal3. If we can provethat it is indeed a fractal (when L →∞), and that this happens for anycodebook and scale factor α, we canthen use fractal theory to verify underwhat conditions this fractal will con-tain the unit ball, so that all values|x | ≤ 1 can be represented as an α-ex-pansion over that codebook.

Iterated Function Systems (IFS)

Let us define x, k, V, and α as inthe section Definition of α-Expansions.Now define the functions

fk(x) = α x + vk (5)and

F(A) = f k (A)

1

M

U (6)

where A is any subset of RRN and fk(A) ={fk(a), a ∈ A}. Note that F is definedon the set4 of the subsets of RRN, andthat fk is extended to apply to elementsof this set also.

Figure 6. αmin

x θmax

.

3 Indeed, this set approximates the Sierpinski tri-angle [28], of fractal theory fame.

4 This set may become a space, if we define a propermetric for it.

cient, but not necessary, providing justan upper bound for α min; the real α min

for a given codebook may be smaller.

Generated Spaces

We will now proceed toward a stillmore general theory of α-expansions.Instead of looking directly for conver-gence conditions, let us try to look atthe problem from another point ofview: we may consider the set of allvectors that can be generated by theright side of Eq. 1; then we can inves-tigate the conditions for this set to con-tain the unit ball on RRN.

Let us consider a 2-D example. LetV be the set defined in Fig. 7, where

0 0.5 1 1.50

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

33

1 0.5 0 0.5 1 1.5 2

1.5

1

0.5

0

0.5

1

1.5

-0.5 0 0.5 1 1.5

-1

-0.5

0

0.5

1

From fractal theory [28], sinceeach fk is a contraction mapping (spe-cifically for 0 < |α | < 1), it follows thatthe set {fk} forms an iterated functionsystem. One of its properties is thatthere is a set Λ, which is the uniquefixed point of F (i.e., F(Λ) = Λ) and iscalled the attractor of the IFS.

Theorem: The set of all points thatcan be represented by the right sideof Eq. 1, is the attractor of the cor-responding IFS.

It is a fact that IFS attractors are gen-erally fractals; they can also degener-ate into “normal” (i.e., non-fractal)sets. For example, in Fig. 8b, Λ is thewhole punctured triangle; each coloredsmaller copy is one of the fk(Λ). Wemust find out the cases in which theattractor will contain the unit ball.

Determining Λ

Geometric background—In the fol-lowing discussion, we will use somereadily available [29] geometric defi-nitions and results. These are summa-rized here.

Combination: Let a1, … ak ∈ RR, andp1, … pk ∈ RRN. The linear combi-nation x = ∑k

i = 1 ai pi is said to be:

Figure 7. A codebook V.

Figure 8. Possible values for x2 and xL.

(a) 2 iterations.

(b) Several iterations.

34

• a positive combination, if ai ≥ 0,∀i ∈ (1, … k);

• an affine combination, if ∑ki = 1 ai = 1;

• a convex combination, if both pre-vious conditions hold.

Convex sets: A set S is said to be con-vex if for every x, y ∈ S, every con-vex combination of x and y is alsoin S.

Convex hull: The convex hull of a setS is the smallest convex set contain-ing S.

Polytope: The convex hull of any fi-nite set S is a polytope; conversely,every polytope is the convex hull ofa subset of its points, called its ver-tices. Polygons and (3-D) polyhedraare examples of polytopes. Note thatinfinite sets may have a convex hullwhich is not a polytope, e.g. a sphere.

Remark: The more generic defini-tion of polyhedron includes un-bounded sets defined as convex in-tersections of half-spaces, andpolytopes as bounded polyhedra;for our purposes, however, we areconsidering only bounded sets.

Vertex: A vertex of a polytope can alsobe defined as those points whichcannot be expressed as a convexcombination of any other points ofthe polytope. A polytope is also theset of all convex combinations of itsvertices. Observe that not all pointsin S (the finite set over which webuild the convex hull) will necessar-ily be vertices; some may be in theinterior of the polytope.

Homothety: An homothety of centerC and scaling factor α is the trans-formation

Figure 9. 5-point codebook and Pα.

V2

V3

V4

1V

O

V5

35

h(x) = C + α (x – C)= α x + (1 – α)C

We can also have h(S) = {h(s), s ∈ S},i.e., extend h to apply to a set.

Homothety of a convex set: Let S bea convex set, α ∈ [0, 1], P ∈ S, andT = α S + (1 – α)P. Then T ⊂ S.Conversely, if α > 1, then T ⊃ S.

The codebook’s polytopes—Let P bethe polytope defined by the convexhull of the vectors {vk}. Likewise, letPα be the convex hull of the set W =

{wk = 1

1−α vk}. Observe that Pα and W

can be obtained from P and V, respec-tively, by the same homothety with cen-

ter at the origin and scaling factor 1

1−α .

By construction, any x ∈ Λ is apositive combination of vectors fromthe codebook V, with coefficients

whose sum is 1

1−α , thus a convex com-

bination of the vectors wk. Therefore,

Λ ⊆ Pα (7)

Observe that we can rewrite fk(x) as

fk(x) = α x − 11 − α

vk

+

11 − α

vk

= α (x – wk) + wk

Therefore, each fk is an homothety ofcenter wk and scaling factor α, whichimplies5 that fk(Pα) ⊂ Pα. This way,F(Pα) is the union of several scaled-down copies of Pα , each of them con-tained in Pα .

That said, two situations may arise:either

F(Pα) = Pα ⇒ Λ = Pα

orF(Pα) ⊂ Pα ⇒ Λ ⊂ Pα

where ⊂ is used here meaning “strictlycontained”. In the first case, all pointsinside Pα (and only these points) canbe represented by the α-expansion; in

the other, only a subset of Pα can beso represented. We must know theconditions under which we willhave Λ = Pα .

Geometrically, the conditionF(Pα) = Pα means that the various fk(Pα)must cover Pα. To better appreciate thiscondition let’s examine a situationwhere it fails, as in Fig. 9. It representsa codebook with N = 2 and M = 5. Ob-serve that Pα has only four vertices, forthe 5th code-vector is inside the con-vex hull of {vk}. In this case, F(Pα)(the union of the copies) does notcover Pα .

As another example, consider thecodebook formed by the four verticesof the inner polygon in Fig. 10. Herewe use α = 0.55, the smallest value forwhich the covering is complete(F(Pα) = Pα).

Determining the α-ExpansionCoefficients

A rule to determine (one) α-expan-sion for x is:

1. Determine k such that x ∈ fk(Pα)(there may exist more than one

Figure 10. A covered Pα.

5 Since 0 < α < 1, wk ∈ Pα and Pα is convex.

36

such k); Λ = Pα guarantees the ex-istence of at least one such k.

2. Determine y = (x – vk) /α ; Λ = Pα

guarantees that y ∈ Pα .

3. Take this y as the next x.

The repeated application of these stepswill generate the desired sequence(k0, k1, …).

Each sequence element localizes xas being into one of the copies of theouter region; since this copy has thesame form as the outer region, it canalso be so divided, thus permitting theprocess to continue indefinitely. Theregions so obtained are smaller andsmaller, thus reducing the possible lo-cations of x. In short, the k-sequencedefines a region in space where x iscontained. Also, this same sequencecan be used to find the approximationxL (see Eq. 2).

Whenever a point pertains to morethan one fk(Pα), any of the correspond-ing k can be selected, leading to dif-ferent but equivalent representations(k-sequences) for the same point.

Binary Expansions as a Special Case

The binary representation of anumber in the interval [0, 1] is a se-quence (b1, b2, …) whose value is

x = bi

12

i

i=1

∞

∑ , bi ∈ {0, 1}

which can also be written as

x = bi

12

i

i=0

∞

∑ , bi ∈ 0,12

In this form, it can immediately be rec-ognized as an α-expansion.

Consider the codebook V = {v0 =0, v1 = 1/2}, and α = 1/2. This leadsto Pα being the closed interval [0, 1],as depicted in Fig. 11. Since Pα is to-tally covered by its two halves, wehave Λ = Pα. Thus, any number in thisinterval can be represented as an α-ex-pansion of this codebook. Take, say,x = 0.1011 … :

x = v1 + 12

(v0 + 12

(v1 + 12

(v1 + …)))

= 12

0

v1 + 12

2

v1 + 12

3

v1 + …

Figure 11. The [0, 1] interval.

A very important conclusion maybe drawn from the above discussion:

For any codebook, convergencefor a given α depends only on itsgeometrical form, not on its size,orientation or position with rela-tion to the origin. By form wemean the relative position of allelements, including those whichmay be inside the convex hull.

The two half-segments intersect at x =0.5; that means that this point can havemore than one representation. Indeed,0.1000 … = 0.0111 … . Note also thatthe point x = 1, contained in Pα , canbe represented as 0.1111 … .

Similarly to the scalar case, anyvector in the unit hyper-cube [0, 1]N

can be coded as an α-expansion over

the codebook V = {[0, 1

2]N} (i.e., a

0 0.5 1

37

2N-element set of N-dimensional vectorswhose coordinates are either 0 or 1/2).The 2-D case is illustrated in Fig. 12.

Any hyper-cube can be covered by2N copies of itself, where the side ofeach copy is 1/2 of the original. Al-though there are non-empty intersec-tions, the total hyper-volume of theseintersections is null. Note that in thiscase α reaches its theoretical lowerbound (see the next section, Bounds onα for Λ = Pα).

Bounds on α for Λ = Pα

For any codebook, there is a criti-cal value α c such that Λα = Pα ⇔ α ≥α c. In addition, we can show boundsfor α c as functions of N and M—thecodebook’s dimension and cardinality.These bounds are stated below.

Theorem: Λ = Pα ⇒ α ≥ M− 1

N

Thus, given only the cardinality M

and dimension N, M− 1

N is the largestpossible such bound for a genericcodebook, since this value of α holdsfor the usual binary (or decimal, andso forth) expansion (see the previoussection, Binary Expansions as a Spe-cial Case). In fact, for the binary ex-

pansion of vectors in RR N, we haveM = 2N vectors, yielding α ≥ 1/2, andconvergence is obtained with α = 1/2.

Theorem: α ≥ N

N + 1 ⇒ Λ = Pα

This bound is effectively reached forall simplices (the N-dimensionalpolytopes with N + 1 vertices). Thus,this is the smallest possible such boundon α c. However, for generic code-books, we may possibly have Λ = Pα

for smaller values of α.The proofs of the above results are

somewhat lengthy and are not in thescope of the present article. A moredetailed discussion of this subject, in-cluding the pertinent demonstrations,is available on-line [30].

The Nearest Point Algorithm

The main step in the algorithm de-scribed in the section Determining theα-Expansion Coefficients involvesknowing whether a point is inside apolytope or not, so that we can findwhich values of k satisfy x ∈ fk(Pα). Donot be deluded by the simplicity of the2-D case; in general, this is a ratherelusive problem, and there seems to beno feasible algorithm to solve it (i.e.,one that runs in polynomial time). So,we will resort once again to the greedyalgorithm of the section The Search forConvergence, with some minor modi-fications to make it more suitable to acomputer implementation, as well asto allow an easier analysis.

Figure 12. 2-D binary expansion.

Any hyper-cube can be covered by 2N

copies of itself, where the side of each copyis one-half of the original. Although thereare non-empty intersections, the total hyper-volume of these intersections is null.

10

1

38

The revised algorithm is as follows:

1. Take r = x.

2. Find vk, the code-vector nearest tor, i.e., the one which minimizes|r – vj |. As all code-vectors are ofunit length, this is equivalent tofinding the code-vector which mini-mizes ang(r, vj), the angle betweenr and vj.

3. Take (r – vk)/α as the next value of r.

4. If |r | is small enough, stop; else, goback to step 2.

Observe that we modified ruleNo. 3 and took out rule No. 4 of thealgorithm in the section The Search forConvergence. This amounts to scalingup the residue at each step instead ofworking with successively smaller ver-sions of the codebook. Here, we saythat the algorithm converges if |r | re-mains bounded.

The price to pay for the use of thissimplified algorithm is that the αneeded for convergence will be at least

as large as it would be otherwise, butgenerally larger.

In the case at hand, the next ele-ment in the coding sequence is chosenby searching for the code-vector near-est the residue of the previous iteration.This will be the “right” decision if andonly if the corresponding fk(Pα) con-tains the point in question. If we wantthis to be always true, we must havefk(Pα) to contain all points which havevk as its nearest code-vector. We nowneed the concept of the Voronoi cell.

The Voronoi Cell

Given a set S = {si ∈ RRN}, we de-fine the Voronoi cell of point si as be-ing the subset Vi = {x ∈ RRN} of pointswhich are at least as close to si as toany other sj [22, 31]. In other words,

x ∈ Vi ⇔ d(x, si) ≤ d(x, sj), ∀ j

As an example, Fig. 13a depictsthe Voronoi cells of a 2-D set of points.For those points which are inside theconvex hull of S, the cells are convexpolytopes. For those which are on thesurface of the convex hull, the cells arethe union of an infinite cone or cylin-der with a convex polytope, alwaysforming an infinite convex region.When referring to Voronoi cells relatedto codebooks, we mean the intersec-tion of the “real” Voronoi cells of theset {wk = vk/(1– α )} with its convexhull, as depicted in Fig. 13b.

Convergence of theNearest Point Algorithm

We can now state the conditionsunder which we will have convergence

Figure 13. Voronoi cells:(a) Points and cells.

39

of the greedy algorithm, i.e., we musthave both

F(Pα) = Pα (8)and

fk(Pα) ⊇ {Vk I Pα} (9)

where Vk is the Voronoi cell of wk. Infact, the second condition implies thefirst, since

Vk PαI{ }

kU = Pα

This way, selecting the wk nearest

to x implies that x ∈ {Vk PαI }, thus

x ∈ fk(Pα); this is sufficient for the al-gorithm to converge.

Observe that the minimum αneeded to obtain Eq. 9 is at least aslarge as the critical value α c defined inthe section Bounds on α for Λ = Pα,needed for convergence of the basicalgorithm. So, in order to use thegreedy algorithm and have the mini-mum possible α, we must look forcodebooks where Eq. 8 implies therelationship 9.

Performance Analysis

Since α-expansions are a generali-zation of the uniform scalar quantizer,we wonder whether we can find acodebook with a better rate x distortionperformance. After all, it would resultin a moot theory indeed if that were notthe case.

In the remainder of this section, wewill deal with this problem. We willshow that, for high rates, the best α-expansion is the one equivalent to uni-form scalar quantization (see the sec-tion Binary Expansions as a Special

Case). However, in low bit-rate imagetransform coding, we have a majorityof small magnitude coefficients; in thiscase, α-expansions using certain mul-tidimensional codebooks can be a bet-ter choice, provided that an efficientcoding method for the zero values isdevised (see the sections Low bit-ratequantization and The EZW Algorithm).

The procedure used to comparecodebooks’ performances was to cal-culate the average MSE for the α-ex-pansions of large numbers of datapoints. This was done implementingthe greedy algorithm of the section TheNearest Point Algorithm. The properα and β values (see the section TheSearch for Convergence) for eachcodebook were found by trial and er-ror, except for some simple codebookswhere those parameters could bereadily determined theoretically. Basi-cally, we run the algorithm describedin the section The Nearest Point Algo-rithm for a large number of data points,while controlling the error behavior; if

Figure 13. Voronoi cells:(b) Convex hull.

40

Successive Approximation QuantizationSuccessive Approximation Quantization

at any moment the error magnitudegrows larger than β, either one or bothparameters are smaller than the neededcritical values. When proper values forα and β are found, we proceed to per-formance measurements.

The results depend strongly on thespatial distribution of the data setsused. The data sets result from the clus-tering of coefficients into vectors of theappropriate dimension, i.e., that of thecodebook. Two kinds of data sets wereused: uniformly distributed points in-side the N-dimensional sphere of ra-dius 1, and vectors generated by theclustering of actual wavelet transformsof natural images.

Rate and Distortion Measurements

Remember that each coded vectorrepresents N coefficients. In order toproperly compare codebooks with pos-sibly different dimensions, we mustdetermine the per-coefficient distortionand rate.

Clearly, both distortion and rate aredependent on the number of iterations,n, through the algorithm. At each step,average total distortion decreases

(since |rn| is bounded by Cα n) and thetotal rate increases simply as RT =n. log2(M) bits per symbol, where M isthe cardinality of the codebook.

Thus, we have that the per-coeffi-cient rate, in bits, is

R = n.log2 ( M )

N = n. log2( M

1

N ).

Naturally, the actual total distor-tion depends on the spatial distributionof the data set; however, the asymp-totic behavior of |rn| depends only onthe form of the codebook: for large n,we will have that | r–n| = C0α n, where C0

is some positive constant. Thus, for thetotal distortion, we will have DT =C0α 2n; the per-coefficient average dis-

tortion will be D = C0

Nα 2n. Average per-

coefficient distortion as a function ofrate will thus be

D(R) = C0

Nα

1

log2 M1

N

2 R

(10)

Eduardo A. B. da Silva was born in Rio de Janeiro, Brazil, in 1963. He received the Engineeringdegree in electronics from Instituto Militar de Engenharia (IME), Brazil, in 1984, the M.Sc. degree in elec-tronics from Universidade Federal do Rio de Janeiro (COPPE/UFRJ) in 1990, and the Ph.D. degree inelectronics from the University of Essex, England, in 1995. In 1987 and 1988 he was with the Departmentof Electrical Engineering at Instituto Militar de Engenharia, Rio de Janeiro, Brazil. Since 1989 he has beenwith the Department of Electronics Engineering (the undergraduate department), UFRJ. He has also beenwith the Department of Electrical Engineering (the graduate studies department), COPPE/UFRJ, since 1996.He has been head of the Department of Electrical Engineering, COPPE/UFRJ, Brazil, in 2002. He won theBritish Telecom Postgraduate Publication Prize in 1995, for his paper on aliasing cancellation in subbandcoding. He is also co-author of the book Digital Signal Processing—System Analysis and Design, pub-lished by Cambridge University Press. He is presently serving as associate editor of the IEEE Transac-tions on Circuits and Systems—I. His research interests lie in the fields of digital signal and image process-ing, especially signal coding, wavelet transforms, mathematical morphology and applications to telecom-munications.

41

for Image Compressionfor Image Compression

So, to minimize the distortion forlarge rates, we must have the minimum

possible α

1

log2 M

1

N

. Since we know that

α ≥ M− 1

N (see the section Bounds on

α for Λ = Pα), we may write M1

N =k

α ≥ 1

α , with k ≥ 1. Analyzing the func-

tion α1

log2k

α

= 2log2 α

log2 k−log2 α , given the re-striction k ≥ 1, we find its minimum atk = 1. In this case, from Eq. 10, D(R) =C0

N 2–2R. For the codebooks correspond-

ing to the binary expansion, M = 2N andα = 1/2, hence k = 1. Thus, we con-clude that for large rates, thesecodebooks are optimal for α-expan-

sions. Since M1

N = 2, the rate x distor-tion performances of these codebooksdo not change with their dimension, soin this case scalar and vector quanti-zation are equivalent.

Experiment Setup

For the experiments described be-low, we determined the resulting av-erage distortion D (MSE) in decibels,that is, 10 log10(D), and the accumu-lated data rate R in bits, up to 20 itera-tions of the α-expansion algorithm.Details about the codebooks anddatasets are given below.

Codebook generation—As we haveseen, convergence will be faster forsmaller α. Also, for given M and N,codebooks whose vectors are more

uniformly spread in RRN tend to havesmaller α c. Note that we have not de-fined “spread” here; we are using theterm in a somewhat intuitive sense.The problem is reminiscent of spherepackings, several variations of whichare tackled by Conway and Sloane in[31]; some related results can also befound in [32, 33].

For these experiments, we haveused a few classes of codebooks; wesometimes refer to them by the nameof the polytopes whose vertices we areusing as codevectors:

• The binary expansion-equivalentcodebook, on several different di-mensions: square, cube, tesseract,and so on (including the 1-dimen-sional “square”, equivalent to uni-form scalar quantization); we referto them as the “binary codebooks”.

• Some 2-D regular polygons;

Décio A. Fonini Jr. received the M.S. degreein electrical engineering from the Instituto Militarde Engenharia, Rio de Janeiro, 1993, and is pres-ently a post-graduate student at the UniversidadeFederal do Rio de Janeiro. He has developed simu-lation software for the Brazilian Army and is nowwith Upknowledge Oy, a company headquarteredin Finland, doing worldwide training and consult-ing related to Internet and cellular technologies.

Marcos Craizer was born in Rio de Janeiro,Brazil, in 1962. He received the mathematics de-gree from Universidade Federal do Rio de Janeiro(UFRJ), Brazil, in 1983, and the Ph.D. degree inmathematics from Instituto de Matemática Pura eAplicada (IMPA), Brazil, in 1989. He is a profes-sor in the mathematics department of the PontifíciaUniversidade Católica do Rio de Janeiro (PUC-Rio)since 1988. His research interest lies in the field ofimage processing, especially image compression,mathematical morphology and applications of par-tial differential equations to images.

42

• Shells of some best known spherepackings [31]. For example, D4 isthe 4-D codebook with the 24 vec-tors of the first shell of the bestknown sphere packing in 4-D.

Most modern methods use uniformscalar quantizers, except for the regionnear the origin, where a double-sizedquantizer bin is used. The reasoningbehind this behavior is that most of the

points to be coded have small magni-tude; the oversized bin near the originhelps to net still more points to becoded as zeros. Remember that mostof the significant information about theimage is concentrated on the largemagnitude coefficients. Accordingly,we tested the codebooks listed abovewith an additional zero-vector as acodevector.

Dataset generation—The datasetsused for the experiments were of threekinds:

• Uniformly distributed (in volume)inside the N-dimensional sphere ofradius 1; those were mostly used forthe convergence tests to find suitablevalues for α and β, but also for per-formance comparison of codebooks.

• Laplace-distributed coefficients.Those resemble more accurately thedatasets obtained by clustering thecoefficients of wavelet transformsof natural images.

• Clusterings of the coefficients ofreal wavelet transforms.

All datasets were normalized so that|x | max = 1.

The coding of the zeros—Usually, inimage and video compression, zerosand other values are coded with differ-ent methods, in order to take advantageof the overwhelming majority of zerosgenerated by the quantization stage;that is why zerotrees and the like lendthose methods such good perfor-mances (see the sections The EZW Al-gorithm and Low bit-rate quantization).So, the resulting bitstream is composedbasically of symbols which representthe non-zero values. In order to simu-late this behavior, in most experimentswe have considered only the rate dueto the coding of non-zero values. Thisis justified because we are assuming

Figure 14. Graph I.

Figure 15. Graph II.

0 5 10 15 20 25-140

-120

-100

-80

-60

-40

-20

0

R[bits]

D[d

b]

2D FR (unif)2D FR (lapl)2D+Z2D+Z FR

0 5 10 15 20 25 30-140

-120

-100

-80

-60

-40

-20

0

R[bits]

D[d

b]

2D FR1D2D4D

43

that an efficient method will be usedfor encoding the zeros, resulting innegligible added rate. In the results thatfollow, the case where the zeros arecoded in the same way as other valuesis shown for comparison; this is indi-cated as “FR” (full rate) in the graphs.

Comparison of CodebookPerformances

The performance of the uniformscalar quantizer on the uniformly dis-tributed dataset is taken as baseline forcomparison; it is seen as the dark-blue,straight line on the graphs.

The graph shown in Fig. 14 depictsthe D x R curves for the followingcases:

2D FR (unif): A uniformly distributeddataset, coded with the 2D binarycodebook; full rate is shown.

2D FR (lapl): Ditto, for a Laplace-dis-tributed dataset.

2D + Z: Laplace-distributed dataset,2D binary codebook plus the originas a codevector.

2D + Z FR: Ditto, full rate shown.

Some interesting features can beperceived from the graph:

• The binary codebook, that isequivalent to scalar uniform quan-tization, has the best asymptoticperformance; this is in accord withthe results from the section Rateand Distortion Measurements.

• The statistical distribution of thedataset influences the results for thefirst steps, but the codebook dictatesthe performance as the residues tendto a distribution which depends onthe codebook; that is why the curvesfor the uniform and laplacian distri-butions merge for high rates.

• The performance improvement ob-tained by the efficient encoding of

zeros is clearly shown by the 3rdand 4th curves (“2D + Z” and “2D +Z FR”).

Graph II (Fig. 15) depicts the D x Rcurves for the 1-, 2- and 4-D binarycodebooks (plus the origin), applied toa Laplace-distributed dataset. We notethat for laplacian distributions, even inthe binary case, the codebook’s dimen-

Figure 16. Graph III.

Figure 17. Graph IV.

0 2 4 6 8 10 12 14 16 18 20-140

-120

-100

-80

-60

-40

-20

0

R[bits]

D[d

b]

2D FR1D2D4DD4

0 1 2 3 4 5 6-90

-85

-80

-75

-70

-65

-60

-55

-50

-45

-40

R[bits]

D[d

b]

2D FR1D2D4DD4

44

sion may have a strong influence onthe D x R performance.

Graphs III and IV (Fig. 16 and 17)show the results obtained on a datasetcomposed by the wavelet coefficientsof the transformed Lena (512 x 512)image (a classical image in this fieldof research). Besides the same binarycodebooks as before, the 4-D binaryand the D4 codebooks (plus the origin)are also used. The results are similarto those obtained on the Laplace-dis-tributed dataset, but the coefficient dis-tribution here is much more concen-trated around the origin, so the advan-tages of including the zero-vector aremuch more pronounced. As can beseen from the graphs, the D4 codebookhas a very good performance on a largerange of data rates. Thus, although thebinary codebooks present the best as-ymptotic behavior, this shows that forpractical data rates other kinds ofcodebooks can outperform them.

Thus, α-expansions as a generali-zation of the usual binary expansion dolead to a rate x distortion performanceimprovement. This explains the resultsobtained in [25–27] where a generali-zation of the EZW [2] algorithm forvectors is shown to yield superior per-formance. We have also conductedsimilar experiments with the MGE [5]algorithm, for a bit-rate of 0.5 bpp. ThePSNR for the scalar case was36.68 dB, while the modified MGEwith the Λ16 codebook (the first shellof the best known sphere packing in 16dimensions [31]) resulted in a PSNRof 37.11 dB, with α = 0.6.

Conclusions

We have discussed successive ap-proximation quantization methods.The classical results of Equitz andCover have been analyzed, as well asthe more recent results of Mallat andFalzon regarding low bit-rate encoding

of transform coefficients. We thenhave shown some practical image en-coding methods based on successiveapproximations. Additionally, we de-veloped the theory of α-expansions, ageneralization of SA quantization. Thistheory has been used here to explainsome published results [25–27]. Wefinish the article by showing that α-expansions do lead to improvements inrate x distortion performance of knownSA-based image compression meth-ods. This requires the use of suitableN-dimensional codebooks, whichopens new avenues for research.

Acknowledgments

We would like to thank Lara C. R.L. Feio for implementing and provid-ing the results of the α-expansionsvariation of the MGE algorithm.

References

[1] P. S. R. Diniz, E. A. B. da Silva, and S. L.Netto, Digital Signal Processing: SystemAnalysis and Design. Cambridge Univer-sity Press, 2002.

[2] J. M. Shapiro, “Embedded Image CodingUsing Zero-Trees of Wavelet Coeffi-cients”, IEEE Transactions on Signal Pro-cessing, vol. 41, pp. 3445–3462, Decem-ber 1993.

[3] A. Said and W. A. Pearlman, “A New, Fastand Efficient Image Codec Based on SetPartitioning in Hierarchical Trees”, IEEETransactions on Circuits and Systems forVideo Technology, vol. 6, no. 3, pp. 243–250, June 1996.

[4] D. Taubman, “High Performance ScalableImage Compression with EBCOT”, IEEETransactions on Image Processing, vol. 9,no. 7, July 2000.

[5] Tse-Hua Lan and A. H. Tewfik, “Multi-grid Embedding (MGE) Image Coding”,Proceedings of the 1999 InternationalConference on Image Processing, Kobe.

[6] JPEG2000 Verification Model 5.3, ISO/IEC JTC1/SC29/WG1 (ITU/T SG28),1999.

[7] W. H. R. Equitz and T. M. Cover, “Suc-cessive Refinement of Information”,IEEE Transactions on InformationTheory, vol. 37, no. 2, March 1991.

45

[8] D. Taubman and A. Zakhor, “Multirate 3-D Subband Coding of Video”, IEEETransactions on Image Processing, vol. 3,no. 5, September 1994.

[9] D. Taubman and A. Zakhor, “A CommonFramework for Rate and DistortionBased Scaling of Highly Scalable Com-pressed Video”, IEEE Transactions onCircuits and Systems for Video Technol-ogy, vol. 46, no. 4, August 1996.

[10] A. K. Jain, Fundamentals of Digital Im-age Processing. Englewood Cliffs, NewJersey: Prentice Hall, 1989.

[11] K. Sayood, Introduction to Data Com-pression, 2nd Edition. San Francisco,California: Morgan Kaufmann Publish-ers, 2000.

[12] N. Jayant, Image Coding Based on Hu-man Visual Models in Image Processing.Academic Press, 1994.

[13] Andrew B. Watson (Ed.), Digital Imagesand Human Vision. MIT Press, 1993.

[14] W. B. Pennebaker and J. L. Mitchell,JPEG: Still Image Compression Stan-dard. New York: Kluwer Academic Pub-lishers, 1992.

[15] Coding of Moving Pictures and Associ-ated Audio for Digital Storage Media upto 1.5Mbit/s, ISO/IEC JTC1/CD 11172,1992.

[16] Generic Coding of Moving Pictures andAssociated Audio, ISO/IEC JTC1/CD13818, 1994.

[17] Coding of Moving Pictures and Audio,ISO/IEC JTC1/SC29/WG11/CD 14496,1997.

[18] T. Berger, Rate Distortion Theory: AMathematical Basis for Data Compres-sion. Englewood Cliffs, New Jersey:Prentice Hall, Inc., 1971.

[19] J. M. Shapiro, “An Embedded WaveletHierarchical Image Coder”, Proceedingsof the 1992 ICASSP Conference, vol 4, pp.657–660, San Francisco, March 1992.

[20] S. G. Mallat, A Wavelet Tour of SignalProcessing. San Diego, California: Aca-demic Press, 1998.

[21] T. C. Bell, J. G. Cleary, and I. H. Witten,Text Compression. Englewood Cliffs,New Jersey: Prentice Hall, 1990.

[22] A. Gersho and R. M. Gray, Vector Quan-tization and Signal Compression. NewYork: Kluwer Academic Publishers,1991.

[23] T. M. Cover and J. A. Thomas, Elementsof Information Theory. New York: JohnWiley and Sons, Inc., 1991.

[24] S. Mallat and F. Falzon, “Analysis of LowBit Rate Image Transform Coding”, IEEETransactions on Signal Processing, vol.46, no. 4, April 1998.

[25] E. A. B. da Silva, D. G. Sampson and M.Ghanbari, “A Successive ApproximationVector Quantizer for Wavelet TransformImage Coding”, IEEE Transactions onImage Processing, Special Issue on Vec-tor Quantization, vol. 5, no. 2, pp. 299–310, February 1996.

[26] E. A. B. da Silva, Wavelet Transforms forImage Coding. Ph.D. Dissertation, Uni-versity of Essex, United Kingdom, June1995.

[27] M. Craizer, E. A. B. da Silva, and E. G.Ramos, “Convergent Algorithms for Suc-cessive Approximation Vector Quantiza-tion with Applications to Wavelet ImageCompression”, IEE Proceedings—Vision,Image and Signal Processing, vol. 146,no. 3, pp. 159–164, July 1999.

[28] M. F. Barnsley, Fractals Everywhere.Academic Press, Inc., 1988.

[29] http://carbon.cudenver.edu/~hgreenbe/glossary/tours/linearalgebra.html.

[30] D. A. Fonini Jr., Quantization with Alpha-Expansions (Internal Report), http://lps.ufrj.br/IR/Fonini/alpha-exp.pdf .

[31] J. H. Conway and N. J. A. Sloane, SpherePackings, Lattices and Groups. Springer-Verlag, 1988.

[32] L. Lovisolo, E. A. B. da Silva, “UniformDistribution of Points on a Hyper-Spherewith Applications to Vector Bit-PlaneEncoding”, IEE Proceedings—Vision, Im-age and Signal Processing, vol. 148, no.3, pp. 187–193, June 2001.

[33] E. B. Saff, A. B. J. Kuijlaars, “Distribut-ing Many Points on a Sphere”, Math-ematical Intelligencer, vol. 19, no. 1, pp.5–11, 1997.

[34] M. Craizer, D. A. Fonini Jr. and E. A. B.da Silva, “Alpha-Expansions: A Class ofFrame Decompositions”, to appear in Ap-plied and Computational HarmonicAnalysis.

[35] M. Craizer, D. A. Fonini Jr. and E. A. B.da Silva, “Quantized Frame Decomposi-tions”, in Curve and Surface Fitting:Saint-Malo 99. Nashville, Tennessee:Vanderbilt University Press, pp. 153–160,2000.

[36] P. L. Zador, Development and Evaluationof Procedures for Quantizing Multivari-ate Distributions. Ph.D. Dissertation,Stanford University, 1963.

46

A Unified Rate-DistortionAnalysis Framework forTransform Coding: A Summary

A Unified Rate-DistortionAnalysis Framework forTransform Coding: A Summaryby Zhihai He and Sanjit K. Mitra

TransactionsTransactionsTransactions

Summary

Recent advances in computing andcommunication technology have

stimulated the research interest in digi-tal techniques for recording and trans-mitting visual information, such asimages and videos. The exponentialgrowth in the amount of visual data tobe stored, transferred, and processedhas created a huge need for data com-pression. The demand for image andvideo compression has triggered thedevelopment of several compressionstandards, such as JPEG andJPEG2000 image coding, MPEG-2,H.263, and MPEG-4 video coding.Besides the standard image/videocompression algorithm, many otheralgorithms have also been reported inthe literature, such as embedded zero-

tree wavelet (EZW) image coding, andset partitioning in hierarchical trees(SPIHT) image coding. In both thecompression standards and the algo-rithms reported in the literature, trans-form coding has become the dominantapproach for image and video com-pression.

In transform coding of images andvideos, the two most important factorsare the coding bit rate and picture qual-ity. The coding bit rate, denoted by R,determines the channel bandwidth re-quired to transfer the coded visual data.One direct and widely used measurefor the picture quality is the meansquare error (MSE) between the codedimage/video and the original one. Thereconstruction error introduced bycompression, often referred as distor-tion, is denoted by D. In typical trans-form coding, both R and D are con-trolled by the quantization parameterof the quantizer, denoted by q. Themajor issue in system design for trans-form coding is how to select the quan-tization parameter to achieve the tar-get coding bit rate, or target picturequality. To this end, we need to ana-lyze and estimate the rate-distortion(R-D) behavior of the image/videoencoder; this behavior is characterizedby its rate-quantization (R-Q) and dis-

Figure 1. Rate and distortion control using rate-distortion functions.

TargetPicture Quality

TargetBit Rate

R(q)

D(q)

qq

47


tortion-quantization (D-Q) functions,denoted by R(q) and D(q), respec-tively. For convenience, they are col-lectively called R-D functions orcurves. Based on the estimated R-Dfunctions, the quantization parameterq can be readily determined to achievethe target bit rate or picture quality, asshown in Fig. 1.

Analysis and estimation of the R-Dfunctions have important applicationsin visual coding and communication.First, with the estimated R-D functionswe can adjust the settings of the en-coder and control the output bit rate orpicture quality according to the chan-nel conditions, the storage capacity, orthe user’s requirement. Second, basedon the estimated R-D functions, opti-mum bit allocation as well as other R-D optimization procedures can be per-formed to improve the efficiency of thecoding algorithm and, consequently, toimprove the image quality or videopresentation quality.

It is well known that the R-D be-havior of an image/video encoder isdetermined both by the characteristicsof the input source data and by the ca-pability of the coding algorithm to ex-

plore these characteristics. PreviousR-D models reported in the literaturetry to use some statistics of the inputsource data, such as variance, to de-scribe the input image or video data.They also try to analyze and modeleach step of the coding algorithms andformulate an explicit expression of thecoding bit rate. To achieve high cod-ing performance, an efficient codingalgorithm must employ a sophisticateddata representation scheme as well asan efficient entropy coding scheme. Toimprove the rate estimation accuracyfor these coding algorithms, the ratemodels become very complex. How-ever, with complex and highly nonlin-ear expressions, the estimation and ratecontrol process becomes increasinglycomplicated and even unstable withthe image-dependent variations.

It should also be noted that, for dif-ferent coding algorithms, previousR-D models and rate control algo-rithms are quite different from eachother. It would be ideal to develop asimple, accurate, and unified ratemodel for all typical transform codingsystems. Based on this simple model,we could then develop a unified rate

“A Unified Rate-Distortion Analysis Framework for Transform Coding”Zhihai He and Sanjit K. Mitra

Abstract—In our previous work, we have developed a rate-distortion (R-D) modeling frame-work H.263 video coding by introducing the new concepts of characteristic rate curves and ratecurve decomposition. In this paper, we further show it is a unified R-D analysis framework forall typical image/video transform coding systems, such as EZW, SPIHT and JPEG image cod-ing; MPEG-2, H.263, and MPEG-4 video coding. Based on this framework, a unified R-D esti-mation and control algorithm is proposed for all typical transform coding systems. We have alsoprovided a theoretical justification for the unique properties of the characteristic rate curves. Alinear rate regulation scheme is designed to further improve the estimation accuracy and robust-ness, as well as to reduce the computational complexity of the R-D estimation algorithm. Ourextensive experimental results show that with the proposed algorithm, we can accurately esti-mate the R-D functions and robustly control the output bit rate or picture quality of the image/video encoder.

IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, no. 12, pp. 1221–1236, December 2001.

CSV

T T

RA

NSA

CT

ION

S

BE

ST P

AP

ER

AW

AR

D

48


and picture quality control algorithmwhich could be applied to all typicaltransform coding systems. To this end,we need to uncover the common rulesthat govern the R-D behaviors of alltransform coding systems. Obviously,this will provide us with valuable theo-retical insights into the mechanism oftransform coding. From a practicalpoint of view, the simple and unifiedrate model and control algorithmwould enable us to control the image/video encoder accurately and robustlywith very low computational complex-ity and implementation cost.

In this work, we have developed anew R-D analysis methodology, called“ρ -domain analysis”. In conventionalR-D analysis, the rate and distortionare studied as functions of the quanti-zation parameter q. In typical trans-form coding, we observe that the per-centage of zeros among the quantizedtransform coefficients, denoted by ρ,monotonically increases with thequantization parameter. This impliesthat there is a one-to-one mapping be-tween them. Therefore, mathemati-cally, the rate and distortion are alsofunctions of ρ. A study of the rate anddistortion as functions of ρ is calledρ -domain analysis. Our theoretical

study and experimental results havedemonstrated that the R-D functionshave unique behaviors in the ρ -domain.

Using the ρ-domain R-D analysismethod, we develop a unified R-Danalysis framework and estimation al-gorithm for all typical transform cod-ing by introducing two new concepts:characteristic rate curves and ratecurve decomposition. In transform cod-ing, coding of zero coefficients is criti-cal to the overall compression efficiency.In the proposed R-D analysis frame-work, we introduce two characteristicrate curves, denoted by Qz(ρ) andQnz(ρ), to describe the coding behav-iors of the zero and non-zero transformcoefficients. We notice that these twofunctions have interesting and uniquebehaviors, which enable us to developa fast algorithm to estimate them.

In Fourier analysis, which is apowerful tool for digital signal pro-cessing, to study the behavior of afunction we represent this function bya linear combination of the basis func-tions sin(nx), cos(nx) which have well-known properties. The Fourier coeffi-cients describe behavior of the func-tion. In this paper, we apply such adecomposition and analysis methodol-ogy to R-D analysis. To be more spe-

Zhihai He received the B.S. degree from Beijing Normal University, Beijing, China, and the M.S.degree from the Institute of Computational Mathematics, Chinese Academy of Sciences, Beijing, China,in 1994 and 1997, respectively, both in mathematics; and the Ph.D. degree in electrical engineering fromUniversity of California, Santa Barbara, California, in 2001. In 2001, he joined Sarnoff Corporation,Princeton, New Jersey, as a member of the technical staff. He received the 2001 IEEE Circuits and Sys-tems Society CSVT Transactions Best Paper Award. His current research interests include image compres-sion, video coding, network transmission, wireless communication, and embedded system design.

IEEE TRANSACTIONS ON

CIRCUITS AND SYSTEMS FORVIDEO TECHNOLOGY

49


cific, in our rate curve decompositionscheme, the actual rate function in theρ-domain, denoted by R(ρ), is repre-sented by a linear combination of thecharacteristic rate curves as follows,

R(ρ) = A(ρ) · Qnz(ρ)+ B(ρ) · Qz(ρ) + C(ρ),

where A(ρ), B(ρ) and C(ρ) are the ratedecomposition coefficients. For agiven input image, Qnz(ρ) and Qz(ρ) aredetermined by their definitions. If weuse different coding algorithms to en-code this image, we should obtain dif-ferent R(ρ). According to the proposeddecomposition scheme, we know thecorresponding decomposition coeffi-cients should also be different. In thisway, different coding algorithms cor-respond to different decompositioncoefficients. Therefore, we can say that{A(ρ), B(ρ), C(ρ)} model the codingalgorithm, while {Qnz(ρ), Qz(ρ)} char-acterize the input source data. As men-tioned above, the R-D performance ofa coding system is determined by thesetwo components. We see that both ofthese components are integrated bylinear combination, which serves as

Figure 2. The proposed rate-distortionanalysis framework.

Sanjit K. Mitra is the recipient of the 1973 F. E. Terman Award and the 1985 AT&T FoundationAward of the American Society of Engineering Education, the 1989 Education Award and the 2000 MacVan Valkenburg Society Award of the IEEE Circuits and Systems Society, the Distinguished Senior U.S.Scientist Award from the Alexander von Humboldt Foundation of Germany in 1989, the 1996 TechnicalAchievement Award and the 2001 Society Award of the IEEE Signal Processing Society, the IEEE Millen-nium Medal in 2000, and the McGraw-Hill/Jacob Millman Award of the IEEE Education Society in 2001.He is co-recipient of the 2000 Blumlein-Browne-Willans Premium of the the Institution of Electrical En-gineers (London), the 2001 IEEE Transactions on Circuits and Systems for Video Technology Best PaperAward, and the 2002 Technical Achievement Award of the European Association for Signal Processing(EURASIP). He is academician of the Academy of Finland and corresponding member of the CroatianAcademy of Sciences and Arts. Dr. Mitra is a Fellow of the IEEE, AAAS, and SPIE.

the framework for our ρ -domain R-Danalysis, as shown in Fig. 2. With theproposed framework, we can estimatethe R-D functions without quantizationand actual coding very accurately androbustly.

Based on estimated rate-qualitybehavior of the encoder, we can opti-mize the output image/video presenta-tion quality, by selecting the best cod-ing parameters for the encoder underthe constraints of network bandwidthand channel conditions.

This technology has potential ap-plications in storage and quality con-trol for digital cameras, scanners, anddigital movies, bandwidth control forlow-delay video communications,such as video conferencing and livevideo streaming, and quality optimiza-tion for storage video and digitalbroadcast.

R(q)

R

DD(q)

R-D functions

q

q

Characteristic Rate curvesDecomposition

Coefficients

Input Picture

EncodingAlgorithm

50

Awards Nominations 2003Awards Nominations 2003Awards Nominations 2003

◊◊ Meritorious Service Award ◊◊Purpose: To honor a person with outstandinglong-term service to the welfare of the CASSociety. The award is based on dedication, ef-fort and contributions. Anyone who is a mem-ber of the CAS Society is eligible. Prize:Plaque and $1,000 check.

◊◊ Technical Achievement Award ◊◊Purpose: To honor a person with outstanding technical contri-butions over a period of years within the scope of the CAS So-ciety as documented by publications (including patents). Theaward is based on the general quality and originality of contri-butions and continuity of effort. Anyone who is a member of theCAS Society is eligible. Prize: Plaque and $1,000 check.

◊◊ Outstanding Young Author Award ◊◊Purpose: To honor an especially meritorious paper published in any one of the CASSociety's Transactions whose author at the date of submission is less than 30 years ofage. The award is based on general quality, originality, contributions, subject matter andtimeliness. Anyone who is an author of papers published in any one of the CAS SocietyTransactions during the two calendar years preceding the award, who at the date of sub-mission of the paper shall be less than 30 years of age is eligible. Prize: Certificate and$500 check for each author (maximum of $2,000 per award).

◊◊ Chapter-of-the-Year Award ◊◊Purpose: To recognize the CAS Society Chapter withthe best yearly activities. The award is based on bestyearly activities in the categories of Chapter-spon-sored technical activities, increase in membership andparticipation in BOG meetings. Anyone who is amember of the CAS Society Chapters is eligible.Prize: Certificates to Chapter Officers.

◊◊ VLSI Transactions Best Paper Award ◊◊Purpose: To recognize the best paper published in the IEEE Transactions onCircuits and Systems for Very Large Scale Integration (VLSI) Systems. Theaward is based on general quality, originality, contributions, subject matter andtimeliness. Anyone who is an author of a paper published in the IEEE Trans-actions on Circuits and Systems for VLSI Systems during the two calendar yearspreceding the award is eligible. Prize: Certificate and $500 check for each au-thor (maximum of $2,000 per award).

◊◊ CAD Transactions Best Paper Award ◊◊Purpose: To recognize the best paper published in the IEEE Transac-tions on Computer-Aided Design of Integrated Circuits and Systems. Theaward is based on general quality, originality, contributions, subject mat-ter and timeliness. Anyone who is an author of a paper published in theIEEE Transactions on Computer-Aided Design of Integrated Circuits andSystems during the two calendar years preceding the award is eligible.Prize: Certificate and $500 check for each author (maximum of $2,000per award).

◊◊ Mac Van Valkenburg Award ◊◊Purpose: To honor a person with outstanding technicalcontributions in a field within the scope of the CAS Soci-ety and outstanding leadership in the field. The award isbased on quality and significance of contribution and con-tinuity of technical leadership. Anyone who is a memberof the CAS Society is eligible. Prize: Plaque and $2,000Check.

◊◊ Guillemin-Cauer Award ◊◊Purpose: To recognize the best paper published in the IEEETransactions on Circuits and Systems. The award is based ongeneral quality, originality, contributions, subject matter andtimeliness. Anyone who is an author of a paper published inthe IEEE Transactions on Circuits and Systems during the twocalendar years preceding the award is eligible. Prize: Certifi-cate and $500 check for each author (maximum of $2,000 peraward).

◊◊ Darlington Award ◊◊Purpose: To recognize the best paper bridging the gap betweentheory and practice published in the IEEE Transactions on Circuitsand Systems. The award is based on general quality, originality, con-tributions, subject matter and timeliness. Anyone who is an authorof papers bridging the gap between theory and practice publishedin the IEEE Transactions on Circuits and Systems during the twocalendar years preceding the award is eligible. Prize: Certificateand $500 check for each author (maximum of $2,000 per award).

◊◊ CSVT Transactions Best Paper Award ◊◊Purpose: To recognize the best paper published in the IEEE Transactions on Cir-cuits and Systems for Video Technology. The award is based on general quality, origi-nality, contributions, subject matter and timeliness. Anyone who is an author of pa-pers published in the IEEE Transactions on Circuits and Systems for Video Tech-nology during the two calendar years preceding the award is eligible. Prize: Cer-tificate and $500 check for each author (maximum of $2,000 per award).

◊◊ Education Award ◊◊Purpose: To honor a person with outstanding contributions to edu-cation in a field within the scope of the CAS Society as documentedby publications of textbooks, research supervision of graduate andundergraduate students, development of short courses and participa-tion in adult education. The award is based on general quality andoriginality of contributions and continuity of effort. Anyone who isa member of the CAS Society is eligible. Prize: Plaque and $1,000check.

◊◊ Industrial Pioneer Award ◊◊Purpose: To honor a person or persons with outstanding and pioneeringcontributions in developing academic and industrial research results intoindustrial applications and/or commerical products.

The award is to be presented annually together with the other awards,and given by the Awards Committee on the basis of quality, originality, andsignificance of contribution. Prize: Plaque and a $1,000 cash award thatwill be divided if there is more than one recipient.

Nominations should be submitted electronically to the 2003 Awards Chair, J. A. Nossek. Forms may also be mailed to J. A. Nossek.Check the Awards page on the CAS website at http://www.ieee-cas.org for up-to-date information on how to submit nominations.

For further information, see the CAShomepage at http://www.ieee-cas.org

51

DUE BY FEBRUARY 1, 2003

IEEE Circuits and Systems Society2001–2002 Outstanding Paper Award Nomination

Paper Award Recommended: _________________________________________________________________Paper Title: _____________________________________________________________________________

______________________________________________________________________________________Paper Authors: ______________________________ ______________________________

______________________________ ______________________________Paper Listing:

Name of Transactions: _______________________________________________________Month: ________________________ Year: ________________ Pages: _________________

Nominator:Name: _________________________ Tel (day): ___________________ Tel (home-opt.): __________________Address: _______________________ Fax: _______________________ E-mail: _________________________

____________________________________________________________

Basis for Nomination:Please give the reasons you believe this paper is deserving of the outstanding paper award. Judging is based upon general quality, originality,contribution, subject matter, and timeliness. Continue on additional page(s).

Outstanding Young Author Award—CAD Transactions Best Paper AwardGuillemin-Cauer Award—CSVT Transactions Best Paper Award

Darlington Award—VLSI Transactions Best Paper Award

Use these forms as a guide to electronically submitnominations to J. A. Nossek (see previous page).

IEEE Circuits and Systems Society2003 Society/Achievement Award Nomination

Name of Award: _____________________________ Date: _________________________________________Nominee:

Name: _____________________________Address: _____________________________ Present Employment Position(s): ____________________

_____________________________ ___________________________________________________________________________ Highest Degree Attained: __________________________

Telephone (day): ________________________________

Nominator:Name: _____________________________ Telephone (day): ________________________________Address: _____________________________ Tel (home-opt.): __________________________________

_____________________________ Fax: _______________________________________________________________________ E-mail: ________________________________________

1. Proposed Citation:Provide a brief statement, not exceeding 50 words, giving the major accomplishments for which the award is being made. This will be used if thenominee is selected as the awardee. Continue on separate page(s).

2. Basis for Nomination:Prepare a statement not exceeding 750 words on why the candidate is being nominated for the award. This statement should then be followed by therecord of accomplishments of the candidate as an educator, and/or as a researcher, and/or as an administrator, and/or as an industrial pioneer, asappropriate. Continue on separate page(s).

3. Short Biography (Not exceeding 2 pages):Include degrees earned (list universities and granting dates); other postgraduate study; record of all positions held (chronologically starting with themost recent position); IEEE activities and offices; other society memberships and offices; awards, honors, patents, inventions and other relevantcontributions. Continue on separate page(s).

4. Publications:List all books, book chapters, and journal papers as well as 10 of the most important publications stating the engineering significance of each.Continue on separate page(s).

Page limitation: Items 1 through 4 are limited to a total of four pages. Any additional pages will be deleted.

5. References:No more than five brief supporting letters from colleagues (and former students for the CAS Society Education Award) should be included with eachaward nomination. List the names of the references on the nomination form. The reference letters can either be collected by the nominator andforwarded unopened to J. A. Nossek, or the references can be instructed to forward their recommendations directly to Prof. Nossek. All referenceletters must be received by the due date of the nominations, February 1, 2003. Please check the Awards page on the CAS website athttp://www.ieee-cas.org on how and where to submit nominations. In the case of a re-nomination, references and other materials can be re-used.

Mac Van Valkenburg Award—Meritorious Service AwardIndustrial Pioneer Award—Chapter-of-the-Year Award

Education Award—Technical Achievement Award

52

IEEE Field Awards

The IEEE Awards Board has announced the 2003 IEEETechnical Field Award recipients. The IEEE Technical

Field Awards were established to provide special recognitionfor outstanding achievements in special fields of electrical and

electronics engineering. Among thisyear’s recipients were three mem-ber of the CAS Society.

Giovanni De Micheli, profes-sor in the Department of ElectricalEngineering at Stanford Universityand president-elect of the CAS So-ciety, received the IEEE EmanuelPiore Award, “for contributions tocomputer-aided synthesis of digitalsystems”.

Keshab K. Parhi, Distin-guished McKnight University Pro-fessor at the University of Minne-

sota and chair of the CAS Technical Committee on VLSI Sys-tems and Applications, was awarded the IEEE Kiyo TomiyasuAward, “for pioneering contribu-tions to high-speed and low-powerdigital signal processing architec-tures for broadband communica-tions systems”.

Receiving the IEEE Leon K.Kirchmayer Graduate TeachingAward was Robert G. Meyer, pro-fessor in the Department of Electri-cal Engineering and Computer Sci-ences at the University of Califor-nia, Berkeley, “for inspirationalclassroom teaching, outstandingmentoring, and developing excel-lent teaching materials in microelectronic circuit design”.

ISI Citation LeaderOn another note, Leon O. Chua, professor at the Univer-

sity of California, Berkeley, was identified by ISI® (foundedas the Institute for Scientific Infor-mation®) as one of the top 15 mosthighly cited authors in the Engi-neering discipline. Using a newevaluation tool, ISI identified thetop 15 researchers in the fields ofengineering, physics, and computerscience based on total citationsfrom papers idexed from 1991 toOctober 31, 2001. Professor Chuawas presented with the award at aspecial reception earlier this year inMontréal, Canada.

RecognitionsRecognitionsRecognitionsIEEE CAS Fellow Profiles 2002Ellen June YoffaFor technical, professional, and business leadershipin electronic design automation.

Ellen J. Yoffa is currently director of Personal & VisualSystems at the IBM T. J. Watson Research Center in YorktownHeights, New York. She is responsible for strategy and man-agement of research activities in computer system hardware,software, and applications,ranging from handheld devicesto large server systems, in ar-eas including digital multime-dia, graphics and visual analy-sis, portable and desktop cli-ents, and large memory sub-systems.

Prior to her current posi-tion at IBM, Dr. Yoffa spent ayear in IBM’s MicroelectronicsDivision managing the Elec-tronic Design AutomationBusiness Strategy organiza-tion, and the year before that she was technical assistant to thedirector of the IBM Research Division. Her research has pri-marily involved the development of tools for VLSI physicaldesign automation.

She is a member of the IEEE Circuits and Systems Soci-ety Board of Governors, and is currently chair of the CAS Dis-tinguished Lecturer Program. For two years, she was techni-cal program chair of the IEEE/ACM Design Automation Con-ference, and in 1997 she was conference general chair. Shehas served on the Editorial Advisory Board of the IEEE Spec-trum magazine.

She received the B.S. and Ph.D. degrees in physics at theMassachusetts Institute of Technology, where her area of studywas theoretical solid state physics. Dr. Yoffa is a member ofIEEE, ACM, Phi Beta Kappa and Sigma Xi.

Arjan van der SchaftFor contributions to the

theory of nonlinear systems.

Arjan van der Schaft re-ceived the undergraduate andPh.D. degrees in mathematicsfrom the University ofGroningen, The Netherlands,in 1979 and 1983, respectively.In 1982 he joined the faculty ofMathematical Sciences, Uni-versity of Twente, Enschede,The Netherlands, where he ispresently full professor inLeon O. Chua

Giovanni De Micheli

Keshab K. Parhi

53

RecognitionsRecognitionsRecognitions

James W. HaslettFor contributions to high

temperature instrumentationand noise in solid-state

electronics.

James W. Haslett receivedthe B.Sc. degree in electricalengineering from the Univer-sity of Saskatchewan, Saska-toon, Saskatchewan, Canada,in 1966, and the M.Sc. andPh.D. degrees from the Univer-sity of Calgary in 1968 and1970, respectively. He thenjoined the Department of Elec-

trical Engineering, University of Calgary, where he is currentlyprofessor. He was head of the Department from 1986 to 1997,and has been president of his own consulting firm since 1981,consulting oilfield instrumentation firms on high temperaturedownhole instrumentation, with Dr. Fred Trofimenkoff, alsoin the Department of Electrical and Computer Engineering atCalgary. He was also a member of several national and inter-national science teams designing satellite instrumentation inthe late 1970’s and 1980’s.

Dr. Haslett joined the TRLabs Industrial Research Con-sortium in 1997. His current research interests include high-temperature electronics for instrumentation applications, solidstate imagers for scientific applications, biomedicalmicrosystems and nanosystems, and RF microelectronics fortelecommunications.

Dr. Haslett has authored or co-authored more than 120publications in the field of analog electronics, has written morethan 40 technical reports to industry, and has won 12 teachingawards in the past 9 years, including the University of CalgaryPresident’s Circle Award for Teaching Excellence in 2001.

Dr. Haslett is a member of the Association of ProfessionalEngineers, Geologists and Geophysicists of Alberta, the Ca-nadian Astronomical Society, the Canadian Society of Explo-ration Geophysicists, and the American Society for Engineer-

Lawrence T. PileggiFor contributions to simulation and modelingof integrated circuits.

Lawrence Pileggi is professor of electrical and computerengineering and director of the Center for Silicon SystemImplementation at Carnegie Mellon University. From 1984through 1986 he worked forWestinghouse Research andDevelopment where in 1986 hewas recognized with thecorporation’s highest engineer-ing achievement award. In1989 he received the Ph.D. inelectrical and computer engi-neering from Carnegie MellonUniversity. From 1989 through1995 he was a faculty memberat the University of Texas atAustin. In January of 1996 hejoined the faculty at CarnegieMellon University. His research interests include various as-pects of circuit-level design automation and analysis. He hasconsulted for several EDA and semiconductor companies, andwhile on leave from Carnegie Mellon in 1999–2000 he wasthe CTO and VP of R&D at Monterey Design Systems. Hereceived the CAD Transactions Best Paper Award in 1991 for“Asymptotic Waveform Evaluation (AWE)”, and again in 1999

mathematical systems and control theory. His research inter-ests include the mathematical modeling of physical and engi-neering systems and the control of nonlinear and hybrid sys-tems. Within these areas he has worked on geometric nonlin-ear control, Hamiltonian control systems, complementarityhybrid systems, and nonlinear robust and H∞ control. Morerecently his research interests have been in the geometrical de-scription of network models of physical systems as port-Hamil-tonian systems, and exploiting this framework for simulationand control. He has served as associate editor for Systems &Control Letters, Journal of Nonlinear Science, and the IEEETransactions on Automatic Control. Currently he is associateeditor for the SIAM Journal on Control and Optimization, as-sociate editor for Systems & Control Letters and editor-at-largefor the European Journal of Control. He is the (co-)author ofthe following books: System Theoretic Descriptions of Physi-cal Systems, Amsterdam, The Netherlands: CWI, 1984; withP. E. Crouch, Variational and Hamiltonian Control Systems,Berlin, Germany: Springer-Verlag, 1987; with H. Nijmeijer,Nonlinear Dynamical Control Systems, Berlin, Germany:Springer-Verlag, 1990; L2-Gain and Passivity Techniques inNonlinear Control, London, UK: Springer-Verlag, 1996, sec-ond edition, Springer Communications and Control Engineer-ing Series, 2000; and with J. M. Schumacher, An Introductionto Hybrid Dynamical Systems, London, UK: Springer-Verlag,LNCIS 251, 2000.

for “Passive Reduced-Order Interconnect Macromodeling Al-gorithm (PRIMA)”. He received a Presidential Young Inves-tigator Award from the National Science Foundation in 1991.In 1991 and again in 1999 he received the Semiconductor Re-search Corporation Technical Excellence Award. In 1993 hereceived an Invention Award from the SRC and subsequentlya U.S. Patent for the RICE simulation tool. In 1994 he receivedthe University of Texas Parent’s Association Centennial Teach-ing Fellowship for excellence in undergraduate instruction. In1995 he received a Faculty Partnership Award from IBM. Heis a co-author of Electronic Circuit and System SimulationMethods, McGraw-Hill, 1995 and IC Interconnect Analysis,Kluwer, 2002. He has published over 125 refereed conferenceand journal papers and holds five U.S. patents.

54


Kaushik RoyFor contributions to the power-aware design of digital circuits.

Kaushik Roy received the B.Tech. degree in electronicsand electrical communications engineering from the Indian In-stitute of Technology, Kharagpur, India, and the Ph.D. degreefrom the electrical and computer engineering department ofthe University of Illinois at Urbana-Champaign in 1990. Hewas with the Semiconductor Process and Design Center ofTexas Instruments, Dallas, where he worked on FPGA archi-tecture development and low-power circuit design. He joinedthe electrical and computer engineering faculty at Purdue Uni-versity, West Lafayette, Indiana, in 1993, where he is currentlyprofessor. His research interestsinclude VLSI design/CAD withparticular emphasis in low-power electronics for portablecomputing and wireless com-munications, VLSI testing andverification, and reconfigurablecomputing. Dr. Roy has pub-lished more than 200 papers inrefereed journals and confer-ences, holds 5 patents, and is aco-author of the book LowPower CMOS VLSI Design(John Wiley).

Dr. Roy received the National Science Foundation CareerDevelopment Award in 1995, the IBM faculty partnershipaward, the ATT/Lucent Foundation award, best paper awardsat the 1997 International Test Conference and the 2000 Inter-national Symposium on Quality of IC Design, and is currentlya Purdue University faculty scholar professor. He is on the edi-torial boards of IEEE Design and Test, the IEEE Transactionson Circuits and Systems, and the IEEE Transactions on VLSISystems. He was guest editor for the special issue on Low-Power VLSI in the IEEE Design and Test in 1994 and the IEEETransactions on VLSI Systems in June 2000.

Sharad MalikFor contributions to electronic design automation tech-

niques in logic and embedded software synthesis.

Sharad Malik received the B. Tech. degree in electricalengineering from the Indian Institute of Technology, NewDelhi, India, in 1985 and the M.S. and Ph.D. degrees in com-puter science from the University of California, Berkeley, in1987 and 1990 respectively.

Avideh ZakhorFor contributions to image and video compression.

Avideh Zakhor received the B.S. degree from CaliforniaInstitute of Technology, Pasadena, and the S.M. and Ph.D.degrees from Massachusetts Institute of Technology, Cam-bridge, all in electrical engi-neering, in 1983, 1985, and1987 respectively. In 1988, shejoined the faculty at U. C. Ber-keley where she is currentlyprofessor in the Department ofElectrical Engineering andComputer Sciences. Her re-search interests are in the gen-eral area of image and videoprocessing, compression, andcommunication. In 1997, shereceived the IEEE Signal Pro-cessing Society Best PaperAward. In 1997 and 1999 she also received the IEEE Circuitsand Systems Society Video Technology Transactions Best Pa-per Awards. She holds 5 U.S. patents, and is the co-author ofthe book, Oversampled A/D Converters with Soren Hein.

Currently he is professor in the Department of ElectricalEngineering, Princeton University. His current research inter-ests are design tools for embedded computer systems, andsynthesis and verification of digital systems.

He has received the Presi-dent of India’s Gold Medal foracademic excellence in 1985,the IBM Faculty DevelopmentAward in 1991, an NSF Re-search Initiation Award in1992, the Princeton UniversityRheinstein Faculty Award andthe NSF Young InvestigatorAward in 1994, Best PaperAwards at the IEEE Interna-tional Conference on Com-puter Design in 1992 and at theACM/IEEE Design Automa-

tion Conference in 1996, the Walter C. Johnson Prize forTeaching Excellence in 1993, and the Princeton UniversityEngineering Council Excellence in Teaching Award in 1993,1994, and 1995. He serves on/has served on the program com-mittees of DAC, ICCAD and ICCD. He served as the techni-cal program co-chair for DAC in 2000 and 2001 and panelschair for 2002. He is on the editorial boards of the Journal ofVLSI Signal Processing, Design Automation for EmbeddedSystems, and IEEE Design and Test.

His research in functional timing analysis and proposi-tional satisfiability has been widely used in industrial electronicdesign automation tools.

ing Education. He is currently a member of the Editorial Re-view Committee of the IEEE Transactions on Instrumentationand Measurement, associate editor of the Canadian Journal ofElectrical and Computer Engineering, and a fellow of the En-gineering Institute of Canada.

55


Carl Matthew SechenFor contributions to automated placement androuting in integrated circuits.

Carl Sechen received the B.E.E. from the University ofMinnesota, the M.S. from M.I.T., and the Ph.D. from Univer-sity of California, Berkeley.Starting in 1986, he was assis-tant and then associate profes-sor in the Department of Elec-trical Engineering at Yale Uni-versity. In 1992 he moved tothe University of Washington,where he is now professor inthe Department of ElectricalEngineering. He is a co-direc-tor of the National ScienceFoundation’s Center for theDesign of Analog and DigitalIntegrated Circuits (CDADIC).

Professor Sechen received the Semiconductor ResearchCorporation’s 1994 SRC Technical Excellence Award. He alsoreceived the Semiconductor Research Corporation’s 1988 SRCInventor’s Award. An SRC Inventor’s Recognition Award wasreceived in 2001 for his development of output predictionlogic. He was a member of the technical program committeefor the IEEE International Conference on Computer AidedDesign (ICCAD) from 1989 to 1993. From 1998 to 2001, hewas chairman of the placement and floorplanning technicalprogram subcommittee for ICCAD.

Professor Sechen developed the first version of theTimberWolf placement and routing package in 1983. Versionsof TimberWolf that Prof. Sechen developed at UC Berkeley,Yale University and the University of Washington were usedin production at Intel Corporation from 1984–1995, at DigitalEquipment Corporation from 1985–1995, at National Semi-conductor from 1985–1995, Crystal Semiconductor from1985–1995, Advanced Micro Devices from 1992–1995, andMotorola from 1992–1995. TimberWolf was used at more than20 companies and more than 25 universities.

In his 15 years as professor, Prof. Sechen has graduated16 Ph.D. students. He has authored one book, and authored orcoauthored over 120 research papers. He is a co-founder ofInternetCAD.com, Inc., a placement and routing tool vendor.His current research interests are primarily in ultra-high-speedand low-power digital integrated circuit design.

Ulrich L. RohdeFor contributions to and leadership in the

development and industrial implementation ofmicrowave computer-aided design technology.

Ulrich L. Rohde studied electrical engineering and radiocommunications at the universities of Munich and Darmstadt,Germany, receiving the Ph.D. in electrical engineering in 1978and the Sc.D. with honors in 1979 in radio communications.

He is president of Communications Consulting Corpora-tion and executive vice president of Ansoft Corporation forStrategic Planning, Pittsburgh, Pennsylvania, after the com-pany successfully merged with Compact Software, Inc. He ischairman of Synergy Microwave Corp., Paterson, New Jer-sey, and a partner of Rohde & Schwarz, Munich, Germany, a

multinational company spe-cializing in advanced test andradio communications sys-tems. Previously, he was presi-dent of Compact Software,Inc., Paterson, New Jersey, andbusiness area director for Ra-dio Systems of RCA, Govern-ment Systems Division,Camden, New Jersey, respon-sible for implementing com-munications approaches formilitary secure and adaptivecommunications.

In 2001 Dr. Rohde was appointed visiting professor of RFand Microwave Technologies at the University of Cottbus,Germany, and in 1998 he was appointed honorary visiting pro-fessor of electronic and microwave engineering by the Uni-versity of Bradford, England. In 1997 he was awarded the hon-orary degree of Doctor Honoris Causa by the University ofGrosswardein and the honorary degree of Doctor HonorisCausa from the Technical University of Klausenburg. He re-ceived a German patent on Method to Measure the One and/or Multi-port Parameters of a Device Under Test (DUT) Us-ing a Network Analyzer, and a U.S. patent on Phase-LockedLoop Circuits and Voltage Controlled Oscillator Circuits. In1997 he was appointed professor of electrical engineering andmicrowave technology at the University of Oradea, Romania.From 1990 to 1992, Dr. Rohde was visiting research profes-sor at New Jersey Institute of Technology, Department of Elec-trical Engineering. Dr. Rohde is also a member of the staff atGeorge Washington University since 1982, and as an adjunctprofessor teaching in the Electrical Engineering and Computer

Sciences departments gave numerous lectures worldwide re-garding communications theory and digital frequency synthe-sizers. In addition, he has taught at the University of Florida,Gainesville since 1977.

Dr. Rohde has published more than 60 scientific papersin professional journals, as well as six books. He is also chair-man of the Electrical and Computer Engineering AdvisoryBoard at New Jersey Institute of Technology.

Professor Zakhor was a General Motors scholar from 1982to 1983, was a Hertz fellow from 1984 to 1988, received thePresidential Young Investigator Award, and the Office of Na-val Research Young Investigator Award in 1992. From 1998 to2001, she was an elected member of IEEE Signal ProcessingBoard of Governors.

56

Calls for Papers and ParticipationCalls for Papers and ParticipationCalls for Papers and Participation

Call for Papers

GLSVLSI 2003Washington, DC April 28–29, 2003

http://www.glsvlsi.org/Sponsored by: ACM SIGDA

With the Technical Support of: IEEE CAS

The 2003 Great Lakes Symposium on VLSI (GLSVLSI) will be in Washington D.C.Original, unpublished papers, describing research in the general area of VLSI are solic-ited. Both theoretical and experimental research results are welcome. Proceedings willbe published by the ACM and will be included on the SIGDA compendium CD-ROM.

The theme of this year’s symposium is “VLSI in the Nanometer Era.”

Submission deadline: January 10, 2003 (5:00pm EST)Acceptance notification: February 14, 2003

Camera ready paper due: March 7, 2003

Paper Submission: Electronic submission to the website is required.

Paper Format: To allow reduced turn-around time for accepted papers,GLSVLSI 2003 submissions should be in camera-ready two-column format,following the ACM proceedings specifications located at:

http://www.acm.org/sigs/pubs/proceed/template.html

Mircea Stan, General ChairUniversity of VirginiaDept. of ECEP.O. Box 400743Charlottesville, VA 22904Tel: 434–924–3960; Fax: 434–924–[email protected]

GLSVLSI 2003

Call for Participation

Asia and South Pacific Design Automation Conference 2003

January 21–24, 2003Kitakyushu International Conference Center, Kitakyushu, JAPAN

ASP-DAC 2003 is the eighth in a series of annual International Conferences on DesignAutomation. The conference aims at providing the Asian and South Pacific CAD/DAand Design community with opportunities of interchanging ideas and collaborativelydiscussing the directions of the technologies related to Embedded System Design, Sys-tem-on-a-Chip, Deep Submicron Technologies and a variety of New Applications.

For detailed information, visit our WEB site: http://www.aspdac.com/ASP-DAC 2003 Chairs: ASP-DAC 2003 SecretariatGeneral Chair: Japan Electronics Show AssociationHiroto Yasuura (Kyushu Univ.) Sumitomo Shibadaimon Bldg. 2-gokan, 5F

1–12–16 ShibadaimonTechnical Program Co-Chairs: Minato-ku, Tokyo, 105–0012 JAPANMasaharu Imai (Osaka Univ.) Phone: +81–3–5402–7601Jason Cong (UCLA) Fax: +81–3–5402–7605

E-mail: [email protected]

International Symposium onSignals, Circuits and Systems

Iasi, Romania, July 10–11, 2003http://www.tuiasi.ro/events/scs2003

CALL FOR PAPERSWe cordially invite you to join us at the Faculty of Electronics and Telecommuni-

cations of the Technical University of Iasi, Romania for SCS 2003.SCS 2003 aims at bringing together scientists and researchers from academia and

industry to present some of their latest ideas and results.

AUTHOR’S SCHEDULE:

Submission of full paper January 15, 2003Notification of acceptance March 30, 2003Camera-ready copy May 1, 2003Registration May 1, 2003

The authors are kindly requested to send their papers in electronic for-mat (PS or PDF) until January 15th, 2003, to the following e-mail address:

[email protected]

SCS 2003 International SymposiumFaculty of Electronics & Telecommunications“Gh. Asachi” Technical University of IasiBd. Carol 11, Iasi, 6600, ROMANIA

Call for Papers

Second Asia-Pacific Workshop onChaos Control and SynchronizationTo be held in conjunction with

The Shanghai International Symposium onNonlinear Science and Applications

Shanghai, June 7–8, 2003

http://www.ee.cityu.edu.hk/~chaos/apwccs03.htmImportant deadlines:March 31, 2003 — Detailed abstract submissionApril 15, 2003 — Acceptance notificationMay 1, 2003 — Advanced registration and paper submission in electronic versionInformation Contact:Prof. Guanrong (Ron) Chen, Chair Professor and DirectorCentre for Chaos Control and SynchronizationDepartment of Electronic Engineering, City University of Hong Kong83 Tat Chee Avenue, KowloonHong Kong SAR, P. R. ChinaPhone: (852) 2788–7922; Fax: (852) 2788–7791; Email: [email protected]

Announcement and Call for PapersSHANGHAI INTERNATIONAL SYMPOSIUM

ON NONLINEAR SCIENCE AND APPLICATIONSShanghai NSA’03, June 9 – 13, 2003

http://www.ssbiophysics.com.cn/snsa/Important deadlines:February 28, 2003 — Detailed abstract submission; mini-symposium and

special session proposals dueMarch 31, 2003 — Acceptance notification

April 15, 2003 — Advanced registration/paper submission in electronic versionContact:Prof. Jiong RuanSecretariat of Shanghai NSA’03Research Center for Nonlinear SciencesDepartment of MathematicsFudan UniversityShanghai 200433, P. R. ChinaTel: +86–21– 65100339Fax: +86–21–56620463E-mail: [email protected]

57

2002 IEEE CIRCUITS AND SYSTEMS SOCIETY ROSTER**(CAS-04, Division I)

Division I Director President President – Elect Past PresidentCary Y. Yang J. A. Nossek G. De Micheli H. C. Reddy

+49 89 2892 8501 +1 650 725 3632 + 1 562 985 5106

Administrative Vice President Vice President – Conferences Vice President – Publications Vice President – Technical ActivitiesI. N. Hajj G. A. De Veirman M. N. S. Swamy M. Hasler

+ 1 961 347 952 +1 714 890 4118 +1 514 848 3091 +41 21 693 2622

Vice President – Regions 1-7 Vice President – Region 8 Vice President – Region 9 Vice President – Region 10W. B. Mikhael M. J. Ogorzalek S. L. Netto N. Fujii

+1 407 823 3210 +48 12 617 3613 +55 21 2562 8164 +81 3 5734 2561

Elected Members of the Board of Governors2002 2003 2004 Ex Officio

G. G. E. Gielen A. G. Andreou J. Cong Chairman of Sponsored orR. Gupta R. J. Marks II M. Flomenhoft Co-Sponsored Conferences

H. Kunieda M. Pedram E. G. Friedman Standing Committee ChairsP. Pirsch L. Trajkovic E. Macii Technical Committee Chairs

H. Yasuura E. Yoffa R. W. Newcomb Chapter ChairpersonsEditors of Society Sponsored

Society Administrator Transactions & MagazineB. Wehner CAS Editor of C&D Magazine

+1 219 871 0210 Division I DirectorD. Senese, M. Ward-Callan

STANDING COMMITTEE CHAIRSAwards H. C. ReddyConstitution and Bylaws R. J. Marks IIFellows W.-K. ChenNominations R. Schaumann

Parliamentarian R. J. Marks II

CONFERENCE ACTIVITIES G. A. De Veirman2002 APCCAS S. Soegijoko2003 DAC I. Getreu2002 ICCAD L. Pileggi2002 ICCD Y. Manoli2002 ICECS A. Baric2003 ISCAS S. Pookaiyaudom/C. Toumazou2002 MWSCAS M. A. Soderstrand

DISTINGUISHED LECTURER PROGRAM E. Yoffa

MEMBERSHIP DEVELOPMENT AFFAIRS G. De Micheli

PUBLICATION ACTIVITIES M. N. S. SwamyCAS Magazine Editor M. K. SainCircuits and Devices Magazine Editor R. W. Waynant

- Editor for CAS Society C.-Y. WuTransactions on Circuits and Systems Part I:

Fundamental Theory and Applications T. RoskaTransactions on Circuits and Systems Part II:

Analog and Digital Signal Processing I. GaltonTransactions on Computer Aided Design of

Integrated Circuits & Systems K. MayaramTransactions on Circuits and Systems for

Video Technology T. SikoraTransactions on Mobile Computing T. La PortaTransactions on Multimedia T. ChenTransactions on VLSI Systems E. G. Friedman

REGIONAL ACTIVITIES G. De Micheli

TECHNICAL ACTIVITIES M. HaslerAnalog Signal Processing T. B. TarimBiomedical Circuits and Systems C. ToumazouBlind Signal Processing R.-W. LiuCellular Neural Networks and Array Computing L. FortunaCircuits and Systems for Communications M. A. Bayoumi

Computer-Aided Network Design M. FlomenhoftDigital Signal Processing T. SaramakiGraph Theory and Computing K. ThulasiramanMultimedia Systems and Applications L.-G. ChenNanoelectronics and Gigascale Systems C.-Y. WuNeural Systems and Applications W.-C. FangNonlinear Circuits and Systems L. GoldgeisserPower Systems and Power Electronics Circuits I. HiskensSensory Systems R. Etienne-CummingsVisual Signal Processing and Communication H. SunVLSI Systems and Applications K. Parhi

REPRESENTATIVESNanotechnology Council H. Reddy, M. N. S. SwamyNeural Networks Council A.R. Stubberud, J. ZuradaSensors Council M. E. ZaghloulSolid-State Circuits Society E. Sanchez-SinencioSociety on Social Implications of Technology R. W. NewcombSociety Education Chairs Committee J. ChomaIEEE Press G. R. ChenIEEE TAB Magazines Committee W. H. WolfIEEE TAB New Technology Directions Committee C.-Y. WuIEEE TAB Newsletters Committee M. HaenggiIEEE TAB Transactions Committee M. HaslerIEEE-USA R. de FigueiredoIEEE-USA Professional Activities Committee

for Engineers (PACE) P. K. Rajan

CORRESPONDING MEMBERSTAB Awards and Recognition Committee L.-G. ChenConference Publications Committee L. GorasP2SB/TAB Electronic Products and

Services Committee A. IoinoviciTAB Finance Committee K. ThulasiramanTAB Periodicals Committee C.-S. LiTAB Periodicals Packages Committee J. Silva-MartinezTAB Products Committee M. OgorzalekTAB Strategic Planning and Review Committee R. Gupta

** To obtain full contact information for any one of the volunteers, pleasecontact the Society Adminstrator, Barbara Wehner at [email protected].

THE INSTITUTE OF ELECTRICAL& ELECTRONICS ENGINEERS, INC.445 HOES LANEPISCATAWAY, NJ 08855

ISCAS 2003: BISCAS 2003: BISCAS 2003: BISCAS 2003: BISCAS 2003: BA N G K O KA N G K O KA N G K O KA N G K O KA N G K O K - T - T - T - T - TH A I L A N DH A I L A N DH A I L A N DH A I L A N DH A I L A N D

LLLLLI F EI F EI F EI F EI F E-----S T Y L ES T Y L ES T Y L ES T Y L ES T Y L E T T T T TECHNOLOG I ESECHNOLOG I ESECHNOLOG I ESECHNOLOG I ESECHNOLOG I ES

AAAAA N E WN E WN E WN E WN E W W A V EW A V EW A V EW A V EW A V E O FO FO FO FO F C C C C CI R C U I T SI R C U I T SI R C U I T SI R C U I T SI R C U I T S A N DA N DA N DA N DA N D S S S S SY S T E M SY S T E M SY S T E M SY S T E M SY S T E M S

WWW.ISCAS2003.ORGWWW.ISCAS2003.ORGWWW.ISCAS2003.ORGWWW.ISCAS2003.ORGWWW.ISCAS2003.ORG

Call for Participation

2003 IEEE International Symposium on Circuits and Systems

Sunday May 25–Wednesday May 28, 2003Imperial Queen’s Park Hotel, Bangkok, Thailand

The 2003 IEEE International Symposium on Circuits and Systems(ISCAS) is the world’s premier networking forum for leading people in thehighly active field of the theory, design and implementation of circuits andsystems. The 2003 ISCAS, sponsored by the IEEE Circuits and SystemsSociety and supported by Mahanakorn University of Technology, will beheld in the exotic and friendly atmosphere of Bangkok, Thailand. Inspiredby lifestyle, the environment and healthcare, the Symposium will focus onnew and next-wave technologies, including third generation mobile, short-distance communications, mixed-signal systems-on-a-chip, low-power au-tonomous devices, sensor interfaces and biosystems.

Complementing the exceptional technical program, the social programof ISCAS 2003 will provide delegates a congenial, sparkling and exoticatmosphere and promises to be a truly unique and unforgettable event. Welook forward to welcoming you in Bangkok. We are certain that you willenjoy an exciting mixture of the technical contents of ISCAS 2003 and theexotic surroundings of the city.

General Co-ChairsSitthichai Pookaiyaudom, Mahanakorn University of

Technology, Thailand [email protected] Toumazou, Imperial College, London

[email protected]

Vice ChairJitkasame Ngarmnil, Mahanakorn University of Technology,

[email protected]

Technical Program ChairTor Sverre Lande, University of Oslo, Norway

[email protected] web page: www.dac.com

CALL FOR PAPERSFOR INFORMATION CALL: 303–530–4333

Anaheim Convention Center, Anaheim, California

June 2–6, 2003DAC is the premier conference devoted to Design Automation (DA) and

the application of DA tools in designing electronic systems. Five types ofsubmissions are invited: regular papers, special topic sessions, panels, tu-torials, and design contest entries. Submissions should be submitted elec-tronically to www.dac.com. Submissions are due NO later than Decem-ber 6, 2002, 5:00 PM MST.

Authors are invited to submit original technical papers describing recentand novel research or engineering developments in all areas of design au-tomation.

All DAC submissions must be made electronically using PDFformat NOlater than December 6, 2002, 5:00 PM MST. Reference the DAC web page(www.dac.com) for instructions on electronic submissions. Please submitone PDF file.

To permit a blind review, do not include name(s) or affiliation(s) ofthe author(s) on the manuscript, abstract or bibliographic citations. Thepapers will be reviewed as finished papers. Preliminary submissionswill be at a disadvantage. Notice of acceptance will be emailed to thecontact person by February 28, 2003.

University Design ContestStudents are invited to submit descriptions of original electronic designs,

either circuit level or system level. These submissions must be electroni-cally submitted NO LATER THAN December 20, 2002 (5:00pm MST).

DeadlinesHands-On Tutorial Proposals January 15, 2003Regular Papers DECEMBER. 6, 2002Student Design Contest DECEMBER 20, 2002

General Chair Conference ManagerIan Getreu Kevin Lepine3450 SW Sherwood Pl. MP Associates, Inc.Portland, OR 97201 5305 Spine Rd., Ste. A(503) 222-7330 Boulder, CO 80301email: [email protected] (303) 530-4562

email: [email protected]

40th DESIGN AUTOMATIONCONFERENCE®

IEEE Circuits MAG SystemsSystems and AZINE · Indonesia (Bali) on October 28–31, 2002 to...

Documents

Transcript of IEEE Circuits MAG SystemsSystems and AZINE · Indonesia (Bali) on October 28–31, 2002 to...