Licensed to -...

Licensed to:

Multimedia Systems: Algorithms, Standards,

and Industry Practices

Parag Havaldar and Gérard Medioni

Executive Editor: Marie Lee

Acquisitions Editor: Amy Jollymore

Senior Product Manager: Alyssa Pratt

Editorial Assistant: Zina Kresin

Marketing Manager: Bryant Chrzan

Content Project Manager: Jennifer Feltri

Technical Editing: Green Pen Quality Assurance

Art Director: Faith Brosnan

Compositor: Integra

Cover Designer: Wing-ip Ngan, Ink design, inc.

Cover Image Credit (left): Digital Vision/Getty Images

(Royalty Free) Image description (left): Video Motif

Image credit (right): Digital Vision/Getty Images

(Royalty Free) Image description (right): Speaker

Image credit: iStockphoto Image descriptions: Urban

Teenagers, Wireless

© 2010 Course Technology, Cengage Learning

ALL RIGHTS RESERVED. No part of this work covered by the

copyright herein may be reproduced, transmitted, stored or used in

any form or by any means graphic, electronic, or mechanical,

including but not limited to photocopying, recording, scanning,

digitizing, taping, Web distribution, information networks, or

information storage and retrieval systems, except as permitted under

Section 107 or 108 of the 1976 United States Copyright Act, without

the prior written permission of the publisher.

For product information and technology assistance, contact us at

Cengage Learning Customer & Sales Support, 1-800-354-9706

For permission to use material from this text or product,

submit all requests online at cengage.com/permissions

Further permissions questions can be emailed to

[email protected]

ISBN-13: 978-1-4188-3594-1

ISBN-10: 1-4188-3594-3

Course Technology

20 Channel Center Street

Boston, MA 02210

USA

Some of the product names and company names used in this book

have been used for identification purposes only and may be

trademarks or registered trademarks of their respective

manufacturers and sellers.

Any fictional data related to persons or companies or URLs used

throughout this book is intended for instructional purposes only. At

the time this book was printed, any such data was fictional and not

belonging to any real persons or companies.

Course Technology, a part of Cengage Learning, reserves the right to

revise this publication and make changes from time to time in its

content without notice.

The programs in this book are for instructional purposes only. They

have been tested with care, but are not guaranteed for any particular

intent beyond educational purposes. The author and the publisher do

not offer any warranties or representations, nor do they accept any

liabilities with respect to the programs.

Cengage Learning is a leading provider of customized learning

solutions with office locations around the globe, including Singapore,

the United Kingdom, Australia, Mexico, Brazil, and Japan. Locate your

local office at: international.cengage.com/region

Cengage Learning products are represented in Canada by Nelson

Education, Ltd.

For your lifelong learning solutions, visit course.cengage.com

Visit our corporate website at cengage.com.

Printed in Canada 1 2 3 4 5 6 7 13 12 11 10 09

Copyright 2009 Cengage Learning, Inc. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part.

Licensed to:

C H A P T E R 1

Introduction to Multimedia—

Past, Present, and Future

The definition of the word multimedia has gone through a large number of evolutionarysteps from the time the concept emerged, to what it signifies today, and it will definitelyevolve into something new tomorrow. Ask a layman or a computer professional aboutthe definition of multimedia and you get answers such as playing computer games,videoconferencing, listening to MP3s, or watching a movie on the Internet. Most ofthese multimedia scenarios are tightly bound to the use of a computer. The truth is,most answers are only partially correct but it is not easy to give the term multimedia aconcrete and accurate definition. You could naively say that, as the name multimediasuggests, it consists of all applications that involve a combined use of different kinds ofmedia, such as text, audio, video, graphics, and animation. A presentation that involvesall these different media types can be termed a multimedia presentation. Software thatinvolves animations, sound, and text is called multimedia software. Also, any systemthat incorporates different flavors of media can be termed as a multimedia system.Based on this initial and informal understanding, let us try to categorize the followingten scenarios or examples as being “multimedia” or “not multimedia.” Give somethought to each and write down your answers (yes or no). Later in the chapter, we willrevisit and analyze these scenarios.

• Watching a Microsoft PowerPoint presentation

• Playing a video game

• Drawing and describing a picture to your friend

• Reading the newspaper in the morning

• Videoconferencing


Licensed to:

2 C H A P T E R 1 • Introduction to Multimedia—Past, Present, and Future

• Watching television or listening to radio

• Going to a movie theater

• Assembling a car in a garage

• Browsing/searching using the Internet

• Having a telephone conversation

In this introductory chapter, we aim to motivate the reader with what multi-media is and how it has, or currently is, changing the world in terms of the way wecommunicate, access information, entertain ourselves, and so on. We start by dis-cussing multimedia from a historical perspective in Section 1. Then, in Section 2, wedefine multimedia information by explaining its inherent properties and its relevantmedia types. In Section 3, we depict what a multimedia system looks like today andcategorize various components that are used. Next, in Section 4 we speak about thetechnological aspects of multimedia systems, the forces that are feeding its revolu-tion, and we convey the importance of industry-established standards. Section 5portrays a few thoughts on how multimedia might continue to shape our future.Finally, in Section 6, we set the stage for the book, how it is organized, how it shouldbe read, and how to approach the exercises.

1 MULTIMEDIA: HISTORICAL PERSPECTIVE

The word multimedia was coined in the beginning of the 1990s. After the success ofthe digital audio recording industry, and the distribution of digital audio in the formof compact discs (CDs), the next anticipated step was to create digital content involv-ing images, text, and video along with audio and distribute it in a similar fashion.Outcomes of this were multimedia CD-ROMs, which included informational contentas well as games. Examples of these include, Encyclopedia Britannica and interactiveCD-ROM games with simple graphics, animations, and audio. These experiences werethen only limited to a single person interacting with the content on a PC computer.But this single person-to-PC experience changed dramatically with the advances indigital networks and digital distribution technologies. In fact, the whole multimediaworld started to deeply alter our ways of communication with the (1) availability oflow-cost capture devices, rendering devices, and smarter software to create content;(2) larger, less expensive storage devices along with research in better compression ofmedia content; and (3) technological advances in digital networks and standardiza-tion of distribution protocols.

The preceding three points directly map to three processes that are now inherentto multimedia systems:

• Multimedia content creation or multimedia authoring—This process involvesdigitizing media (audio, images, video) using capture devices andassembling/processing them using smart software and hardware.

• Storage and compression—Multimedia content created today has significantmemory requirements and has to be engineered so as to minimize necessities for


Licensed to:

Multimedia: Historical Perspective 3

storage and distribution. The process mostly involves state-of-the-artcompression algorithms and standards for audio, video, images, and graphics.

• Distribution—Distribution involves how multimedia content is distributed viavarious media, such as wired cables, optical networks, satellite, wirelessnetworks, or any combination thereof, to specific platforms ranging fromtelevision, computers, personal digital assistants (PDAs), and so on.

This threefold view is not new and has been used for information in general—creating or gathering information, storing or recording it, and distributing it tothe end user. The table shown in Figure 1-1 gives an evolutionary perspective onthe type of information that people have grown accustomed to through the ages, the

Time and Type of Storage Mode of Age era information medium distribution

Prehistoric 15,000 BC Sounds to Rock –communicate, surfaces,gestures, cave wallspainting

Ancient 500 BC Alphabets, Invention of People drawing paper delivering

messages,horseback

Middle Ages 400–1000 AD Letters, Books Beginning ofwriting a postal

system

Renaissance 1300–1800 AD News, Books, Printing paintings, libraries press, steammagazine engines,

automobiles

Modern world 1900 AD Morse code, Film, Telegramradio, magnetic service,photographs, tapes, wirelessmovies phonograph radio waves

Electronic 1950–1980 Telephone, Electronic Radio and TVtelevision, fax, memory, broadcasting,computers cassette tapes, satellite

LP records communication

Digital 1980 to Computers, Hard disks, Ethernet,present day digital video, CD-ROMs, wireless

surround DVDs networks,sound optical

networks

Figure 1-1 A brief evolution of information


Licensed to:


various ways in which information was captured or stored, and the means used todistribute it.

As you go down the table from olden times to recent times, the column showingthe type of information suggests that the informational variety that people havegrown accustomed to has significantly increased. In the olden days, we had hand-written messages and letters or just word of mouth being propagated, but todaypeople are habituated to information that contains video, audio, images, and text.Simultaneously, the speed at which information has been distributed to the end userhas increased. For example, in the event of a war, in olden days it would suffice fora king to know that the battle ended when an emissary reached him carrying a noteto that effect, which could often take two or three days. However, in today’s digitalworld, you want to see video/audio/text coverage of a war taking place in foreignareas. People expect to see this coverage in real time on all kinds of devices, includ-ing computers, television, PDAs, and so on. From this generic trend involvingincreasing quantity of information content, its growing medium of storage, and itsaccelerated distribution, you might wonder what a multimedia system would corre-spond to at a certain time. For example, if you were to learn about multimedia sys-tems in the 1700s, it would entail learning how the printing press worked, how youwould use it to print newspapers, and how efficiently you would distribute thatprinted information to people. Today, however, if you learn about multimedia sys-tems, it entails dealing with digital data, where all the information is digital and dis-tributed using digital networks to end terminals and rendering devices that are alsodigital in nature.

2 MULTIMEDIA DATA AND MULTIMEDIA SYSTEMS

Multimedia information can be defined as information that consists of one or moredifferent media types. This definition, however, is a changing one because media typesthemselves are constantly changing. Today, multimedia information consists of text,audio, video, 2D graphics, and 3D graphics. These are the media types that are usedextensively today because of the availability of devices to capture them, as well ascapabilities of authoring software applications to combine them to produce a variety ofinformational and entertaining content. Other more “futuristic” media types are beingresearched today (and more will be invented tomorrow), but have not yet made it intomainstream multimedia, such as holographs and haptics. Whereas holography dealswith the creation of experiences in 3D, haptics deals with providing feedback andinteractivity using a sense of touch. Thus, this definition of multimedia information isa changing one.

2.1 Inherent Qualities of Multimedia Data

Before we delve into each media type in detail and the way they can be combined toproduce multimedia content, it should be noted that there are certain inherent qualities


Licensed to:

Multimedia Data and Multimedia Systems 5

generic to all media, which, in turn, define its multimedia nature. These qualities areas follows:

• Digital—Multimedia information is always digital.1 In fact, it is the digitalnature of the information that allows it to be combined together (or to keep itsown identity) to produce rich content. Whether it is digital images, video,audio, or text, the underlying representation of the information is always bitsand bytes.

• Voluminous—The size of the data resulting from combining video, audio, andimages together is understandably large and voluminous. This causes problemswhen such high volume data has to be stored, searched, and, worse, when it hasto be transmitted over bandwidths, which might be narrow, wide, and evenvarying. The storage and transmission bandwidth limitations require that thedata be compressed.

• Interactive—Multimedia content can be interacted with from a high-levelapplication point of view, such as choosing a video to watch or a set of imagesto browse down to a low level, where you can click on areas of an image causingan action to be taken. For example, on a Web site consisting of hyperlinkedtext, images, or video, you can read, jump to different Web sites, or browsevideo in any order you want. Another practical example of interactivity is thenavigational capability to jump to chapters as well as browse additional relatedcontent in a DVD experience.

• Real-time and synchronization—When transmitting content involving differentmedia types, real-time requirements and resulting synchronization issues play acrucial role in the system’s architecture and design. Real-time needs imply thatthere can be only a very small and bounded delay while transmitting informationto the end client. Synchronization imposes time-respected rendering of themedia, which might be self-contained or interdependent. For instance, video hasto play at a certain rate (intramedia) while the accompanying sound must matchthe video playback rate (intermedia).

Understanding these properties is core to designing good working multimediaapplications. For example, suppose you want to capture a live football game and trans-mit it over the Internet. There are signal-processing issues that stem from capturing(or converting to) digital media, which directly relate to the quality and quantity ofdata recorded. The quantity of data dictates what type of compression you impose onit to transmit it over a fixed bandwidth network. The real-time transmission require-ments need to take into account the available bandwidth, and network traffic. Furthermore the design and architecture of such a real time, end-to-end system will needbuffering, caching, monitoring data throughput, maintaining synchronization and so

1 It is also possible to talk about multimedia information in an analog form. However, we believe that thedigital nature of multimedia makes it possible to easily combine the different media types and interactwith it to create purposeful content. Hence, for the purpose of this text, we assume multimediainformation to be digital.


Licensed to:


on at the sender and receiver ends. Moreover, this system architecture should be scal-able and able to serve multiple viewers that can connect over varying bandwidths.

2.2 Different Media Types Used Today

As mentioned earlier, the different types of media used to create information is chang-ing. The following sections describing these different types of media are, then, anincomplete taxonomy. These definitions and descriptions for media types are briefand introductory explanations. Detailed explanations of these media types are thesubject matter of Chapters 2 and 3.

2.2.1 Text

“This is a line of text to explain that text does convey information!” Text has beencommonly used to express information not just today but from the early days.Literature, news, and information being archived today and accessed by browsing onthe Internet include a large amount of text. The representation and writing of textinformation has evolved from simple text to more meaningful and easy-to-read for-matted text, using a variety of fonts. Today, hypertext is commonly used in digitaldocuments, allowing nonlinear access to information.

One aspect that needs mention is the role text plays in multimedia. It is very nat-ural to downplay the role textual information plays in the context of multimedia, per-haps because of its simplicity, especially when compression, display, and distributiontechnologies all concentrate on serving the video, audio, and graphical media types.Text has been—and still is—the single most widely used media type to store informa-tion, and it has been attributed with aspects of revolutionizing society similar to whatmultimedia is doing today. You might draw an analogy between the beginning of thedigital era, where computer experience was limited to a single person-to-PC situation,and the time when text was contained in handwritten notes and books were kept atspecific places, such as libraries and monasteries. The invention of the printing pressrevolutionized this limited access by making it possible to easily duplicate or printtext and send it to various people in different regions, similar to digital duplicationand distribution via digital networks today. The printing press opened the way forsmaller and more portable texts, lowered the cost of books, and encouraged a greatsurge in literacy. The resulting ease of access to information globalized Europe in the1700s, changed people’s social and political ways, and ultimately led to the industrialrevolution. In addition, the text printing and typography industry invented auto-mated ways to efficiently duplicate and distribute printed information. It was the firstautomated process that spawned methodical step-by-step processes, which became ablueprint of all automation that followed—from the creation of the assembly line forproduct manufacturing to the digital world of disseminating information.

2.2.2 Images

Images consist of a set of units called pixels organized in the form of a two-dimensional array. The two dimensions specify the width and height of the images.Each pixel has bit depth, which defines how many bits are used to represent an image.


Licensed to:

Multimedia Data and Multimedia Systems 7

There are various kinds of images, which can be characterized into groups dependingon the following:

• Bit depth—Bit depth represents the number of bits assigned to each pixel.Accordingly, images are categorized by the bit depth as binary images whereevery pixel is represented by one bit or gray-level images where every pixel isrepresented by a number of bits (typically 8) or color images, where each pixelis represented by three color channels.

• Formats—Formats are application-specific, for example, faxes are also imagesthat have a format different from digital photographs.

• Dimensionality—Images can be enjoyed singularly or combined in a variety ofways. Stereo images are commonly used for depth-perception effects. Images canalso be stitched together to form mosaics and panoramas.

2.2.3 Video

Video is represented as a sequence of images. Each image in the sequence typicallyhas the same properties of width, height, and pixel depth. All of these parameters canbe termed as spatial parameters. Additionally, there is one more temporal parameterknown as frames per second or fps. This parameter describes how fast the images needto be shown per second for the user to perceive continuous motion. Apart from thisbasic definition, video can be classified depending on the following:

• Aspect ratio—A common aspect ratio for video is 4:3, which defines the ratio ofthe width to height. This has been the adopted standard for the major part ofthe last century. Today, however, we have a variety of different aspect ratios forhigh definition, cinemascope, and so on.

• Scanning format—Scanning helps convert the frames of video into a one-dimensional signal for broadcast. The interlaced scanning format was inventedto make television work in the middle of the last century. Today, in the digitalworld, display devices can support progressive scanning and provide betterquality for visual information.

2.2.4 Audio

Digital audio is characterized by a sampling rate in hertz, which gives the number ofsamples per second. A sample can be defined as an individual unit of audio information.Each sample also has a size, the sample size, which typically is anywhere from 8-bits to16-bits depending on the application. Apart from these properties, audio is alsodescribed by:

• Dimensionality—The dimensions of an audio signal signify the number of channelsthat are contained in the signal. These may be mono (one channel), stereo (twochannels), which is by far the most common. Recent standards also use surroundsound which consists of many channels, for example 5.1 surround sound systemshave one low frequency speaker and five spatially-located speakers.

• Frequency Range—Audio signals are also described by the frequency range orfrequency band that they contain. For example, audio voice signals are referred


Licensed to:


to as narrow band because they contain lower frequency content. Music isnormally referred to as wide band.

2.2.5 2D Graphics

2D graphical elements have become commonplace in multimedia presentations toenhance the message to be conveyed. A 2D graphic element is represented by 2Dvector coordinates and normally has properties such as a fill color, boundary thick-ness, and so on. Additionally, 2D graphical elements are effectively used to create 2Danimations to better illustrate information.

2.2.6 3D Graphics

3D graphics are primarily used today for high-end content in movies, computer games,and advertising. Like 2D graphics, 3D graphics largely make use of vector coordinatespaces. 3D graphics concepts and practices have advanced considerably as a science but,until recently, were not a commonplace media type. This is now changing with afford-able consumer-level software, scanning devices, and powerful graphics cards nowbecoming available.

2.3 Classification of Multimedia Systems

A multimedia system, defined end to end, is a system that takes care of all contentcreation, storage, and distribution issues to various platforms. Depending on theapplication, multimedia systems can be classified in a variety of ways, such as interac-tion style, the number of users interacting, when the content is live, and so on. A fewcommon classifications are discussed in the following list:

• Static versus dynamic—This differentiation, although rarely used, refers to caseswhen the multimedia data remains the same within a certain finite time, for example,one slide of a Microsoft PowerPoint presentation or one HTML Web page. Comparethis with the dynamic case when the data is changing, for example watching a video.

• Real-time versus orchestrated—This is a more common classification.Orchestrated refers to cases when there is no real-time requirement. Forexample, compressing content on a DVD and distributing it has no real-timerequirement. The most important constraint here is the quality of thecompressed data. However, showing a game in a live broadcast over the Internetimposes a whole new set of engineering constraints in addition to compressionquality, which relate to on-time data delivery and synchronization.

• Linear versus nonlinear—Here, the method of interaction with the multimediadata is used to differentiate the system. In a linear system, you would proceedlinearly through the information, for example reading an eBook or watching avideo. However, if you want to interact with the data in a nonlinear fashion,you would have to make use of links that map one part of the data to another.A well-known example of this is hypertext. You could extend this analogy fromtext to other media types—images, video, and audio. The term hypermediageneralizes the concept of accessing media nonlinearly.


Licensed to:

A Multimedia System Today 9

• Person-to-machine versus person-to-person—In this case, the classification is based on whether the end user is interacting with a machine or with another person. For example, playing a CD-ROM game is a simple person-to-machine experience. However, videoconferencing is a person-to-personexperience.

• Single user, peer-to-peer, peer-to-multipeer, and broadcast—Here, the manner of information distribution is used as a means to classify a multimedia system.You might have a single-user scenario such as browsing the Web, or it could be a peer-to-peer scenario when the information is exchanged from oneperson/computer to another, for example two friends instant messaging over the Internet. A peer-to-multipeer scenario extends the paradigm to sendingmessages to a limited set of intended viewers such as in a chat room.Broadcasting is the most general-purpose scenario, where information is sentnot to any specific listener(s) but available to all those who want to listen, suchas television and radio broadcasts.

3 A MULTIMEDIA SYSTEM TODAY

Multimedia systems can be logically grouped into three parts whose primary func-tionalities are (1) content production, (2) compression and storage, and (3) distribu-tion to various end users and platforms. The multimedia experience today hastranscended a simplistic one person-to-PC scenario to become a very sophisticatedone, which involves a distributed and collaborative medium. This has been madepossible because of sophisticated, inexpensive devices for capturing and renderingcontent, as well as smarter software to create content and the availability of increas-ing digital bandwidth. A typical end-to-end multimedia system today has beengraphically depicted in Figure 1-2. It consists of three logical sections, which asexplained earlier, correspond to content creation, compression, and distribution.

The content creation section shows a variety of different instruments, whichcapture different media types in a digital format. These include digital cameras,camcorders or video cameras, sound recording devices, scanners to scan images, and3D graphical objects. Once the individual media elements are in their digital represen-tations, they may be further combined to create coherent, interactive presentationsusing software (S/W) applications and hardware (H/W) elements. This content can bestored to disk, or in the case of real-time applications, the content can be sent directlyto the end user via digital networks.

The second section deals with the compression of multimedia content. This entailsthe use of various compression technologies to compress video, audio, graphics, and soon. Shown in the Figure 1-2 are hardware and software elements, such as mediaencoders and storage devices.

The last section deals with media distribution across a variety of low-bandwidthand high-bandwidth networks. This ranges from cellular, to wireless networks, tocable, to digital subscriber line (DSL), to satellite networks. Distribution normallyfollows standards protocols, which are responsible for collating and reliably sending


Licensed to:


information to end receivers. The commonly used end receivers are computers, televi-sions, set-top boxes, cell phones, or even more application- or entertainment-specificitems, such as video game consoles.

Now that we know what a multimedia system looks like, and what its generalrequirements are, let us revisit the ten examples from the beginning of this chapter,and analyze them. In each case, we provide a percentage number that roughly corre-sponds to the way our students answered on the first day of class. The class includedentry level graduate students in computer science and electrical engineering.

1. A PowerPoint presentation. Yes—95%. Of course, PowerPoint presentationsinvolve all types of media and have tools to make it interactive.

2. Playing a video game. Yes—100%. Video games are inherently interactive.

3. Describing a picture to your friend. Yes—10%. What if you are in a chat roomwhere the picture makes up the backdrop and you and your friend talkinteractively?

4. Reading the newspaper in the morning. Yes—20%. What if you were readingwww.latimes.com?

5. Videoconferencing. Yes—100%. Almost all students unanimously agreed thatvideo conferencing is multimedia because video conferencing is considered tobe one of first digital multimedia applications.

6. Watching television or listening to radio. Yes—80%. Most said yes becauseTV comprises audio and video, with channel surfing as interactivity.

Computer

Game ConsoleWireless

DSL Modem

Cable Television

Wi-Fi - HotspotsSatellite

Distributionvia

networks

Mediaserver

Compression andmedia encoding

Multimediacontent creation

MediaAcquisition

Storage

S/W & H/WAssembly

AdvertisingDigital RightsMangement

Subscribermanagement

Watermarking

Encryption

Cell Phone PDA

Figure 1-2 Components of a multimedia system today


Licensed to:

The Multimedia Revolution 11

However, the multimedia experience becomes clear when you experiencedigital transmission over cable networks with a DVR (Digital VideoRecording; for example, TiVo) that allows you to nonlinearly choose andwatch what you want.

7. Going to a movie theater. Yes—90%. Again, most of the students agreed to thisbeing multimedia because movies today entail digital video and audio.

8. Assembling a car in a garage. Yes—0%. Almost all said no. What if the garage isa metaphor for a “3D-room” in an application where designers from differentgeographic locations get together virtually to assemble 3D car parts?

9. Browsing/searching using the Internet. Yes—100%. Surfing the Internet toread, watch videos, listen to music are applications that involve different media.

10. Having a telephone conversation. Yes—60%. What if you were making use ofVoice over IP?

The truth is that all of these scenarios can be defined as multimedia experi-ences, depending on the devices used, the interactivity involved, and the mediumof delivery. One commonality of all these experiences is that they are digital and theend user could always interact with the content or the other end user.

4 THE MULTIMEDIA REVOLUTION

The creation of multimedia information and the ability to exchange it among variousdevices has clearly created conveniences in our lives. Two decades ago, it was hard tofathom a wireless telephone. Today, whether it is using a cell phone to make a callfrom any place, or browsing the Internet for information for which you previouslyhad to drive down to a library, or watching live video and audio coverage of an eventhalfway across the world on your mobile laptop, the rich content of information alongwith its mobility across various devices has revolutionized our habits of creating,exchanging, browsing, and interacting with information. It is difficult to quantifyone definite reason for this revolution to have happened globally but we can defi-nitely attribute a few causes for it to have taken place:

• Digitization of virtually any and every device—Today, you have digital cameras,camcorders, sound recorders that make good-quality digital media available forprocessing and exchange. At the same time, digital displays such as high-performance cathode ray tubes (CRTs), liquid crystal displays, plasma screens,and so on allow us to view information at good resolutions.

• Digitization of libraries of information—Virtually all libraries, whether general-purpose or specific, are making their way to be digital.

• Evolution of communication and data networks—The research in digital networksand networking protocols have made it possible to exchange huge amounts ofdata over wired, optical, and wireless mediums. Deployments in this area aremaking availability of bandwidth on demand.


Licensed to:


• New algorithms for compression—Because multimedia information is veryvoluminous, abilities to compress information prior to sending it over networksallow us to engineer applications that perform in real time and with a highfidelity.

• Better hardware performance—Microprocessors, along with graphical processingunits (GPU) are both getting faster and perform better. Also, large capacitystorage devices are becoming common place now, not just with computers butalso other hardware devices such as digital cameras, camcorders and so on.

• Smarter user interface paradigms to view/interact with multimedia information ona variety of terminals—As personal communication devices get compact andsmaller in size, the role of user interfaces becomes important when it is expectedfor them to have information access capabilities similar to a computers. Userinterface designs based on touch screens are now playing an increasing role inhow we access information on our cell phones, PDAs and so on.

Although this list should not be considered comprehensive, it does show a definitetrend, that of increasing our capabilities and capacities to gain ubiquitous access toinformation. In addition to this, industrial companies along with research/academiccommunities have formed international bodies to regularly set standards. These stan-dards have gone a long way toward allowing us to experience information exchangeon a variety of platforms. Standards bodies such as the International Organization forStandardization (ISO) and the International Telecommunication Union (ITU) are prima-rily responsible for ratifying specifications that the industry uses for interoperabilityand exchange of information. Within these standards bodies there are groups and com-mittees responsible for each of the different media types. Examples of such committeesthat have set forth standards for digital media are Joint Pictures Expert Group (JPEG)for images, Motion Pictures Expert Group (MPEG) for video, DVD standards for audio-visual content, Open Systems Interconnection (OSI) for networking, SynchronizedMultimedia Integration Language (SMIL), Virtual Reality Modeling Language (VRML)for graphics and so on. There is a reason why you can buy any DVD content created bya movie studio such as Warner Brothers or Universal Pictures; insert it into any DVDplayer manufactured by Panasonic, SONY, and so on; view the video signal on anyHDTV or standard definition TV manufactured by Samsung, RCA, and so on; andenjoy the sound experience on a surround sound system by BOSE or Yamaha. This ispossible because everyone adheres to the set standards that allow for interoperabilityand information exchange.

One aspect previously mentioned that needs more elaboration is the role that userinterface paradigms have played in the enabling of our access to and interaction withmedia information. User interface paradigms have always encouraged users to explorethe range of features that a multimedia application offers. The ease and conveniencewith which any user can access information, manipulate it, and interact with it hasalways been defined by the interface that allows the user to do so. Graphical userinterface paradigms such as buttons, drop-down lists, context-sensitive menus,and other spatial/temporal browsing metaphors on computers have been around forsome time now. These metaphors have become more critical today as the devices foraccess and interaction become more portable, such as cell phones, PDAs, kiosks, and


Licensed to:

A Possible Future 13

so on, where the device’s capabilities are far less compared with that of a traditionaldesktop computer. Good examples of current consumer devices today where the roleof the user interface has been revisited and rethought are Apple’s iPhone and Google’sG1 phone. Although both boast of a variety of features, they both have moved towardthe use of touch-sensitive screens to have simple but efficient metaphors to browsethrough and select multimedia information.

5 A POSSIBLE FUTURE

Information has existed in a digital form for more than a decade. The distributednature of closed and open networks, including the massive Internet, has made thisdigital information available on a very broad scale. With proper network access proto-cols, it is now possible to post, access, analyze, search, view, copy, and own digitalinformation from virtually any place without any regard to geopolitical boundaries.Additionally, with the digitization of devices, you do not necessarily need a computerto access information—smaller, smarter devices such as cell phones, PDAs, and so oncan do the job just as well. This means that as long as there is network access, virtuallyanyone—person, group, or organization—located physically anywhere can post infor-mation and practically anyone can use it. This digital phase is an ongoing reality inmost of the developed nations and will very soon be all over the world.

The digital change has already affected people’s everyday life and will do so moreeffectively in different walks of life. This is seen in the various digital modes in whichpeople primarily communicate, such as cell phones, e-mail, instant messaging, blogs,sharing documents, voice and videoconferencing, chat rooms, social networking sites,and other more interesting avenues yet to come. Aspects of this and improvementsthere upon will naturally get imbibed into different applications such as distance learn-ing, media entertainment, health care, commerce, defense/military, advertising, and soon. The marriage of Global Positioning Systems (GPS) and communication devices hasmade it possible to add a new dimension to information analysis. Although the initialuses of GPS were restricted to military applications and commercial aircrafts to improveand automate navigational accuracy, systems are now in place for consumer-levelcommerce. Novel applications include systems to aid farmers in fertilizer distributionover less-fertile regions; tracking people, vehicles, and cargo; consumer vehicle naviga-tion; and plotting road routes based on traffic, road, closures, and so on.

Although the application areas and communication improvements seem endless,a few common hurdles might need to be solved to make the suggested media andinformation exchange technologies usable, viable, and commercially practical. Thefollowing paragraphs mention a few of these hurdles.

First, there will be much-needed applications that can search quickly throughthis digital myriad of information. Currently, a number of search engines efficientlysearch Web pages on the Internet (for example, Google), but all of them are limitedto searching text or text-annotated media objects. There will be a need to search,index, and browse through media information such as images, audio, and video.This media information might also be hyperlinked to other media objects, creatinghypermedia.


Licensed to:


Searching and browsing abilities will not be enough. With the enormous amountof information available for any topic, it is improbable not to be in an “informationoverload” state. One common example is the amount of e-mail that you need to siftthrough on a daily basis. Also, more important, when you have a purposeful searchabout a topic, you get a multitude of information, most of which might not be relevantor even valid, and definitely not easy to efficiently sift through with all the hyper-links. Current research, which could unfold into a practical future, involves the use ofartificial intelligence to create autonomous agents and software robots whose task is toorganize information for the user. The set of tasks or applications an agent can assistwith is virtually unlimited: information filtering; information retrieval; mail manage-ment; meeting scheduling; selection of books, movies, and music; and so forth.

With the availability of information also comes the need to have specific, limited,and restricted access to it. This is another area, which will need to play an effective rolein the future—digital rights management or DRM. DRM refers to protecting the owner-ship/copyright of digital content by restricting what actions an authorized recipientmay take in regard to that content. DRM is a fairly nascent field with implementationlimited to specific areas/businesses, most notably with the distribution of movies viaDVDs, perhaps because of the large revenue streams and ease of duplication that go withdigital movies. But as media and text information become customarily distributed vianetworking, a variety of businesses in publishing, health, finance, music, and movieswill need standard ways to control and authenticate digital information.

6 MAP OF THIS BOOK

This book has been written with one objective in mind: to educate students with thetheory and industry practices that are increasingly used to create real applicationsinvolving multimedia content. To achieve this goal, we have divided the book intofour parts; they are related and should be read sequentially, but depending on thecomfort level with each section, you could also read them independently. The chap-ters in each part provide a description of the fundamental aspects, illustrative exam-ples, and a set of comprehensive exercises, both in theory and programming.

The first part deals with relevant aspects of the creation of multimedia content,which includes capturing media into digital form and related signal-processing issues.It discusses representational issues of each media type as well as various formats thatare prevalently used to store each media type. It also presents color theory and how itrelates to creating display devices from monitors, televisions, and printers. Also,selective techniques and algorithms widely used to combine various media types tocreate multimedia content are discussed. These include image processing techniquesto enhance images, chroma-keying and compositing for video, simple audio filtering,creating graphical animations, and so forth. This section also touches on the impor-tance of user interfaces to interact with multimedia content with a few important userinterface paradigms.

The second part of the book analyzes the quantity of multimedia information anddiscusses issues related to compression and storage. This section starts with formalanalysis of information representation, the theoretical limits of information, leading to


Licensed to:

How to Approach the Exercises 15

information compression. We give a taxonomy of generic lossless and lossy encodingalgorithms. We show how these generic techniques are specifically applied to eachmedia domain—text, images (DCT for JPEG, wavelets for JPEG2000), video (motioncompensation for MPEG), audio (MP3, Dolby AC3), and graphics (TopologicalSurgery). We also discuss a variety of standards established around compression andthe different issues related to storage of media.

The third part deals with architectures and protocols used for the distribution ofmultimedia information. It addresses and analyzes distribution-related issues such asmedium access protocols, unicast versus multicast, constant bit rate traffic, and mediastreaming. It also formalizes Quality of Service (QoS) issues for different media andexplains how they are controlled using flow control, congestion control, and latencyissues. Standards used for non-real-time and real-time media distribution are alsodiscussed—TCP/IP, UDP, HTTP, RTP, RTSP. We also discuss wireless access protocols(WAP) implemented on the Global System for Mobile (GSM) communications as well asthe current generation G3 networks. We also discuss issues and solutions that need tobe addressed to make the next generation G4 networks a practical reality. This sectionalso addresses the design requirements for end-to-end architectures for commerciallydeployed applications, such as video on demand, wireless content distribution, GPSwith media, and so on. Also explained here are security issues related to distribution,which involves digital watermarking and media encryption.

The last section deals with recent trends in multimedia, including a discussion ofreal-world applications and standards for multimedia. Among the topics elucidatedhere are the latest MPEG-4 standard and multimedia frameworks using the emergingMPEG-21 standard. The section also discusses issues related to multimedia databasesand the use of MPEG-7. Finally, this section concludes describing many industrydeployments that have been spawned out of this theory and technology. Examples ofsuch deployments include HDTV, DVD, HD-DVD, Blu-ray computer game engines andgame content, special effects for movies, Wi-Fi hot spots, and so on.

7 HOW TO APPROACH THE EXERCISES

At the end of each chapter, we provide comprehensive exercises both in theory andprogramming. Each question is rated by a number from 1 to 10. This number relates tothe difficulty level for that question—1 being very easy, requiring only a fewmoments to solve the question, and 10 being hard enough to involve an entire week-end. Solutions to the exercises are also available in instructional copies. Also, wepropose programming exercises, which are really projects, and we do provide a goodcode base to get started in the form of skeletal frameworks. These are written in C��,and run on both Microsoft Windows and Linux environments. All source code isavailable under the Student Downloads section of www.cengage.com.


Licensed to:

Licensed to -...

Documents

Transcript of Licensed to -...