0055 Introduction to Multimedia Systems Notes

8/12/2019 0055 Introduction to Multimedia Systems Notes

1/25

Introduction to Multimedia About This Course

o Aims of Moduleo Objectives of Moduleo Syllabus Outlineo Recommended Course Books

Introductiono History of Multimedia Systemso Multimedia Hy!ermedia

"hat is Multimedia#"hat is Hy!erTe$t and Hy!erMedia#

o Multimedia SystemsCharacteristics of a Multimedia SystemChallen%es for Multimedia Systems

&esirable 'eatures for a Multimedia SystemCom!onents of a Multimedia System

o A!!licationso Trends in Multimediao 'urther Readin% ($!loration

Aims of ModuleTo %ive students a broad %roundin% in issues surroundin% multimedia) includin% the role of anddesi%n of multimedia Systems *hich incor!orate di%ital audio) %ra!hics and video) underlyin%

conce!ts and re!resentations of sound) !ictures and video) data com!ression and transmission)inte%ration of media) multimedia authorin%) and delivery of multimedia+

Objectives of ModuleStudents should be able to:

,nderstand the relevance and underlyin% infrastructure of the multimedia systems+ ,nderstand core multimedia technolo%ies and standards -&i%ital Audio) .ra!hics) /ideo)

/R) data transmission com!ression0 Be a*are of factors involved in multimedia systems !erformance) inte%ration and

evaluation

1
http://www.cs.cf.ac.uk/Dave/Multimedia/node2.html#SECTION01100000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node3.html#SECTION01110000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node4.html#SECTION01120000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node5.html#SECTION01130000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node6.html#SECTION01140000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node7.html#SECTION01200000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node8.html#SECTION01210000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node9.html#SECTION01220000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node10.html#SECTION01221000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node11.html#SECTION01222000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node12.html#SECTION01230000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node13.html#SECTION01231000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node14.html#SECTION01232000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node15.html#SECTION01233000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node16.html#SECTION01234000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node17.html#SECTION01240000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node18.html#SECTION01250000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node19.html#SECTION01260000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node3.html#SECTION01110000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node4.html#SECTION01120000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node5.html#SECTION01130000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node6.html#SECTION01140000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node7.html#SECTION01200000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node8.html#SECTION01210000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node9.html#SECTION01220000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node10.html#SECTION01221000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node11.html#SECTION01222000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node12.html#SECTION01230000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node13.html#SECTION01231000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node14.html#SECTION01232000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node15.html#SECTION01233000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node16.html#SECTION01234000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node17.html#SECTION01240000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node18.html#SECTION01250000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node19.html#SECTION01260000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node2.html#SECTION01100000000000000000


2/25

Syllabus Outline To!ics in the module include the follo*in%2

1. Introduction 2 Multimedia a!!lications and re3uirements -e+%+) overvie* ofmultimedia systems) video4on4demand) interactive television) video conferencin%)hy!ermedia course*are) %rou!*are) "orld "ide "eb) and di%ital libraries0+2. Audio/Video fundamentals includin% analo% and di%ital re!resentations) human

!erce!tion) and audio video e3ui!ment) a!!lications+. Audio and video com!ression includin% !erce!tual transform coders for

ima%es video -e+%+) 56(.) M6(.) H+789) etc+0) scalable coders -e+%+) !yramid coders0)and !erce!tual audio encoders+ A!!lication and !erformance com!arison of variouscodin% al%orithms includin% hard*are soft*are trade4offs+ Ima%e and video !rocessin%a!!lications and al%orithms+". Multimedia #ro$rammin$ %rame&or's 2 5ava for :uicktime) 5ava Media

'rame*ork

7


3/25

1. Introduction History of Multimedia Systems Multimedia Hy!ermedia

o

"hat is Multimedia#o "hat is Hy!erTe$t and Hy!erMedia#

Multimedia Systemso Characteristics of a Multimedia Systemo Challen%es for Multimedia Systemso &esirable 'eatures for a Multimedia Systemo Com!onents of a Multimedia System

A!!lications Trends in Multimedia 'urther Readin% ($!loration

(istory of Multimedia Systems ;e*s!a!ers *ere !erha!s the first mass communication medium to em!loy Multimedia 44 theyused mostly te$t) %ra!hics) and ima%es+

In 1) .u%liemo Marconi sent his first *ireless radio transmission at 6ontecchio) Italy+ A fe*years later -in 1=?10 he detected radio *aves beamed across the Atlantic+ Initially invented fortele%ra!h) radio is no* a major medium for audio broadcastin%+

Television *as the ne* media for the 7?th century+ It brin%s the video and has since chan%ed the*orld of mass communications+

Some of the im!ortant events in relation to Multimedia in Com!utin% include2 1=@> 4 Bush *rote about Meme$ 1=8 4 ;e%ro!onte formed the Architecture Machine .rou! at MIT 1=8= 4 ;elson /an &am hy!erte$t editor at Bro*n Birth of The Internet 1= 1 4 (mail 1= 8 4 Architecture Machine .rou! !ro!osal to &AR6A2 Multi!le Media 1=


4/25

Multimedia/(y!ermedia "hat is Multimedia# "hat is Hy!erTe$t and Hy!erMedia#

)hat is Multimedia*

Multimedia can have many definitions+ These include2

Multimedia means) com!uter information that can be re!resented throu%h audio) video) andanimation in addition to traditional media -i+e+) te$t) %ra!hics dra*in%s) ima%es0+

A %ood %eneral definition is2

Multimedia is the field concerned *ith the com!uter+controlled inte%ration of te$t) %ra!hics)dra*in%s) still and movin% ima%es -/ideo0) animation) audio) and any other media *here everyty!e of information can be re!resented) stored) transmitted and !rocessed di%itally+

A Multimedia Application is an a!!lication *hich uses a collection of multi!le media sourcese+%+ te$t) %ra!hics) ima%es) sound audio) animation and or video+

(y!ermedia can be considered as one of the multimedia a!!lications+

)hat is (y!er,e-t and (y!erMedia*

Hypertext is a te$t *hich contains links to other te$ts+ The term *as invented by Ted ;elsonaround 1=8>+ Hy!erte$t is therefore usually non4linear -as indicated belo*0+

efinition of (y!erte-t HyperMedia is not constrained to be te$t4based+ It can include other media) e+%+) %ra!hics)ima%es) and es!ecially the continuous media 4 sound and video+ A!!arently) Ted ;elson *asalso the first to use this term+

@
http://www.cs.cf.ac.uk/Dave/Multimedia/node10.html#SECTION01221000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node11.html#SECTION01222000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/htdefn.gifhttp://www.cs.cf.ac.uk/Dave/Multimedia/hypertext.gifhttp://www.cs.cf.ac.uk/Dave/Multimedia/node10.html#SECTION01221000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node11.html#SECTION01222000000000000000


5/25

efinition of (y!erMedia The "orld "ide "eb -"""0 is the best e$am!le of hy!ermedia a!!lications+

-am!les (y!ermedia A!!lications The "orld "ide "eb -"""0 is the best e$am!le of a hy!ermedia a!!lication 6o*er!oint Adobe Acrobat Many Others#

Multimedia SystemsA Multimedia System is a system ca!able of !rocessin% multimedia data and a!!lications+ AMultimedia System is characterised by the !rocessin%) stora%e) %eneration) mani!ulation andrendition of Multimedia information+

Characteristics of a Multimedia System Challen%es for Multimedia Systems &esirable 'eatures for a Multimedia System Com!onents of a Multimedia System

0haracteristics of a Multimedia SystemA Multimedia system has four basic characteristics2

Multimedia systems must be computer controlled . Multimedia systems are integrated + The information they handle must be re!resented digitally + The interface to the final !resentation of media is usually interactive +

0hallen$es for Multimedia Systems istributed et&or's

Su!!ortin% multimedia a!!lications over a com!uter net*ork renders the a!!lication distributed +This *ill involve many s!ecial com!utin% techni3ues+

,em!oral relationshi! bet&een dataMultimedia systems may have to render a variety of media at the same instant 4 a distinction fromnormal a!!lications+ There is a tem!oral relationshi! bet*een many forms of media - e.g. /ideoand Audio+ There t*o are forms of !roblems here24

Render different data at same time E continuously+

>
http://www.cs.cf.ac.uk/Dave/Multimedia/node13.html#SECTION01231000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node14.html#SECTION01232000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node15.html#SECTION01233000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node16.html#SECTION01234000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/hypermedia.gifhttp://www.cs.cf.ac.uk/Dave/Multimedia/node13.html#SECTION01231000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node14.html#SECTION01232000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node15.html#SECTION01233000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node16.html#SECTION01234000000000000000


6/25

Sequencing *ithin the media 4 playing frames in correct order/time frame in video Synchronization 4 inter4media schedulin% - e.g. /ideo and Audio0+ i! synchronisation isclearly im!ortant for humans to *atch !layback of video and audio and even animationand audio+ (ver tried *atchin% an out of -li!0 sync film for a lon% time#

,he 'ey issues multimedia systems need to deal &ith here are:

Ho* to re!resent and store tem!oral information+ Ho* to strictly maintain the tem!oral relationshi!s on !lay back retrieval "hat !rocesses are involved in the above+ &ata has to be re!resented di%itally F Analo% to &i%ital) Conversion) and Sam!lin% etc+ ar%e &ata Re3uirements 4 band*idth) stora%e+ Therefore &ata com!ression is usually mandatory

&ata has to re!resented digitally + So many initial source of data needs to be digitised 4 translated fromanalo$ source to di$ital re!resentation usin% the !rocess called Analo%4to &i%ital -A &0 conversion+ TheA & soft*are hard*are are called A/ converters +

These *ill involve scanning -%ra!hics) still ima%es0) sampling -audio video0 althou%h di%ital camerasno* e$ist for direct scene to di%ital ca!ture of ima%es and video+

The di%ital data is large -several Mb for audio and video0 4 therefore storage , transfer -band*idth0 and processing overheads are high + Conse3uently data compression techni3ues are necessary and common+

esirable %eatures for a Multimedia System.iven the above challen%es the follo*in% features are desirable -if not a !rere3uisite0 for a MultimediaSystem2

i. Very (i$h #rocessin$ #o&er

;eeded to deal *ith lar%e data !rocessin% and real time delivery of media+ S!ecial hard*are iscommon!lace+

ii. Multimedia 0a!able %ile System ;eeded to deliver real4time media 4 e.g. /ideo Audio Streamin%+ S!ecial Hard*are Soft*areneeded e.g RAI& technolo%y+

iii. ata 3e!resentations/%ile %ormats that su!!ort multimedia &ata re!resentations file formats should be easy to handle yet allo* forcom!ression decom!ression in real4time+

iv. fficient and (i$h I/O In!ut and out!ut to the file subsystem needs to be efficient and fast+ ;eeds to allo* for real4timerecordin% as *ell as !layback of data+ e.g. &irect to &isk recordin% systems+

v. S!ecial O!eratin$ System To allo* access to file system and !rocess data efficiently and 3uickly+ ;eeds to su!!ort directtransfers to disk) real4time schedulin%) fast interru!t !rocessin%) I O streamin% etc.

vi. Stora$e and Memory ar%e stora%e units -of the order of >?41?? .B or more0 and lar%e memory ->? 41?? Mb or more0+ar%e Caches also re3uired and fre3uently of evel 7 and 9 hierarchy for efficient mana%ement+

vii. et&or' Su!!ort

8


7/25

Client4server systems as *ell as distributed systems common+viii. Soft&are ,ools

,ser friendly tools needed to handle media) desi%n and develo! a!!lications) deliver media+

0om!onents of a Multimedia System

;o* let us consider the com!onents - (ard&are and Soft&are 0 re3uired for a multimedia system2i. 0a!ture/0a!turin$ devices

/ideo Camera) /ideo Recorder) Audio Micro!hone) Deyboards) mice) %ra!hics tablets) 9& in!utdevices) tactile sensors) /R devices+ &i%itisin% Sam!lin% Hard*are

ii. Stora$e evices Hard disks) C&4ROMs) 5aG i! drives) &/&) ,SB flash &isks etc

iii. 0ommunication et&or's (thernet) Token Rin%) '&&I) ATM) Intranets) Internets+

iv. 0om!uter Systems Multimedia &eskto! machines) "orkstations) M6(. /I&(O &S6 Hard*are

v. is!lay evices C&43uality s!eakers) H&T/) S/.A) Hi4Res monitors) Colour !rinters etc.

A!!lications of Multimedia Systems($am!les of Multimedia A!!lications include2

"orld "ide "eb Hy!ermedia course*are /ideo conferencin% /ideo4on4demand -/o&0 Interactive T/ .rou!*are

Home sho!!in% .ames /irtual reality &i%ital video editin% and !roduction systems Multimedia &atabase systems

,rends in MultimediaCurrent bi% a!!lications areas in Multimedia include2)orld )ide )eb

4 Hy!ermedia systems 4 embrace nearly all multimedia technolo%ies and a!!lication areas+ (verincreasin% !o!ularity+

M4one 4 Multicast 4ac'bone 2 (3uivalent of conventional T/ and Radio on the Internet+

nablin$ ,echnolo$ies 4 &evelo!in% at a ra!id rate to su!!ort ever increasin% need for Multimedia+ Carrier) S*itchin%)

6rotocol) A!!lication) Codin% Com!ression) &atabase) 6rocessin%) and System Inte%rationTechnolo%ies at the forefront of this+


8/25

5lobal structure of Multimedia System:-a0 A!!lication domain-b0 System domain-c0 &evice domain

A!!lication domain6rovides functions to the user to develo! and !resent multimedia !rojects+ This includes soft*aretools) and multimedia !rojects develo!ment methodolo%y+

System omainIncludin% all su!!orts for usin% the function of the device domain) e+%+ o!eratin% system)communication systems -net*orkin%0 and database systems+

evice domainBasic conce!ts and skill for !rocessin% various multimedia elements and for handlin% !hysicaldevice+

2. Multimedia ,echnolo$yMultimedia Systems Technolo%y

&iscrete v continuous media Analo% and &i%ital Si%nals In!ut &evices and Stora%e

o Te$t and Static &atao .ra!hicso Ima%es

o Audioo /ideo

Out!ut &evices Stora%e Media

o Hi%h !erformance I Oo Basic Stora%eo RAI& 44 Redundant Array of Ine$!ensive &iskso O!tical Stora%eo C& Stora%e

C& Standardso

&/& "hat are the features of &/&4/ideo#:uality of &/&4/ideo"hat are the disadvanta%es of &/Com!atibility of &/&SiGes and ca!acities of &/&&/& video details&/& audio


9/25

Interactive &/& featureso &/& and com!uters

'urther Information

2.1 Multimedia Systems ,echnolo$y

2.1.1 iscrete v continuous mediaMultimedia systems deal *ith the %eneration) mani!ulation) stora%e) !resentation) and communication ofinformation in di%ital form+ The data may be in a variety of formats2 te$t) %ra!hics) ima%es) audio) andvideo+

Majority of this data is lar%e and the different media may need synchronisation F the data may havetem!oral relationshi!s as an inte%ral !ro!erty+

Some media is time independent or static or discrete media2 e+%+ normal data) te$t) sin%le ima%es)%ra!hics are e$am!les+

/ideo) animation and audio are e$am!les of continuous media+

2.1.2 Analo$ and i$ital Si$nalsThere are mechanisms and issues involved in transformin% si%nals from analo%4di%ital+Here *e *ill introduce some basic definitions before discussin% the technolo%y re3uired to !erform suchtasks+

The *orld *e sense is full of analog si%nals+ (lectrical sensors such as transducers) thermocou!les)micro!hones convert the medium they sense into electrical si%nals+ These are usually continuous and still

analo%+ These analo% si%nals must be converted or digitised into discrete di%ital si%nals that com!uter canreadily deal *ith+S!ecial hard*are devices called Analog-to-Digital Converters (A/D !erform this task+'or !layback Digital-to-Analog must !erform a converse o!eration+

4 2 Te$t) .ra!hics and some ima%es are %enerated directly by com!uter and do not re3uire di%itiGin%2they are %enerated directly in binary format+ Hand*ritten te$t *ould have to be di%itiGed either byelectronic !en sensin% or scannin% of !a!er based form+2.1. In!ut evices and Stora$e

et us no* consider each media in turn and summariGe ho* it may be in!ut into a Multimedia system+"e also briefly analyGe the basic stora%e re3uirements for each ty!e of data+ "e do not yet consider anyeffect of com!ression on the files+ ;ote that stora%e re3uirements are lar%e for many forms of media+

Te$t and Static &ata .ra!hics Ima%es Audio /ideo

=
http://www.cs.cf.ac.uk/Dave/Multimedia/node138.html#SECTION03156800000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node139.html#SECTION03157000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node140.html#SECTION03157100000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node117.html#SECTION03131000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node118.html#SECTION03132000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node119.html#SECTION03133000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node120.html#SECTION03134000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node121.html#SECTION03135000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node138.html#SECTION03156800000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node139.html#SECTION03157000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node140.html#SECTION03157100000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node117.html#SECTION03131000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node118.html#SECTION03132000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node119.html#SECTION03133000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node120.html#SECTION03134000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node121.html#SECTION03135000000000000000


10/25

2.1. .1 ,e-t and Static ataThe sources of this media are the keyboard) flo!!ies) disks and ta!es+ Te$t files are usually stored andin!ut character by character+ 'iles may contain ra* te$t or formatted te$t e.g HTM ) Rich Te$t 'ormat-RT'0 or a !ro%ram lan%ua%e source -C) 6ascal) etc. 0+The basic stora%e of te$t is 1 byte !er character -te$t or format character0+ 'or other forms of data e.g.

S!readsheet files some formats may store format as te$t -*ith formattin%0 others may use binaryencodin%+The stora%e re3uirements of this data -te$t0 are never hi%h *hen data is stored on disk+ Small files maytake lar%er disk stora%e re3uirements due to block and sector siGes of disk !artitions+

2.1. .1 5ra!hics.ra!hics are usually constructed by the com!osition of !rimitive objects such as lines, polygons, circles,curves and arcs + .ra!hics are usually %enerated by a %ra!hics editor !ro%ram - e.g. 'reehand0 orautomatically by a !ro%ram - e.g. 6ostscri!t usually %enerated this *ay0+ .ra!hics are usually editable orrevisable -unlike Ima%es0+

.ra!hics in!ut devices include2 keyboard -for te$t and cursor control0) mouse) trackball or %ra!hics tablet+

.ra!hics files may adhere to a %ra!hics standard -O!en. ) 6HI.S) .DS0 Te$t may need to stored also+

.ra!hics files usually store the !rimitive assembly and do not take u! a very hi%h overhead+

2.1. .2 Ima$esIma%es are still !ictures *hich -uncom!ressed0 are re!resented as a bitma! -a %rid of !i$els0+ Ima%esmay be %enerated by !ro%rams similar to %ra!hics or animation !ro%rams+ A bitma! -sometimes called araster ima%e44is made u! of nothin% but !i-els + It is the format most commonly used for !hoto%ra!hs orhand dra*in%s+ Bitma! ima%es ty!ically re3uire lots of stora%e s!ace -hei%ht $ *idth $ colour de!th0+Bitma! ima%es also do not e$!and *ell2 each !i$el is stretched into lar%er and lar%er blocks) %ivin% the

overall ima%e a chunky look -this is called aliasin$ 0+

Ima%es may also be obtained by scannin% of !hoto%ra!hs or !ictures usin% a di%ital scanner or ca!tured by a di%ital camera+ Some /ideo cameras allo* for still ima%e ca!ture also+ Analo% sources *ill re3uiredi%itiGin%+Ima%es may be stored at 1 bit !er !i$el -Black and "hite0) < Bits !er !i$el -.rey Scale) Colour Ma!0 or7@ Bits !er !i$el -True Colour0+

Thus a >17$>17 .rey scale ima%e takes u! ?+7> Mb) a >17$>17 7@ bit ima%e takes ?+ >Mb *ith nocom!ression+ This overhead soon increases *ith ima%e siGe so com!ression is commonly+

2.1. . AudioAudio si%nals are continuous analo% si%nals+ They are first ca!tured by a micro!hone and then di%itiGedand stored4 usually com!ressed as C& 3uality) audio re3uires 184bit sam!lin% at @@+1 DHG -There areother audio sam!lin% rates0+ So 1 Minute of Mono C& 3uality audio re3uires 8?J@@1??J7 Bytes *hich isa!!ro$imately > Mb+

2.1. ." Video

1?


11/25

&i%ital video is essentially a se3uence of bitma! ima%es) dis!layed at a certain fre3uency+ Most film isshot at 7@ frames !er second) so thatKs a common fre3uency+ Obviously) di%ital video re3uires a hu%enumber of bits+ 'or e$am!le) a 8@? by @


12/25

44 &ata is hi%h volume) maybe continuous and may re3uire conti%uous stora%e+ There s a directrelationshi! bet*een siGe of data and ho* lon% it takes to handle+ Com!ression and also distributedstora%e is necessary+

ata Stora$e 44 The strate%y for data stora%e de!ends of the stora%e hard*are and the nature of the data+ The follo*in%

stora%e !arameters affect ho* data is stored2 Stora%e Ca!acity Read and "rite O!erations of hard*are ,nit of transfer of Read and "rite 6hysical or%aniGation of stora%e units Read "rite heads) Cylinders !er disk) Tracks !er cylinder) Sectors !er Track Read time Seek time

ata ,ransfer &e!ends on ho* data is %enerated and *ritten to disk) and in *hat se3uence it needs to retrieved+"ritin% .eneration of Multimedia data is usually se3uential e.g. streamin% di%ital audio video direct todisk+ Individual data - e.g. audio video file0 is usually streamed+

RAI& architecture can be em!loyed to accom!lish hi%h I O rates by e$!loitin% !arallel disk access

O!eratin$ System Su!!ort Schedulin% of !rocesses *hen I O is initiated+ Time critical o!erations can ado!t s!ecial !rocedures+&irect disk transfer o!erations free u! C6, O!eratin% system s!ace+

4asic Stora$eBasic stora%e units have !roblems dealin% *ith lar%e multimedia data

Sin%le Hard &rives 44 SCSI I&( &rives+ So called AV -Audio4/isual0 drives) *hich avoidthermal recalibration bet*een read *rites) are suitable for deskto! multimedia+ ;e* drives arefast enou%h for direct to disk audio and video ca!ture+ But not ade3uate forcommercial !rofessional Multimedia+ (m!loyed in RAI& architectures -Section >+>+90

Removable Media 44 5aG i! &rives) C&4ROM) &/&+ Conventional -dyin% out#0 flo!!ies not ade3uatedue 1+@ Mb ca!acity+ Other media usually ok for backu! but usually suffer from *orse !erformance thansin%le hard drives

. ata 0om!ressionata 0om!ression

,ncom!ressed %ra!hics) audio and video data re3uire considerable stora%e ca!acity *hich in the case ofuncom!ressed video is not often feasible in todayKs C& technolo%y+ The same is for multimediacommunication+ The data transfer of uncom!ressed video data over di%ital net*ork re3uires very hi%h

band*idth to be !rovided for a sin%le !oint to !oint communication+ To !rovide visible and cost effectivesolution) most multimedia system handle com!ressed di%ital and audio stream data+

17
http://www.cs.cf.ac.uk/Dave/Multimedia/node126.html#sec:raidhttp://www.cs.cf.ac.uk/Dave/Multimedia/node126.html#sec:raid


13/25

UncompressedPicture

Compressed data

0odin$ re7uirement: Stora%e siGe and band*idth minimiGation

1. ntro!y 0odin$:Run len%th codin%Huffman codin%Arithmetic codin%

2. Source codin$:6rediction codin%

- &6CM- &M

Transformation- ''T -'ast 'ourier Transformation0- &CT -&iscrete Cosine Transform0

i+ ayered Codin%Bit !osition

Sub sam!lin%Sub4band codin%

ii+ /ector :uantiGation

. (ybrid codin$

- 56(.- M6(.- H+781- &/I- 6 /

ntro!y 0odin$ It is used re%ardless of media s!ecific characteristics+ The data stream to be com!ressed is considered to

be a sim!le di%ital se3uence and the semantics of the data is i%nored+ (ntro!y codin% is an e$am!le of thelossless encodin% as the decom!ression !rocess re%enerates the data com!letely+ (+%+ are Run len%thcodin%) Huffman codin%) arithmetic codin%) etc+Source codin$: Source codin% take into an account the semantics of the data+ The de%ree of com!ressionthat can be reached by source codin% de!end on the data contained+ In the case of source codin% a one*ay relation bet*een the ori%inal data stream and encoded data streams e$ists+ The data streams aresimilar but not identical+ (+%+2 delta !ulse code modulation) delta modulation) etc+

(ybrid 0odin$It is the com!ression techni3ue+ That is the *ell kno*n al%orithm and transformation techni3ue+ (+%+2

56(. codin%) M6(. codin%) etc+

Major ste!s of data com!ression:

fi%2 Major ste!s of data com!ression

19

Picturepreparation

preparation

Pictureprocessing

Processing

Quantization

EntropyCoding

Coding


14/25

1. #icture !re!aration: this includes analo% to di%ital conversion and %eneratin% an a!!ro!riatedi%ital re!resentation of the information+ An ima%e is divided into block of < N < !i$els andre!resented by fi$ed number of bits !i$el

2. #icture !rocessin$: 6rocessin% is actually the first ste! of com!ression !rocess *hich makes useof so!histicated al%orithm of transformation from time domain to fre3uency domain+ It can be

!erformed usin% &CT -&iscrete Cosine Transform0+3. 8uanti9ation #rocess: :uantiGation !rocess the result of the !revious ste!+ It s!ecifies the

ma!!in% of the real number into inte%er+ This can be considered as the e3uivalent of -law and A- law *hich a!!ly the audio data+ :uantiGation is !erformed usin% the different number of bits !er coefficient+

4. ntro!y 0odin$: (ntro!y codin% is usually the last ste! of data com!ression+ It com!resses ase3uence of di%ital stream *ithout loss+ 'or e+%+ a se3uence of GeroKs in the data stream can becom!ressed by s!ecifyin% the number of occurrence follo*ed by Gero it+

The !rocessin% and 3uantiGation can be re!eated iteratively several times in feedback loo!+The term s!atial domain -time domain0 refer to the ima%e !lane itself and a!!roaches in this cate%oryare based on discrete mani!ulation of !i$el in an ima%e+'re3uency domain !rocessin% techni3ue is based on modifyin% the 'ourier transform of an ima%e+

Some basic com!ression ,echni7ues:1. 3unlen$th 0odin$Sam!le ima%e) audio and videos data stream often contain se3uence of same bytes+ By re!lacin%these re!eated byte se3uence *ith the number of occurrence) a substantial reduction of data can beachieved+ This is called Run len%th codin%+

'or e$am!le if a byte occur at least four consecutive times) the number of occurrence is counted+ Thecom!ressed data contain these byte follo*ed by the s!ecial fla% and number of its occurrence+ Itconverts byte bet*een four and 7>= to into three byte+

,ncom!ressed data2 A B C C C C C C C & & & ( ( ( (Com!ressed data2 A B C & & & ( @

Run len%th encodin% is the %eneraliGation of the Gero su!!ression *hich assumes that just one symbola!!ears !articularly often in se3uence+ The blank in the te$t such as symbol or !air of blanks arei%nored+ Startin% *ith a se3uence of three blanks) they are re!laced by an M4byte and a byte thats!ecifies the number of blanks of this se3uence+ Se3uence of three to a ma$imum 7>< bytes can bereduced to 7 bytes+&iatomic encodin% is a variation of run len%th codin% based on combination of t*o bytes+ Thistechni3ue determines most fre3uently occurrin% !air of bytes+

2. (uffman 0odin$:Huffman codin% is one ty!e of entro!y codin% *here a %iven character must be encoded to%ether*ith the !robability of their occurrence+ The Huffman Codin% Al%orithm determines the o!timalcode usin% the minimum number of bits+ The len%th -number of bits0 of the coded character *ill

be differin%+ To determine Huffman code) it is useful to construct a binary tree+ The leaves-nodes0 of the tree re!resent the characters that are to be encoded+ (very nodes contains theoccurrence of !robability ? and 1 are assi%ned to the branches of the tree+ (very character hasassociated *ei%ht e3ual to number of times the character occurs in a data stream+

6-A0 L ?+18

1@


15/25

6-B0 L ?+>16-C0 L ?+?=6-&0 L ?+196-(0 L ?+11

.ray evel 6robabilitya1 ?+@a7 ?+9a9 ?+1a@ ?+1a> ?+?8a8 ?+?@

. Arithmetic 0odin$: Arithmetic codin% is one ty!e of the (ntro!y codin% bet*een the code symboland code *ord doesnKt e$ist because it doesnKt encode each symbol se!arately+ (ach symbol is insteadcoded by considerin% the !eriod data+ Therefore coded data stream must al*ays be read from be%innin%+Random access is not available+ In !ractice the avera%e com!ression rate achieved by arithmetic andHoffman codin% are similar+". ,ransformation ncodin$: &ata is transformed into another mathematical domain suitable forcom!ression+ The inverse transformation most e$ists and is kno*n to the encodin% !rocess+ The most*idely kno*n e$am!le is the 'ourier Transformation *hich transforms data from the time into fre3uencydomain+ The most effective transformation for ima%e com!ression is discrete cosine transformation andfast 'ourier transformation+

4. Multimedia Authorin$ "hat is an Authorin% System#

o "hy should you use an authorin% system# Multimedia Authorin% 6aradi%ms Multimedia 6ro%rammin% vs Multimedia Authorin% Issues in Multimedia A!!lications &esi%n

o Content &esi%n7+1+1 Scri!tin% -writing 0

1>
http://www.cs.cf.ac.uk/Dave/Multimedia/node22.html#SECTION02110000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node23.html#SECTION02111000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node24.html#SECTION02120000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node25.html#SECTION02130000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node26.html#SECTION02140000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node27.html#SECTION02141000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node28.html#SECTION02141100000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node28.html#SECTION02141100000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node28.html#SECTION02141100000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node28.html#SECTION02141100000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node22.html#SECTION02110000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node23.html#SECTION02111000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node24.html#SECTION02120000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node25.html#SECTION02130000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node26.html#SECTION02140000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node27.html#SECTION02141000000000000000http://www.cs.cf.ac.uk/Dave/Multimedia/node28.html#SECTION02141100000000000000


16/25

Rules for %ood *ritin%27+1+7 .ra!hics - illustrating 0

.ra!hics Styles7+1+9 Animation -wiggling 07+1+@ Audio -hearing 0

Ty!es of Audio in Multimedia A!!lications27+1+> Interactivity - interacting 0

Ty!es of Interactive Multimedia A!!lications2o Technical &esi%no /isual &esi%n

Storyboardin% Overvie* of Multimedia Soft*are Tools

o &i%ital Audioo Music Se3uencin% and ;otationo Ima%e .ra!hics (ditin%o

Ima%e .ra!hics (ditin%o Animationo Multimedia Authorin%

'urther Information

".1 )hat s an Authorin$ System*An Authorin% System is a !ro%ram *hich has !re4!ro%rammed elements for the develo!ment ofinteractive multimedia soft*are titles+ Authorin% systems vary *idely in orientation) ca!abilities) andlearnin% curve+ There is no such thin% -at this time0 as a com!letely !oint4and4click automated authorin%systemP some kno*led%e of heuristic thinkin% and al%orithm desi%n is necessary+ "hether you realiGe itor not) authorin% is actually just a s!eeded4u! form of !ro%rammin%P you donKt need to kno* the

intricacies of a !ro%rammin% lan%ua%e) or *orse) an A6I) but you do need to understand ho* !ro%rams*ork+

".1.1 )hy should you use an authorin$ system*It %enerally takes about 1


17/25

". Issues in Multimedia A!!lications esi$nThere are various issues in Multimedia authorin%+ Belo* *e summaries issues involved in Multimediacontent and technical desi%n+ In multimedia) there are five *ays to format and deliver your messa%e+Qou can write it) illustrate it) wiggle it) hear it) and interact *ith it+

1. Scri!tin$ ; !riting < "ules for good !riting#1+ ,nderstand your audience and correctly address them+7+ Dee! your *ritin% as sim!le as !ossible+ -e+%+) *rite out the full messa%e-s0 first) then shorten it+09+ Make sure technolo%ies used com!lement each other+

2. 5ra!hics ; illustrating nicorn #erformer

Cubase o%ic Audio Com!etitor) similar functionality

".". Ima$e/5ra!hics ditin$;a< Adobe #hotosho!

Allo*s layers of ima%es) %ra!hics and te$t Includes many %ra!hics dra*in% and !aintin% tools So!histicated li%htin% effects filter A %ood %ra!hics) ima%e !rocessin% and mani!ulation tool

;b< Adobe #remiere 6rovides lar%e number -u! to ==0 of video and audio tracks) su!erim!ositions and virtual

cli!s Su!!orts various transitions) filters and motions for cli!s A reasonable deskto! video editin% tool

(c) Macromedia %reehand .ra!hics dra*in% editin% !acka%e

4: ,here are many other editors in !ublic domain ;free< and others are commercial

"."." Video/5ra!hics ditin$Many commercial !acka%es available

79
http://www.cs.cf.ac.uk/Dave/Multimedia/cubase_score.gif


24/25

Adobe 6remier /ideosho! Avid Cinema S.I MovieMaker

".".6 AnimationMany !acka%es available includin%2

Avid SoftIma%e Animated .if buildin% !acka%es e.g. "if#uilder

".".? Multimedia Authorin$Tools for makin% a com!lete multimedia !resentation *here users usually have a lot of interactivecontrols+

".".?.1 Macromedia irector Movie meta!hor -the cast includes bitma!!ed s!rites) scri!ts) music) sounds) and !alettes) etc+0 Can acce!t almost any bitma!!ed file formats in%o scri!t lan%ua%e *ith o*n debu%%er allo*s more control includin% e$ternal devices) e+%+)

/CRs and video disk !layers Ready for buildin% more interactivities -buttons) etc+0 Currently in version +?) this !o!ular %eneral market !roduct follo*s the cast score scri!tin%

!aradi%m) *hich makes it the tool of choice for animation content+ Its roots as a cel4 and s!rite4animation !ro%ram are unmistakableP and its inclusion of in%o) its object4based scri!tin%lan%ua%e) has made it the animation4ca!able !ro%ram to beat+ The AfterBurner com!ression Ntracreates Shock*ave files) allo*in% "eb !layback+

".".?.2 Author&are 6rofessional multimedia authorin% tool Su!!orts interactive a!!lications *ith hy!erlinks) dra%4and4dro! controls) and inte%rated

animation Com!atibility bet*een files !roduced from 6C version and MAC version

".".?. Many Others:

Microcosm 2 Multicosm) td+ P &OS) "indo*s Microcosm is a Hy!ermedia inka%e authorin%system+

:uestion Mark 2 :uestion Mark Com!utin% td P &OS) Mac) "indo*sP """ -via 6erce!tion0:uestion Mark is o!timiGed for (lectronic Assessment !roduction+

(mblaGe Creator 2 .eo International P 5avaScri!t) Mac) "indo*s=>) """+

(mblaGe Creator 7+> is a cast score scri!tin% tool *hich is desi%ned for "eb4based !layback ofinteractive multimedia+

'lash 2 Macromedia P Mac) "indo*s=>) ;T) """ -via 'lash 6layer0+

7@


25/25

'lash 9+? is a cast score scri!tin% tool) *hich !rimarily uses vector %ra!hics -and can createvector %ra!hics from im!orted bitma!s0+ It is o!timiGed for "eb delivery) and is es!eciallycommon for banner adds and small interactive *eb deliverables+

Hy!erCard 2 A!!le Com!uter P Mac) """ -via iveCard 0+

Hy!erCard is a card scri!tin% authorin% system currently in version 7+@+1+ It runs natively on both8

0055 Introduction to Multimedia Systems Notes

Documents

Transcript of 0055 Introduction to Multimedia Systems Notes