Colour and Sound: Part 1: Introduced and Compressed Dan Eaves CSE5060 – Multimedia Applications on...

Colour and Sound: Part 1: Introduced and Compressed Dan Eaves CSE5060 Multimedia Applications on the Web, Semester 2, 2006 Lecture 2 Note: Non-anime images are from Teds Photographics: http://www.ted.photographer.org.uk Very highly recommended! Please spend some time at this wonderful site.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced2 Colour onto Screen Computers use RGB colour Red Green Blue The process is additive. There are three electron guns at the back of the TV tube. There are three types of phosphors (rare earths) which coat the inside of the front of the monitor. There is a grid (mask) or set of vertical wires (aperture grill) through which the electrons go on their way from the gun to the face of the monitor. (Aperture grill is better.) Each position on the screen has the three phosphors. The colour of the position is their combined light.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced3 Colour onto Screen: Additive Colour

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced4 Additive Colour: RGB to CMYK

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced5 Monitor Phosphors versus Eye Sensitivity (The lines indicate the 3 different cones)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced6 Monitor (With Shadow Mask)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced7 Monitor (With Aperture Grill)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced8 Monitors: Aperture Grill versus Shadow Mask

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced9 LCD Displays For liquid crystal displays A freak of nature. Such crystals are Liquid Present a twisted path (blocks light) But electricity untwists it, so light can get through! Voila

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced10 From Digital Home Magazine: How LCDs work

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced11 Plasma Displays 3 florescent lights (red, green, blue) in each pixel position

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced12 Plasma Display Problems 3 florescent lights are big There is no likelihood of making them really small So plasma displays always have to be big and low resolution But for a home theatre wall display at 16x9 HDTV resolution theyre wonderful. Note that 16x9 aspect ratio Note that HDTV resolution is (in theory) up to 1920 x 1080 pixels(i) at 16x9, but a current 50 inch Sony $9,500 plasma display has a 1280 x 768(p) resolution Note the i and p: interlaced or progressive: both those last are actually (sort of) the same resolution.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced13 Aspect Ratios When TV developed in 1940s, took the then standard motion picture ratio: 4x3 Movie industry responded to perceived threat from TV by inventing a new standard, 16x9 Other countries responded to US TV standard (NTIS) by specifying alternative incompatible standards to protect their electronics industries, e.g., the PAL standard used in Australia Different standards have different number of horizontal lines, different screen refresh rates Aint competition wonderful!

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced14 Getting Colour onto the Screen We can specify an intensity of each of the three colours in the range of 0 to 255. 255 by 255 by 255 = about 16,700,000. This takes 24 bits, 8 for each colour If we want pure red, we set: Red = 255 Green = 0 Blue = 0 This translates to the intensity of the beams of electrons striking the different phosphors. What do we get when: Red = 255 Green = 255 Blue = 255

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced15 But Wait, Theres More... True colour can be either 24 or 32 bits deep. Both of these have a palette of 16.7 million colours. If present, the 8 extra bit are called the alpha channel. This contains transparency information: How well can you see through this pixel, on a scale of 0-255 Primarily used as masks in animation where images are overlaid to build the final result. (Note: you can also call 32 bit colour 4 channel colour) Alpha channels dont exist in transmitted image. Here just because you will be using then in Flash

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced16 Anti-Aliasing (I use a 16 colour picture so the effect is clear)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced17 Anti Aliasing When digitized (and compressed) curved lines get the jagged edges. You can make these look less bad by dithering in some pixels in colours between those of the line and the background where the jagged parts are.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced18 Anti Aliasing The Actual Pixels The Visual Effect

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced19 Anti Aliasing Used all the time Built into digital cameras Built into compression programs Built into products like PhotoShop Built into products like Flash/Shockwave Even with PhotoShop, hand retouching is done with great frequency for high (perceived) quality images Foxing (fooling) the eye is the basis for much of the substance of computer graphics, including compression!

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced20 Compression Introduced We Shall Return! Because pictures take up so much space, we have to compress them Reduce the number of colours Store constant information only once Reduce the size of images Use vector graphics Used by Flash/Shockwave Used for animations Used for simple static images There are a number of different static image raster compression formats

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced21 Raster versus Vector Graphics Raster = The invisible dot that moves across the (TV, Monitor) screen, Puts something in each pixel position. Then does a flyback from the bottom right to top left positions. Driven by storage of data (2-24 bits) for each pixel position. Supports highly detailed images, e.g., photographs File sizes are typically big, sometimes very big. Vector = A line (any line, not necessarily straight, but which can be described by a mathematical formula. Formulas are converted into bit maps (raster images) These are displayed on the screen Supports low detail images, e.g., cartoons and drawings File sizes are typically small, sometimes very small. Vector graphics trade computer processing for file size.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced22 GIF/PNG GIF Graphics Interchange Format is: A lossless compression (so not very compressed) Maximum of 8 bit colour (256 different colours) One colour can be invisible (transparent) Very fast to decompress Universally used for headings, buttons, etc. Works by recognizing areas of rows of pixels of the same colour and not storing actual pixel by pixel data but start/stop/colour information That is, where possible works like a scanner and sends addresses of changes rather than all pixels. Sends all pixels where necessary. Useless for photographs

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced23 JPG/JPEG JPG - Joint Photographers Expert Group Lossy compression What you get after decompression isnt what you started with Tuned to the human vision system: Starts by eliminating colour differences the eye cant see Degree of compression under your control You specify the degree of compression, from unnoticeable to horrible Many colours (up to 24 bits) No transparent colour Used for photographs

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced24 Raster Graphics Data is stored for each pixel position. With an image that is 500 x 300 there are 150,000 pixel positions. 500 * 300 = 150,000 * 3 (bytes) = 450,000 * 24 images per second * 60 seconds per minute * 60 minutes per hour And then, of course, theres sound

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced25 The Tragedy of Raster Graphics Remember, modems are serial (1 bit at a time) so thats, say, 1-3 mbits a second Weep! Bash head against wall! Bewail fate! If a page contains 400 words @ 5 letters per word and 5 characters of extra HTML per word we get: (400 * 5) + (400 * 5) = 4,000 characters = 4,000 bytes no time at all Drat, drat, drat. Well need to compress, use vector graphics, reduce the number of colours, compress again.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced26 Vector Graphics Like Corel Draw, Flash and Illustrator files (note: can have raster background image(s) in these. Lines and outline shapes are stored as mathematical formulas Fills are stored as a few bytes. A program reconstructs the always necessary raster image from the vectors, stores the results in the video cards frame buffer, and shows the results on a raster device (your screen). Image size can be increased or decreased without loss of detail or introduction of artefacts (e.g., moire effects) So we have very small files but lots of processing work on the way to displaying the results. Flash/Shockwave are currently the only standard ways of using vector graphics on the web. But new standards are being developed to extend web use of vector graphics their chance of success is ????? TrueType and PostScript fonts are vector graphics formats.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced27 A Shift: Digital Sound Music on CD works as follows: Each chunk of data represents the amplitude of the sound at a moment in time with a number ranging from 0 to 65,535 -- 44,000 samples are taken per second per stereo channel. So data storage per second is 44,100 samples * 2 bytes per sample * 2 channels = 176,000 bytes per second. So a 3 and a half minute song takes 176,000 * 60 * 3.5 = 36,960,000 bytes of storage space Hummmmm. Were going to have problems with sound, too. Your CD music is the equivalent to uncompressed raster graphics MP3 is the equivalent to.JPG/.JPEG compression, as it is both lossy and the degree of loss (and compression) is under user control. Standard MP3 gets a standard length song down to, say 3- 4 meg. (Estimate using a 90% compression factor)

Part 2: Speeding Things Up

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced29 Good Sources As always, Wikipedia.Wikipedia Good Australian broadband site: Whirlpool.Whirlpool Note especially the Broadband Choice link. Recent OECD study (Dec., 2005) of worldwide broadband usageOECD study

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced30 Broadening the Bandwidth Alternatives are: ISDN (international standard digital network) Cable modems (fiber optics to local hub, coaxial to your house) Cable (point to point) DSL (digital subscriber line), aka ADSL (A = asymmetrical) BoP (broadband over powerlines) Satellite WebTV - Combining your TV, a specialised web computer and a remote control with keyboard, using existing cable connections.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced31 ISDN (International Standard Digital Network) Comes in two flavours: Uses existing POTS for digital (not analog) transmission. Primarily for home use Initially provided two 64kbit/s data links and 1 voice link Available in selected suburbs Not very expensive at lower speeds, very expensive at higher ones. Uses special lines (fiber/coaxial) Primarily for business use Many speeds and combinations available Available in most CBDs Predates Web and Internet

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced32 Cable Modems (Australian) pay TV wiring works like this: Fiber optics into near neighborhood hub. Coaxial cable from hub to home. All channels are broadcast all the time. There is extra bandwidth available (in both links) So a.5 - 1.5 mbyte/s connection possible But: Less in local link, which would be shared across all cable modem users in immediate neighborhood. Pay TV less popular that anticipated, so wiring incomplete No economies of scale in modem replacements, due to Australias unique position re cable TV Some suppliers (e.g., Optus) may be rationing bandwidth. Note: Pay TV elsewhere is a mix of usually less sophisticated technologies, but 9 million cable modems in use in USA.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced33 DSL Digital Subscriber Line or ADSL for Asymmetric Digital Subscriber Line Uses the existing TWP (twisted wire pair) telephone line, which is split (ADSL): 0-4,000 kHertz for voice. 4,000 25.875 kHertz up link (varies by standard) 25.875 1104 kKertz down link (varies by standard) Intended for The Last Mile (from local exchange to customer) Line/signal quality falls with distance Signal is faster on the down link and slower on the up link. Data rate is distance to exchange dependent: Theoretical downlink maximum speed varies from 8 to 24 mbps, with different protocols and standards. But max speed may be down to 128 kbps in some (bad) situations. Above ca. 5.5km, DSL isnt possible. Hummmmmmm...

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced34 ADSL in Australia From Telstra/Big Pond From Telstra competition Telstra has been criticised for limiting their ADSL DSLAM speed to a maximum of 1.5 Mbit and 256 kbit/s (download and upload respectively), whereas the maximum ADSL speed possible is 8 Mbit and 1 Mbit (download and upload respectively). The reason Telstra limit their DSLAM speed is so as to not kill off its own ISDN service which charges hugely expensive premiums for 2 Mbit/s speeds. Wikipedia, DSL Around the World, accessed April 13, 2006. DSLAM = Digital subscriber line access multiplexer

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced35 ADSL in Australia 2: From Whirlpools Broadband ChoiceWhirlpools Broadband Choice I supplied my phone number and requirements (speed, volume, price willing to pay, etc) I got: Your telephone line is connected to the Warranwood exchange. This exchange has been enabled for ADSL service. In terms of pricing, Warranwood is classed as Metro. Please note that while the Warranwood exchange supports ADSL, this is no guarantee that your physical phone line is up to scratch. A conclusive test can only be done upon registering for a service. And a list of 20+ DSL and cable plans available (including the cable service Im using. I set this up before DSL was available.)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced36 ADSL Take-up Currently 2nd broadband technology of choice in USA ADSL = 6.5 per 100 inhabitants by Dec., 2005 (versus 9 per 100 inhabitants with cable modems, 1.3 per 100 other mainly satellite) Currently 1 st in Australia: relative figures are: ADSL = 10.8 per 100 inhabitants by Dec., 2005 (versus 2.6 per 100 inhabitants with cable modems, 0.4 per 100 other mainly satellite) Speed in theory averages about 1.5mbits / second down, 256kbits / second up. (So 25+ times faster than voice modem) Less congestion problem than cable models, at least in theory.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced37 Near Future Technologies Wi-Fi Wi-Fi Wireless Broadband Works without phone lines G3 phones limited to ca. 384 kbps. Unwired Is examplem available in Sydney and (from April, 2006) Melbourne CBDsUnwired Speeds from 256/64 to 1024/256 kpbs for $55 $120 per month Highly limited coverage really just CBD Requires line-of-sight link, so just like cell phone, with dead spots Google is Wi-Fiing up all of San Francisco in 2005-6 for free as an experiment.

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced38 Near Future Technologies BOP (Broadband over Powerwires Different Scopes Complete path Last mile Within site (home or office) Primary Problem Very noisy environment Transformer problem In US, most homes have own high to low voltage transformer In Europe, each 10-100 homes have own transformer As there must be a relay station around each transformer, cost is much higher per customer in US. Much promised over last 15 years

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced39 Satellites Two classes of connection: One-way satellite link: Landline-based upload, with Direct satellite download That is, traffic from you to the satellite goes down the phone line of the ISP, to the satellite company and up to the satellite from there. The response goes directly from satellite to the antenna on your roof. The traditional internet satellite link. Two-way satellite link Data goes up from your antenna, as well as being received by it. Requests relayed down by satellite. (4 trips) Requires a slightly larger antenna (.75 x 1.1 m.) Requires a very accurately aimed antenna. Such a service recently introduced in the US by StarBand 55,000 customers after 4 months (March, 2001)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced40 Satellites, 2

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced41 Satellites, 3 Three classes of satellites: Geosynchronous (GEO) always over exactly the same place Has to be at altitude of exactly 35,786 km over the equator. (Slots are running out, but you only need three for full coverage.) And the speed of light comes in! 40,000+ km up, 40,000km down, maybe 20,000 across = 100,000 km trip, = a half of a second -- called latency Problem is in lack of synchronisation for conversations Problem is in acknowledgment of successful reception of packets. Trend is to use undersea cables for voice, satellite for data transfers (in non-packet form.) Medium Earth Orbit (MEO) Less latency, but still too much Require a number of satellites because dont stay over the same location

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced42 Satellites, 4 Medium Earth Orbit (MEO) Less latency, but still too much Require a number of satellites because dont stay over the same location

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced43 Satellites, 5 Low Earth Orbit (LEO) Need lots because moving relative to the surface, so say 288 (plus spares) satellites at 1,375 km altitude Speed of light isnt a problem Coordinating across satellites is a problem Will handle voice, data & video all as Internet IP packets Was initial plan of Teledesic (from McCaw and Gates)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced44 Satellites, 6 Additional Problem: the IP in TCP/IP TCP/IP assumes Low latency Low bit error rates Relatively symmetric bandwidth (All those little packets have to be acknowledged.) Satellite Telecommunications are characterised by: High latency High bit error rates Highly asymmetric bandwidth (esp. with landline based uploading) Solution of choice Convert to a non-IP protocol designed for satellite communications. (Whoops)

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced45 Satellites, 7 Some conclusions LEO eliminates latency but requires an enormous investment. (And nobody is going to invest in satellites for awhile) For file transfers, any satellite is great, without TCP/IP. If you are isolated, and need bandwidth...

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced46 The Dark Fiber Problem In the late 1990s (Internet boom) A huge amount of optical fiber was laid. Within countries (esp. US) Between countries The assumption was that Computer line use would follow exponential growth curve of 1995-2000. The boom ended and wavelength division multiplexing increased existing fiber capacity by 100x There is currently (2006) still huge over capacity in fiber This has greatly slowed the investment in other technologies

Rev. 26 July, 2006CSE5060 Lecture 2: Colour and Sound Introduced47 Shrinking the Content To fit more through the eye of the needle, we can Make the eye bigger (increase bandwidth) or Shrink the content Better compression The vector graphics approach, e.g., Send instructions to build things, not the things themselves. We trade (much) faster transmission for (much) increased local processing. It is hard (but not impossible) to digest a photograph or movie. Background transmission Send stuff in advance of need behind the scenes Examples, streaming audio, video, media Streaming means: It keeps being sent automatically after you first ask for it It starts playing before it is fully transmitted Has fallbacks for not keeping up (dropped frames, etc.)

Colour and Sound: Part 1: Introduced and Compressed Dan Eaves CSE5060 – Multimedia Applications on...

Documents

Transcript of Colour and Sound: Part 1: Introduced and Compressed Dan Eaves CSE5060 – Multimedia Applications on...