Colour and Sound Introduced with Too Little on Computing and the Handicapped Dan Eaves CSE5900 –...

Colour and Sound Introducedwith

Too Little on Computing and the Handicapped

Dan Eaves

CSE5900 – Introduction to Multimedia Computing, Semester 1, 2006

Lecture 2

Note: Non-anime images are from Ted’s Photographics:

http://www.ted.photographer.org.uk

Very highly recommended! Please spend some time at this wonderful site.

Rev. 3 March, 2006 CSE5900 Lecture 2: Colour and Sound Introduced 2

Colour onto Paper

• Magazine printing uses 4 colours: CYMK– Cyan (a shade of light blue)

– Yellow

– Magenta (a shade of purplish red)

– Black

• Scanning or photographing through filters, four different printing plates are made.

• The process is subtractive: Magenta is white light less the cyan and yellow.

• The paper is run through 4 presses, one doing each colour

• Inks consist of pigment and a solvent

• Each adjacent colour blends in the eye/brain: what you see is not what’s there!


Subtractive Colour


Subtractive ColourRed, Green and Blue

No Filter



Cyan Filter



Magenta Filter



Yellow Filter


But There’s More

• Colour (or black) aren’t continuous but in an array of dots, a screen (as in screen door).

• (There are types of colour printing, e.g., lithography, where this isn’t the case.)

• Dots (of colour or black) in a position on a screen can be of different sizes. This means that any combination of CMYK colours are possible in a particular position.

• Screens and varying dot sizes are best seen looking at a low resolution black and white photograph. In a black area, the dot is big, in a light grey area the dot is small.


Black and White “Screen”


Colour “Screen”The 4 Printing Plates:

Cyan, Yellow, Magenta & Black: CMYK


Colour “Screen”Final Appearance, with Closeup


The Horrors of a Colourful Life

• Ink varies from batch to batch.

• Ink changes in characteristics as it gets older.

• Manufacturers’ formulas for the “same” colour vary.

• Papers can be coated or uncoated.

• There are different kinds of coatings.

• Different kinds of paper absorb at different rates.

• The same paper absorbs ink at different rates depending on the amount of water in it.

• There are several proprietary ways of specifying exactly what colour you want (for spot colour printing). Pantone is the best known. None work perfectly.


Colour onto Screen

• Computers use RGB colour– Red– Green – Blue

• The process is additive.

• There are three electron guns at the back of the TV tube.

• There are three types of phosphors (rare earths) which coat the inside of the front of the monitor.

• There is a grid (mask) or set of vertical wires (aperture grill) through which the electrons go on their way from the gun to the face of the monitor. (Aperture grill is better.)

• Each position on the screen has the three phosphors. The colour of the position is their combined light.


Colour onto Screen: Additive Colour


Additive Colour: RGB to CMYK


Monitor Phosphors versus

Eye Sensitivity(The lines indicate the 3 different cones)


Monitor(With Shadow Mask)


Monitor(With Aperture Grill)


Monitors: Aperture Grill versus Shadow Mask


LCD Displays• For liquid crystal displays

– A freak of nature. Such crystals are• Liquid

• Present a twisted path (blocks light)

• But electricity untwists it, so light can get through!

– Voila


From Digital Home Magazine: How LCDs work


LCD Problems: Traditional and Contemporary

• Limited viewing angle – but greatly improved• Colour intensity not too good – but greatly improved• Limited response time precluded using for games – but

greatly improved … for a price

• Colour accuracy not good enough for artists, graphic applications, but fine otherwise

• Only one optimal resolution, e.g., 1280 x 1024 on my LCD monitors

• Aspect ratio difference: • Monitors = 1024/768 = 1.333 = 4:3 aspect ratio – like your

(old) TV• 1280/1024 = 1.25 = 5:4 aspect ratio

• Were more expensive than CRTs, but not now except for games/artist quality models.


Plasma Displays

• 3 florescent lights (red, green, blue) in each pixel position


Plasma Display Problems

• 3 florescent lights are big• There is no likelihood of making them really small

• So plasma displays always have to be – big and

– “low resolution”

• But for a home theatre wall display at 16x9 HDTV resolution they’re wonderful.

• Note that 16x9 aspect ratio

• Note that HDTV resolution is (in theory) up to 1920 x 1080 pixels at 16x9, but a current 50 inch – Sony – $9,500 plasma display has a 1280 x 768 resolution


Aspect Ratios

• When TV developed in 1940s, took the then standard motion picture ratio: 4x3

• Movie industry responded to perceived threat from TV by inventing a new standard, 16x9

• Other countries responded to US TV standard (NTIS) by specifying alternative incompatible standards to protect their electronics industries, e.g., the PAL standard used in Australia

• Different standards have different number of horizontal lines, different screen refresh rates

• Ain’t competition wonderful!


Getting Colour onto the Screen

• We can specify an intensity of each of the three colours in the range of 0 to 255.

• 255 by 255 by 255 = about 16,700,000. This takes 24 bits, 8 for each colour

• If we want pure red, we set:– Red = 255

– Green = 0

– Blue = 0

• This translates to the intensity of the beams of electrons striking the different phosphors.

• What do we get when:– Red = 255

– Green = 255

– Blue = 255


But Wait, There’s More...

• “True colour” can be either 24 or 32 bits deep.

• Both of these have a palette of 16.7 million colours.

• If present, the 8 extra bit are called the alpha channel.

• This contains transparency information: How well can you see through this pixel, on a scale of 0-255

• Primarily used as masks in animation where images are overlaid to build the final result.

• (Note: you can also call 32 bit colour “4 channel colour”)


The Horrors of the Colourful Life, Mark II

• Different monitors use different rare earths as phosphors.

• Electron guns wear out.

• Phosphors wear out.

• Ambient light plays an important role in colour as seen:– Natural light

– Incandescent light

– Fluorescent light.

• What if we are building coloured images on the screen for printing????

• Pantone helps here too, but there are so many variables that the colours actually printed may be surprising.


But What if We Don’t Have 24/32 Bits

• We use either 8 bits (256 colours) or 16 bits (65,000 - 64k colours)

• We fake the colours in-between … BUT

• We can’t blend colours like when printing.

• Instead, we dither.

• We take a square of pixels (2x2, 3x3, 4x4) and put colours into them which (we hope) blend in the eye to look like the colour we want.

• Usually they don’t.


Or, We Can “Spend” Our CLUT on Greys– 2 5 8 256


Dithering and Anti-Aliasing(here’s the 16 colour picture the examples come from)


To Dither or Not To Dither

• We can carefully pick 256 colours from 16.7 million, creating a CLUT (colour look up table).

• The CLUT is specific to the page, not to a particular image on the page.

• So selecting the right CLUT is an art, not a science.

• Or we can dither.

• Or both. If you aren’t careful, dithering may happen automatically, and look horrible.


Dithering

The Actual Pixels

The Visual Effect


Anti Aliasing• The screen is a very, very, very low resolution device: say

92x92 dots per inch as opposed to 600x600+ dpi for laser printers (10,000 dots versus 360,000: 36 times as many)

• Curved lines get the jaggedies

• You can make these look less bad by dithering in some pixels in colours between those of the line and the background where the jagged parts are.

• But, if this is done automatically, a number of your 256 colours will be used up without your noticing.


Anti Aliasing

The Actual Pixels

The Visual Effect


Anti Aliasing

• Used all the time

• Built into digital cameras

• Built into products like PhotoShop

• Built into products like Flash/Shockwave

• Done by hand with regularity, especially with GIFs– Remember, GIFs limited to 256 colours

• Even with PhotoShop, hand retouching is done with great frequency for high (perceived) quality images

• “Foxing” (fooling) the eye is the basis for much of the substance of computer graphics!


Compression – Static Images

• Because pictures take up so much space, we have to compress them– Reduce the number of colours

– Store constant information only once

– Reduce the size of images

– Use vector graphics• Used by Flash/Shockwave

• Used for animations

• Used for simple static images

• There are a number of different static image raster compression formats


Raster versus Vector Graphics• Raster = The invisible dot that moves across the (TV,

Monitor) screen, – Puts something in each pixel position.– Then does a flyback from the bottom right to top left positions.– Driven by storage of data (2-32 bits) for each pixel position.– Supports highly detailed images, e.g., photographs– File sizes are typically big, sometimes very big.

• Vector = A line (any line, not necessarily straight, but which can be described by a mathematical formula.– Formulas are converted into bit maps (raster images)– These are displayed on the screen– Supports low detail images, e.g., cartoons and drawings– File sizes are typically small, sometimes very small.

• Vector graphics trade computer processing for file size.


GIF/PNG

• GIF – Graphics Interchange Format is: – A lossless compression (so not very compressed)– Maximum of 8 bit colour (256 different colours)– One colour can be invisible (“transparent”)– Very fast to decompress– Universally used for headings, buttons, etc.– Works by recognizing areas of rows of pixels of the

same colour and not storing actual pixel by pixel data but start/stop/colour information

– That is, where possible works like a scanner and sends addresses of changes rather than all pixels.

– Sends all pixels where necessary.– Useless for photographs


JPG/JPEG

• JPG - Joint Photographers Expert Group– Lossy compression

• What you get after decompression isn’t what you started with

– Tuned to the human vision system:• Starts by eliminating colour differences the eye can’t see

– Degree of compression under your control• You specify the degree of compression, from unnoticeable to

horrible

– Many colours (up to 32 bits)

– No transparent colour

– Used for photographs


Raster Graphics

• Data is stored for each pixel position.

• With an image that is 500 x 300 there are 150,000 pixel positions.– If it’s in two colours, we store either a 0 or a 1 for each position.

So the number of bytes is 150,000 / 8 = 18,750 bytes

– If there’s 4 colours, we store 00, 01, 10 or 11 for each pixel position -- 150,000 * 2 / 8 = 37,500 bytes

– Note: They don’t have to be standard colours. We can say, using the CLUT, that 0 = puce, 1 = tangerine, 2 = maroon and 4 = black…or whatever colours we want.

– With a palette of 256 colours, we need 8 bits for each pixel --- 150,000 * 8 / 8 = 150,000 bytes

– With a palette of 16.7 million colours, we need 24 bits for each pixel -- 150,000 * 24 / 8 = 450,000 bytes

– (With a 56 kb modem, how fast will this picture load?)


The Tragedy of Raster Graphics• Remember, modems are serial (1 bit at a time) so that’s 56

* 1024 /8 bytes = 7,000 bytes per second at best so our 450,000 byte picture will take 65 seconds to load. Or 100 seconds with a 33k modem. Or 6.5 seconds with good broadband. Consider that animation/movies require 24 picture per second.

• Weep! Bash head against wall! Bewail fate!

• If a page contains 400 words @ 5 letters per word and 5 characters of extra HTML per word we get: (400 * 5) + (400 * 5) = 4,000 characters = 4,000 bytes = .6 seconds to load. It’s not called HTML for nothing

• Drat, drat, drat. We’ll need to compress, use vector graphics, reduce the number of colours, compress again.


The Necessity of Raster Graphics

• HTML allows you to select size in terms of Heading 1 - Heading 8, not in terms of font, weight or point size.

• The user’s default font is used in the display.

• This is useless and hopeless from a design perspective.

• So headings and buttons have to be raster graphics (usually .GIF files with a limited colour palette).

• Build heading in Corel Draw or Illustrator (vector)

• Pass it to Photoshop (raster)

• Reduce the colour depth (bits per pixel)

• Dither and anti-alias and adjust both

• Save as .GIF/PNG


Palettes – Somewhat Historical – Relates to GIF Only

• We have 256 colours to play with (say), represented by the numbers 0 to 255. That is, we store 8 bits of colour information for each pixel.

• But the screen can display 64k different colours (16 bit colour) or 16.7 million colours (24 bit colour).

• So we pick our particular 256 colours out of all the available ones, using a CLUT (colour look up table).

• This shows our 256 colour numbers and the larger colour number to display for that number– 0 = red of 12, green of 45 and blue of 210– 1 = red of 25, green of 90 and blue of 255– and so on to 255

• But we can have only one such CLUT for everything on the HTML page


Vector Graphics

• Like Corel Draw or Illustrator files– Lines and outline shapes are stored as mathematical formulas

– Fills are stored as a few bytes.

– A program reconstructs the always necessary raster image from the vectors, stores the results in the video card’s frame buffer, and shows the results on a raster device (your screen).

– Image size can be increased or decreased without loss of detail or introduction of artefacts (e.g., moire effects)

– So we have very small files but lots of processing work on the way to displaying the results.

– Flash/Shockwave are currently the only standard ways of using vector graphics on the web.

– But new standards are being developed to extend web use of vector graphics – their chance of success is ?????

• TrueType and PostScript fonts are vector graphics formats.


A Shift: Digital Sound

• Music on CD works as follows:– Each chunk of data represents the amplitude of the sound at a

moment in time with a number ranging from 0 to 65,535 -- Whoopee, you now instantly say that each sample is a 16 bit number.

– 44,000 samples are taken per second per stereo channel.

– So data storage per second is 44,000 samples * 2 bytes per sample * 2 channels = 176,000 bytes per second.

– So a 3 and a half minute song takes 176,000 * 60 * 3.5 = 36,960,000 bytes of storage space

– Hummmmm. We’re going to have problems with sound, too.

– Your CD music is the equivalent to uncompressed raster graphics

– MP3 is the equivalent to .JPG/.JPEG compression, as it is both lossy and the degree of loss (and compression) is under user control. Standard MP3 gets a standard length song down to, say 3-4 meg. (Estimate using a 90% compression factor)


MP3 Assessed

• “MP3 -- MP3 was the disappointment I expected it to be. This does not mean it has no application - for the PC, MP3 is ideal. It has a good compression factor, allows easy transfer over the Internet, and sets new standards in audio compression (I said new - not better). MP3 is instantly recognised without an A/B switching test. It introduces some distortion to highs in particular. Also, in complex passages the instruments get lost - you cannot follow each one - it is as if the encoder could not work out what parts to keep and which to throw away. This hypothesis was confirmed when I went back and recorded the same songs at higher MP3 bit rates (128kbps was the one I used initially, as it is the most widely used today). At 256kbps and 320kbps there were some remarkable improvements. I do believe MP3 will stay mainly in the computing field…. For car applications, background music, or played over typical speakers found on most PC systems, MP3 is fine. As a serious audio storage medium to be used with quality equipment however, I cannot recommend it at all. “

• Guy Churchill, 19 Mar 1999: http://www.minidisc.org/format_comparison.html

http://www.minidisc.org/format_comparison.html


MIDI

• MIDI is the music equivalent to vector graphics.

• Music is stored as a set of instructions, a score, giving the note, the instrument to play the note, the way the note is to be played, etc.

• Many notes (voices) can be played at the same time.

• Sound is generated either via “FM Synthesis” or by using ROM or RAM based samples of how the instruments sound.

• Files are quite small but lots of processing is necessary to generate the sound.


Computing for the DisabledA Short Overview


Is This Trip Necessary?

• The WWW is an “information appliance”– It is available and used everywhere

– People are expected to access and use it.

– But what if you are spastic or blind?

• The contemporary ethic is that disabled people must not be denied access to public resources just because they are disabled.

• The WWW is a public resource• So we must …..

– (Saying ‘It’s too hard’ isn’t acceptable!)


Good Sources

• UK supplier of products: http://www.keytools.com/default.asp

• Solutions for Humans: http://www.sforh.com/

• New Zealand/Canadian company specializing in hardware and software, esp. for the blind. Was Pulse Ware: http://www.humanware.com

• Freedom Scientific, maker of JAWS, etc.: http://www.freedomscientific.com/

• Australian supplier. Check out the Technology for Life section: http://www.quantech.com.au/index.html

• Haptic interfaces. A good starting place is the Wikipedia entry, with lots of good links at the end: http://en.wikipedia.org/wiki/Haptic

http://www.keytools.com/default.asp

http://www.keytools.com/default.asp

http://www.sforh.com/



http://www.humanware.com/

http://www.humanware.com/

http://www.freedomscientific.com/



http://www.quantech.com.au/index.html

http://www.quantech.com.au/index.html

http://en.wikipedia.org/wiki/Haptic

http://en.wikipedia.org/wiki/Haptic


(?)

• Do we need to deal with all computer systems here, or just the WWW?

• Legally, it’s all publicly available systems.

• Practically (and economically), in some circumstances it may be better simply to supply a human intermediary.

• But doing so may be unacceptable to the disabled!


Interfaces for Spastics

• (The two bravest people I’ve ever known were badly spastic women who are productive members of society. And you didn’t dare try to help them.)

• Spastics obviously can’t use mice, so the interface must support non-mouse-driven use.

• A number of very helpful special keyboards are available.


Big Keyboards


“The Grid” On-Screen Touch Sensitive Screen Overlays

• Symbol Communication


“The Grid” On-Screen Touch Sensitive Overlays

• Typing -- Note suggestions on left. The user has entered ‘h’ and ‘o’


Intelikeys

• A special keyboard with overlays


For the Blind• The leading screen reader is Jaws from Freedom Scientific

– interactive talking installation

– Supports Office and other standard products

– Supports Internet Explorer including:

• links lists, frames lists, forms mode, reading HTML tables and graphic labels, etc.

– Provides scripting language to extend to additional programs

– Can output to refreshable Braille displays – $US895 to 1,295 per copy (ouch) + 120-200

maintenance per year.


Connect Outloud• Also from Freedom Scientific

– Cutdown, specialised version of the WWW part of Jaws

– interactive talking installation

– Supports Internet Explorer including:

• links lists,

• frames lists,

• forms mode,

• reading HTML tables and graphic labels, etc.

– Can output to refreshable Braille displays – $US249 per copy (no ouch)


Refreshable Braille Displays

• Braille Star from Pulse Data (now Humandata)– 40 or 80 Braille characters in line

– Can take notes, annotate, store, work off line, etc

– $UD6,000 or 11,000 (40 or 80 characters) (ouch, ouch)


One Hand Keyboards


Keyboard for Head/Mouth Pointer


Head Movement Mouse

Colour and Sound Introduced with Too Little on Computing and the Handicapped Dan Eaves CSE5900 –...

Documents

Transcript of Colour and Sound Introduced with Too Little on Computing and the Handicapped Dan Eaves CSE5900 –...