Theoretical Foundations of Multimedia

Theoretical Foundations of Multimedia Chapter 3

Theoretical FoundationsTheoretical Foundationsof Multimediaof Multimedia

Dr. Robert S. TannenbaumDirector, Academic Computing Services

University of [email protected]


Hardware that Enables Hardware that Enables MultimediaMultimedia

Input and Output DevicesInput and Output Devices

Virtual Reality DevicesVirtual Reality Devices

Modems and Network InterfacesModems and Network Interfaces


Input and Output DevicesInput and Output Devices

MonitorsMonitors Speakers and MIDI interfacesSpeakers and MIDI interfaces VR helmets and immersive displaysVR helmets and immersive displays Keyboards and OCR devicesKeyboards and OCR devices Digital cameras, scanners, & CD-ROMsDigital cameras, scanners, & CD-ROMs MIDI keyboards and microphonesMIDI keyboards and microphones Video cameras and frame grabbersVideo cameras and frame grabbers Mice, track balls, joysticks, and VRMice, track balls, joysticks, and VR gloves and gloves and

wandswands

OutputOutput

InputInput


MonitorsMonitors

A simplified cathode ray tube (CRT)A simplified cathode ray tube (CRT)


MonitorsMonitors

Pixel — a picture element; a dot Pixel — a picture element; a dot of color on the screenof color on the screen

Three different phosphors at Three different phosphors at each pixel to create the coloreach pixel to create the color

RGB (RGB (Red, Red, Green, Green, BlueBlue)) CYM (CYM (CyanCyan, , YellowYellow, , MagentaMagenta))


MonitorsMonitors

Raster ScanningRaster Scanning


MonitorsMonitors

Refresh rate — the frequency atRefresh rate — the frequency atwhich the phosphors are excitedwhich the phosphors are excited

Normally the refresh rate is givenNormally the refresh rate is given in Hertzin Hertz For flicker-free images 75 Hz orFor flicker-free images 75 Hz or

faster is desirablefaster is desirable The refresh rate for a projector The refresh rate for a projector

needs to be coordinated with the needs to be coordinated with the monitormonitor


MonitorsMonitors

The digitized image to be displayed The digitized image to be displayed must be must be stored in a bufferstored in a buffer

The stored image is said to beThe stored image is said to be “bit-“bit-mapped,” because, for mapped,” because, for monochrome monochrome images, the map images, the map used just one bit per used just one bit per pixelpixel

Multimedia monitors use 24 bitsMultimedia monitors use 24 bits per per pixel (8 for each color); canpixel (8 for each color); can define define >16 million colors>16 million colors


A Good Multimedia MonitorA Good Multimedia Monitor

Large enough for comfortableLarge enough for comfortable viewing, viewing, probably 15” or greaterprobably 15” or greater

Pixel size of no more than 0.28mmPixel size of no more than 0.28mm Refresh rate of at least 75 HzRefresh rate of at least 75 Hz Capable of displaying 24-bit colorCapable of displaying 24-bit color Designed for the CPU and operatingDesigned for the CPU and operating systemsystem Ergonomically comfortable andErgonomically comfortable and

attractiveattractive


Speakers and Speakers and MIDI Interfaces MIDI Interfaces

Storage of digitized sound filesStorage of digitized sound files Reproduction via digital-to-analog conversion sent to a Reproduction via digital-to-analog conversion sent to a

loudspeakerloudspeaker Built-in speakers often do not have sufficient fidelityBuilt-in speakers often do not have sufficient fidelity Low-powered (3- to 5-watt) external speakers or head-Low-powered (3- to 5-watt) external speakers or head-

phones will serve a single user and provide excellent phones will serve a single user and provide excellent fidelityfidelity


Storage of synthesizer command files Storage of synthesizer command files Create the sounds by sending theCreate the sounds by sending the commands commands

to a synthesizerto a synthesizer Musical Instrument Digital Interface (MIDI) Musical Instrument Digital Interface (MIDI)

standard (1982)standard (1982) MIDI includes both a hardware and a message MIDI includes both a hardware and a message

standardstandard

Speakers and Speakers and MIDI Interfaces MIDI Interfaces


Speakers and MIDI InterfacesSpeakers and MIDI Interfaces

MIDI hardware standard defines cables, MIDI hardware standard defines cables, connectors, circuits, andconnectors, circuits, and electrical electrical signalssignals

MIDI message standard defines aMIDI message standard defines a Device number for multiple device systems Control segment that gives an instrument-specific

command Data segment containing the information needed by

that instrument for that command


Alphanumeric KeyboardsAlphanumeric Keyboards

For entering commands, text, For entering commands, text, and and datadata

Each key is a switch that closes when it is Each key is a switch that closes when it is depressed, sending a code to the CPUdepressed, sending a code to the CPU

The arrangement of the keys may varyThe arrangement of the keys may vary The most common is QWERTYThe most common is QWERTY Another arrangement is DvorakAnother arrangement is Dvorak


Choosing a KeyboardChoosing a Keyboard

Does it include Does it include allall of the needed of the neededcharacters, including command characters, including command keys?keys?

Is it ergonomically comfortableIs it ergonomically comfortableand safe, preventing repetitive and safe, preventing repetitive stress stress syndrome?syndrome?


Optical Character Recognition Optical Character Recognition (OCR)(OCR)

Hardware — scans the text imageHardware — scans the text image Software — systematically checksSoftware — systematically checks the the

entire image for patterns of light and dark entire image for patterns of light and dark that it recognizes as alphabetic, numeric, that it recognizes as alphabetic, numeric, or punctuation charactersor punctuation characters

OCR software entails pattern OCR software entails pattern recognition, a sophisticated logic problemrecognition, a sophisticated logic problem


E e E E e e E E Ee e E E E e e E e e

It is relatively easy for a human to recognize each of these characters as the letter “e.” For the pattern recognition logic in OCR software, this is very difficult.

Optical Character Recognition Optical Character Recognition (OCR)(OCR)


Digital Cameras and ScannersDigital Cameras and Scanners

Real Image Real Image —— a portion of what is physically present in nature

Digital Image Digital Image — — a representation of a real image in which individual points are encoded to represent the wavelength and intensity of light at that point

Still Image Still Image — — a single snapshot of an instant; may be real or digital

Motion Image Motion Image — — a sequence of images that, when viewed consecutively at the appropriate rate, gives the impression of continuous motion; may be digital or analog


ScannersScanners

Schematic Drawing of a ScannerSchematic Drawing of a Scanner


Digital CamerasDigital Cameras

Schematic Drawing of a Digital CameraSchematic Drawing of a Digital Camera


Digital Cameras and ScannersDigital Cameras and Scanners

Quality of the optics and the scanning Quality of the optics and the scanning mechanism, mechanism, which determines focuswhich determines focus

Precision of the photosensitive cells, which determines Precision of the photosensitive cells, which determines the accuracy of the encoding of intensity and wavelength the accuracy of the encoding of intensity and wavelength datadata

Resolution of the instrument in dots per inch, which Resolution of the instrument in dots per inch, which determines graininessdetermines graininess

Amount of storage available, which determines the total Amount of storage available, which determines the total size of an image that can be digitizedsize of an image that can be digitized

Image quality depends on the:Image quality depends on the:


Inputting ImagesInputting Images

Memory required to store a 5” x 7” snapshotMemory required to store a 5” x 7” snapshot

Dots/inch resolutionof snapshot image

100300600

1200

Bytes requiredfor storage

1.05 Mb9.45 Mb37.8 Mb

151.2 Mb

Assuming no compression, 24 bits per pixel


Video Cameras and Frame Video Cameras and Frame GrabbersGrabbers

Video cameras are similar to digital camerasVideo cameras are similar to digital cameras Except that a video camera takes image after image Except that a video camera takes image after image

continuouslycontinuously The output from many video cameras is analog and requires The output from many video cameras is analog and requires

digitizing circuitry to make the image usable in a computerdigitizing circuitry to make the image usable in a computer Digital camcorders are now available Digital camcorders are now available Frame grabber software allows the capture ofFrame grabber software allows the capture of a single a single

still image from the video streamstill image from the video stream Frame grabbed images are of rather low resolution, Frame grabbed images are of rather low resolution,

however, <80-90 dots/inchhowever, <80-90 dots/inch


Microphones and Microphones and MIDI KeyboardsMIDI Keyboards

For input of soundFor input of sound Microphones capture sound waves from the air as an analog Microphones capture sound waves from the air as an analog

signalsignal The analog signal must be digitized to be stored and then The analog signal must be digitized to be stored and then

replayed by the computerreplayed by the computer Digitizing at <10,000 Hz is adequate forDigitizing at <10,000 Hz is adequate for

speech; 20,000-40,000 Hz is needed for musicspeech; 20,000-40,000 Hz is needed for music MIDI keyboards usually look like piano key-MIDI keyboards usually look like piano key-

boards with extra switches and controlsboards with extra switches and controls MIDI keyboards encode and transmit musical MIDI keyboards encode and transmit musical

information according to the MIDI standardinformation according to the MIDI standard

Theoretical Foundations of Multimedia

Documents

Transcript of Theoretical Foundations of Multimedia