520251: Multimedia Systems

!"#$%&"'()(*+,-%./01/

20 2(34/15/ 2554

!"#$%&'()*+,-,./)

"#01&)+,-,./)

$12163/7!8"29,21*7:16

;<,=&5/><,#?'8@'5

:662A(B=*1#C"2-(A;D"6E?'

8F,1569<>GG

"#01&)+,-,./)

$12163/7!8"29,21*7:16

;<,=&5/><,#?'8@'5

:662A(B=*1#C"2-(A;D"6E?'

8F,1569<>GG

!8"29,$12163/7?<)+';:HG?'8"5I1#2=<6J$(*B(K1-

"#01&)+,-,./)

$12163/7!8"29,21*7:16

;<,=&5/><,#?'8@'5

:662A(B=*1#C"2-(A;D"6E?'

8F,1569<>GG

!8"29,$12163/7?<)+';:HG?'8"5I1#2=<6J$(*B(K1-

!8"29,$12163/7$I#?<

D126JGG$%&"$+LL1M

'()(*+,

"#01&)+,-,./)

"#01&)+,-,./)

*G*A/

G(D (bit)

?GDE (byte)

;,!01/$"# (binary)

;,!01/$(GF: (hexadecimal)

D+A>*/!8"29, (Data Representation)

D+A>*/!8"29,D+A"+:N6[TEXT]

00110000

00110000

00110000

01000001

>GG>O/!"#G(D{!8"29,} (Bit Pattern)

01000001 $12163;<P/"+:!6J A F6%";<P/CI1"

%&/Q':H?'8

"1);<P/CI1;R'$=;*1 25%

:16'()(?*SE!8"29,(DIGITIZATION)

:16?'821!"# digital media

QT8@<6>:62C"2-(A;D"6E$681#!U./

@'5D6# ;TI/ ;61QT8@<6>:62 Paint

Q/:16A1'69<*=&;<P/'()(*+,

*7:16><,#CI1;T(#:15K1- F6%"CI1

analogue QF8;<P/CI1 digital ;6=5:

:662A(B=/=.AI1 digitization

Digitization

K1-3I15

Digitization

G+/*U:;$=5#

Digitization

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

$+LL1M analogue

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

sampling

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris quantization

:6JGA/:16 Analogue to Digital Converter (ADC)

Digitization

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

$+LL1M analogue

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

sampling

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris quantization

:6JGA/:16 Analogue to Digital Converter (ADC)

$+LL1M'()(*+,)J

G66;*1<VLF1;6%&"#

noise

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris quantization

Digitization

23DIGITAL DATACHAPTER 2

fluctuations in thermal energy of the transmission medium. This noise will cause the signal value to be changed. If the signal is an analogue one, these changes will not be detectable – any analogue value is legitimate, and so a signal polluted by noise cannot be distinguished from a clean one. If the signal is a digital one, though, any minor fluctuations caused by noise will usually transform a legal value into an illegal one that lies between quantization levels. It is then a simple matter to restore the original signal by quantizing again. Only if the noise is sufficiently bad to alter the signal to a different level will an error in the transmission occur. Even then, because digital signals can be described numerically, schemes to detect and correct errors on the basis of arithmetic properties of groups of bits can be devised and implemented. Digital signals are therefore much more robust than analogue ones, and do not suffer degradation when they are copied, or trans-mitted over noisy media.

However, looking at Figure 2.3, you can see that some information must have been lost during the digitization process. How can we claim that the digitized result is in any sense an accurate representation of the original analogue signal? The only meaningful measure of accuracy must be how closely the original can be reconstructed. In order to reconstruct an analogue signal from a set of samples, what we need to do, informally speaking, is decide what to put in the gaps between the samples. We can describe the reconstruction process precisely in mathematical terms, and that description provides an exact specification of the theoretically best way to generate the required signal. In practice, we use methods that are simpler than the theoretical optimum but which can easily be implemented in fast hardware.

One possibility is to “sample and hold”: that is, the value of a sample is used for the entire extent between it and the following sample. As Figure 2.4 shows, this produces a signal with abrupt transitions, which is not really a very good approximation to the original (shown dotted). However, when such a signal is passed to an output device – such as a monitor or a loudspeaker – for display or playback, the lags and imperfections inherent in the physical device will cause these discon-tinuities to be smoothed out, and the result actually approximates the theoretical optimum quite well. (However, in the future, improvements in the technology for the playback of sound and picture will demand matching improvements in the signal.)

Clearly, though, if the original samples were too far apart, any reconstruction is going to be inadequate, because there may be details in the analogue signal that, as it were, slipped between samples. Figure 2.5 shows an example: the values of the consecutive samples taken at si and si+1 are identical, and there cannot possibly be any way of inferring the presence of the spike in between those two points from them – the signal could as easily have dipped down or stayed at

Sample and hold Figure 2.4. reconstruction

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

sample & hold

reconstruction

information lost

Digitization

undersampling

24 FUNDAMENTALS

the same level. The effects of such undersampling on the way in which the reconstructed signal will be perceived depend on what the signal represents – sound, image, and so on – and whether it is time-varying or space-varying. We will describe specific instances later. Suffice it to say, for now, that they are manifested as distortions and artefacts which are always undesirable.

It is easy enough to see that if the sampling rate is too low some detail will be lost in the sampling. It is less easy to see whether there is ever any rate at which we can be sure that the samples are close enough together to allow the signal to be accurately reconstructed, and if there is, how close is close enough. To get a better understanding of these matters, we need to consider an alternative way of representing a signal. Later, this will also help us to understand some related aspects of sound and image processing.

You are probably familiar with the idea that a musical note played on an instrument consists of waveforms of several different frequencies added together. There is the fundamental, which is the pitch associated with the note, but depending on the instrument, different numbers of harmonics, or overtones, are also present, and these give the note its distinctive timbre. The fundamental and each harmonic are pure tones – sine waves of a single frequency, so the note is made from the superposition of a set of sine waves.

Any periodic waveform can be decomposed into a collection of different frequency components, each a pure sine wave, in a similar way; in fact, with a little clever mathematics, assuming you don’t mind infinite frequencies, any waveform, periodic or not, can be decomposed into its frequency components. We are using the word “frequency” in a generalized sense, to go with our generalized concept of a signal. Normally, we think of frequency only in the context of time-varying signals, such as sound, radio, or light waves, when it is the number of times a periodic signal repeats during a unit of time. When we consider signals that vary periodically in space, such as the one shown in Figure 2.6, it makes equal sense to consider its frequency as the number of times it repeats over a unit of distance, and we shall often do so. Hence, in general discussions, frequencies, like signals, may be either temporal or spatial.

si si+1

UndersamplingFigure 2.5.

A periodic fluctuation Figure 2.6. of brightness

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

:16 undersampling !"#>F,I#:7;/(' :I"QF8;:('<VLF1Q/:16 reconstruct $+LL1M*=&)J>$'#O,

;-%&"<6J:+/C4MK1-!"# o

utput )J

D8"#QT8"+D61:16 sampling '8A5

Nyquist rate SU&#;*I1:+G 2fh

D+A"5I1#;$=5#*=& sampling

'8A5"+D61*=&;F21J$2

D+A"5I1#;$=5#*=&

undersampling

Q/;6%&"#K1- :16 undersampling *7QF8;:('!"GF5+: F6%";:(' Moiré pattern

22 FUNDAMENTALS

In multimedia, we encounter values of several kinds that change continuously, either because they originate in physical phenomena or because they exist in some analogue representation. For example, the amplitude (volume) of a sound wave varies continuously over time, as does the amplitude of an electrical signal produced by a microphone in response to a sound wave. The colour of the image formed inside a camera by its lens varies continuously across the image plane. As you see, we may be measuring different quantities, and they may be varying either over time or over space (or perhaps, as in the case of moving pictures, both). For this general discussion, we will follow tradition, and refer to the value we are measuring, whatever it may be, as a “signal”, not usually distinguishing between time-varying and space-varying signals.

When we have a continuously varying signal, such as the one shown in Figure 2.2, both the value we measure, and the intervals at which we can measure it, can vary infinitesimally. In contrast, if we were to convert it to a digital signal, we would have to restrict both of these to a set of discrete values that could be represented in some fixed number of bits. That is, digitization – the process of converting a signal from analogue to digital form – consists of two steps: sampling, when we measure the signal’s value at discrete intervals, and quantization, when we restrict the value to a fixed set of levels. Sampling and quantization can be carried out in either order; Figure 2.3 shows a signal being first sampled and then quantized. In the sampling step, you see the continuous signal reduced to a sequence of equally spaced values; in the quantization step, some of these values are chopped off so that every one lies on one of the lines defining the allowed levels.

These processes are normally carried out by special hardware devices, called analogue to digital converters (ADCs), whose internal workings we will not examine. We will only consider the (almost invariable) case where the interval between successive samples is fixed; the number of samples in a fixed amount of time or space is known as the sampling rate. Similarly, we will generally assume that the levels to which a signal is quantized – the quantization levels – are equally spaced.

One of the great advantages that digital representations have over analogue ones stems from the fact that only certain signal values – those at the quantization levels – are valid. If a signal is transmitted over a wire or stored on some physical medium such as magnetic tape, inevitably some random noise is introduced, either because of interfer-ence from stray magnetic fields, or simply because of the unavoidable

An analogue signalFigure 2.2.

Sampling and Figure 2.3. quantization

apisake@gmail.com...................................................... © MacAvon MediaEx Libris quantization

Digitization

Q/:6M=*=&;612=CI16J'+GQ/:16*7 quantization *=&D&7)J;<P/D8/;FD4QF8;:('CA12?2I$2G96ME!"# output

381;612=CI16J'+GQ/:16*7 quantization DI1#W:+/ ;61)J?'8C4MK1-!"#K1-*=&DI1#:+/

381;612=CI16J'+GQ/:16*7 quantization DI1#W:+/ ;61)J?'8C4MK1-!"#K1-*=&DI1#:+/

381;612=CI16J'+GQ/:16*7 quantization DI1#W:+/ ;61)J?'8C4MK1-!"#K1-*=&DI1#:+/

381;612=CI16J'+GQ/:16*7 quantization DI1#W:+/ ;61)J?'8C4MK1-!"#K1-*=&DI1#:+/

381;612=CI16J'+GQ/:16*7 quantization DI1#W:+/ ;61)J?'8C4MK1-!"#K1-*=&DI1#:+/

29DIGITAL DATACHAPTER 2

between areas of those colours would be elided. The effect on black and white images can be seen clearly in Figure 2.11, which shows a gradient swatch using 256, 128, 64, 32, 16, 8, 4, and 2 different grey levels. The original gradient varies linearly from pure white to pure black, and as we reduce the number of different greys, you can see how values band together as they are quantized increasingly coarsely. In a less regular image, the effect manifests itself as posterization, more formally known as brightness contouring, where coloured areas coalesce, somewhat like a cheaply printed poster. Figure 2.12 shows how posterization manifests itself on a colour image: the version on the left is a digital photograph with millions of colours; on the right, we have reduced it to just four colours, and the posterization effect can be seen clearly, especially in the background, where nearly all detail of the landscape has been lost. (Sometimes you may want to posterize an image deliberately – the result can be quite pleasing.)

The numbers of grey levels in our first example were not chosen at random. The most common reason for limiting the number of quantization levels (in any type of signal) is to reduce the amount of memory occupied by the digitized data by restricting the number of bits used to hold each sample value. Here we have used 8, 7, 6, 5, 4, 3, and 2 bits, and finally a single bit. As you can see, although the effect is just about visible with 128 (7 bits) greys, it only becomes intrusive when the number of levels falls to 32 (5 bits), after which deterioration is rapid.

When sound is quantized to too few amplitude levels, the result is perceived as a form of distor-tion, sometimes referred to as quantization noise, because its worst manifestation is a coarse hiss. It also leads to the loss of quiet passages, and a general fuzziness in sound (rather like a mobile phone in an area of low signal strength). Quantization noise is clearly discernible when sound is

PosterizationFigure 2.12.

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

:16G=G"+'!8"29,(Data Compression)

Compression

30 FUNDAMENTALS

sampled to 8 bits (256 levels), but not (except in the opinion of some audiophiles) at the 16 bits (65536 levels) used for audio CDs.

CompressionYou will learn, as we examine the individual media types in detail, that a characteristic property of media data is that it occupies a lot of storage. This means, in turn, that it needs a lot of band-width when it is transferred over networks. Storage and bandwidth are limited resources, so the high demands of media data can pose a problem. The common response to this problem is to apply some form of compression, which means any operation that can be performed on data to reduce the amount of storage required to represent it. If data has been compressed, an inverse decompression operation will be required to restore it to a form in which it can be displayed or used. Software that performs compression and decompression is often called a codec (short for compressor/decompressor), especially in the context of video and audio.

Compression algorithms can be divided into two classes: lossless and lossy. A lossless algorithm has the property that it is always possible to decompress data that has been compressed and retrieve an exact copy of the original data, as indicated in Figure 2.13. Any compression algo-rithm that is not lossless is lossy, which means that some data has been discarded in the compression process and cannot be restored, so that the decompressed data is only an approximation to the original, as shown in Figure 2.14. The discarded data will represent information that is not significant, and lossy algorithms which are in common use do a remarkable job at preserving the quality of images, video and sound, even though a considerable amount of data has been discarded. Lossless algorithms are generally less effective than lossy ones, so for most multi-media applications some lossy compression will be used. However, for

text the loss of even a single bit of information would be significant, so there is no such thing as lossy text compression.

It may not be apparent that any sort of data can be compressed at all without loss. If no informa-tion is being discarded, how can space be saved? In the case of text, lossless compression works because the usual representation of characters does not make the most efficient use of storage – it’s efficient for the operations most often applied to text, but wasteful of storage. As we explained earlier, text can be represented just by arbitrarily mapping characters to numbers. It is normal to store each of these numbers in the same number of bits. If, instead, we were to use a different number of bits for each character, and assign fewer bits to the most commonly encountered char-acters, the total amount of space required would be less. Lossless compression works by reorgan-izing data in this and other ways. The resulting compressed files are not at all convenient to work

original data

compress

decompress

compressed data

Lossless compressionFigure 2.13.

apisake@gmail.com...................................................... © MacAvon MediaEx Libris

31DIGITAL DATACHAPTER 2

with directly, so storage schemes which make less than optimal use of the available bits are used most of the time instead.

Lossless and lossy compression are not entirely separate techniques. Most lossy algorithms make use of some lossless technique as part of the total compression process. Generally, once insignificant information has been discarded, the resulting data is more amenable to lossless compression. This is particularly true in the case of image compression, as we will explain in Chapter 4.

Ideally, lossy compression will only be applied at the latest possible stage in the preparation of the media for delivery. Any processing that is required should be done on uncompressed or losslessly compressed data whenever possible. There are two reasons for this. When data is lossily compressed the lost information can never be retrieved, which means that if data is repeatedly compressed and decompressed in this way its quality will gradually deteriorate. Additionally, some processing operations can exaggerate the loss of quality caused by some types of compression. For both these reasons, it is best to work with uncom-pressed data, and only compress it for final delivery.

This ideal cannot always be achieved. Video data is usually compressed in the camera, and although digital still cameras that produce uncompressed images are increasingly common, many cheaper cameras will compress photographs fairly severely when the pictures are being taken. It may be necessary for a photographer to allow the camera to compress images, in order to fit them onto the available storage. Under these circumstances, data should be decompressed once, and an uncompressed version should be used for working. This may not, however, be practical for video.

The differing characteristics of different types of media mean that the sort of data that can be discarded is different for each type. There are therefore lossy algorithms specific to each type – JPEG for image compression, MP3 for audio compression, and so on. Lossless algorithms can be applied to any type of data but the amount of compression they lead to will depend on the type of data, any structure it may have and the actual data. For instance, lossless compression is usually fairly ineffective when applied to photographs, but computer-generated images that feature large areas of flat single colour will compress well. We will expand on this point in Chapter 4.

No compression algorithm can work magic. It is a general property of any compression scheme that there will be some data for which the “compressed” version is actually larger than the

original data

decompressed data

compressed data

compress

decompress

Lossy compressionFigure 2.14.

apisake@gmail.com...................................................... © MacAvon MediaEx Libris