Avlm 2009 Compression Erik Luyten
-
Upload
avlm2009avnet -
Category
Technology
-
view
1.219 -
download
2
description
Transcript of Avlm 2009 Compression Erik Luyten
compression
from production format to distribution format
What's this ?
2AVLM Training26/05/09
René Margritte: La Trahison des Images (1928-29)
3AVLM Training26/05/09
Moving images !?
4AVLM Training26/05/09
Temporal resolution
2 images/second 5 images/second
12,5 images/second 25 images/second
5AVLM Training26/05/09
Spatial resolution
7 x 10 = 7028 x 40 = 1120
14 x 20 = 280
112 x 160 = 1792056 x 80 = 4480
448 x 640 = 309120224 x 320 = 71680
6AVLM Training26/05/09
Color resolution
24 bit color pallet244 KB file
8 bit color pallet28 KB file
4 bit color pallet16 KB file4 bit color palletNo “dithering”
8 bit optimized pallet28 KB file
7AVLM Training26/05/09
Video signal(uncompressed)768 (720) pixels
576
pixe
ls
25 images/secondEach pixel has 3 values: (3 bytes)• Red: 256 steps (1 byte)• Green: 256 steps (1 byte)• Blue: 256 steps (1 byte)
Datarate for the component video signal:720(pix) x 576(lines) x 2(byte) x 25(images)=20 MByte/sec(160 Mbit/sec)
Datarate for the RGB video signal:768(pix) x 576(lines) x 3(byte) x 25(images)=32 MByte/sec(256 Mbit/sec)
8AVLM Training26/05/09
Audio signalSampling Rate: 44,1 kHz (CD)Bit Depth: 16 bit (65.536 steps)Stereo: x 216 bit x 2 x 44.100 = 1.411.200 bps 1,4 Mbit/sec
Production systems normally use a sampling rate of 48 kHz16 bit x 2 x 48.000 = 1.536.000 bps
9AVLM Training26/05/09
Video distribution ?
Disk
floppy disk = 0,05 seconds
CD-rom = 20 seconds
1 Gbyte hard disk = 32 seconds
16 Gbyte array = 8 min 30 sec
Network
Ethernet 100 Mbit/sec (shared) LAN1/2.5
Ethernet 10 Mbit/sec (shared !) LAN1/25
ISDN lijn ( 2B + D) : 128Kbit/sec 1/2000
Modem connection : 56 Kbit/sec 1/4600
1 second video: 32 Mbyte (256 Mbit/sec) 1 minute video: 1,9 GByte 1 hour video: 111 GByte
10AVLM Training26/05/09
Compression techniques
Intraframe (spatial compression)Compression within a single frame
Interframe (temporal compression)Compression between successive frames
LosslessNo information loss
Lossy(Important) information is lost (f.i. medical applications)
SymmetricCoding and decoding equally processor intensive
AsymmetricCoding is more processor intensive than decoding
11AVLM Training26/05/09
Intraframe compression (in one frame)
Run Length Encoding• without compression:
one line of 100 (white) pixels 100 x 3 bytes = 300 bytes
• RLE compression 100 identical white pixels in a row 2 bytes: total amount of pixels involved 3 bytes for the colour information 2 + 3 = 5 bytes of info for 100 pixels
• Compression factor of 60:1 without information loss
100 pixels
12AVLM Training26/05/09
Other Intraframe techniquesDiscrete Cosine Transformation
• JPEG, MPEG, DV,…• Type of Fourier Transformation • Blocks of 8 x 8 pixels and/or 16 x 16
Wavelet Compression• MPEG-4 (still image), JPEG2000• Improving image resolution as more data are received
Vector QuantizationFractal Compression
13AVLM Training26/05/09
Interframe compression
Image 124
Image 125
14AVLM Training26/05/09
Interframe: MPEG (Moving Picture Expert Group)
I B B P B B P B B P B B I
GOP Group Of PicturesI-pictures: (Intra pictures) reference image with complete information
for one frame (scene changes, ...)
P-pictures: (Predicted pictures) images are “predicted” using the information in earlier I or P pictures.
B-pictures: (Bi-directional predicted) pictures are recomposed from information in earlier or later I or P pictures. B-pictures are never used as reference for other images.
Group Of Pictures (GOP)
15AVLM Training26/05/09
Movie Formats
File Format:Specifies how data are arranged in a file
(interleaving video and audio)Codec:
Specifies how (image) data is compressedArchitecture:
Synchronisation of different media components (video, audio, language tracks, text,…. tracks)
Control for interactivity, alternate tracks,…
16AVLM Training26/05/09
Movie Architectures
• QuickTime• AVI• Windows Media Player• Real Media• Flash ?• MPEG-4
Media platformplatform distribution cost alternates remarks
QuickTimeMacWindows(Linux)
CD romDownload (HTTP)Streaming (RTSP)
Player = freePro = €30Server= open srce
RTSP alternatesHTTP alternatesMixed alternatesSkip prot.,instant -on
QTVR, text, midi, SMIL,..production and distribution
H.264
RealWindowsMacLinux
Streaming (RTSP)Download (HTTP)
Player = freeSuperpass=$13/mServer=payable
RTSP alternates(Surestream techn.)Dynamic
only distribution and mainly for streaming
WindowsMedia
Windows(Mac)(Linux)
Streaming ( )Download (HTTP)CD rom
Player=freeServer=Windows2003
RTSP alternatesDynamic
Production anddistributionVersion 9 on windows platform
Flash(Spark,
On2, h264)
WindowsMacLinux
CD romDownload (HTTP)Streaming
Player=freeServer=€4759
? H.264
MPEG-4WindowsMacLinux
Streaming (RTSP)Download (HTTP)CD romDVD, GSM, ….
Player=QT, Real,…Server=open srceLicense fee
scalable (in the future)
Industrie standaardfrom handheld to HD
H.264
17AVLM training24/05/09
18AVLM Training26/05/09
File Formats
• DV• MPEG-1• MPEG-2• MPEG-4
19AVLM Training26/05/09
Codec’s• Picture:
• Cinepak• Indeo (Intel Indeo)• H.261, H.263, H.264• Sorenson• M-JPEG• MPEG-1, MPEG-2, MPEG-4• DV
• Sound:• IMA • Qualcomm PureVoice, Qdesign Music,… • MP3• AAC (Advanced Audio Coding)• WMA, RealAudio
20AVLM Training26/05/09
H.264
Codec’s and bitrate
MPEG-2MPEG-2
MPEG-4
56 kbit 256 Mbit1 Mbit 9 Mbit 25 Mbit200 kbit
MPEG-1
H-261H-263
SorensonDV
uncompr.
Distribution codec’s Production codec’s
profile 0 profile 1
MPEG-4
HD distribution Studio productionPIM, GSM, ...
Web ?
21AVLM Training26/05/09
MPEG-4 differenceMPEG-1, MPEG-2video
subtitles
background
graphics
Ren
derin
g an
dC
ompo
sitio
nC
ompr
essi
ontransport
Dec
ompr
essi
on
displayand
scaling
video
subtitles
background
graphics
transport
transport
transport
transport
com
pres
sion
Sca
ling,
Ren
derin
g an
dC
ompo
sitio
n
Dec
ompr
essi
on
display
Bitmap stream
ASCI code
Bitmap image
Vector code
MPEG-4
displayand
scaling
display
22AVLM Training26/05/09
Video to Computer• Video recording
• Interlaced, 25 frames - 50 fields per second
• Restricted to 576 effective lines (PAL) and 3x4 picture aspect ratio.
• Computer image• Non-interlaced, 60 to 80 frames per
second• No restrictions on picture format and
aspect ratio• De-interlacing of video material is
desirable.
23AVLM Training26/05/09
De-interlacing
1
2
3
311
312
313
313
314
315
623
624
625
Field 1Field 2
24AVLM Training26/05/09
Delivery methods• DVD, CD, USB stick,…• Internet
• Progressive download (HTTP, FTP)• “file transport” type (TCP protocol)• No data loss: resent lost packages• No real-time events• Complete file must be downloaded to allow random
access. • No special server needed (normal web service over
http)• Streaming media (RTP)
• RTSP transport (UDP protocol)• Lost data (probably) will not be recovered• Real-time events possible• “Random access” possibility (only needed fragments will be
downloaded)• Needs one or more(worldwide proxy delivery)
streaming servers
25AVLM Training26/05/09
IP distributionHTTPProgressive Download
RTSP StreamingUnicast
RTSP StreamingMulticast
Protocol/Port TCP (http)Port 80
UDP (RTP)Port 554, 7070,... (80)
UDP (RTP)Port 554, 7070,...
Firewall Same as Web accessno problems
UDP(RTP) accessProxy server, http tunnel
Only in multicast enabled segments
Live broadcast No Ja (through relay server) Yes
Recorded mat. Yes Yes Yes
Video on Demand Yes(after complete download) Yes No
Alternates Yes Yes Yes
Copy protected Can be circumvented Can not be saved (easily) Can not be saved (easily)
Packet loss (quality) No (resent) Yes (discard) Yes (discard)
Buffering 100 %complete file
5-8 secondslatency
5-8 secondslatency
Servers WWW serverApache, NT,... Streaming Server RTP Streaming Server RTP
26AVLM Training26/05/09
QuickTime ?• QuickTime is a container that allows other
media to work in synchronization with video and audio.
• QuickTime is more than video and provides functionality for:• QuickTime VR• Interactivity (Flash, JavaScript, Wired Sprites, Pop-up
menu, chapter track, http track, ....)• Components (Zoomify, Pulse, Kaydara, ...)
• FastStart download and/or RTSP streaming• Alternates for:
• Network connection(datarates)• Language versions (audio track and/or subtitle track)• Processor speed, QT version, choke speed
27AVLM Training26/05/09
Compression settings ?The download data rate for progressive download can be higher than the
network connection limit. The client application simply waits until enough data have been received before activating the play button. (no information is lost)
With streaming the stream data rate has to be lower than the network connection limit. There will be no time to recover lost or late packages and the client application will only receive partial information resulting in frozen images and broken sound.
download streamingsmall mediu
mlarge 112 256 384
Image size 240x180
320x240
480x360
240x180
240x180
320x240
Images/sec 12,5 25 25 12,5 12,5 25Codec h.264
VBRh.264VBR
h.264VBR
h.264CBR
h.264CBR
h.264CBR
Audiocodecsample rate
MPEG-416000
MPEG-432000
MPEG-448000
MPEG-422kMon
oMPEG-416000
MPEG-416000
Target bitrateV+A kbits/sec
168+32200
436+64500
896+128
102464+16
80176+32
208276+32
308
28AVLM Training26/05/09
External web server
Toledo server
Reference movie with alternates
QT Streaming server
client
29AVLM Training26/05/09
Toolswindows mac OSX
Digitalisation
Adobe PremiereSorenson Squeeze (499$)Movie Maker, AVID,...
Final Cut ProSorenson Squeeze (499$)iMovie, AVID,....
Editing Adobe PremiereMovie Maker, AvidQT Pro (€40)
Final Cut ProiMovie, AVIDQT Pro(€40)
Compression
Episode ($495) Sorenson SqueezeCleaner XL 1.5 ($595)Premiere, QT Pro,Super, ffmpeg,....
Episode ($495) Sorenson SqueezeCleaner 6.5 ($595)Compressor (FCP), QT Pro,VisualHub, ffmpegX, ...
Interaction LiveStage, Flash CS4Cleaner XLQT Pro
LiveStage, CS4Cleaner 6QT Pro
30AVLM Training26/05/09
Books• Compression for Great Digital Video (©
2002)Ben Waggoner, CMP Books, ISBN 1-57820-111-X
• Video Compression Demystified (© 2001)Peter Symes, McGraw-Hill, ISBN 0-07-136324-6
• Streaming Media (© 2003)Tobias Künkel,Wiley Computer Publishing,ISBN 0-470-84724-7
• QuickTime for the Web 3th ed. (© 2004)Apple,Morgan Kaufmann Publishers,ISBN 1-55860-904-0