Video Compression 101

Video Compression 101

Howard StatemanPQD, ITD

Sony Electronics

05/01/23 Video Compression 101 2

Why Compress?

• Bandwidth

• Disk Space


Streaming Media

• Data must be delivered in a timely manner– Text data is “burst” data

• Does not matter when packets arrive• Or in what order

– Audio/Video data is “streaming” data• Must arrive on time and in the right order

• Challenges– Noisy phone lines, low connection speeds– High network traffic, collisions, router hops


The Numbers - VideoVideo Bit Rates

Type Width Height fps Pixels/sec Bits/sec* Mbps**DVD 720 480 30 10,368,000 165,888,000 162

Broadcast 640 480 30 9,216,000 147,456,000 144

CIF (VHS) 320 240 30 2,304,000 36,864,000 36

QCIF 160 120 15 288,000 4,608,000 4.5

Postage Stamp 80 60 15 72,000 1,152,000 1.125

* Captured video has 2 bytes per pixel, and 8 bits per byte

** 1Mb = 1024000 bits


The Numbers - AudioAudio Bit Rates

Type High Freq Sample rate Bit depth Channels bps*CD 22,050 44,100 16 2 1,411,200

FM Radio 15,000 30,000 8 2 480,000

AM Radio 4,000 8,000 8 1 64,000

Telephone 2,500 5,000 4 1 20,000

*Audio Bit rate = number of channels x samples/sec x bit depth

This presentation will focus on video – audio is a minor part of video files


Basic Video Compression

• Reduce image size

• Reduce frame rate


Reduce Image Size

• 320x240 is ¼ the pixels of 640x480

• 160x120 is 1/16 the pixels of 640x480

• Compression must maintain aspect ratio


Aspect Ratio Examples

4:3 Original

4:2 (squeezed vertically)3:5 (squeezed horizontally)


Aspect Ratios• The Ratio of Width to Height

• 4:3 is standard TV and computer screen ratio– 640 x 480

• 16:9 is standard cinema ratio (“letterbox”)– 1920 x 1080

• Out of Aspect Squeezed Image– Video compression codecs must maintain aspect ratio


4:3 Aspect Ratio SizesSize W x H PixelsNTSC Full 640x480 307200NTSC 1/4 320x240 76800NTSC 1/16 160x120 19200PAL Full 768x576 442368PAL 1/4 384x288 110592PAL /16 192x144 27648CIF 352x288 101376QCIF 176x144 25344DVD (PC) 720 x 486 349920


Reduce Frame Rate

• Computer Video (NTSC)– 30 frames per second (fps)

– Most people perceive motion at 8 fps• < 8 fps is seen as a slide show

– 15 fps is target for dial-up streaming video• ½ the data of 30 fps• Still appears as a movie


Advanced Techniques

• Require analysis and computation• “Lossy” compression

– Data is pulled out of the video file• Noise• Insignificant color changes• High frequency data

• “Lossless” compression– No data is lost, but file size is reduced

• Shorthand used for repetitive data


Codecs

• Encode-Decode Algorithms• Use “codebook” to increase compression

– Analyze color and brightness changes– Motion prediction– Repetitive data

• < 10% of data changes from frame to frame• Codec encodes only changed data

– 90% compression!

– Noise filtering


Noise

• GIGO– Garbage in, Garbage out– Best to start with clean

video source

• Codec can filter some– Takes time, CPU– Mixed results

• Depends on content


Compressing Color

• 16.8 Million colors supported by PC– Human eye/brain can’t tell adjacent colors apart– Especially not at 1/30th or 1/15th of a second

• Reduce color map to fewer colors– 65,536 colors in 16-bit SVGA mode– 256 Colors in 8-bit VGA mode

• Means fewer changes in color• Higher compression


Can You Tell The Difference?R-G-B

24-bit color map

64-128-192 65-128-192 66-128-192 67-128-192

64-129-192 64-130-192 64-131-192 64-132-192

64-128-193 64-128-194 64-128-195 64-128-196

Can You Tell The Difference?


High Frequency Filtering

• Takes the average of neighboring pixels to produce a lower data rate or frequency– Softens the image– Makes video look more analog

• Studies show most streaming media viewers prefer soft, smooth image


Frame Compression

• Predictive Coding • Motion Estimation and Compensation• Picture Interpolation

The greatest amount of data redundancy in video is in the vast number of similar pixels in successive frames that change

very little, or not at all.


Compressed Frame Types

• I-Frame (Intra-frame)– Full JPEG data frame

• P-Frame (Predictive)– “Delta” from previous frame

• B-Frame (bidirectional)– “Delta” from next and previous (I or P) frame– CPU-intensive

• Not recommended in live streaming


I-Frames

• A.K.A. “Key Frames”• Contain 100% of the data for that frame• Used as milestones for compressed data• Recovery points in streaming video

– Dropped I-frame can mean no video until next I-frame is received

• First frame in a set is an I-frame


P-Frames

• Only has data which differs from I-Frame. – Each P-Frame has a "motion offset" to the

corresponding I-Frame– Each pixel in a P-Frame is encoded as a delta

from the corresponding I-Frame pixel– Ignores data in B-frames

• Dropped P-frame may not be noticed– Contains very little data


P-Frame Encoding

Frame 1 Frame 2

P-Frame


B-Frames

• Bidirectional version of P-frame– Only sends data which differs from preceding

or successive (I or P) frame. – Each B-Frame has a "motion offset" to the

preceding (I or P) frame. – Each B-Frame has a "motion offset" to the

successive (I or P) frame. – Each pixel in a B-Frame is encoded as a delta

from the average of the corresponding pixel in the preceding and successive frames.


B-Frame references

1 2 3 4 5 6

• B-frame 2 includes data from I-frame 1 and P-frame 3

• B-frames 4 and 5 include data from P-frames 3 and 6

• B-frames 4 and 5 are redundant

I B P B B P


Framing Trade-offs

• More I-Frames– Bigger file– Uses more bandwidth– Quicker recovery from dropped frames

• More P-Frames– Smaller file– Uses less bandwidth– Longer recovery from dropped frames


Framing Trade-offs (cont.)

• More B-Frames– Uses more CPU– Slower file creation

• Requires I-frames and P-frames to be built first– More robust

• Less damage from dropped frames– Too many B-frames in a row is a waste


Streaming Video Demo 1

300kbps LAN Video•320 x 240 pixels

•30fps

•File size = 960 kb

56kbps Modem Video•160 x 120 pixels

•15fps


Requires Windows Media Player 9.x or higher

http://www.howeird.com/Video/LAN/oz1.wmv

http://www.howeird.com/Video/modem/oz1.wmv

http://www.howeird.com/Video/LAN/oz1.wmv

http://www.howeird.com/Video/modem/oz1.wmv


Streaming Video Demo 2

300kbps LAN Video•320 x 240 pixels

•30fps


56kbps Modem Video•160 x 120 pixels

•15fps


Requires Windows Media Player 9.x or higher

http://www.howeird.com/Video/LAN/kate1.wmv

http://www.howeird.com/Video/LAN/kate1.wmv


Notes on Streaming Demos

• Demo 1– High motion

• Less compression• Jerky at modem speeds

– High color saturation• Less compression• Good viewing

experience

• Demo 2– Low Motion

• More compression• Okay at modem speeds

– Low color saturation• Excellent compression• Diminished viewing

experience

Both videos demonstrate “GIGO” principle – both were digitized from low-quality analog camcorder tapes which had been copied to VHS


Streaming Video SitesSome examples can be found on the author’s site:

http://www.howeird.com/theaterVid.htmlhttp://www.howeird.com/whale.html

Movie Trailers:http://www.sonypictures.com/previews/index.htmlhttp://trailers.warnerbros.com/web/index.jsphttp://www.paramount.com/motionpictures/http://www.netbroadcaster.com/movies/trailers.html

TV News:http://www.msnbc.com/m/v/video_news.asp?0cv=c641http://www.cbsnews.com/sections/i_video/main500251.shtml


Recording for Compression [1]

“Talking Heads”• Use a solid backdrop

– Minimize pixel changes away from main focus• Use tripods

– Minimize camera motion• Keep “talent” seated

– Ideally, only the lips should move– Especially important in close-ups



Sports• Not much you can do

– Every head in the crowd is a moving pixel• Close-ups, following the athlete

– Hand-held camera motion– Background is in motion

• Stay in focus– Blurred moving images don’t compress well



Advertisements• Avoid motion in the background

– Fountains, ocean, people, moving vehicles– Best to provide a backdrop

• Use wide shots– Smaller subject means fewer pixels in motion

• Use static shots– Close-ups of product


A Note About Audio• Good sound will “fool” the viewer

– People take their cue from audio– If the audio is excellent, most people will think the

video is better than it actually is• Poor sound hurts

– If the audio is poor, most people will think the video is worse than it actually is

Audio bandwidth is an insignificant part of a video clip – use as much as you need


Conclusions• Video compression saves:

– Disk space– Bandwidth

• Compression techniques– Basic:

• Reduce the image size• Reduce the number of frames per second

– Advanced:• Codecs using codebooks

– Eliminate redundant data– Provide a shorthand for repeating data

Video Compression 101

Documents

Transcript of Video Compression 101