Post on 23-Jan-2017
Video Compression 101
Howard StatemanPQD, ITD
Sony Electronics
05/01/23 Video Compression 101 2
Why Compress?
• Bandwidth
• Disk Space
05/01/23 Video Compression 101 3
Streaming Media
• Data must be delivered in a timely manner– Text data is “burst” data
• Does not matter when packets arrive• Or in what order
– Audio/Video data is “streaming” data• Must arrive on time and in the right order
• Challenges– Noisy phone lines, low connection speeds– High network traffic, collisions, router hops
05/01/23 Video Compression 101 4
The Numbers - VideoVideo Bit Rates
Type Width Height fps Pixels/sec Bits/sec* Mbps**DVD 720 480 30 10,368,000 165,888,000 162
Broadcast 640 480 30 9,216,000 147,456,000 144
CIF (VHS) 320 240 30 2,304,000 36,864,000 36
QCIF 160 120 15 288,000 4,608,000 4.5
Postage Stamp 80 60 15 72,000 1,152,000 1.125
* Captured video has 2 bytes per pixel, and 8 bits per byte
** 1Mb = 1024000 bits
05/01/23 Video Compression 101 5
The Numbers - AudioAudio Bit Rates
Type High Freq Sample rate Bit depth Channels bps*CD 22,050 44,100 16 2 1,411,200
FM Radio 15,000 30,000 8 2 480,000
AM Radio 4,000 8,000 8 1 64,000
Telephone 2,500 5,000 4 1 20,000
*Audio Bit rate = number of channels x samples/sec x bit depth
This presentation will focus on video – audio is a minor part of video files
05/01/23 Video Compression 101 6
Basic Video Compression
• Reduce image size
• Reduce frame rate
05/01/23 Video Compression 101 7
Reduce Image Size
• 320x240 is ¼ the pixels of 640x480
• 160x120 is 1/16 the pixels of 640x480
• Compression must maintain aspect ratio
05/01/23 Video Compression 101 8
Aspect Ratio Examples
4:3 Original
4:2 (squeezed vertically)3:5 (squeezed horizontally)
05/01/23 Video Compression 101 9
Aspect Ratios• The Ratio of Width to Height
• 4:3 is standard TV and computer screen ratio– 640 x 480
• 16:9 is standard cinema ratio (“letterbox”)– 1920 x 1080
• Out of Aspect Squeezed Image– Video compression codecs must maintain aspect ratio
05/01/23 Video Compression 101 10
4:3 Aspect Ratio SizesSize W x H PixelsNTSC Full 640x480 307200NTSC 1/4 320x240 76800NTSC 1/16 160x120 19200PAL Full 768x576 442368PAL 1/4 384x288 110592PAL /16 192x144 27648CIF 352x288 101376QCIF 176x144 25344DVD (PC) 720 x 486 349920
05/01/23 Video Compression 101 11
Reduce Frame Rate
• Computer Video (NTSC)– 30 frames per second (fps)
– Most people perceive motion at 8 fps• < 8 fps is seen as a slide show
– 15 fps is target for dial-up streaming video• ½ the data of 30 fps• Still appears as a movie
05/01/23 Video Compression 101 12
Advanced Techniques
• Require analysis and computation• “Lossy” compression
– Data is pulled out of the video file• Noise• Insignificant color changes• High frequency data
• “Lossless” compression– No data is lost, but file size is reduced
• Shorthand used for repetitive data
05/01/23 Video Compression 101 13
Codecs
• Encode-Decode Algorithms• Use “codebook” to increase compression
– Analyze color and brightness changes– Motion prediction– Repetitive data
• < 10% of data changes from frame to frame• Codec encodes only changed data
– 90% compression!
– Noise filtering
05/01/23 Video Compression 101 14
Noise
• GIGO– Garbage in, Garbage out– Best to start with clean
video source
• Codec can filter some– Takes time, CPU– Mixed results
• Depends on content
05/01/23 Video Compression 101 15
Compressing Color
• 16.8 Million colors supported by PC– Human eye/brain can’t tell adjacent colors apart– Especially not at 1/30th or 1/15th of a second
• Reduce color map to fewer colors– 65,536 colors in 16-bit SVGA mode– 256 Colors in 8-bit VGA mode
• Means fewer changes in color• Higher compression
05/01/23 Video Compression 101 16
Can You Tell The Difference?R-G-B
24-bit color map
64-128-192 65-128-192 66-128-192 67-128-192
64-129-192 64-130-192 64-131-192 64-132-192
64-128-193 64-128-194 64-128-195 64-128-196
Can You Tell The Difference?
05/01/23 Video Compression 101 17
High Frequency Filtering
• Takes the average of neighboring pixels to produce a lower data rate or frequency– Softens the image– Makes video look more analog
• Studies show most streaming media viewers prefer soft, smooth image
05/01/23 Video Compression 101 18
Frame Compression
• Predictive Coding • Motion Estimation and Compensation• Picture Interpolation
The greatest amount of data redundancy in video is in the vast number of similar pixels in successive frames that change
very little, or not at all.
05/01/23 Video Compression 101 19
Compressed Frame Types
• I-Frame (Intra-frame)– Full JPEG data frame
• P-Frame (Predictive)– “Delta” from previous frame
• B-Frame (bidirectional)– “Delta” from next and previous (I or P) frame– CPU-intensive
• Not recommended in live streaming
05/01/23 Video Compression 101 20
I-Frames
• A.K.A. “Key Frames”• Contain 100% of the data for that frame• Used as milestones for compressed data• Recovery points in streaming video
– Dropped I-frame can mean no video until next I-frame is received
• First frame in a set is an I-frame
05/01/23 Video Compression 101 21
P-Frames
• Only has data which differs from I-Frame. – Each P-Frame has a "motion offset" to the
corresponding I-Frame– Each pixel in a P-Frame is encoded as a delta
from the corresponding I-Frame pixel– Ignores data in B-frames
• Dropped P-frame may not be noticed– Contains very little data
05/01/23 Video Compression 101 22
P-Frame Encoding
Frame 1 Frame 2
P-Frame
05/01/23 Video Compression 101 23
B-Frames
• Bidirectional version of P-frame– Only sends data which differs from preceding
or successive (I or P) frame. – Each B-Frame has a "motion offset" to the
preceding (I or P) frame. – Each B-Frame has a "motion offset" to the
successive (I or P) frame. – Each pixel in a B-Frame is encoded as a delta
from the average of the corresponding pixel in the preceding and successive frames.
05/01/23 Video Compression 101 24
B-Frame references
1 2 3 4 5 6
• B-frame 2 includes data from I-frame 1 and P-frame 3
• B-frames 4 and 5 include data from P-frames 3 and 6
• B-frames 4 and 5 are redundant
I B P B B P
05/01/23 Video Compression 101 25
Framing Trade-offs
• More I-Frames– Bigger file– Uses more bandwidth– Quicker recovery from dropped frames
• More P-Frames– Smaller file– Uses less bandwidth– Longer recovery from dropped frames
05/01/23 Video Compression 101 26
Framing Trade-offs (cont.)
• More B-Frames– Uses more CPU– Slower file creation
• Requires I-frames and P-frames to be built first– More robust
• Less damage from dropped frames– Too many B-frames in a row is a waste
05/01/23 Video Compression 101 27
Streaming Video Demo 1
300kbps LAN Video•320 x 240 pixels
•30fps
•File size = 960 kb
56kbps Modem Video•160 x 120 pixels
•15fps
•File size = 149 kb
Requires Windows Media Player 9.x or higher
05/01/23 Video Compression 101 28
Streaming Video Demo 2
300kbps LAN Video•320 x 240 pixels
•30fps
•File size = 540 kb
56kbps Modem Video•160 x 120 pixels
•15fps
•File size = 55 kb
Requires Windows Media Player 9.x or higher
05/01/23 Video Compression 101 29
Notes on Streaming Demos
• Demo 1– High motion
• Less compression• Jerky at modem speeds
– High color saturation• Less compression• Good viewing
experience
• Demo 2– Low Motion
• More compression• Okay at modem speeds
– Low color saturation• Excellent compression• Diminished viewing
experience
Both videos demonstrate “GIGO” principle – both were digitized from low-quality analog camcorder tapes which had been copied to VHS
05/01/23 Video Compression 101 30
Streaming Video SitesSome examples can be found on the author’s site:
http://www.howeird.com/theaterVid.htmlhttp://www.howeird.com/whale.html
Movie Trailers:http://www.sonypictures.com/previews/index.htmlhttp://trailers.warnerbros.com/web/index.jsphttp://www.paramount.com/motionpictures/http://www.netbroadcaster.com/movies/trailers.html
TV News:http://www.msnbc.com/m/v/video_news.asp?0cv=c641http://www.cbsnews.com/sections/i_video/main500251.shtml
05/01/23 Video Compression 101 31
Recording for Compression [1]
“Talking Heads”• Use a solid backdrop
– Minimize pixel changes away from main focus• Use tripods
– Minimize camera motion• Keep “talent” seated
– Ideally, only the lips should move– Especially important in close-ups
05/01/23 Video Compression 101 32
Recording for Compression [2]
Sports• Not much you can do
– Every head in the crowd is a moving pixel• Close-ups, following the athlete
– Hand-held camera motion– Background is in motion
• Stay in focus– Blurred moving images don’t compress well
05/01/23 Video Compression 101 33
Recording for Compression [3]
Advertisements• Avoid motion in the background
– Fountains, ocean, people, moving vehicles– Best to provide a backdrop
• Use wide shots– Smaller subject means fewer pixels in motion
• Use static shots– Close-ups of product
05/01/23 Video Compression 101 34
A Note About Audio• Good sound will “fool” the viewer
– People take their cue from audio– If the audio is excellent, most people will think the
video is better than it actually is• Poor sound hurts
– If the audio is poor, most people will think the video is worse than it actually is
Audio bandwidth is an insignificant part of a video clip – use as much as you need
05/01/23 Video Compression 101 35
Conclusions• Video compression saves:
– Disk space– Bandwidth
• Compression techniques– Basic:
• Reduce the image size• Reduce the number of frames per second
– Advanced:• Codecs using codebooks
– Eliminate redundant data– Provide a shorthand for repeating data