COM 597 Streaming Media Class 2 June 28, 2007. Today’s topics: The nuts and bolts of Audio The...
-
Upload
annabelle-robbins -
Category
Documents
-
view
217 -
download
0
description
Transcript of COM 597 Streaming Media Class 2 June 28, 2007. Today’s topics: The nuts and bolts of Audio The...
COM 597Streaming Media
Class 2
June 28, 2007
Today’s topics:
• The nuts and bolts of Audio• The encoding and delivery process (sampling
& resampling)• Aliasing• Audio Workshop• Codecs and Players• Compression demonstration• Compression workshop
The process of compressing a file from 40,000 feet up (the big picture)
What is Video?
Video is basically a three-dimensional array of color pixels. Two dimensions serve as spatial (horizontal and vertical) directions of the moving pictures, and one dimension represents the time domain.
A frame is a set of all pixels that (approximately) correspond to a single point in time.
Digitizing• This is the process of taking an analog signal and turning
it into an approximate digital representation of the original image and sound.
• Examples of digitizing: scanning a photograph, capturing video into a computer, capturing a recording
• Ripping a CD is not technically “digitizing” because it is already data. It is usually referred to as “capturing”
Compression in simple terms is reducing the data used to display an image, play an audio file or present video. It is used throughout the industry.
• Cable TV• Editing platforms• DVD• TiVo / PVD• Video Acquisition
Data can not be put back once it has been removed.
Spatial Compression• Spatial encoding is performed by taking
advantage of the fact that the human eye is unable to distinguish small differences in color as easily as it can changes in brightness and so very similar areas of color can be "averaged out"
• Common image file examples are .jpg, .tiff, .gif and .png
Spatial Compression
Points are usually described with Cartesian values (X-Y)
• X side to side• Y up and down• Z close and far away
Temporal Compression
With temporal compression only the changes from one frame to the next are encoded as often a large number of the pixels will be the same on a series of frames
Low motion example 276 KbpsMedium motion example 344 KbpsFast motion example 354 Kpms
When not to compress
• Acquisition and origination• Archiving• Mastering
You (your client or boss) have to decide what is an acceptable level of compression based upon the variables we will be discussing in class.
Other nifty terms we will be using:• Metafile: A small file on a web server that includes
information (metadata) that informs a player where to locate a file on a media server. A roadmap, if you will
• Encoder: a software or hardware application that transforms a source media file into a file that can be streamed efficiently.
• FTP Client: software on a client computer or server that uses File Transfer Protocol to upload or download files from another computer at a remote location.
Additional websites for reference:
http://www.streamingmedia.com/magazine/http://www.digitalproducer.com/http://www.streamingmediabible.com/http://www.w3.org/AudioVideo/
Audio
• How do you deal with legacy material?
Audio sources:RecordsOpen reel tapeCDDVDOff-air recordingDownloaded filesOptical sound from a filmSurround soundEmbedded audio
Analog v. digital audio
What is the difference?Analog material must be played in real timeYou will need to use a software tool or hardware device to convert these different physical representations of sound (grooves in a record for example) into a digital representation of the sound. This process can happen in a number of different ways during your production path.A-to-D converters (scanners)
Analog v. digital audio
What is the difference?
Sample rate will determine the fidelity of your final product
Remember you cannot add detail to a sampled media file so it is best to start with a file that is as large as is reasonable. (RAW v. JPEG for example)
Raw Image .jpg Image
Resolution Comparison Source v. Delivery
Raw Image .jpg Image
Resolution Comparison Source v. Delivery
Compression and Bitrate
• To easily understand how compression can impact the fidelity (quality) of a file listen to this demo.
Click me… please
At a certain threshold there is not a perceptible difference in the sound
• Sample rate is not bits-per-second, although they are related
• Sample rate is the number of “slices” of sound per second where bps is about how much data is begin delivered in the signal. Bps can be 22KHz or 96KHz or even 192KHz. For this class most of our work will be either in 44.1KHz or 48KHz.
• When you sample a sound you need to record both the wave peaks and troughs
• Human ear is limited to detecting frequencies
no higher than 22KHz. • It was believed that simply doubling this
number would give a sample rate that would reproduce all the sounds we can hear. Therefore, CDs run at 44.1KHz
Resampling a fileAnd the problems inherent in the
process
The difficulty with 44.1 as a standard is that it does not integrate easily into a
standard production path.• Most audio files are converted using integer-
based averaging. • This becomes apparent when you see that the
common rates used for mastering are 48, 96 and 192 Khz. Moving down in quality involves simple averaging and not a complex resampling of the file.
Aliasing Artifacts
Aliasing Artifacts
Aliasing Artifacts
Aliasing Artifacts
Aliasing Artifacts
Aliasing Artifacts
• These quantization errors become evident when you convert between CD rates and film/television production rates. It is important to know your delivery before you start the process of sampling your sound.
• This is also why it is important to make certain all the tools in your production path are in agreement. (set to the same sample rate)
Audio formats you are likely to encounter
• Mono one track of audio• Dual Mono (this is not stereo) identical audio on
both tracks• Stereo multi-channel audio on two tracks• Mix Minus production audio with two mono tracks
containing different sources (dialogue on one and effects on the other is one
common example)• 5.1 surround Home theater is the most common
application with three front speakers (left, right and center), two rear speakers and once low-frequency speaker (sub-woofer)
• AC3 An alternative to Dolby Labs 5.1 surround format.
Codecs
• Which is best? There really is no one-stop solution.
• All of the major standards do a very good job of compressing most media
• The differences are subtle
There are real advantages to choosing one standard as your default distribution method
• Users will only need to rely on one playback solution
• Hardware will be optimized to that solution (servers and production tools)
• Licensing of technology is easier• Staff is trained with emphasis on only one
platform• Support is easier
There are real disadvantages to choosing one standard as your default distribution method
• Not a large user base for the technology• Format of files is not competitive with
changes in the marketplace• High licensing fees• Flaky software• Challenges with using the files on all types of
hardware (WM on a Mac)• You are tied to the fortunes of the company
that created your codec
Format compatibilities and incompatibilities
• QuickTime Mac and PC are OK, no Linux support • Windows Media PC is great, Mac support is
limited to older versions, no Linux• Real Networks Supports most every operating
system• Flash Supports most every operating system
because it runs in a browser• MPEG will play in many of the proprietary
systems H.264 shows promise as a common standard.
How might you examine a codec to see if it will work for your needs?
You might choose one compression solution for your production and another for delivery to your users.
Property Production Codec Delivery Codec
Compression Ratio Low usually between 1:1 and 4:1
High typically 50:1 or more
Quality Lossless (perfect) or close Good to acceptable, will denigrate to sub-VHS quality
Lossy Zero for some codecs, for others very minimal
Very lossy
Bit rate 25-50 and up to 150 Mbps As low as 300 Kbps for video and 24Kbps for audio
File Size :20 seconds per GB (180 GB per hour)
As low as .4 GB per hour
Editable Yes Not really, although many editors will accept a compressed file as a source. The quality will be dubious at best
Compositable Yes Only if file has alpha channel
Conversion to other formats Easy Additional degradation of quality is likely
General rule is that as compression schemas improve the files get smaller and the images either improve or stay the same.
Let’s look at how we compress a file to get an understanding of all the variables we can manage
Prior to compression you need to get your media in the best shape possible. Often it is easier in production to prep the media than it is to heal bad video during compression.
Video Preparation
• Deinterlacing the footage
• Frame rate agreement (2-3 cadence, 15fps, PAL v. NTSC, HD)
• Noise filters (dubious benefits at best)
• Grain removal
Video Preparation
• Color correction (gamma adjustments)
• Scaling (size your canvas can speed up compression significantly)
• Aspect ratio adjustment including anamorphic adjustments
Compression Example
• What the heck do those letters mean? And why do I care?
• I-frame Intra frame or key frame• P-frame predicted difference frame• B-frame bi-directional difference frame
• Compression Demo