Considerations for Higher Frame Rates - Society of … for Higher Frame Rates Nick Mitchell V.P....

Digital Cinema Mastering

Considerations for Higher Frame Rates

Nick Mitchell V.P. Mastering Services Technicolor Creative Services - Digital Cinema

© 2012 SMPTE · The 2012 Annual Technical Conference & Exhibition · www.smpte2012.org

Agenda

• Acronyms, TLAs, FLAs and Important Info

• The Digital Cinema Mastering Workflow

• How Higher Frame Rates changes things


Important Information

• DCP – Digital Cinema Package

• CPL – Composition Playlist – XML file within a DCP that defines how a composition is played and uniquely

identifies the version

– Specifies what portion of the track files are to be played

• HFR – High Frame Rate

• MXF – Material Exchange Format – Container format for media defined by SMPTE


Important Information

• DCDM – Digital Cinema Distribution Master – Uncompressed source, specific to DC Mastering:

2K or 4K - 16bit X’Y’Z’ tiff files

24bit/48K Broadcast wav files

DC specific subtitle XML files

Typically delivered as reels


Ingest

~25,000 frames (24 fps @ 2K) = ~250GB 3D-24 fps ~50,000 frames (48 fps @ 2K) = ~500GB 3D-48 fps ~100,000 frames (96 fps @ 2K) = ~1TB 3D-60 fps ~125,000 frames (120 fps @ 2K)= ~1.25TB

DCDM Picture Reels:

Ingest Conform Create

DCP QC

Create distribution packages

Replication Distribution


Ingest

2D ~2 TB

3D 24fps 4.5fL ~4 TB

3D 24fps 7fL ~4 TB

HFR 3D 48fps

4.5fL ~8 TB -

DCI Deprecated

HFR 3D 48fps

7fL ~8 TB

More data…

Total ~26 TB


Conform

Sync Picture and Sound Define the entry points and durations of the track files that make up the composition Single frame delays are twice as hard to detect in HFR


DCP QC




Compression/Packaging

HFR Stereo – SMPTE DCP only 250Mbps – Defined as the current maximum bitrate for 2K and 4K @24fps DCI recommendation for HFR Stereo (Near-term) – 500Mbps Because of the increased temporal resolution HFR, 500Mbps is not enough for all content – Stress tests will show artifacts are visible at 2K 48fps 2D


DCP QC






DCP QC



Feature DCDM ~25,000*.tiff files per reel

@ 2K ~ 250 GB

2X = ~500GB for Stereoscopic

4X = ~1TB for 4K & HFR

Stereoscopic

~25,000*.j2c files @ 2K ~20 GBytes @ 4K ~21 GBytes

120 min 2K movie:

~120 GBytes

~150GB for Stereoscopic

~450GB for HFR 48fps Stereoscopic @ 500 Mbps

JPEG2000 Compression




DCP QC



HFR not much different aside from Frame Rate and Edit Rate MXF file Creation – Proposed UL for High Frame Rate track files P-HFR JP2K @ 2k: 06.0E.2B.34.04.01.01.0D.0E.16.02.02.03.01.01.03 Signals a device that the track file contains high bit-rate material


QC

Requires projection and playback equipment that can support HFR HFR/High Bit-Rate is a paid upgrade for most devices HFR is less fatiguing - 2D and 3D


DCP QC




Creating distribution packages


DCP QC



HFR support requires a transition from the JPEG Interop (ad-hoc) to SMPTE format DCPs During the transition - in order to support legacy Interop content - mixed format (SMPTE and JPEG Interop) media will be required Changes the architecture of distribution media


Replication


DCP QC



Time required to create and replicate hard drives is heavily dependent on Bit-Rate HFR Inventory will need to be manually managed on separate media until all devices can handle HFR appropriately


Distribution


DCP QC



Distribution of HFR content will need to be managed manually on separate media until all devices are updated to handle the content appropriately Some smaller HDDs may need to be replaced with higher capacity devices Satellite distribution of HFR DCP at high bit-rates is challenging

Thank you

[email protected]

2012 SMPTE Symposium Program:High Frame Rates for Digital Cinema

Compression and Distribution Challenges for High Frame Rate Digital Cinema

Peter Wilsonwww.hddc.co.uk www.edcf.net

© 2012 SMPT E · T he 2012 Annual Technical C onference & Exhibition · www.smpte2012.org

Contents

• Sampling Theory - Friend or Foe?

• JPEG 2000

• MTF = Energy

• HFR + Short Shutter = Energy

• Energy is the enemy of compression

• Temporal Axis Distortion in Display

• Converting HFR to LFR


Sampling Theory - Friend or Foe?• Sampling was used before the Digital Age

• SDTV is sampled at line rate and frame rate

• Undersampling creates artefacts

• Oversampling promotes quality at a cost

• Spatially content decides over or undersampling

• Temporally content decides over or undersampling

• Nyquist and Shannon were smart

• 24 Fps 85 years of Temporal Undersampling


JPEG 2000 (J2K)

• Digital Cinema uses JPEG 2000 CompressionThe Free licence version is Part 1

• J2K is a wavelet compression system- The wavelet system is layered groups of HP LP- 5 levels max for 2K, 6 levels max for 4K- This gives us the 2K & 4K compatibility

• J2K is Intra Frame only, each frame is separate- VBR or variable bit rate allows us to spread data over time but the frames are still separate

More info here: http://www2.engr.arizona.edu/~bilgin/publications/ISCAS2006.pdf


Stages of a wavelet transform


MTF = Energy• Digital Data Channels have Energy templates

• Energy drops to zero at cut off

• Energy not at zero at cut off will Alias

• Film and Digital Cameras Energy slopes down

• 35 mm energy relatively low at 2K cut off

• New Cameras will blast through to 4K

• Top layer in J2K will be exercised increasing bit rate requirement


HFR + Short Shutter = Energy• HFR improves motion rendition

• LFR is a big compromise on motion v blur

• HFR has less compromise

• Sharper images with less blur generate more energy

• High energy images need more bits

• The next slide illustrates this

• Top image 50 Fps, bottom image 300 Fps


Effect of a short shutter @ HFR

1/50 s

1/300 sWith thanks toRichard Salmon


Energy is the Enemy of Compression

• New generation cameras generate more information

• HFR generates more information

• J2K is limited in future development

• Either bit rates increase or quality metrics need improvingSharper images at higher frame rates will need proportionally more bits

• Who is the arbiter of quality?DCPs today are being reduced in size to reduce satellite transmission costs


Quality Metrics No need to develop new compression techniques or standards

Existing compressors can do the job, but additional built-in meters and test point links should be provided

Nearly all important parameters are known internally, but not reported Built-in meters are much simpler and much more cost-effective

because all necessary data (eg; statistics or list of currently used quantization matrices) are already there and in the appropriate formats – no need to format these data and transfer them to remote measurement devices

Availability of additional measurement dataallows more efficient compression in terms of automatic profile optimization and pre-processor settings control

Most importantly a demand value can best set by the Distributor.


Temporal Axis Distortion in Display

• Many Cinemas use Single lens 3D systems- These systems time sequentially display L + R- These systems use multiple flashing to reduce flicker- These Systems use multiple flashing to preserve Depth information

• Currently all systems will have difficulty meeting the latest DCI RP for brightness without doubling up

• Multiple flashing causes temporal axis distortion which results in Judder and object replication.


Temporal Axis Distortion in Display

BACKGROUND IS PRESENTEDTO TRACKING EYE IN A

DIFFERENT PLACE IN EACH FRAME

=BACKGROUND

STROBING


Converting HFR to LFR

• For regular release, HFR movies need a LFR version- Converting from HFR to LFR is not trivial- Compromising on shutter angle and motion for LFR dilutes the value of HFR

• Making multiples doesn't guarantee a simple conversion (eg; 96 - 24)- There are often secondary effects

• Global motion blur addition can be highly damaging- Sharp backgrounds should be preserved while objects in motion are blurred in proportion to their speed


Effect of a short shutter @ HFR

1/50 s

1/300 sWith thanks toRichard Salmon

The End

Peter Wilsonwww.hddc.co.uk www.edcf.net

2012 SMPTE Symposium Program: High Frame Rates for Digital Cinema

Interna'onal Versioning

Laurence Claydon www.deluxedigital.co.uk

© 2012 SMPTE · e 2012 Annual Technical Conference & Exhibition · www.smpte2012.org

Contents •  Data Management

•  DCDM Data Volumes

•  Data i/o implica2ons

•  Inventory Management

•  SubFtling PosiFoning •  Dynamic Z-‐Axis PosiFoning

•  Future for In-‐Theatre 3D SubFtles


Data Management • Moving and Storing Data

•  48FPS 2K 3D DCDM is ~8TB

•  60FPS 2K 3D DCDM is ~12TB

•  Data Throughput •  48FPS 2K 3D DCDM replay requires disk i/o 7.6Gb/sec

•  60FPS 2K 3D DCDM replay requires disk i/o 9.6Gb/sec

•  60FPS requires 16Gb/sec facility fibre infrastructure


Data Management • Mastering -‐ Compression

•  At Least 500Mb/sec required for 48FPS

•  At Least 625Mb/sec required for 60FPS

•  Larger DCP Package Sizes •  250-‐500Gb

•  Increased Cloning Time

•  Increased Delivery Time via Satellite/Network

•  Increased Load-‐Time in the Cinema


Inventory Management •  Adds to the MulFple Inventory ConsideraFons

•  7.1/5.1/Atmos Audio

•  4.5XL/7XL/10XL Light Levels

•  24FPS and 48/60FPS Versions

•  2D and 3D


SubFtle PosiFoning •  Current Hardware Overview

1920

DVI-‐2K SDI

1920 DVI-‐2K SDI

Background

Foreground

Background

Foreground

DCP Player

Scalers

2K SDI Keyers

Dual Head Workstation

LeT YCzCx

Right YCzCx

LeT SubFtle

Right SubFtle

LeT YCzCx

Right YCzCx

LTC Network (Control)

3D Image Processor

Series II Projector Dual-DVI or 3G-SDI

Interface

HDMI 1.4a Plasma


•  Hardware Issues for HFR 3D •  1.5Gb SMPTE 292 Interface Standards will not support HFR 3D

•  Only Series-‐II projectors capable of displaying HFR 3D

•  HFR 3D capable replay system expensive (e.g. Clipster)

•  Require 3G-‐SDI capable Scalers/Keyers

SubFtle PosiFoning


•  SoluFon 1 – PosiFon at 24FPS and Convert… CONSIDERATIONS

•  Is the HFR cut same as 24FPS Cut?

•  Do HFR cuts always occur on ‘A-‐frame’?

•  Trim in points +1 frame, Trim out points -‐1 frame?

•  HFR preview not possible un2l DCP is made

BENEFITS

•  24/48, 24/60 and 30/60 sub2tle file conversions easily made

•  Current 24FPS workflow may be used up to MXF-‐Wrap stage

`

SubFtle PosiFoning

1

A1 B1

2

A2 B2

3

A3 B3

4

A4 B4

24/30FPS

48/60FPS


•  SoluFon 2 – True HFR SimulaFon •  Integrate HFR DCP Replay into Worksta2on

•  Output Interface Dual DVI or 3G-‐SDI

•  Interim ‘Frame Drop’ mode for connec2on to HDMI 1.4a displays

•  48FPS -‐> output 24P

•  60FPS -‐> output 30P

•  Replay from DCP rather than DCDM to minimise disk i/o and capacity

•  Requires dual GPU’s for HFR 3D JPEG2000 Replay

SubFtle PosiFoning

1

A1 B1

2

A2 B2

3

A3 B3

4

A4 B4

24/30FPS

48/60FPS


•  Single WorkstaFon SoluFon

D-CINEMA XML (SMPTE 428-7)

24FPS

OV DCP (HFR 3D)

Create Single-Reel DCP’s (Including Head/Tail)

3D D-CINEMA XML 48 or 60 FPS

PLUS SUBTITLE PNG GRAPHICS

SubFtle PosiFoning

Series II Projector Dual-DVI or 3G-SDI

Interface

Import and convert to HFR timeline

HDMI 1.4a Plasma


•  HFR provides greater temporal resoluFon •  Dynamic posi2oning an increasing requirement due to content

•  Integer Pixel values insufficient for smooth Z-‐axis transi2ons at HFR

•  Sub-‐pixel accuracy required

•  Suitable An2-‐aliasing must be applied to transi2ons

Dynamic Z-‐axis PosiFoning


•  Dynamic posiFon implementaFon – Rendered SubFtles •  Keyframes provide Z-‐axis posi2ons in 2meline

•  SoXware interpolates posi2on for each frame

•  Linear or S-‐Transi2on

•  Output Sub2tle file has 1 sub2tle instance per-‐frame



•  Dynamic posiFon implementaFon – Rendered SubFtles •  Keyframes provide Z-‐axis posi2ons in 2meline

•  SoXware interpolates posi2on for each frame

•  Linear or S-‐move

•  Output Sub2tle file has 1 sub2tle instance per-‐frame

•  Z-‐posi2on must be redefined to 0.001% accuracy for Subpixel Placement

•  Sub2tle file TC must be HH:MM:SS:FF and not HH:MM:SS:Ticks



•  Dynamic posiFon implementaFon : In-‐Theatre SubFtles •  Define Keyframes

•  XML ‘Keyframe’ Anribute with Z-‐Posi2on Value and Timecode

•  Define TransiFon types

•  XML ‘Z-‐Transi2on’ Anribute with Type e.g. ‘Linear’, ‘Log1’, ‘Log2’ etc

•  Projector/Media Block will need to be capable of:

•  Posi2onal Interpola2on on Frame-‐by-‐Frame basis

•  Sub-‐Pixel Placement

•  Generic Sub2tle File for 24/48 and 30/60?



Summary

•  Significant data impact to post producFon and distribuFon

•  Inventory needs to be carefully managed against install base

•  SubFtling process improvements required

•  Current rendered-‐in soluFon workable •  Further work required to extend SMPTE 429-‐7 specificaFon

•  Further work needed to define in-‐theatre subFtle render-‐engine requirements

Thank You

Laurence Claydon www.bydeluxe.com www.deluxedigital.co.uk

Considerations for Higher Frame Rates - Society of … for Higher Frame Rates Nick Mitchell V.P....

Documents

Transcript of Considerations for Higher Frame Rates - Society of … for Higher Frame Rates Nick Mitchell V.P....