The Image Interchange Framework...

The Image Interchange Framework Demystified:

What it is, and why it matters

Andy Maltz Director Science and Technology Council Academy of Motion Picture Arts and Sciences

®2012 A.M.P.A.S.

The Image Interchange Framework • HPA Tech Retreat February 2012

From urbandictionary.com

Special Snowflake:

“A product created specifically for a customer or client who cannot or will not conform their

requirements or needs to the general specifications causing said vendor to customize uniqueness to an

established product thereby creating more work, time, energy, and effort to be expended on the vendor’s

employees' part…”

®2012 A.M.P.A.S.


®2012 A.M.P.A.S.


The Last Snowflake Workflow

®2012 A.M.P.A.S.


Agenda

•  What’s changing •  Why the Academy is supporting IIF •  IIF overview, core technical features •  Demo room pointers •  Your call to action

®2012 A.M.P.A.S.


This is a period of great change and instability

•  Film is going away •  Digital motion picture cameras are getting better, and

generating LOTS of data •  Cinema may not stay the highest “quality” for long:

–  Brighter displays in the home –  Higher framerates coming –  4K, UHDTV (wider color gamut than HDTV)

•  Low cost technology lets anyone do nearly anything they want

®2012 A.M.P.A.S.


Problems in Motion Picture Mastering

•  It’s a mess out there: –  Many image file formats and implementations –  Color management is slathered in “secret sauces” that make inter-facility

interchange difficult –  Snowflake Workflows: good for HPA, bad for filmmakers

•  10-bit Cineon isn’t good enough anymore –  Insufficient precision and dynamic range –  Based on an end-of-life technology (film) –  Extending to 16 bits won’t fix the other problems

®2012 A.M.P.A.S.


The Problems - 2

•  There is no defined “Digital Source Master” –  And therefore no digital archiving standard

•  The traditional sources of industry leadership are absent

–  81% of a movie’s total revenue is from non-theatrical distribution –  A well-designed, system-wide solution won’t happen on its own.

®2012 A.M.P.A.S.


Why the Academy?

•  Mission since 1927: To Advance the Art and Sciences of Motion Pictures

•  Image quality is important and shouldn’t be so difficult to achieve

•  The Academy is a neutral forum –  Represents the interests of the creators of motion pictures –  Takes on problems that are not commercially feasible –  Historical role in development of industry standards

®2012 A.M.P.A.S.


Academy-led standards

®2012 A.M.P.A.S.


®2012 A.M.P.A.S.


Other Academy efforts: Digital Preservation

� � � � � ��

� � � � � � ��

� � ��

®2012 A.M.P.A.S.


Just published (free copies outside)

�THE SCIENCE AND TECHNOLOGY COUNCIL

®2012 A.M.P.A.S.


Back to the matter at hand…

•  Next-generation cinema is impossible without a next-generation imaging architecture

•  This is complicated, and it won’t happen all by itself

®2012 A.M.P.A.S.


The Image Interchange Framework (IIF-ACES)

•  An architecture for high fidelity motion picture imaging to support numerous workflows –  Digital Intermediate –  Visual Effects –  Animation –  Restoration –  Film and/or digital capture –  Scene- and output-referred methodologies

®2012 A.M.P.A.S.


Key IIF components

•  Standardized image encoding specifications: –  Colorimetric: 16 bit half-float, wide gamut, HDR –  Densitometric: 16 bit integer, fully specified metrics

•  Standardized transforms and methodologies: –  Image input (digital and film) –  Rendering and display

•  Standardized file formats –  “Constrained” OpenEXR –  Extended, fully specified DPX

•  Straightforward adoption path

®2012 A.M.P.A.S.


A Brief History of IIF

•  2004: –  Siggraph BOF: film scans are inconsistent, there needs to be better “image interchange” –  Academy File Format Project initiated

•  2005: –  problem study, requirements document written (42 pages long) –  Conclusion: color management is the toughest nut to crack –  Ed Giorgianni commissioned to write Digital Cinema color management whitepaper

•  2006: –  Project Committee begins designing, viewing images –  Follow-on whitepaper by Giorgianni –  Academy hires Senior Imaging Engineer

®2012 A.M.P.A.S.


A Brief History of IIF – 2

•  2007: –  Draft image encoding specifications published for comment and test –  Core transforms prototyped –  Proof-of-concept work launched –  Color Transform Language (CTL) published as SMPTE RDD-15

•  2008: –  IIF introduced at HPA, Call for Participation issued –  Color corrector implementations begin –  Container and metadata work begins –  IIF papers at SMPTE Fall Conference

®2012 A.M.P.A.S.



•  2009: –  Working Groups continue working –  Academy Metadata Symposium –  IIF introduced at IBC Technical Sessions –  Draft encoding specifications finalized

•  2010: –  Transform development and testing continues –  SMPTE standards work initiated

•  2011: –  Industry trials begin –  First “Designed With IIF In Mind” products appear at NAB”

®2012 A.M.P.A.S.



•  2011 (cont’d): –  Key transforms updated with results from trials –  Television discovers IIF: at least a dozen shows (pilots & series)

–  IIF at NAB – Justified panel –  Academy hires two more imaging engineers and builds state-of-the art imaging lab –  First 3 SMPTE standards elevated to Draft Publication

Approved <TBD>

Warning

This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication.

Contents Page

Foreword ............................................................................................................................................................ 2!

Intellectual Property............................................................................................................................................ 2!

Introduction......................................................................................................................................................... 3!

1! Scope ........................................................................................................................................................... 5!

2! Conformance Notation ................................................................................................................................. 5!

3! Normative References.................................................................................................................................. 5!

4! Terms and Definitions .................................................................................................................................. 6!

5! Academy Color Encoding Specification (ACES) .......................................................................................... 8!

Annex A (Normative) ACES Reference Input Capture Device spectral sensitivities ........................................ 12!

Annex B (Informative) Characteristics of ACES RGB color encoding .............................................................. 19!

Annex C (Informative) ACES RICD Camera Flare ........................................................................................... 21!

Annex D (Informative) ACES RGB values of common stimuli as produced by the ACES RICD ..................... 22!

Annex E (Informative) Bibliography .................................................................................................................. 25!

Page 1 of 20 pages

SMPTE ST 2065-1:2011 SMPTE STANDARD

Academy Color Encoding Specification (ACES)

Approved <TBD>

Warning


Contents Page

Foreword ............................................................................................................................................................ 2!


Introduction......................................................................................................................................................... 3!

1! Scope ........................................................................................................................................................... 4!




5! Academy Printing Density ............................................................................................................................ 6!

Annex A (Normative) Spectral Responsivities !APD ........................................................................................... 9!

Annex B (Informative) Influx Spectrum SAPD..................................................................................................... 12!

Annex C (Informative) Bibliography.................................................................................................................. 15!

Page 1 of 20 pages


Academy Printing Density (APD) —Spectral Responsivities, Reference Measurement Device, and Spectral Calculation

Approved <TBD>

Warning


Contents Page

Foreword ............................................................................................................................................................ 2!


Introduction......................................................................................................................................................... 3!

1! Scope ........................................................................................................................................................... 4!




5! Academy Density Exchange Encoding ........................................................................................................ 6!

Annex A (Informative) Bibliography .................................................................................................................... 9!

Page 1 of 20 pages


Academy Density Exchange Encoding (ADX) — Encoding Academy Printing Density (APD) Values

®2012 A.M.P.A.S.



•  2012 (so far): –  Engineering nearing completion on key transforms –  Next round of industry trials in preparation –  Collaborative software development site (GitHub)

®2012 A.M.P.A.S.


Industry-wide collaborative development

•  More than 50 leading practitioners, engineers and scientists •  Leading equipment manufacturers •  Major post-production, VFX and animation facilities •  Major technical contributions from:

–  Adobe, Arri, Autodesk, Dolby, FilmLight, Fujifilm, ILM, Kodak, Pixar, Sony Electronics and top color science and imaging experts

®2012 A.M.P.A.S.


The IIF Eye Chart (Idealized System Block Diagram)

ADX: Academy Density eXchange ACES: Academy Color Encoding Specification

RRT: Reference Rendering Transform RDT: Reference Display Transform IDT: Input Device Transform

®2012 A.M.P.A.S.


Motion picture workflow: “legacy”

digital color

correction

release print film emulation

hd mon

rec 709

xform

on-set timing session

digital acquisition

source cam lut

Full-range raw digital acquisition

dist master

color correction session

digital color

correction

release print film emulation

DCDM xform

cam lut

®2012 A.M.P.A.S.


Motion picture workflow - IIF

ACES RRT hd mon ODT

on-set timing session

digital acquisition

source IDT

Full-range raw digital acquisition

dist master

color correction session

ACES RRT RDT IDT

®2012 A.M.P.A.S.


ACES: Key features

•  Enhanced Precision •  Wide Color Gamut •  High Dynamic Range •  IMF-ready

®2012 A.M.P.A.S.


Precision: why 10 bits isn’t enough

Steps visible

Steps not visible

®2012 A.M.P.A.S.


Exposure latitude of film negative exceeds 10 bit Cineon - that’s why ACES is 16 bit half-float

Video Conversion Range Video Conversion

Beyond Video Conversion Range

®2012 A.M.P.A.S.


Wide gamut: ACES covers all visible colors

ACES color primaries in perspective

CIE x CIE y Red 0.73470 0.26530

Green 0.00000 1.00000 Blue 0.00010 -0.07700

ACES color primaries in perspective

®2012 A.M.P.A.S.


Color gamut is also increased with greater dynamic range

®2012 A.M.P.A.S. 32

®2012 A.M.P.A.S. 32 “Normal” exposure

®2012 A.M.P.A.S. ®2012 A.M.P.A.S. 33 Rec. 709 processing - 2 stops under

®2012 A.M.P.A.S. ®2012 A.M.P.A.S. 34 ACES – 2 stops under

®2012 A.M.P.A.S.


What the colorists say

“It’s All There”

®2012 A.M.P.A.S.


What a famous Hollywood director says

“I want my whole cow. Not just the steak you serve me.”

®2012 A.M.P.A.S.

®2012 A.M.P.A.S.


Why we render

•  Film and digital sensors capture scene exposures (proportional to # of photons hitting the imaging plane)

•  Unprocessed scene exposures don’t look good on any display medium

•  “Rendering” applies an image processing algorithm to scene exposure values to generate a “pleasing” image on a target display device or medium

®2012 A.M.P.A.S.

®2012 A.M.P.A.S.


Scene exposure vs. “output referred”

42

Accurate Reproduction of Scene Colorimetry Image Rendered for Pleasing Reproduction

®2012 A.M.P.A.S. Scene exposure – Printing Density

®2012 A.M.P.A.S. 44 Scene exposure – ACES

®2012 A.M.P.A.S. 45 Rendered image (“Output-referred”)

®2012 A.M.P.A.S.


Good rendering is hard to do

•  Film manufacturers spent 70 years working on it •  Digital Intermediate facilities spent 15+ years on it (and they all do it

differently) •  The IIF Project Committee spent 5 years working on a “baseline

renderer”: the Reference Rendering Transform, or RRT

46

®2012 A.M.P.A.S.


RRT Facts

•  Designed to go several stops beyond film and today’s digital displays (>1,000,000:1)

•  Designed to make the colorist’s job easier: a better starting point

•  Designed to not get in the way of the creative vision •  Mathematically sound

®2012 A.M.P.A.S.

Print Film Emulation (PFE)

Digital Cinema Projector

Print Film Projector

Video Display

Dolby PRM

Lab HDR Display

IIF Theoretical Display

®2012 A.M.P.A.S.

RRT Release Candidate (February, 2012)

Digital Cinema Projector

Print Film Projector

Video Display Dolby

PRM

Lab HDR Display

IIF Theoretical Display

®2012 A.M.P.A.S.


Mythbusting

®2012 A.M.P.A.S.


IIF Myths

•  “It’s not baked yet” –  First standards are done –  Field trials taught us a lot, and they resulted in significant

improvements –  Continued comprehensive, industry-wide testing is crucial

•  “It forces the ‘Film Look’ ” –  Rendering is required –  All “pleasing images” have some things in common –  The RRT does not go to all the places PFEs go

®2012 A.M.P.A.S.


IIF Myths - 2

•  “IIF will never work in production” –  Every trial project found some issues, and they all were resolved –  Issue: smaller, under-resourced VFX facilities may not be good early

adopters •  “IIF will never work for everything”

–  How do you prove a negative? –  We think it will –  Even if we’re wrong, huge benefits for the projects it does work for

®2012 A.M.P.A.S.


IIF Myths - 3

•  “The Academy is dictating that you use IIF” - Early SMPTE proponents include:

• ASC • Paramount Pictures • Sony Pictures • Pixar • Several manufacturers

•  Visit the Demo Room

®2012 A.M.P.A.S.


The cost of waiting

•  SMPTE standards are publishing now –  Very difficult to change after publication –  The time to ask for changes is now

®2012 A.M.P.A.S.


What’s next

•  RRT release candidate: March 1, 2012 •  Next round of industry trials •  Complete file container and base metadata

standardization •  Support plan development

®2012 A.M.P.A.S.


Demo Room: IIF in Real Life

•  Dolby, Image Systems: High dynamic range display driven by IIF-enabled color corrector

•  FilmLight: Baselight with IIF support, matching multiple digital motion picture cameras

•  Fujifilm: IIF-based on-set color management tool •  Image Essence: IIF software implementation

®2012 A.M.P.A.S.

Try it out – let us know what you think!

For more information on The Image Interchange Framework

please visit http://www.oscars.org/iff

The Image Interchange Framework...

Documents

Transcript of The Image Interchange Framework...