Layered Compression Technologies for Digital ... fileUML activity diagrams were extracted from the...
Transcript of Layered Compression Technologies for Digital ... fileUML activity diagrams were extracted from the...
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 1
Layered Compression Technologies for Digital Cinematography and Cross Media Conversion
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 2
Motivation
• Optimized workflow for the complete production chain for Digital Cinema in 4k does not exist
• DCI specification does not deal with production (only deals with the backend of the chain)
• Digital Cinema is a chance for the European film industry
• It requires an international consortium to solve the fundamental issues together
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 3
Project Overview WorldscreenTM
• Objective: Investigation of the use of scalable video coding in digital cinema workflows
• Duration: 09/2004 – 04/2007
• Partners: 6 + 2 advisory (ARRI, Warner Bros.)
• Project Coordinator:
• Budget: 6 Mio. €, Funding: 3 Mio. €
• Funded by the European Community (6th Framework)
• Website: http://www.worldscreen.org
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 5
Scalable Video Coding
SNR Scalability
Spatial Scalability
Scalable Bitstream
Encoding
For acquisition, postproduction,distribution andarchiving
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 6
WP3.1 Compression for Data Acquisition
WP3 Simulation and Tests
WP3.2 Postproduction for Cross Media Distribution
WP3.3 Transmission, Distribution
WP4 Implementation and Integration
WP4.1 Compression for Data Acquisition
WP4.2 Transmission, Distribution
WP4.3 Integration and System Test
WP5
Assess-mentand Evaluation
Technical Workpackages
WP2
User Requirements, Specifications and Concepts
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 7
Acquisition Postproduction Distribution Exhibition
Layered scheme data compression in Digital Cinema workflows
JPEG2000-Encoder
Fieldrecorder
Use of Scalabilityfor processing /
for output to targetformats
JPEG2000/MXF Standardisation
Transmission
JPEG2000 Decoder
Quality tests
Project Details
User RequirementsBusiness Models
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 8
Acquisition Postproduction Distribution Exhibition
Layered scheme data compression in Digital Cinema workflows
JPEG2000-Encoder
Fieldrecorder
Use of Scalabilityfor processing /
for output to targetformats
JPEG2000/MXF Standardisation
Transmission
JPEG2000 Decoder
Quality tests
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 9
Acquisition Postproduction Distribution Exhibition
Layered scheme data compression in Digital Cinema workflows
A Research Project in the 6thFramework of the EuropeanCommunity
15 Feb 07 10
• Developed workflow and metadata models for the entire production chain with LSC
Project Achievements
• Developed a field recorder for providing a portable storage solution for future D-Cinema cameras including a JPEG2000 encoder
• Developed postproduction and transcoding methods (e.g. for (HD-)DVD, TV, streaming formats) from LSC master
• Developed scalable distribution and decoding systems (DCI compliant)
• Developed a quality assessment methodology for Digital Cinema
• Supported research and standardization of layered scheme data compression (both intra and inter) for D-Cinema
A Research Project in the 6thFramework of the EuropeanCommunity
11
Layered Compression Technologies for Digital Cinematography and Cross Media Conversion
A Research Project in the 6thFramework of the EuropeanCommunity
12
Goals – Methodology - Results
• The investigation of the use of layered scheme data compression throughout the entire DC chain and for rich media archives required:
– Identification of user requirements– Specification of workflows and metadata models
(current & future)– Evaluation of business models
• Methodology:– Interviews with 52 users and manufacturers– Structuring & Consolidation of market feedback
• Results– Deliverables with specifications for workflows and metadata– Deliverables covering cost comparisons and business
models for the value chain
A Research Project in the 6thFramework of the EuropeanCommunity
13
UML activity diagrams were extracted from the information from the interviews – these formally describe the most typical workflows.
Activity Diagrams
LAB Telecine Scan
telecine
film scannegative developing
Digital Intermediate Post Production
film rolls develop originalnegative
DAT sound
rolldateformatscene numberstype
telecine & cc (P3)
video tape 444 10bit log
synchronizeaudio tapes
negative scan
getrough cut EDL
digital scans
selected takes
"circled take"numbers
HD & SD deliverables e.g. for rough cut
generate pull list
pull list
clean
timing report
DCI compliant DC files for projection review encoding
A Research Project in the 6thFramework of the EuropeanCommunity
14
Acquisition with JPEG2000 (J2K)
J2KJ2K--ExportExport
CameraCamera FieldrecorderFieldrecorder& &
J2K J2K EncoderEncoder
J2KJ2K--ImportImport
PostproductionPostproduction--SWSWwithwith
PluginsPlugins
Dual HD-SDI
(future 10G)
J2K-Realtime Encoder
(near lossless)
I/O-Modul
Storage-Modules
(HDD or Flash)
Display, Keyboad
Dual HD-SDI
Dual HD-SDI
DVI/Audio
Fibrechannel
IEEE1394
VTR-Control
Power
J2K-Encoder
Uncompressed Fieldrecorder (Megacine)
A Research Project in the 6thFramework of the EuropeanCommunity
15
JPEG2000 Workflow
J2KJ2K--ExportExport DCP/KDMDCP/KDM--GeneratorGenerator
DCPDCP--ValidatorValidator
DCP-Generator
DCP-Analyser
PostproductionPostproduction Mastering/ValidationMastering/Validation
• Development of Softwaretools for JPEG2000– Input/Output-Plugins for Digitalfusion– DCP/KDM Generation Tools– DCP/KDM Validation Tools
• Concepts for Archives based on JPEG2000(will be continued in EU-Project EDCINE)
• Future Need: J2K-Hardwaresupport in Postproduction
A Research Project in the 6thFramework of the EuropeanCommunity
16
What is Transcoding all about?
• Conversion of theatrical content into entertainment video formats, e.g.:– SDTV (PAL, NTSC)– HDTV – Streaming video (internet, mobile)– etc.
Additional Sourcesof Revenue
Theatrical Content Entertainment Video
A Research Project in the 6thFramework of the EuropeanCommunity
17
Transcoding Workflow – Example
Tran
scod
ing
Universal 2K Master(e.g., 2048x1080 pel, R‘G‘B‘ 4:4:4, 12 bit)
Entertainment Video Master(1280x720 pel, YUV 4:2:0, 8 bit)
A Research Project in the 6thFramework of the EuropeanCommunity
18
Digital Cinema Packages – DCI Model
TheaterStorage
2K Projection
4K Projection
DCP
JPEG 2000 Encoded+
AES Encrypted+
MXF Packaged
Decryption Keys sent in parallel
Hard disk, Satellite...
Key delivery messages protected with RSAOnly the target projection system is able to decode the messages
Both Projectionsfrom the same
DCP
< 250 Mbps=
< 400 GBytes for a 3 hour Movie
A Research Project in the 6thFramework of the EuropeanCommunity
19
AssetMap to reconstruct
each Track file
PackingList to verify
integrity of all needed
files
Composition PlayList
to put the reels together
MXF Files
Reel 1ImageMXF
AudioMXF
Reel 2 Reel 3Reel 1...
...
...
Advertisement
Trailer
Movie
Chunks of Data
TheaterStorage
Reel 2 Reel 3Reel 1
Reel 2 Reel 3Reel 1Projection
Show Playlistschedules
presentation
Digital Cinema Packages – DCI Model
A Research Project in the 6thFramework of the EuropeanCommunity
20
Deutsche Telekom @ WorldScreen
T-Systems is a full service provider for Digital Cinema
Digital Cinema Factory™: Reliable delivery of Digital Cinema Packages via satellite
Digital Cinema Feature Film™: Server Solution for Multiplex Theatres
powered by:
A Research Project in the 6thFramework of the EuropeanCommunity
21
Deutsche Telekom @ WorldScreen
Ingest-ServerIngest-Server Delivery-ServerDelivery-Server
Communication-Gateway
Communication-GatewayDatabase-ServerDatabase-Server InternetInternet
LANLAN
VPN
Ingest-Archive Operational-Archive
Operation-ServerOperation-Server
Digital Cinema Factory™
Within the WorldScreen project, software components for 4K DCP delivery have been developed, co-funded by the EC and
the Deutsche Telekom Laboratories
A Research Project in the 6thFramework of the EuropeanCommunity
22
J2K Decoder Architecture
Input Module
JP2 Module
#1
Output Module
“2k Board”
GigabitEthernet
Connection
HD-SDI/GbEor Projector
JP2 Module
#2
JP2 Module
#3
JP2 Module
#5
JP2 Module
#7
JP2 Module
#6
JP2 Module
#8
JP2 Module
#4
BUS PCI 32bits@66MHz
Serial rapid IO
Audio Connector
Audio data
This architecture must to be able to decode the 2K at 24fps, 2K at 48fps and 4K at 24fps profiles; for the 4K, or future higher profile, some parallel 2K-Boards will be necessary.
A Research Project in the 6thFramework of the EuropeanCommunity
23
DSP-FPGA Coprocessor in the Decoder Chain
• FPGA acts as coprocessor for DSP,in a loosely coupled architecture.
• FPGA performs bit-oriented operation on Tier 1 (computationally expensive) and Inverse Quantization.
• DSP completes the decoding chain, performing IDWT on large images.
A Research Project in the 6thFramework of the EuropeanCommunity
24
Image Quality Assessment
Kodak Digital Cinema stands for highest image quality, analog AND digital
– Multi-format Cineserver JMN3000 with 3D support– Digital Processing / Delivery of over 40 full feature films – Comprehensive Theater Management System– Over 2100 pre-show advertising systems installed
• Motion Quality Ruler (MQR) is a new method for assessing image quality of motion sequences
– Can be used to judge wide quality range efficiently and precisely
• MQR method was implemented and validated at a special Kodak test facility in Paris
• MQRs were created for a wide range of scene content and origination sources that would be encountered in practice
A Research Project in the 6thFramework of the EuropeanCommunity
25
Motion Quality Rulers
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
16:1 12:1 25:1 50:1 75:1 100:1 6:1 12:1 25:1 50:1 75:1 100:1
Fire Engine - 1.4 JND sensitivity Bike - 1.4 JND sensitivity
JND
s
Error bars are 95% Student's t-confidence intervals with respect to the actual number of judges
• MQRs were verified and subsequently applied to test scenes compressed with JPEG2000 at compression ratios of 6:1 to 100:1
• Results indicate feasibility to devise MQRs for individual scenes that can be used to judge quality degradation from compression over large quality range
A Research Project in the 6thFramework of the EuropeanCommunity
26
Acquisition Postproduction Distribution Exhibition
Layered scheme data compression in Digital Cinema workflows
JPEG2000-Encoder
Fieldrecorder
Use of Scalabilityfor processing /
for output to targetformats
JPEG2000/MXF Standardisation
Transmission
JPEG2000 Decoder
Quality tests
User RequirementsBusiness Models
A Research Project in the 6thFramework of the EuropeanCommunity
27
Screening - Content
• Advertising Clips– ZKM-Spot (ZKM)– BMW X5 (BMW)– Rimowa (Rimowa)
• Trailer– Happy Feet (Warner Bros.)– Harry Potter (Warner Bros.)– We are Marshall (Warner Bros.)
• Excerpts from Feature Films/Testmovies– Cool Hand Luke* (Warner Bros.)– Ocean´s 11* (Warner Bros.)– As good as it gets (ARRI/Bill Benett)– Mystic India (BAPS/Keith Melton)
* available on DVD