Client Server methods for Future Cinema

Compositing Multiple 4K Cinema Streams

1

Scalable City: Massive Scale Virtual Worlds

• Massively multiplayer continuous world

• Hundreds of thousands of interactive objects

• Large aggregate bandwidth requirements

Challenges/IssuesOriginally develop multicore accelerated single user, immersive system, to complex virtual world. Create compute and conceptual ecosystem.

2

The environment can be extensible with endless variations, computed at run time.

3 10gb interfaces to compute accelerators

Development Server Framework 2/2011

IBM Z10 mainframe computer at San Diego Supercomputer Center2- IFL’s with 128mb Ram, zVM virtual OS manager with Linux guests6 tb storage fast local storage – 15K disks4 SR and 2 LR 10gb ethernet interfaces

8HS22 blades- 16 Xeons – 96 cores

4 QS20 blades- 8 Cell CPU’s

2 QS22 blades- 4 Cell CPU’s4 way Xeon Server – 32 core

10gb interface to internet

nVidia Tesla accelerator – 4 GPU’s on linux host

Many Clients

7

Virtual World Director

Sheldon Brown, UCSD Site Director CHMPRKristen Kho, Programmer, UCSD Experimental Game Lab

Future Cinema –

Virtual World Director

Allows Virtual World to be platform for creating Cinema.

Enables very flexible methods for creating cinema.

Allows cinematic production to take place in a broadly distributed and many user environment.

Allows cinematic data to be derived from the experiences of many users.

Interface – Machinima Menu

Interface - Editing

4K Cinema Generation

Automate re-rendering of data into 3D 4K frames.

Sony ZEGO BCU-100 Computing Unit• Cell/B.E. Processor and RSX technology • Prototype system with 4 HD-SDI inputs/outputs

• Use to process real-time 4K video• Procedural 4K animations – (ala Galileo films)

• Initial experiments to use BCU-100 as a 4K video player:• Not fast enough to load 4K frames from disk and maintain a

reasonable frame rate. • Must process input and immediately send to output.• Create procedural 4K movies from data sources – i.e. bitmap

compositing, algorithms, etc.

Ultra high resolution projection technologies 4K – 3840 x 2160 pixel displays today8K and beyond tomorrow Digital Cinema Auditorium

ultra-high resolution cinema, 3-D sound; tele-presence conferencing; stereo imaging, Cine-Grid optical ultra-high bandwidth

Digital Cinema –Galileo Movies, 24 short 4K Movies – Sheldon Brown, Kim Stanley-Robinson

Leveraging global lambda networks to improve performance for extremely high bandwidth and time dependent applications such as cinema, teleconferencing and virtual reality.

Collaboration with University of Sao Paulo, Mackenzie University, Keio University, Sony, Zaxel, IBM, Intel

Stream Ultra High Fidelity Cinema on Global Lambda Grid

17

Stream real-time uncompressed 4K to our compositing and annotation server

Extending TritonSort* systemHybrid Disk/Flash Memory Storage Array

100’s of Terabytes

Multiple 10gb interfaces

*George Porter Center for Networked Systems, UCSD

Data-intensive Cinema• Data-intensive Cinema (DI-Cinema) Project

– Encompasses video, scientific visualization, motion picture production, medical imagery, …

– Higher resolution formats + 3D = big data challenge• 100s of TB to multiple PBs of data per project• Not just streaming

– Subjected to increasing amount of computation– Media is the result of staged, pipeline-oriented computing

Digital Cinema as a Computing ProblemBatch-oriented video

computing:– Color adjustment– Digital watermarks– Compositing (join operation)– Format, resolution, frame-rate

adjustments– Rendering– Captioning– Transcoding

Requires high resolution, and thus large data sizes, at each of these stages as well as in the final product

I/O I/O I/O

Compute

Compute

ComputeCompute

Compute

I/OI/O I/O Streaming

Data-intensive video is a problem today…

• Netflix transcodes each of its movies 50+ times for each type of end-user device and resolution, relying on the cloud for scalability (Adrian Cockcroft, CTO Netflix)

• 4+ million “assets,” or pieces of content, must be orchestrated together to form a single project (“What’s up Prof? Current Issues in the Visual Effects & Post-Production Industry”, Leonardo, vol 43, no 1, Feb 2010).

• On digital intermediate:“…the files are so massive they are difficult to display, edit or color correct”, “Real time playback…is extremely difficult on even the fastest RAIDs”, “This is a lot of data to push around…requiring proprietary software and hardware solutions” (Filmlook, Inc., 2008)

* Image used with permission (GNU Free Documentation License)

…and a bigger problem tomorrow

4K/Stereo High Resolution Cinema development with same asset pipeline as game environment

WAAG, AmsterdamCalit2, UCSD

Calit2 and NSF Optiputer Project Allow for High Speed Networks and Ultra-High Fidelity environments.

Potential DI-Cinema system architecture

1. Supporting efficient computation is focus2. Supporting low-latency streaming3. Supporting efficient integration with networked storage

Themis

Multi-stage,iterative

computationu

NAS/SANLong-term Storage

w

vSoftware remote display emulator

Hardwareremote display

Streaming Support

Storage networking challenges• Giant gather/scatter problem

– Read from ~100 disks and aggregate in-network at 60 fps (100+ MB/frame)

– Disk seek latency of 15ms, yet new frames must be loaded within 16 ms

2. Storage for batch and low-latency

Potential New Industrial Connections• Technicolor:

– Major provider of motion picture toolchain

– Acquiring content, video storage, computing, viewing

• Disney– Major source of content– Large computing

infrastructure; large pipeline-oriented computing platform

• Cisco– Datacenter networking

• HP– Provider of computation

and storage to motion picture industry

– Carries out data-intensive computing research

• Google– World’s largest video

processing system– YouTube– Just announced support

for 4K video

Next Steps

• Resolve server/compute system infrastructure

• Develop client interface mixing virtual world 4K and 4K camera based

• Integrate streaming storage system with server/compute