XLcloud 3-d remote rendering
-
Upload
marius-preda -
Category
Science
-
view
193 -
download
3
description
Transcript of XLcloud 3-d remote rendering
![Page 1: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/1.jpg)
XLcloud – 3D Rendering in the Cloud
Open Stack Summit, Paris, November, 2014
Marius PREDA, TELECOM SudParis
Philippe GRAVEZ, CEA
![Page 2: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/2.jpg)
Summary
Some words about the project
Remote rendering principle and optimizations
Remote Rendering middleware
Demos
![Page 3: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/3.jpg)
ConsortiumATEME: H264/MPEG-4/AVC low latency compression algorithms
Bull: cloud infrastructure, distributed systems architecture and HPC cluster
CEA List: interactive simulation of natural phenomenons, virtual reality
EISTI: marketplace, OpenERP integration
Inria Reso: energy efficiency
TSP (Artemis Lab): remote rendering, video compression, multimodality
Silkan: massively distributed (HPC) software architecture for interactive simulation
OW2: communication, dissemination
Origin of the project
Winner of 1st Call for Projects “Cloud Computing” by the FSN in 2011
Project length: 36 months
Official Start: 01/01/2012
Coordinator: Bull
Open Source (Apache V2)
http://gitorious.ow2.org/xlcloud
http://xlcloud.org/
http://gitorious.ow2.org/xlcloud
![Page 4: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/4.jpg)
XLcloud objective
Establish the demonstration of a High Performance Cloud Computing (HPCC) platform designed to run effectively a
representative set of compute intensive workloads including interactive games and 3D graphics applications
![Page 5: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/5.jpg)
Capabilities of the XLcloud platform
The platform exposes an on-demand Platform-as-a-Service service endpoint (and GUI) which developers can use to manage software stacks of nearly any scale and complexity without sacrificing control
The platform's main design paradigm is to make it easy to manage the complete application lifecycle, including resource provisioning, configuration management, application deployment, software updates, monitoring, and access control.
The software stack artefacts produced by XLcloud are primarily targeted toward delivering two kinds of virtual clusters, but could easily be extended to produce other types of virtual platforms:
● An HPC Virtual Cluster based on SLURM
● A Remote Rendering Virtual Cluster based on Ogre3D Today’s topic
![Page 6: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/6.jpg)
Remote Rendering
![Page 7: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/7.jpg)
Scene analyzer
User interaction
Scene description
DataAI
Local Input
Network parameters
Remote Input
Segmentation map
Depth map
Motion vectors
Application Engine
Scene Graph
Rendering engine
2D/3D graphics
Video Encoder
Graphics Encoder
RTP/UDPPacketizer
Streamer
Network ready packets
Compressed stream
PlayerInteractivity
Terminal parameters
Network
Server
Client
Network
How remote rendering works?
![Page 8: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/8.jpg)
Scene analyzer
User interaction
Scene description
DataAI
Local Input
Network parameters
Remote Input
Segmentation map
Depthmap
Motion vectors
Application Engine
Scene Graph
Rendering engine
2D/3D graphics
Video Encoder
Graphics Encoder
RTP/UDPPacketizer
Streamer
Network ready packets
Compressed stream
PlayerInteractivity
Terminal parameters
Network
Server
Client
Network
How remote rendering works?
xlcloud added value
Transmoder
Depth MAPQuality control
Buffers extraction
optimizations
Pre-Processing
![Page 9: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/9.jpg)
Scene analyzer
User interaction
Scene description
DataAI
Local Input
Network parameters
Remote Input
Segmentation map
Depthmap
Motion vectors
Application Engine
Scene Graph
Rendering engine
2D/3D graphics
Video Encoder
Graphics Encoder
RTP/UDPPacketizer
Streamer
Network ready packets
Compressed stream
PlayerInteractivity
Terminal parameters
Network
Server
Client
Network
How remote rendering works?
xlcloud added value
Transmoder
Depth MAPQuality control
RoAQuality control
Region Of Attention
Map
Buffers extraction
optimizations
Pre-Processing
![Page 10: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/10.jpg)
Examples of applications in Remote Rendering
![Page 11: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/11.jpg)
The problem that the Cloud can solve for Remote Rendering
Video bitrateNetwork Bitrate (kb/s)
Time (s)
Frozenimage
Frozenimage
Frozenimage
Frozenimage
![Page 12: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/12.jpg)
Reduce the video bitrate to keep it under the network bitrate
Network Bitrate (kb/s)
Time (s)
The problem that the Cloud can solve for Remote Rendering
![Page 13: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/13.jpg)
The objective is simple, the results when applying traditional rate control are just … horrible
![Page 14: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/14.jpg)
Problem Statement
Find new representation spaces of video sequences that allows to reduce the debit beyond the theoretical limit of the pixel space while preserving the user experience
![Page 15: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/15.jpg)
Proposed Approach
Statistically detect regions of interest (ROI)
Create importance maps
Concentrate the coding error in “less important” areas
Correlate importance maps with the characteristics of the image, measured in real time
Perform a segmentation of the image in relation to these characteristics
Build a hybrid space
Compress the hybrid space
Transmit, decode and display the rendered hybrid space
![Page 16: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/16.jpg)
What is the “important” info?
Player’s attention is game specific and it depends on the graphical content, action, interaction
![Page 17: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/17.jpg)
What is the “important” info? “Ask” the user …
5 players, 2 games (Doom3 and 0AD), 1 eye-tracker and many playing hours
To get this …
![Page 18: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/18.jpg)
What is the “important” info? “Ask” the user …
![Page 19: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/19.jpg)
Intra-player analysis
Strong correlation if rp>0,5
![Page 20: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/20.jpg)
Inter-player analysis
Doom3, rp > 0,85 (between ¯xDoom3 and all the tests)
0 A.D., rp > 0,69 (between ¯x0AD and all the tests)
![Page 21: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/21.jpg)
Region of Attention Model
Modulate the quality of the video encoding based on RoA
![Page 22: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/22.jpg)
Additional control: adaptive video coding based on depth map
Control of region quality based on distance from viewpoint
Inverse exponential function to correct non-linear depth values
Variable range support
![Page 23: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/23.jpg)
𝑧5 ,𝑚2
𝑧 6 ,𝑚1𝑧 3 ,𝑚1
𝑧1 ,𝑚3
𝑧 2 ,𝑚2
𝑧 4 ,𝑚2
𝑧7 ,𝑚4
Displacing the errors in the video frame is fine, however it has theoretical limits imposed by the “Pixel” space
Additional improvement: represent the image by using two spaces
Regions containing edges – represented as color pixels
Edge free regions – represented as polynomial coefficients
![Page 24: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/24.jpg)
Hybrid Space – Visual Results
Original Image
Hybrid Space
Pixel Information
Polynomial Information
Reconstructed Image
![Page 25: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/25.jpg)
Remote Rendering Middleware
![Page 26: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/26.jpg)
Node 2Node 1
Remote Rendering – 3 Layers structure
Lobby
Lobby Agent Lobby Agent
RR Session 1
RR Session 2
RR Session 3
RR Session 1
RR Session 2
RR Session 3
Application Catalog
Client
Video Stream
Interactivity
Remote Rendering Virtual Cluster
Cluster : Lobby Component
Node : Lobby Agent Component
Application
Load Table
![Page 27: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/27.jpg)
Remote Rendering – 3 Layers Structure
Sequence Diagram for Remote Rendering Session Submission
![Page 28: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/28.jpg)
Node 1
Remote Rendering – 3 Layers StructureLobby
Lobby Agent
X server
Application 1
Application Catalog
Client
Video Stream
Interactivity
Remote Rendering Virtual Cluster
Session Initialization
X Server launch
RR Middleware components launch
Application launch
Load Table
Streaming server
X server
Application 2
Streaming server
![Page 29: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/29.jpg)
Remote Rendering Middleware - Components
Remote Rendering Library
Graphics Engine Video Output (GEVO)
Interaction Module (NetInput)
Open Source Graphics Engine (OGRE)
Ogre-GEVO Plugin
Video Encoder
Local Dependencies
Streaming Server
Lobby Agent
![Page 30: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/30.jpg)
RR Middleware – Application Integration
RR Middleware
RR Dependencies
Remote Rendering Library
GEVO - RRStream Interactivity (NetInput)
VideoEncoder
Ogre – GEVO Plugin
3D ApplicationOgre Application
StreamingServer
![Page 31: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/31.jpg)
Demo
![Page 32: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/32.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 cores
![Page 33: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/33.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 coresCPU
![Page 34: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/34.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 coresCPU
![Page 35: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/35.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 coresGPU
![Page 36: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/36.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 coresGPU
![Page 37: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/37.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 coresNetwork
![Page 38: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/38.jpg)
Games
4 sessions, 1280x 720Nvidia K340 grid
Intel Xeon 2 coresNetwork
![Page 39: XLcloud 3-d remote rendering](https://reader037.fdocuments.us/reader037/viewer/2022110308/557d5e1fd8b42abf3d8b4ed2/html5/thumbnails/39.jpg)
A demonstration of a High Performance Cloud Computing platform
More computation is available, better compression is possible
Modulate the compression quality with respect to user attention and application behavior
Generate in real time complementary representation spaces for images
Concluding Remarks