MPEG-4- based adaptive remote rendering for video games
17
MPEG-4-based adaptive remote rendering for video games Authors: N. Tizon, C.Moreno, M.Cernea, M.Preda Dept. ARTEMIS: http://www-artemis.it- sudparis.eu/ Contact: [email protected]
description
MPEG-4- based adaptive remote rendering for video games. Authors : N. Tizon , C.Moreno , M.Cernea , M.Preda Dept . ARTEMIS: http://www-artemis.it-sudparis.eu/ Contact: [email protected]. Outline. Remote rendering approaches The Kusanagi framework The Kusanagi plug-in - PowerPoint PPT Presentation
Transcript of MPEG-4- based adaptive remote rendering for video games
MPEG-4-based adaptive remote rendering for video games
Authors: N. Tizon, C.Moreno, M.Cernea, M.Preda
Dept. ARTEMIS: http://www-artemis.it-sudparis.eu/
Contact: [email protected]
direction ou services <pied de page>
Outline
Remote rendering approaches
The Kusanagi framework• The Kusanagi plug-in• The lobby server• The MPEG-4 client
Experimentation and optimizations• User experience• Latency measurement and adaptation
page 2
direction ou services <pied de page>page 3
Remote rendering approaches
direction ou services <pied de page>
Video streaming based solution ?
Six main families of solutions:
page 4
Approach
Graphic Commands
2D Pixels
2D Primitives
3D Vectors
Simple 3D Object
Multiples 3D Objects
Thin Client
-
+
+
+
-
-
High Quality
+
+
-
-
+
+
LimitedBandwidth
+
-
+
+
+
+
Constraint ConstantBitrate
-
+
+
+
+
+
~
direction ou services <pied de page>
The Kusanagi framework
I. The Kusanagi plug-in
II. The lobby server
III. The MPEG-4 client
page 5
direction ou services <pied de page>page 6
Architecture overview
direction ou services <pied de page>
Kusanagi plugin
From the 3D graphic application to the remote controlled software:• Image & Audio Capture• Interactivity commands between the client and the serverNet-input HTTP server
• Audio and video data from server to client.
page 7
Game + Stream Lib. Client
Send data to: localhost
Open: rtsp://IP_server/sdp_file
RTSP server
Send data to: IP_client
direction ou services <pied de page>
The lobby server
Interface between:• the users connected through the
network,• the 3D graphic application,• the hardware components and/or
third applications.
Resource manager:• the network interface for the user
connections and data transport,• the CPU and the graphic card for
the rendering and the encoding,• the audio card for the audio mixing and capture.
page 8
direction ou services <pied de page>
The MPEG-4 client
Full multimedia player with networking, media,user interactions management:• MPEG-4 scene description language• H.264/AVC and RTP/RTSP
Standard solution => Cross platform approach
page 9
direction ou services <pied de page>
Experimentation and optimizations
I. User experience
II. Latency measurement and adaptation
page 10
direction ou services <pied de page>
User experience
« Beta tests » approach (subjective evaluation)• Area: Parisian region, 53 users,1 month.• Point&click game => low constraint for the latency.• TRA (Theory of Reasoned Action, Fishbein and Ajen): 92%
of positive attitude towards the Kusanagi service.
Improvements• Availability on different devices (TV, mobile phone).• People are sensitive to image quality and latency => adaptive
video streaming.
page 11
direction ou services <pied de page>
Latency measurment
RTT(t) estimated from RTCP receiver reports Round Trip Time (RTT) estimation:
RTTi update: • RTT relative standard deviation: RSD = σ/µ
• If RTT variability is low (RSD< RSDth) => RTT(t) ~ RTTi
page 12
)()( tRTTRTTtRTT ci
Intrinsic component, provided at the beginning by the lobby server (ping)
Congestion based component
Network infrastructure based parameter (routers, transport channel..)
Depends on users activity.=> prone to adaptation
direction ou services <pied de page>
Bitrate adaptation
page 13
RTTi update
Qp factor adaptation when congestion is detected
Qp increase
Qp decrease
direction ou services <pied de page>
Expirmental results: background
Testing conditions• Video encoding: VGA, 30fps, H.264 low latency encoding (no B-
frames, CAVLC, no rate control buffering), (~5Mbps, 50dB)• Network: bandwidth and delay(RTT) monitoring • Algorithm parameters: α=0.1, β=1 and θ=0.9
RTT measurments without adaptation (RTTi=20ms, 5Mbps)
page 14
200 pQ
Peak bitrates > 5Mbps => Congestion=> Playability not acceptable
direction ou services <pied de page>
Intrinsic delay estimation
Measured RTT and PSNR (RTTi = 20ms and 80ms, bandwidth:5Mbps)
page 15
70s to discover the new RTTi value
direction ou services <pied de page>
Bitrate adaptation
Measured RTT and PSNR (RTTi = 20ms, bandwidth:2.5Mbps)
page 16
30s to reach a stable state: RTT=RTTi
direction ou services <pied de page>
Conclusions
A complete framework for remote rendering applied on video games.
Three main modules: the Kusanagi plug-in, the lobby server, the MPEG-4 client.
Beta tests: need of more adaptability in order to improve the user QoE.
An efficient method to detect congestion based latency and to adapt the video bitrate adequately.
Futur works:• To optimize the resource sharing• To optimize video encoding with side information from the 3D engine.
page 17
