A.G. Malamos Multimedia Lab Dept. of Informatics Engineering

RESTful APIs and Delivery Formats for 3D ResourcesKristian Sons (DFKI) Rémi Arnauld (AMD)

Johannes Behr (Fraunhofer IGD)

The multimedia point of view

A.G. MalamosMultimedia Lab

Dept. of Informatics EngineeringTEI of Crete, Heraklion, Crete, Greece

([email protected])

19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA

The multimedia dimension of graphics

Graphics and especially virtual worlds have a clear multimedia dimension in the sense of coexistence of multiple medias presented together in real time to provide an integrated audiovisual result.

Performance in that case is a convolutional result of several factors applied to graphics, video and image textures and sound. Such performance factors might be...LOD (resolution), Delay, Jitter and of course.... user expectations.


Our experiments in real-time 3D delivery

1. HTTP vs Websockets (The two most popular approaches on application networking)

2. The MPEG DASH (Dynamic Adaptive Streaming over HTTP)


HTTP vs Websockets

The 1st experiment


HTTP vs Websockets (1/5)

What is websockets?

WebSocket is a protocol providing full-duplex communications over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API is being standardized by the W3C.

Client Server

Handshake (upgrade HTTP)

Full-duplex Data Channel

Persistent Connection

Time


HTTP vs Websockets (2/5)

TCP/IP

WEBSOCKETSSTOMP

Simple (or Streaming) Text Orientated Messaging Protocol

HTTP

The websockets testbed


HTTP vs Websockets (3/5)The data-set

We have currently uploaded several models from tens of thousands to millions of point coordinates, including colour, faces and normals information

1. The dome of the church of St. Ioannis in the TEI area, in Crete2. Traditional Cretan musical instrument 3. Ancient Theater Demetriados4. An olive tree5. Egyptian Cat

from the The Classic Archaeology Collectionin the Museum of Archeology and History of Art of the University of Athens

6. an amphora7. a wheel-made plate8. Corinthian oinochoe 9. a wheel-made lekythos) 19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY

AUGUST 8-10 – VANCOUVER, CANADA

HTTP vs Websockets (4/5)Models are in X3D-X3DOM format

Three Web-based distribution platforms available to compare:

1. Direct display

2. progressive streaming over HTTP requests

3. Websocket-STOMP progressive transmission

Optional use of LZW compression into streams

Two progression modes:1. Serial (neighborhood) progression2. Area sort (biggest triangles first)


HTTP vs Websockets (5/5)a. Serial progression b. Area sort progression

Webservices have better performance than HTTP requests

Entropy Compression (LZW) increase performance in Webservices and it degrades performance of http requests

Streaming has better performance than model downloading19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY

AUGUST 8-10 – VANCOUVER, CANADA

MPEG DASH in real-time graphics delivery?

The 2nd experiment


MPEG DASH (1/4)

The MPEG-DASH streaming adaptation approach is conceptually simple.

A client (player) application accesses the Media Presentation structure of a media, normally stored in an XML file with the extension .mpd, and selects the appropriate Adaptation Sets corresponding to the characteristics of the end-device and the end-user (e.g. audio tracks in specific languages, video tracks in specific encodings).

Then, the client begins downloading and playing Representations from these adaptation sets, while at the same time probing for QoE metrics (e.g. achieved FPS rate, remaining power, computational resources available).

Source: MPEG DASH IF


MPEG DASH 2/4

Mesh parameters

Texture Image/Vide

o

3D Model

MPD Structure

…

Representation #1

Representation #2

Representation #3

Multimedia Presentation Description

Stream preparation

X3D VR World

The media (3D) file preparation


MPEG DASH 3/4

Host Server1

Host Server2

Host Server(n)

Presentation Quality (LOD)

Decision

Client

Network

Memory

Battery

FPS

Timer

Metrics Probing

X3DOMDisplay

Web Server

The MPEG DASH player


MPEG DASH 4/4The test-bed and results

http://mclab1.medialab.teicrete.gr:8081/indexdash.html


http://mclab1.medialab.teicrete.gr:8081/indexdash.html

A.G. Malamos Multimedia Lab Dept. of Informatics Engineering

Documents

Transcript of A.G. Malamos Multimedia Lab Dept. of Informatics Engineering