Document Number: 608831-001
Intel® Media Accelerator Reference
Software for Linux*
White Paper
Document Number: 608831-001
Document Revision: 1.0
05 2019
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
2 Document Number: 608831-001
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR
OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS
OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING
TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE,
MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD
YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS
SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS
COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT
LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS
SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of
any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for
conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with
this information.
You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel® products
described herein. You agree to grant Intel® a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter
disclosed herein.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
All information provided here is subject to change without notice. Contact your Intel® representative to obtain the latest Intel® product
specifications and roadmaps.
The products described may contain design defects or errors known as errata which may cause the product to deviate from published
specifications. Current characterized errata are available on request.
Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting:
http://www.intel.com/design/literature.htm
Intel® technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn
more at http://www.intel.com/ or from the OEM or retailer.
No computer system can be absolutely secure.
Intel® and the Intel® logo are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2019, Intel® Corporation. All rights reserved.
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 3
Contents
1.0 Introduction ............................................................................................................................. 6
1.1 What is Intel® Media Accelerator Reference Software? .................................................... 6
1.1.1 Definition .......................................................................................................................................... 6
1.2 Scope of this Document ............................................................................................................................ 7
1.3 Terminology ...................................................................................................................................................... 7
1.4 Reference Documents ................................................................................................................................ 8
2.0 Intel® Media Accelerator Reference Software – Linux* ................................ 9
2.1 Overview of Intel® Media Accelerator Reference Software ............................................ 9
2.2 Technologies Used ........................................................................................................................................ 9
2.2.1 Advanced Linux* Sound Architecture (ALSA*) ...................................................... 9 2.2.2 FFmpeg* ........................................................................................................................................ 10 2.2.3 Intel® Media SDK ................................................................................................................... 10 2.2.4 Intel® Automotive Solution Compositor for Wayland* Protocol
Weston ............................................................................................................................................ 10
2.3 Intel® Media Accelerator Reference Software High-Level Architecture .............. 10
2.3.1 Intel® Media Accelerator Reference Software Video Manager ................ 12
3.0 A High Level Look at Popular Linux* Media APIs and Frameworks ......14
3.1 Advantages and Disadvantages of Using Higher Level Programming APIs such
as FFmpeg* vs Intel® Media SDK .................................................................................................. 14
3.2 Advantages and Disadvantages of Using FFmpeg*-QSV vs FFmpeg*-VAAPI .. 15
3.3 Advantages and Disadvantages of Migrating to iHD .......................................................... 16
3.4 Advantages and Disadvantages of Using FFmpeg vs. GStreamer ........................... 17
4.0 A High Level Look at Weston vs. X11 Rendering ..............................................19
4.1 Advantages and disadvantages of using Weston vs X11 ............................................... 19
4.2 Advantages and Disadvantages of Using Intel® Automotive Solution
Compositor for Wayland* Protocol Weston vs. Other Wayland Reference
Implementations ......................................................................................................................................... 19
5.0 Conclusion ...............................................................................................................................22
6.0 A Look Ahead at Future Enhancements .................................................................23
7.0 How to Acquire the Intel® Media Acceleator Reference Software ........24
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
4 Document Number: 608831-001
Figures
Figure 1. Intel ® Media Accelerator Reference Software for Linux* High Level
Architecture .................................................................................................................................................... 11 Figure 2 Video Manager .............................................................................................................................................. 13
Tables
Table 1. Terminology ...................................................................................................................................................... 7 Table 2. Reference Documents ................................................................................................................................ 8 Table 3. Advantages and disadvantages: FFmpeg vs Intel® Media SDK ............................... 14 Table 4. Advantages and Disadvantages: QSV vs. VAAPI ................................................................. 15 Table 5. i965 vs. Intel® HD Graphics Drivercomparison ................................................................... 16 Table 6. Advantages and disadvantages: FFmpeg vs. GStreamer .............................................. 17 Table 7. Advantages and disadvantages: Wayland vs. X11 ............................................................. 19 Table 8. Advantages and disadvantages: Intel® Automotive Solution Compositor for
Wayland* Protocol Weston vs. others ......................................................................................... 20
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 5
Revision History
Date Revision Description
May 2019 1.0 Initial release of this document.
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
6 Document Number: 608831-001
1.0 Introduction
The Intel® Media Accelerator Reference Software is a reference media player
application for digital signage use cases. It harnesses fixed-function hardware
acceleration for superior media streaming performance, reduced Central Processing
Unit (CPU) loading, and customizable Graphics Processing Unit (GPU) pipeline
solutions.
The architecture of the software stack includes the FFmpeg* VAAPI, FFmpeg* QSV, and
Intel® Automotive Solution Compositor for Wayland* Protocol Weston which improves
developer experience by promoting portability and compatibility with a range of
processors.
This white paper describes the architecture of Intel® Media Accelerator Reference
Software, a software stack applicable to many different market use cases, including
digital signage, kiosks, and more.
1.1 What is Intel® Media Accelerator Reference Software?
1.1.1 Definition
The main goal of the Intel® Media Accelerator Reference Software is to demonstrate
video content playback using VAAPI and QSV via FFmpeg and Weston Wayland for
rendering for retail, banking, hospitality, and education use cases. Intel® Media
Accelerator Reference Software also provides a configurable multi-zone composited UI
that enables the user to customize the number and types of playout zones .
Intel® Media Accelerator Reference Software is licensed reference code
showcasing best practices for video decode, playback, compositing, blending,
and rendering on Intel® platforms. Cross-platform support for access to Intel®
Graphics hardware acceleration and video processing pipeline.
Offloads graphics functions from CPU freeing up valuable processing for
analytics and other media player functions.
Code once and use on multiple current and future Intel® platforms.
Support for the latest formats and containers out of the box: JPG, GIF, BMP,
H.265/HEVC, H.264/AVC, VC-1, MJPEG, VP9, MKV, MP4, and MPEG2 TS.
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 7
1.2 Scope of this Document
This document provides a brief overview of Intel® Media Accelerator Reference
Software for Linux* along with the details about the advantages and disadvantages of
the Linux tools and frameworks selected for Intel® Media Accelerator Reference
Software.
1.3 Terminology
Table 1. Terminology
Term Description
ALSA* Advanced Linux Sound Architecture
DSG Digital Signage
FFmpeg* Fast Forward mpeg: A free software project that is a leader with their multimedia framework used to decode, encode, transcode, mux, demux, stream, filter and play.
HDR High Dynamic Range
GStreamer* An open source project that provides a library for constructing graphs of media-handling components. GStreamer playback, audio/video streaming as well as audio (mixing) and video (non-linear editing) processing.
GUI Graphical User Interface
IAS Reference implementation of a Wayland compositor called Intel® Automotive Solution Compositor for Wayland* Protocol
ICL IceLake
IWB Interactive White Board
Linux GNU Linux Operating System
OS Operating System
Qt* A cross-platform application framework and widget toolkit
SKL Intel® microarchitecture code name Skylake
TSD Transportation Solutions Division
VAAPI* Video Acceleration API
VPP Video Post/Pre Processing
VR Visual Retail
Wayland* Display Server Protocol
Weston* Reference implementation of a Wayland compositor
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
8 Document Number: 608831-001
1.4 Reference Documents
Table 2. Reference Documents
Document Document No./Location
Intel® Media Accelerator Reference Software for Windows* 608837-001
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 9
2.0 Intel® Media Accelerator Reference Software –
Linux*
2.1 Overview of Intel® Media Accelerator Reference Software
The Intel® Media Accelerator Reference Software is an application that delivers digital
content across media signage devices such as digital posters, interactive kiosks, menu
boards, dynamic shelf labels, intelligent vending machines, point-of-sale devices, and
more. Intel® Media Accelerator Reference Software works to display the contents on the
screen, built by an Extensible Markup Language (XML) configuration file.
At a high-level, Intel® Media Accelerator Reference Software supports the features such
as multi-monitors, multiple zones, and a configuration Graphical User Interface (GUI).
Other features at a high-level include the following:
Video Acceleration API (VAAPI) and QSV via FFmpeg for decode
GPU vs CPU operations for decode, color conversion, scaling, and compositing.
Qt* based zone editor for display configuration
Hybrid multi-monitors.
Identical, Tiled, Different, Portrait, and Landscape display topologies.
Multiple zones: Each display could be configured with various media contents
via zones.
Media content: Video, audio, image, text ticker, RSS, benchmark zone.
Zone carousel, Zone border
Zone overlay and alpha blend
2.2 Technologies Used
The technologies used with Intel® Media Accelerator Reference Software include Intel®
Media SDK/Intel® Quick Sync Video and VAAPI via FFmpeg, Qt, Weston and Advanced
Linux* Sound Architecture (ALSA*).
2.2.1 Advanced Linux* Sound Architecture (ALSA*)
ALSA is a kernel driver providing audio software components access to the Linux kernel
for decoding audio frames.
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
10 Document Number: 608831-001
2.2.2 FFmpeg*
FFmpeg is a community project that is a leader with their multimedia framework used
to decode, encode, transcode, mux, demux, stream, filter and play. FFmpeg is portable
and compiles and runs on multiple platforms and hardware architectures.
2.2.3 Intel® Media SDK
Intel® Media SDK accelerates rich media performance and speeds up video playback.
Use Intel® Media SDK to develop professional-grade media and video applications for
digital surveillance, retail, industrial, smart home, video conferencing, with the libraries,
tools, and samples included in this SDK. Its single API enables hardware acceleration
for fast video transcoding, image processing, and media workflows while helping you
use the capabilities in Intel® Quick Sync Video.
Accelerate rich media performance. Speed up video playback, encoding,
processing, and media formatting conversion.
Cut product time to market. Prototype, optimize, and productize your media
pipelines with a comprehensive, convenient API.
Speed up transition to new codecs. Use efficient codecs with hardware
acceleration to increase speed, compression, and quality across AVC, HEVC, and
MPEG-2.
Debug and customize your products quickly. Find and solve application bugs,
port applications to new operating systems, and add new features using source
code for open source Linux*.
2.2.4 Intel® Automotive Solution Compositor for Wayland* Protocol Weston
Weston is the reference implementation of a Wayland compositor, and a useful
compositor in its own right. Weston has various backends that lets it run on Linux
kernel modesetting and evdev input as well as under X11. According to the Weston
main page, “Weston is the reference implementation of a Wayland server. Wayland
server is a display server, a window manager and a compositor all in one.” Visit
https://github.com/intel/ias for more information about this project.
2.3 Intel® Media Accelerator Reference Software High-Level Architecture
The high-level architecture diagram in Figure 1 below depicts the major software
modules in the Intel® Media Accelerator Reference Software application.
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 11
Figure 1. Intel® Media Accelerator Reference Software for Linux* High Level Architecture
Intel® Media Accelerator Reference Software Linux Install
Config ManagerXML Config File
Display Manager
Display
SysInit Manager
Wayland
Video Manager
Image Manager
RSS ManagerAudio Manager
FFMPEG API
Input
Info
DecodeInitialize Codecs
FFMPEG SW Library VA API via libVA
Software Decode Hardware Decode
IAS Shell, Compositor
Initialize/Configure Modules
Ticker Manager
Composite & Render
Video / Audio / Image
RSS / Text
URL
Qt Module
QSV via libVA
Hardware Decode
FFmpeg API
FFmpeg SW Library
SysInit Manager is the software module that manages the initialization of the
application by first reading the configuration file and creating the configured zones via
manager instances. For example, if the configuration file has a video zone with five
videos for playback and a ticker zone to scroll text horiizonatally across the video it will
start and initialize a video manager and a ticker manager. As already mentioned the
SysInit Manger is responsible for instantiating appropriate modules based on the
configuration by the user. Each zone instance will execute as a separate thread in the
system. Certain zones like Video may have more than one thread (e.g. to read file data,
decode audio data, decode video data, rendering thread etc.).
Note: The C++11 standard is used for threads, mutexes and semaphores and managed via a
common software block called the KAL (Kernel Abstraction Layer) as illustrated in
Figure 2.
Once all the zone managers, per the user defined configuration, are initialized the
SysInit Manger will start the media playback. The playback can continue for a finite or
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
12 Document Number: 608831-001
infinite duration based on the user configuration. In case of finite duration, the media
playback will end after the specified time.
For image, video ,and audio media types, Intel® Media Accelerator Reference Software
uses FFmpeg media APIs to demux, decode, filter, etc. the data. An abstraction layer is
created to abstract FFmpeg API calls from the application.
Finally, the SysInit Manager is responsible for starting and initializaing the Intel®
Automotive Solution Compositor for Wayland* Protocol Weston compositor for
rendering. All the media content will be passed to a compositor module which is
responsible for creating a composited scene and rendering on a display.
Note that Qt* is only used for the Configuration UI as part of Intel® Media Accelerator
Reference Software, and all manager modules are instantiated as follows:
Different configuration – One instance of each manager module per display.
Identical and Tiled configuration – One instance of each manager across
displays.
2.3.1 Intel® Media Accelerator Reference Software Video Manager
The below diagram is the high level architecture for the Video Manager as introduced in
Figure 1 above. As mentioned above, the SysInit manager starts the Video Manager
which in turn starts three concurrent threads for demuxing and parsing media files,
decoding audio and video media data via FFmpeg, and after, uses Intel® Automotive
Solution Compositor for Wayland* Protocol Weston for rendering the media data to the
display(s).
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 13
Figure 2 Video Manager
FFmpeg Decode Engine
Audio DecodeVideo Decode
Player Engine
System Manger
Media Manger Interface
File Read Thread
Video Decode Thread
AudioDecodeThread
Video Manager Interface(Main Task)
(KAL)
Kernel
Abstraction
Layer
(Threads, Semaphores,
Message queues
etc)
Video Client Thread EventLoop
FFmpeg APIs
S/W Decode VAAPI
S/W DecodeHardware
Decode (GPU)
Display Manager
ConfigurationManager
Renderer
Third Party Modules
Intel® Media Accelerator Reference Software Developed
Legends
Intel® Automotive Solution Compositor
for Wayland* Protocol
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
14 Document Number: 608831-001
3.0 A High Level Look at Popular Linux* Media
APIs and Frameworks
This section presents a high level comparison of key advantages of:
Using higher level programming like FFmpeg vs Intel® Media SDK
Using Intel® Quick Sync Video (QSV) technology vs VAAPI
Moving to iHD
Using FFmpeg vs GStreamer
3.1 Advantages and Disadvantages of Using Higher Level Programming APIs such as FFmpeg* vs Intel® Media SDK
FFmpeg* is a very popular library which has been integrated into various applications
(such as MPV, VLC, Google Chromium/Chrome, Handbrake, and more). It can be
leveraged without much development to enable FFmpeg-qsv vs. FFmpeg-VAAPI.
Intel® Media SDK is focused on Intel® video decoding, encoding, processing which is not
enough for transcoding. With FFmpeg, you can have streaming/mux/demux, and also
various software audio and video codecs such as AAC, x264, 265, libvpx, and libaom.
Table 3. Advantages and disadvantages: FFmpeg vs Intel® Media SDK
Advantages Disadvantages
FFmpeg Intel® Media SDK FFmpeg Intel® Media SDK
More higher level
features such as
streaming, mux,
demux, RTP/RTSP, etc
while still having
access to MediaSDK
features via FFmpeg-
qsv.
Calls to Intel® Media
SDK directly gives
more control to use
the Intel® Media SDK
API without
FFmpeg/others
Unfamiliarity
or
complexity
with
licensing
Video processing of
elementary streams
only
Various audio and
video codecs
Not all features of Intel® Media SDK have
been exposed by
FFmpeg-qsv
More
restrictive
licensing
terms
Popular with
community support
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 15
3.2 Advantages and Disadvantages of Using FFmpeg*-QSV vs FFmpeg*-VAAPI
Intel® Quick Sync Video technology on Intel® Iris Pro Graphics and Intel® HD graphics
provides transcode acceleration on Linux* systems with FFmpeg* v2.8 and later.
FFmpeg* v2.8+ includes Intel® Quick Sync Video accelerated h264_qsv, mpeg2_qsv
and hevc_qsv codecs. These provide a fast time-to-market path to server transcode
solutions based on Intel® Xeon® processors with processor graphics.
VAAPI (Video Acceleration API) is an open-source library and API specification, which
provides access to graphics hardware acceleration capabilities for video processing. It
consists of a main library and driver-specific acceleration backends for each supported
hardware vendor.
Advantages of ffmpeg-qsv v ffmpeg-vaapi are listed in Table 4.
Table 4. Advantages and Disadvantages: QSV vs. VAAPI
Advantages Disadvantages
QSV VAAPI QSV VAAPI
Can support Windows. Common API for Linux and Windows.
Linux* only - Can support different vaapi-drivers, including i965 driver, iHD driver. And also AMD/NVidia mesa driver.
Less popular in the FFmpeg community and Linux* players.
The libva driver needs to be attached to a DRM device to work
Better encoding quality with specified feature (such as LookAhead)
Easier secondary development due to focusing on Linux and less dependency on third-party such as Intel® Media SDK.
Better encoding performance with specified feature (such as MFE for 1:N/N:N transcoding on high performance GPU).
Wider codec support such as VP9 decoding and encoding. (Just current status. FFmpeg-qsv will support vp9 soon).
Some features have less dependency on FFmpeg framework and focusing on Intel® platform. Thus making these features easier to upstream.
Co-work closer with FFmpeg software codecs.
Automatically switches to FFmpeg software decoder if HW decoding failed.
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
16 Document Number: 608831-001
Advantages Disadvantages
QSV VAAPI QSV VAAPI
More popular in FFmpeg community, and more popular by various Linux player (Players don’t care encoder very much).
Automatically switches to FFmpeg software decoder if HW decoding failed.
3.3 Advantages and Disadvantages of Migrating to iHD
The i965-va-driver is the Intel® G45 and HD Graphics family driver for VAAPI. The
dependencies include at least 2 DRM packages, libdrm-intel1 and libdrm. Since this
driver has a long history being open source, the i965 driver can support legacy Intel®
GPUs, a legacy upstream Linux kernel, and various media frameworks such as ffmpeg
and GStreamer. I965 is convenient as you can just apt-get install on Ubuntu* or yum
install it on CentOS*/Fedora*.
The iHD driver was distributed with MediaServerStudio. Currently, it is open source
also, but just supports Generation 6 and above (Skylake+) platforms. The Intel® HD
Graphics Driver has some limitations with the upstream Linux kernel.
The iHD driver can work with Intel® Media SDK quite well, and supports more video
processing features such as tone-mapping/rotation.
Table 5. i965 vs. Intel® HD Graphics Drivercomparison
Advantages Disadvantages
I965 Intel® HD Graphics Driver
I965 Intel® HD Graphics Driver
Open Source Open Source Deprecated soon. i965 driver won’t support 10th or 11th generation and/or Ice Lake+ (ICL+) platforms. This means the Intel® HD Graphics Driver will be the only driver for 10th generation and above platforms
Limited platform support . See supported platforms at https://github.com/intel/media-driver
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 17
Advantages Disadvantages
I965 Intel® HD Graphics Driver
I965 Intel® HD Graphics Driver
Supports legacy GPUs
Works with Intel® Media SDK
Limitations with upstream kernel prior to 4.16
Supports legacy kernel versions
Supports more video processing features
Supports various media frameworks – ffmpeg and gstreamer
Soon to be the only video driver for Intel®
3.4 Advantages and Disadvantages of Using FFmpeg vs. GStreamer
FFmpeg is a useful transcoding library
(streaming/decoding/encoding/mux/demux/processing) with a very simple and easy to
use API. FFmpeg supports a large codec library which is why FFmpeg can seem more
attractive than using GStreamer.
GStreamer is closer to a media framework and is very flexible and distributes various plugins. Besides the pre-packaged plugins GStreamer can be extended to support new codecs/filters by creating new plugins using basic GSteamer programming concepts.
Table 6. Advantages and disadvantages: FFmpeg vs. GStreamer
Advantages Disadvantages
FFmpeg GStreamer FFmpeg GStreamer
Transcoding easily performed through command line or various popular GUI implementations readily available
GStreamer FFmpeg plugin available with a set of elements using the FFmpeg library
An API vs. media framework.
Easiser to create a new application which desires more of a media framework architecture
Media framework vs. APIs.
Easier to integrate APIs vs. adopting a new media framework into an existing software application
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
18 Document Number: 608831-001
Advantages Disadvantages
FFmpeg GStreamer FFmpeg GStreamer
Various audio and video codecs
Various plugins for codecs, filters, sinks and sources. GStreamer provides a framework to control different elements that perform different tasks and synchronized data flow.
Large redistributable framework size
Supported in a various popular applications (VLC, YouTube, MPV, Handbreak, Google Chrome/Chromium,
and more)
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 19
4.0 A High Level Look at Weston vs. X11 Rendering
4.1 Advantages and disadvantages of using Weston vs X11
Weston is a compositing window manager for Wayland. Weston is lightweight, fast and
can be used for embedded or mobile devices. Wayland is a protocol between a
compositor and clients.
The table below describes the advantages and disadvantages of using Weston vs. X11
architecture.
Table 7. Advantages and disadvantages: Wayland vs. X11
Advantages Disadvantages
Wayland X11 Wayland X11
Small/Minimal redistributable size
Protocol for communication between the display server and the clients
X was developed over two decades ago and code base is very large (much of it is unused)
Designed with the premise that every frame is perfect. Essentially only completed frames from applications are rendered and timed with vsync updates.
No guarantee to prevent tearing (rendering of frames timed with vsync)
HDR support in progress
4.2 Advantages and Disadvantages of Using Intel® Automotive Solution Compositor for Wayland* Protocol Weston vs. Other Wayland Reference Implementations
The Intel® Automotive Solution Compositor for Wayland* Protocol Weston compositor
was developed to suit the Intel Transportation Solutions Division (TSD) use cases, but
also support features which are far broader. For example, Intel® Automotive Solution
Compositor for Wayland* Protocol Weston already supports where applications will be
positioned, how they will be z-layered, and how big windows be or whether they will
even be shown or not. For features not already supported easy-to-write plugins can be
developed and integrated.
To simplify the control of the the look and feel of a system the Intel® Automotive
Solution Compositor for Wayland* Protocol shell exposes an interface to the HMI
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
20 Document Number: 608831-001
controller (a common central controller of the system Intel® Automotive Solution
Compositor for Wayland* Protocol in the automotive world) to modify the properties
for all other applications and Intel® Automotive Solution Compositor for Wayland*
Protocol would just follow this guidance. The goal of Intel® Automotive Solution
Compositor for Wayland* Protocol is to provide key hardware functionality to our
customers as fast as possible.
The goal of Weston is to provide a reference compositor to showcase Wayland
protocol and also something that works equally well on all hardware (.
For customers to use open source Weston (upstream non- Intel® Automotive Solution
Compositor for Wayland* Protocol Weston) they would need to essentially modify the
policies within the compositor and therefore own the entire compositor.
Table 8. Advantages and disadvantages: Intel® Automotive Solution
Compositor for Wayland* Protocol Weston vs. others
Advantages Disadvantages
IAS Weston Intel® Automotive Solution
Compositor for Wayland* Protocol
Weston
HMI operations management
No plan to upstream Intel® Automotive Solution Compositor for Wayland* Protocol features into open source Weston such as rendering to a region of intrest (window position)
Written specifically for TSD/automotive use cases
No focus in the desktop world so definitely no support for Ubuntu/Fedora distros
Time to edit policies within compositor – thus owning the compositor
Support for multiple hw planes
IA (Intel Architecture) display controller functionality like display controller based rotation and scaling
No provision for window
Positioning
Z-layering
Size
Visibility
Supports remote sharing
Major gaps for upstream Weston (e.g. remote sharing, lack of desktop
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 21
As noted above, the Intel® Automotive Solution Compositor for Wayland* Protocol
Weston Wayland compositor implementation spans beyond the Intel Transportation
Solutions Division (TSD) use cases. Thus, the Intel® Media Accelerator Reference
Software takes advantage of Intel® Automotive Solution Compositor for Wayland*
Protocol Weston to efficiently composite videos, images, and/or tickers at specific
positions and z-layers on one more displays.
support for some major desktops like Ubuntu* 16LTS, no ability to draw to a region of interest on a display, no multi-display support for Tiled/Different/Clone modes)
Boots faster (optimized for Intel Architecture)
No support for multiple hw planes
Support for rendering to multiple diplays and rich graphics (3D as well)
No IA display controller functionality like display controller based rotation and scaling
Supports hypervisor based systems and can share buffers created from one domain with another domain
No support for rendering to multiple displays
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
22 Document Number: 608831-001
5.0 Conclusion
The Intel® Media Accelerator Reference Software demonstrates using FFmpeg*-VAAPI
or FFmpeg*-QSV and Intel® Automotive Solution Compositor for Wayland* Protocol
Weston. It was designed to accelerate time to market for implementing key features in
each of the above technologies such as FFmpeg-QSV for decode and VPP operations
(using hardware or software codecs depending on the configuration) such as
deinterlace and Intel® Automotive Solution Compositor for Wayland* Protocol Weston
for compositing multiple videos/images/scrolling text(s) across multiple displays at
specific locations. Given FFmpeg’s popularity (by both the community and application
implementations), API architecture, relatively minimal footprint, and vast readily
available features and rich capabilities ( such as RTSP decode for IP cameras, MJPEG
decode for USB cameras, and more), it was decided as a key component in the Intel®
Media Accelerator Reference Software. Similarily, Intel® Automotive Solution
Compositor for Wayland* Protocol Weston as opposed to X11 and upstream Weston,
was chosen due to its rich capabilities that span beyond automotive use cases such as
rendering to specific locations on a one more displays and multi-display support such
as clone and extended modes. Intel® Automotive Solution Compositor for Wayland*
Protocol Weston (along with the upstream Weston) will also support High Dynamic
Range (HDR) in the near the future.
Intel® Media Accelerator Reference Software for Linux* May 2019 White Paper
Document Number: 608831-001 23
6.0 A Look Ahead at Future Enhancements
In addition to the performance enhancements created by GPU-based hardware
acceleration and reduced CPU loading, future versions of Intel® Media Accelerator
Reference Software will contain more advance features such as multi-zone compositing
for encode/broadcast, real-time transcode and broadcasting, and the integration of
deep learning analytics via the Intel® distribution of OpenVINO™ toolkit.
The integration of the Intel® distribution of the OpenVINO™ toolkit will allow the
creation of customized use cases such as how to monitor audience demographics in
front of a digital signage solution (e.g. the number of customers and their approximate
age and gender). All of the existing models in Intel Model Zoo packaged with
OpenVINO™ will be supported along with any future models which are supported by
OpenVINO™ Model Optimizer.
Additional use cases spanning retail, hospitality, banking, and education which could be
relialzed via easy-to-perform configuration changes in the Intel® Media Accelerator
Reference Software are:
Emotion detection, gaze tracking, and facial reidentification. With these
behavioral data references solution providers can gain insight into user
responses and craft improved messaging, personalization, prediction, and
more.
Student and teacher analytics in a classroom such as raising hands, sitting,
standing, attendance tacking, etc). Over time, the collected data can be used to
design and calibrate course content and interactions to be used by the factulty,
administration, and parents.
Autonomous/frictionless stores could use object detection models for tracking
inventory in a store, vending machine, or cooler as well.
Intel® Media Accelerator Reference Software for Linux*
White Paper May 2019
24 Document Number: 608831-001
7.0 How to Acquire the Intel® Media Acceleator
Reference Software
In order to acquire the Intel® Media Accelerator Reference Software, you’ll need to
contact your Intel representative to see if you qualify for access.
Top Related