Terminal mode architecture

Copyright © 2010 Nokia Corporation, Audi AG, BMW AG, Daimler AG, Porsche AG, Volkswagen AG. All rights reserved.

Terminal Mode Technical Architecture

Release Candidate v0.9

This document is an integral part of the Terminal Mode Specification, and together with “TmSer-verDevice:1 Device Template, version 0.9”, “TmApplicationServer:1 Service Template, version 0.9”, “TmClientProfile:1 Service Template, version 0.9”, “TmClientDevice:1 Device Template, ver-sion 0.9”, “TmApplicationClient:1 Service Template, version 0.9”, “TmConnectionManager:1 Ser-vice Template, version 0.9” define the Specification version 0.9.

Copyright © 2010 Nokia Corporation, Audi AG, BMW AG, Daimler AG, Porsche AG, Volkswa-gen AG. All rights reserved.

The copyright holders hereby grant you the right to make verbatim copies of the Specification and to make available and distribute such copies. You may not distribute, make available or copy only parts of the Specification, nor modify or create any derivative works of the Specification. No patent rights or other rights not expressly stated as granted, are granted herein.

The above copyright notice and these terms and the following disclaimer must be retained in all copies of the Specification.

THE SPECIFICATION IS PROVIDED “AS IS”. THE COPYRIGHT HOLDERS GIVE NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL PROPERTY RIGHTS REGARDING THE SPECIFICATION.

Document editor:

Jörg Brakensiek [email protected]

Nokia Inc. 955 Page Mill Road 94304 Palo Alto, CA

USA

- 2 -


List of Contributors

Name Affiliation

Dennis Fernahl Carmeq (for Volkswagen AG)

Jörg Brakensiek Nokia Corporation

Holger Grandy BMW AG

Kari Kostiainen Nokia Corporation

Keun-Young Park Nokia Corporation

Mark Beckmann Volkswagen AG

Martin Fesefeldt Volkswagen AG

Martti Ala-Rantala Nokia Corporation

Matthias Benesch Daimler AG

Michael Wolf Daimler AG

N. Asokan Nokia Corporation

Raja Bose Nokia Corporation

- 3 -


TABLE OF CONTENTS

TABLE OF CONTENTS .............................................................................................................................. 3

LIST OF FIGURES ....................................................................................................................................... 5

LIST OF TABLES ......................................................................................................................................... 7

TERMS AND ABBREVIATIONS ............................................................................................................... 9

1 ABOUT ................................................................................................................................................ 10

2 INTRODUCTION TO TERMINAL MODE ..................... ............................................................... 11

3 CONNECTIVITY PROTOCOL STACK......................................................................................... 13

3.1 PHYSICAL & LINK LAYER ............................................................................................................. 13

3.1.1 Universal Serial Bus (USB) ..................................................................................................... 13 3.1.2 Wireless Local Area Networks (WLAN)................................................................................... 14

3.2 NETWORKING AND TRANSPORT LAYER ........................................................................................ 14

3.2.1 USB Networking ...................................................................................................................... 15

3.2.2 WLAN Networking ................................................................................................................... 16 3.2.3 Transport Layer ....................................................................................................................... 16

3.3 SESSION & APPLICATION LAYER .................................................................................................. 16

4 AUTHENTICATION AND SECURITY .......................................................................................... 17

4.1 HOST AUTHENTICATION MECHANISMS......................................................................................... 17

4.1.1 USB Networking ...................................................................................................................... 17

4.1.2 WLAN Networking ................................................................................................................... 17 4.2 CONFIDENTIALITY AND INTEGRITY MECHANISMS ........................................................................ 17

4.2.1 USB Networking ...................................................................................................................... 17

4.2.2 WLAN Networking ................................................................................................................... 17 4.3 DEVICE IDENTIFICATION MECHANISM .......................................................................................... 17

5 DISPLAY OUTPUT AND CONTROL INPUT ............................................................................... 19

5.1 CONNECTION SETUP ..................................................................................................................... 20

5.2 TRADITIONAL VNC PROTOCOL PHASES ....................................................................................... 21

5.2.1 Handshaking Phase ................................................................................................................. 21 5.2.2 Initialization Phase .................................................................................................................. 22

5.2.3 Framebuffer Update and Event Phase ..................................................................................... 23 5.3 VNC TERMINAL MODE EXTENSION MESSAGES ........................................................................... 26

5.3.1 Display Configuration Messages ............................................................................................. 28 5.3.2 Event Configuration Messages ................................................................................................ 31 5.3.3 Event Mapping Messages ........................................................................................................ 34 5.3.4 Key Event Listing Message ...................................................................................................... 36 5.3.5 Virtual Keyboard Trigger Messages ........................................................................................ 39 5.3.6 Device Status Messages ........................................................................................................... 41 5.3.7 Content Attestation Messages .................................................................................................. 44 5.3.8 Framebuffer Blocking Notification .......................................................................................... 47

5.4 ADDITIONAL ENCODINGS AND PSEUDO ENCODINGS ..................................................................... 49

5.4.1 Terminal Mode Pseudo Encoding ............................................................................................ 49 5.4.2 Context Information Pseudo Encoding .................................................................................... 49

6 AUDIO OUTPUT AND INPUT......................................................................................................... 51

6.1 RTP PACKET STRUCTURE AND HEADER DEFINITION .................................................................... 52

- 4 -


6.2 RTP AUDIO PAYLOAD DEFINITION ............................................................................................... 53

6.2.1 16 Bit Audio Payload (Mono) .................................................................................................. 53 6.2.2 16 Bit Audio Payload (Stereo) ................................................................................................. 54

6.3 ESTABLISHING THE RTP CONNECTION ......................................................................................... 54

6.4 RECOMMENDATION FOR CLIENT AND SERVER IMPLEMENTATION ................................................ 55

6.5 INTEROPERABILITY WITH BLUETOOTH .......................................................................................... 56

6.5.1 Bluetooth Profiles relevant for Terminal Mode ....................................................................... 56 6.5.2 Interoperability States –Terminal Mode Server Perspective ................................................... 56 6.5.3 Interoperability States –Terminal Mode Client Perspective .................................................... 58

7 SERVICE NEGOTIATION FRAMEWORK ..................... ............................................................. 61

7.1 UPNP USAGE MODELS ................................................................................................................. 61

7.1.1 2-Box Pull Model ..................................................................................................................... 61 7.1.2 2-Box Push Model .................................................................................................................... 61 7.1.3 3-Box Model............................................................................................................................. 62

7.2 UPNP DEVICE ARCHITECTURE ..................................................................................................... 63

7.3 TMSERVERDEVICE:1 DEVICE ....................................................................................................... 63

7.3.1 TmApplicationServer:1 Service ............................................................................................... 63 7.3.2 TmClientProfile:1 Service ....................................................................................................... 64

7.4 TMCLIENTDEVICE:1 DEVICE ........................................................................................................ 64

8 AUDIO LINK SELECTION .............................................................................................................. 66

8.1 AUDIO LINK OPTIONS ................................................................................................................... 66

8.2 AUDIO LINK SELECTION ............................................................................................................... 67

8.2.1 LaunchApplication (AppID, ProfileID) ................................................................................... 67 8.2.2 TerminateApplication (AppID, ProfileID) ............................................................................... 69 8.2.3 GetApplicationStatus (AppID, ProfileID) ................................................................................ 70

9 DEVICE ATTESTATION ................................................................................................................. 71

9.1 DEVICE ATTESTATION LAUNCH .................................................................................................... 71

9.2 DEVICE ATTESTATION TERMINATION ........................................................................................... 72

9.3 DEVICE ATTESTATION PROTOCOL ................................................................................................ 72

10 REFERENCES .................................................................................................................................... 78

APPENDIX A – EVENT MAPPING ......................................................................................................... 80

KNOB SHIFT AND ROTATE EVENTS ............................................................................................................ 80

KEY EVENT MAPPING ................................................................................................................................ 81

APPENDIX B – APPLICATION CONTEXT INFORMATION ....... ..................................................... 85

TRUST LEVEL ............................................................................................................................................. 85

APPLICATION CATEGORIES ........................................................................................................................ 85

CONTENT CATEGORIES .............................................................................................................................. 86

CONTENT RULES ........................................................................................................................................ 86

- 5 -


LIST OF FIGURES

Figure 1: Terminal Mode Concept ............................................................................................................ 11

Figure 2: Terminal Mode Networking and Transport Stack................................................................. 15

Figure 3: Session Layer............................................................................................................................... 16

Figure 4: Terminal Mode VNC Setup ...................................................................................................... 19

Figure 5: VNC Connection Setup ............................................................................................................. 20

Figure 6: VNC Handshaking Phase ......................................................................................................... 21

Figure 7: Initialization Phase ..................................................................................................................... 22

Figure 8: Framebuffer Update Phase ....................................................................................................... 23

Figure 9: Server and Client Display Configuration ............................................................................... 28

Figure 10: VNC Server Options for non-scalable Clients ...................................................................... 30

Figure 11: Server and Client Event Configuration ................................................................................. 31

Figure 12: Example Event Mapping Message Flow ............................................................................... 34

Figure 13: Key Event Listing Messages ................................................................................................... 36

Figure 14: Key Event Listing – Incremental Updates ............................................................................ 37

Figure 15: Key Event Listing – Non-Incremental Updates ................................................................... 38

Figure 16: Example Virtual Keyboard Trigger Message Flow ............................................................. 39

Figure 17: Example Device Status Message Flow .................................................................................. 41

Figure 18: Example Content Attestation Message Flow ........................................................................ 44

Figure 19: Example Framebuffer Blocking Notification Message Flow .............................................. 47

Figure 20: Context Information (Example) .............................................................................................. 49

Figure 21: Audio Setup .............................................................................................................................. 51

Figure 22 Sequence for RTP connection: RTP Server by TM Server .................................................... 55

Figure 23: Bluetooth / Terminal Mode IOP States – Terminal Mode Server Perspective ................ 57

Figure 24: Bluetooth / Terminal Mode IOP States – Terminal Mode Client Perspective ................. 59

Figure 25: General UPnP Device Architecture........................................................................................ 61

Figure 26: 2-Box Pull Model ...................................................................................................................... 61

Figure 27: 2-Box Push Model .................................................................................................................... 62

Figure 28: 3-Box Model .............................................................................................................................. 62

Figure 29: Terminal Mode UPnP Service Architecture .......................................................................... 63

Figure 30: Terminal Mode Client Device Architecture .......................................................................... 64

Figure 30: Message Flow – Launch Audio Link ..................................................................................... 68

Figure 31: Message Flow – Terminate Audio Link ................................................................................ 69

Figure 32: Message Flow – Launch Device Attestation Protocol ......................................................... 71

- 6 -


Figure 33: Message Flow – Terminate Device Attestation Protocol .................................................... 72

Figure 34: Device Attestation certification infrastructure ..................................................................... 72

Figure 35: Device and software attestation protocol overview ............................................................ 73

Figure 36: Coordinate System for Knob Events ...................................................................................... 80

- 7 -


LIST OF TABLES

Table 1: Bandwidth Requirements vs. Display Update Rate ................................................................ 13

Table 2: Requirements for Handshaking Phase Messages .................................................................... 22

Table 3: Requirements for Initialization Phase Messages ..................................................................... 23

Table 4: Requirements for Framebuffer Update and Event Phase Messages ..................................... 24

Table 5: VNC Extension Type Message Structure .................................................................................. 26

Table 6: New Extension Types for Terminal Mode Messages .............................................................. 26

Table 7: Server Display Configuration Message..................................................................................... 29

Table 8: Client Display Configuration Message ..................................................................................... 29

Table 9: Server Event Configuration Message ........................................................................................ 32

Table 10: Client Event Configuration Message ....................................................................................... 33

Table 11: Event Mapping Message ........................................................................................................... 34

Table 12: Event Mapping Request Message ............................................................................................ 34

Table 13: Key Event Listing Message ....................................................................................................... 37

Table 14: Key Event Listing Request Message ........................................................................................ 38

Table 15: Virtual Keyboard Trigger Message ......................................................................................... 40

Table 16: Virtual Keyboard Trigger Request Message .......................................................................... 40

Table 17: Device Status Message .............................................................................................................. 42

Table 18: Device Status Request Message ............................................................................................... 42

Table 19: Terminal Mode Server Device Status Default Values ........................................................... 43

Table 20: Content Attestation Response Message .................................................................................. 45

Table 21: Content attestation signature content ..................................................................................... 45

Table 22: Content Attestation Request Message ..................................................................................... 46

Table 23: Framebuffer Blocking Notification Message .......................................................................... 47

Table 24: Blocked Rectangle from Framebuffer Update ........................................................................ 48

Table 25: Additional VNC Encodings ...................................................................................................... 49

Table 26: Context Information Pseudo Encoding ................................................................................... 50

Table 27: RTP Packet Header Definition ................................................................................................. 52

Table 28: RTP Payload Format – 16 Bit Mono ......................................................................................... 54

Table 29: RTP Payload Format – 16 Bit Stereo ........................................................................................ 54

Table 30: IOP Transition (from Terminal Mode Server perspective) ................................................... 58

Table 31: IOP Transition (from Terminal Mode Client perspective) ................................................... 60

Table 32: UPnP Negotiation for Audio Selection ................................................................................... 67

Table 33: Device Attestation – attestationRequest elements ................................................................. 74

- 8 -


Table 34: Device Attestation – Component List ..................................................................................... 74

Table 35: Device Attestation – attestationResponse Elements .............................................................. 75

Table 36: Device Attestation – Possible Attestation Result Values ...................................................... 76

Table 37: Knob Shift and Rotate Configuration Settings ....................................................................... 80

Table 38: Key Event Mapping ................................................................................................................... 84

Table 39: Trust Level .................................................................................................................................. 85

Table 40: Application Categories .............................................................................................................. 86

Table 41: Content Categories ..................................................................................................................... 86

Table 42: Content Rules to tackle Driver Distraction ............................................................................. 87

- 9 -


TERMS AND ABBREVIATIONS

A2DP Bluetooth Advanced Audio Distribution Profile

ARP Address Resolution Protocol

BT Bluetooth

CDC Communications Device Class; specified from USB Device Working Group

CE Consumer Electronics; CE devices are referred to as mobile devices within this specification

DHCP Dynamic Host Configuration Protocol

ECM Ethernet Control Model; part of the CDC device class

HFP Bluetooth Hands-free Profile

HSP Bluetooth Handset Profile

HMI Human Machine Interface"

HU Head-unit (this term is used interchangeably with the Terminal Mode client)

HS Head-set

IP Internet Protocol

NCM Network Control Model; part of the CDC device class

RFB Remote Framebuffer

RTP Real-time Transport Protocol

TCP Transmission Control Protocol

TM Terminal Mode

UDP User Datagram Protocol

UI User Interface

UPnP Universal Plug and Play

USB Universal Serial Bus

VNC Virtual Network Computing

- 10 -


1 ABOUT

This document specifies an interface for enabling remote user interaction of a mobile device via another device. This specification is written having a car head-unit to interact with the mobile de-vice in mind, but it will similarly apply for other devices, which do provide a colored display, audio input/output and user input mechanisms.

This document is aimed at people going to design and develop compliant solutions. The docu-ment will provide all necessary interface functionality and requirements to implement a fully compliant device, on both the mobile device and the head-unit side.

The specification lists a series of requirements, either explicitly or within the text, which are man-datory elements for a compliant solutions. Recommendations are given, to ensure optimal usage and to provide suitable performance. All recommendations are optional.

The document will focus on the interface functionality, its parameters and protocols only. It does not provide any guidelines for implementing the protocol. If there is a reference towards an im-plementation, this is of informative nature only.

- 11 -


2 INTRODUCTION TO TERMINAL MODE

The Terminal Mode provides a concept for integrating the mobile device (hereinafter referred to

as the “Terminal Mode Server”) and the vehicle head-unit (hereinafter referred to as the

“Terminal Mode Client”). In a Terminal Mode context, the control and interaction of applications and services running on the mobile device will be replicated into the car environment. Diverting display and audio output to the car head-unit come together with receiving key and voice control input from it are the main interaction streams, as shown in the following figure.

Consumer Electronics

Device

Automotive Head Unit

Display Control

Audio / Voice

Internet

Applications & ServicesContent User

InputDisplay Speaker & Micro

Figure 1: Terminal Mode Concept

The result is a concept somewhere between running the applications natively in the mobile phone or in the car unit. From the user experience point of view it can offer "the best of the both worlds" where the large variety of mobile phone applications is complemented and enhanced by the car system providing convenient and safe means for using (i.e. controlling) these applications.

It is easier to add new consumer electronic functionalities into the vehicle environment via a mobile device than integrating them into the car infotainment system. In any case, the usage of those applications will become more convenient if the same device with the same content stored in it can be used in all the different environments from home to car, and providing Internet connectivity at the same time. On the other hand, the large displays of the car units can enhance the user experience from what the mobile device can offer by itself.

In addition the mobile device typically provides the latest technologies, from radio connectivity, to multimedia codecs. At the same time, the openness of the platforms, allows delivery of new applications and services at any time.

There are no standard methods currently defined for Terminal Mode connectivity. However, when creating the required solutions, technologies provided by existing open, non-proprietary standards - like USB, TCP/IP, VNC, UPnP etc. - should be used as the basis. The needed additional elements should then be developed and agreed in cooperation between the related industry sectors.

The car systems comprise of several different methods for user interaction, like individual keys, rotating knobs, touch screen and even voice-activated control. For proper interoperability, the control method towards the mobile device should be the same regardless of the actual input mechanism on the car side. Furthermore, to ensure that Terminal Mode does provide

- 12 -


interoperability independent of any application, even legacy ones, it hooks into low-level abstraction.

- 13 -


3 CONNECTIVITY PROTOCOL STACK

The connectivity between the terminal mode server and client is the basis to provide interopera-bility between both. The Connectivity stack is specified in the following, starting from the low layer and going up the protocol stack.

It is not the objective of this specification to provide a detailed overview of the different protocols. Instead this document should highlight the components and parameter required to ensure proper connectivity. The connectivity solution is built purely on existing wireless and wired standards. Therefore detailed information is available in the respective documents.

3.1 Physical & Link Layer

In principle this specification does not intend to limit the use of any wireless and wired technolo-gy. Nevertheless the connectivity solution must provide reasonable high bandwidth. Minimum bandwidth on link layer cannot be given, as the user experience depends on the networking & transport layer performance, as well as on the parameter of the display (resolution and color for-mat).

The following table gives some indication of the required bandwidth on the display level, i.e. on top of any transport mechanism. These values assume non-incremental, uncompressed updates.

Full Display Update / s

Example: QVGA 320 x 240 x 4

Example: QHD 640 x 360 x 4

Example: WVGA 800 x 480 x 2

Example: WVGA 800 x 480 x 4

2 614 000 Byte/s 1 843 200 Byte/s 1 536 000 Byte/s 3 072 000 Byte/s

5 1 536 000 Byte/s 4 608 000 Byte/s 3 840 000 Byte/s 7 680 000 Byte/s



Table 1: Bandwidth Requirements vs. Display Update Rate

Wired technologies do have advantages with regard to achievable bandwidth and security over wireless technologies. In addition wired USB nowadays provides charging capabilities and is in-deed the preferred charging interface in the mobile industry.

3.1.1 Universal Serial Bus (USB)

USB provides a high-bandwidth connection while allowing charging of the mobile device at the same time.

Requirement: The USB host must at least support USB 2.0 high-speed.

To simplify the user intervention on the terminal mode server, it should set the right USB profile automatically1, once connected to the terminal mode client. To facilitate the automatic personality selection, the USB host should send a specific identification message to the device, prior configur-ing the device, according the following format.

bmRequestType = 0x40 bRequest = 0xF0

1 A USB personality may include multiple USB device classes, which can be then used from the USB host simultaneously.

- 14 -


wValue[1] = Terminal Mode major version (e.g. 1) wValue[2] = Terminal Mode minor version (e.g. 0) wIndex = USB Host vendorID wLength = 0 Data = None

The message should be sent before set configuration, since the phone may have wrong personali-ty loaded before that2. It must also be understood, that as a result of this, if wrong personality is loaded, the phone will disconnect and reconnect again with a new personality loaded. This will restart the enumeration3.

Requirement: The USB device must recognize Terminal Mode request message and must select the respective USB personality.

Requirement: If the Terminal Mode client does not send the described identification mes-sage, the mobile device must present the user with a list of available USB per-sonalities.

The Terminal Mode specification does not attempt to specify, which other USB profiles should be available under the Terminal Mode personality, and which USB personalities are available and supported from the device. But to support multiple USB profiles under one personality, USB Host needs to support composite device including IAD (Interface Association Descriptor). IAD is re-quired to support USB function which requires multiple interfaces, and without IAD, it is not possible to associate multiple interfaces with single USB functionality.

Requirement: USB host (TM client) must support composite device including IAD.

Recommendation: It is recommended for USB device (TM server) to provide MTP and ACM as part of the Terminal Mode personality.

3.1.2 Wireless Local Area Networks (WLAN)

Support for Wireless LAN is optional.

3.2 Networking and Transport Layer

Networking mechanisms are used to abstract the different physical transport mechanisms. The Internet Protocol is a well-established and known networking solution. IP protocol packets are encapsulated into Ethernet packages.

Requirement: The networking layer must support IPv4. Support for IPv6 is optional.

This specification does anticipate only USB and WLAN networking. Other wired or wireless links are supported optionally, as long as they allow carrying IP packets, as shown in Figure 2.

2 According to USB 2.0 specification, a USB device, which does not support a message, must re-turn a STALL PID (Request Error). As the endpoint is control endpoint, there is no further action required. USB host can consider device returning STALL as non-terminal mode device and can proceed with non-terminal mode behavior.

3 A user will be able to manually switch between USB device personalities at any time.

- 15 -


USB 2.0WLAN <Other Link Layer>

UDP TCP

DHCP

IPv4

Figure 2: Terminal Mode Networking and Transport Stack

3.2.1 USB Networking

Support for USB Networking is mandatory. Three competing Communication Device Class sub-class drivers are available. In all cases, the USB connection basically looks like an Ethernet 802.3 networking card.

• RNDIS: Remote Network Device Interface Specification is a Microsoft proprietary specifi-

cation. RNDIS is partly Operating System dependent.

• CDC/ECM [3]: Communications Device Class/Ethernet Networking Control Model al-

lows one Ethernet package inside a single USB transfer. ECM has been developed for USB full-speed.

• CDC/NCM [4]: Communications Device Class/Network Control Model allows multiple Ethernet packages inside a single USB transfer. NCM can therefore offer a much higher performance. NCM has been particular developed for high-bandwidth.

Requirement: The USB networking must follow CDC/NCM device class, revision 1.0 speci-fication.4

Recommendation: The host and client should support a Maximum Transmission Unit (MTU) size bigger than 1,500 Bytes. It is recommended to support MTU sizes up to 9000 Byte.

Requirement: The USB host must follow the maximum Ethernet frame size supported from the USB device as discovered from the value wMaxSegmentSize in the Ether-net Networking Functional Descriptor (For details refer to [3]) and supported from the host (Least common denominator).

The Dynamic Host Configuration Protocol (DHCP) is used by the terminal mode client (DHCP client) to obtain configuration information for operation in an IP network from the mobile device (DHCP server). This protocol reduces system administration workload, allowing devices to be added to the network with little or no manual intervention. DHCP is using UDP for the negotia-tion.

• Packets sent from the client have source port 68 and destination port 67.

• Packets sent from the server have source port 67 and destination port 68.

Requirement: A DHCP Server must be available on the mobile device, connected to the USB interface. The DCHP client must use the standard DHCP port.

4 According to USB CDC/NCM specification, the device and host MUST support 16-bit NTB structures (NTB-16) and MAY also support 32-bit NTB structures (NTB-32).

- 16 -


Requirement: To minimize IP address conflicts on the terminal mode client, the DHCP Server must provide an IP address within the 192.168.x.y with x in the range of 2 to 127 and y in the range of 0 to 254. The netmask must be 255.255.255.0.

Requirement: The DHCP Server may provide a default gateway address for the DHCP client. Provisioning of the default gateway address must not be interpreted as if the Terminal Mode server provides Internet connectivity. The Terminal Mode specification does not intend to specify the setup of IP routing functio-nality on the Terminal Mode server.

Recommendation: Use ARP to resolve potential IP conflicts on client side.

3.2.2 WLAN Networking

Support for Wireless Local Area Network (WLAN) is optional. IP packets are carried over WLAN connections, using the Ethernet LLC/SNAP framing. On other network types than Ethernet, LLC and SNAP headers are required to multiplex different protocols on the link layer.

Requirement: In case WLAN is supported, the mobile device must support ad-hoc and in-frastructure networks. In latter case, the access point must be implemented in the terminal mode client.

3.2.3 Transport Layer

The IP protocol enables two transport mechanisms,

• User Datagram Protocol (UDP) to provide connectionless communication, used for ser-vice advertising, multi-casting, and most real-time streaming protocols

• Transmission Control Protocol (TCP) to provide connection-oriented communication

Requirement: The transport layer must support UDP and TCP transport protocols on top of IP.

3.3 Session & Application Layer

The Terminal Mode application layer consists of three basic session layer components using ei-ther UDP or TCP sockets to interact as shown in the figure below.

• Audio, responsible for providing and exchanging audio content, using UDP sockets.

• VNC, responsible for exchanging display and control information, using TCP sockets.

• UPnP, responsible for service negotiation and remote application control, using UDP broadcasting and TCP sockets.

SSDP HTTP 1.1

SOAP

RTP

Audio

RTCP

UPnP

UDP TCP

VNC

VNC

Figure 3: Session Layer

The application layer is specified in the following chapters.

- 17 -


4 AUTHENTICATION AND SECURITY

Giving the terminal mode client control over the server requires addressing security and authen-tication concerns. Potential threats in the Terminal Mode setup include

1. Attacker can read and/or modify communication between terminal mode client and server.

2. Terminal mode server (e.g. a mobile device) does not connect to the intended terminal mode client (e.g. a vehicle head-unit) or vice versa.

3. Terminal mode client connects to a non-compliant server.

Threats 1 is addressed via confidentiality and integrity mechanisms, where as threats 2 is ad-

dressed via host authentication. Threat 3 is addressed with device attestation. The term “security

mechanisms” is used to refer to confidentiality, integrity and authentication mechanisms. Those mechanisms are described in the following.

4.1 Host Authentication Mechanisms

Host authentication in this context refers to the terminal mode client authenticating itself towards the terminal mode server, to mitigate the threat that the server connects to unintended client (or vice versa).


Additional authentication and integrity mechanisms in wired USB networking are not required.


Authentication and integrity mechanisms in WLAN networks are available on Link Layer.

Requirement: Link-Layer authentication mechanisms, like WPA, must be used, if mandated from the terminal mode server or from the terminal mode client.

4.2 Confidentiality and Integrity Mechanisms

Confidentiality and integrity mechanisms mitigate the threat that an external attacker can read, change or inject any data.


Additional confidentiality and integrity mechanisms in wired USB networking are not required.


Confidentiality and integrity mechanisms in WLAN networks are available on Link Layer.

Requirement: Link-Layer confidentiality mechanisms, like WPA, must be used if mandated from the terminal mode server or from the terminal mode client.

4.3 Device Identification Mechanism

Device identification in this context refers to the mobile device identifying itself to the terminal mode client.

- 18 -


Device identification (proving the identity of the device and or the device manufacturer are avail-able from different sources during the connection setup.

1. USB standard device descriptor entries idVendor and idProduct

2. UPnP XML device description: <manufacturer> and <modelName>

Requirement: The Device must set the USB vendor and product ID as well as UPnP device manufacturer and model name accordingly.

Requirement: The device must prevent 3rd party software to change USB vendor ID and UPnP device manufacturer according to the state-of-the-art of the particular device platform.

- 19 -


5 DISPLAY OUTPUT AND CONTROL INPUT

The contents of the Terminal Mode server device’s screen are transferred to the Terminal Mode client device. The control inputs are transferred from the Terminal Mode client to the Terminal Mode server. Screen copy methods can be used to copy the content of the Terminal Mode server's display (frame buffer copy) to the Terminal Mode client’s display. The copy operation may in-clude rotation or color conversion. The frame buffer is used as an abstraction layer, which avoid any changes to the applications and services running on the mobile device can be avoided. For this purpose the Virtual Networking Computing (VNC) protocol is used.

The Virtual Networking Computing (VNC) uses the Remote Framebuffer Protocol (RFB) as a simple protocol for remote access to any sort of framebuffer based user interfaces. The remote endpoint is called the VNC Client, whereas the endpoint driving the framebuffer is called the VNC Server. In the Terminal Mode context, the VNC Client resides in the car head-unit (Terminal Mode client) and the VNC Server is in the mobile device (Terminal Mode server). The VNC client will show the remote display either on the entire local display or on a subset of it, as shown in Figure 4.


Device

Automotive Head Unit

Display ControlRFB Protocol

VNC Server

VNC Client

CE Display

CE Display

HU Display

User Input

Figure 4: Terminal Mode VNC Setup

The command and control input is handled as part of the VNC protocol by key and pointer events. A key or pointer event on terminal mode client will be signaled to the terminal mode server via a specific key symbol value, which uniquely identifies the event. The mobile device and/or its application will not necessarily support all possible keys defined. Some applications may even have a dynamic behavior on the selection of key inputs they expect.

The RFB protocol is originating from the desktop computing world and has been designed as a thin client protocol, i.e. it assumes a client with only a few requirements, and a server having access to more processing capabilities. The protocol allows the client to be as simple as possible. In the Terminal Mode context this assumption needs to be reconsidered, as mobile devices are experiencing performance limitations as well.

Requirement: The terminal mode client must implement the VNC client functionality.

Requirement: The terminal mode server must implement the VNC server functionality

- 20 -


5.1 Connection Setup

The connection setup is facilitated via UPnP. Once the VNC server is up and running, its service is announced using UPnP protocol mechanisms. The VNC Server must listen for the VNC client to connect. Connection is done via establishment of a TCP/IP socket. The Server IP address and VNC port number are provided as part of the UPnP protocol (see respective chapter).

The established connection is facilitating the execution of the VNC protocol.

Once the TCP/IP connection is disconnected, the VNC server will wait for the VNC client to re-connect. The VNC protocol does not provide specific messaging to shut down the connection. Be-fore reconnection, the VNC client has to verify, whether the VNC server is still available (using UPnP mechanisms).

The connection setup is high-lighted in the following picture. The red dotted lines indicate trigger points between the Server and Client operation.

Wait for VNC Client to (Re-)Connect

VNC Operation

Still connected

?

Start VNC Server

No Yes

Wait for VNC Server to become available

Connect to VNC Server

Start VNC Client

VNC Operation

Close Connection

VNC Server available

Connect

VNC Protocol

Disconnect

Figure 5: VNC Connection Setup

The Server IP address and VNC port number are provided as part of the UPnP protocol (see re-spective chapter).

Requirement: The VNC Server must listen for the VNC client to connect. If the VNC client disconnects, the VNC Server must listen for the client to reconnect.

Requirement: The VNC Server must listen on a TCP/IP socket.

- 21 -


5.2 Traditional VNC Protocol Phases

After the connection between the VNC server and client has been established, the VNC protocol processing will start according to the VNC specification. The VNC protocol basically consists of three main steps

(1) Exchange of handshaking messages. In this step, the VNC connection between both end-points is set up. After the handshaking phase, the VNC connection parameters are nego-tiated and the connection is established.

(2) Exchange of initialization messages. After this phase, both ends have agreed on all needed parameter for the following operational phase.

(3) Client to server and server to client messages are used to reflect changes of the framebuf-fer content on the local endpoint and user interaction on the remote endpoint.

These three VNC protocol phases are specified in more detail in the following.

5.2.1 Handshaking Phase

The handshaking phase defines a couple of messages, which are being exchanges between the VNC Client and the VNC Server, as shown in the following figure. In general the VNC Server presents its capabilities and the VNC Client selects the best option with regard to its own capabili-ties.

Server Protocol Version

VNC Server VNC Client

Client Protocol Version

Security Type Support

Security Type Selection

Security Failure ReasonSecurity Result

Security Failure Reason (3.8 only)

Security_type = 0

Figure 6: VNC Handshaking Phase

The following parameters have to be supported from the VNC Client and the Server.

Message Origin Parameter Mandatory Values

Protocol Version Server Max. protocol version At least 3.7

Protocol Version Client Max. protocol version At least 3.7

Security Type Support Server # of security types (as specified in RFB)

Security types 1 (None)

Security Type Selection Client Security type (as specified in RFB)

Security Failure Reason Client Reason length

(as specified in RFB) Reason string

- 22 -



Security Result Server Security status (as specified in RFB)

Security Failure Reason (RFB version 3.8 only) Server

Reason length (as specified in RFB)

Reason string

Table 2: Requirements for Handshaking Phase Messages

Authentication and security is handled outside the VNC protocol on link-layer and transport-layer. The VNC Client cannot expect the VNC Server to offer additional security or authentication features.

5.2.2 Initialization Phase

The initialization phase defines a couple of messages, which are being exchanged between the VNC Client and the VNC Server, as shown in the following figure. In general the VNC Server presents its capabilities and the VNC Client selects the best option with regard to its own capabili-ties.


Client Init

Server Init

Set Encodings

Set Pixel Format

Figure 7: Initialization Phase



Client Init Client Shared-flag (as specified in RFB)

Server Init (using native framebuf-fer configuration)

Server

Framebuffer-width

(as specified in RFB)

Framebuffer-height

Name-length

Name-string

Bits-per-pixel

Depth

Big-endian-flag

True-colour-flag

Red-/Green-/Blue max

Red-/Green-/Blue shift

Set Encodings Client

Number of encodings (as specified in RFB)

Encoding-type -223 (Desktop Size Pseudo Encoding – optional for client)

- 23 -


Message Origin Parameter Mandatory Values -523 (Terminal Mode Pseu-do Encoding)

Set Pixel Format Client

Bits-per-pixel 32, 16

Depth 24, 16

Big-endian-flag (as specified in RFB)

True-colour-flag 1 (true color)

Red-/Green-/Blue max RGB888, RGB565

Red-/Green-/Blue shift

Table 3: Requirements for Initialization Phase Messages

The Terminal Mode limits the RFB protocol, as shown in the above table with regard to supported color formats, to allow for efficient implementations. Some more specific recommendations and requirements are given below.

Requirement: The VNC Client must not select any other pixel format, unless the server has indicated support, using the ServerDisplayConfiguration VNC extension message.

Requirement: The VNC Client must send a Set Pixel Format message, prior to any Frame-buffer Update Request message.

Recommendation: It is recommended that the VNC Client selects the native color format of the VNC server. On device color conversion will lead to a significant reduction of achievable frame rate.

5.2.3 Framebuffer Update and Event Phase

The update and event phase defines a couple of messages, which are being exchanged between the VNC Client and the VNC Server. The VNC Server only responds to framebuffer update re-quests, as shown in Figure 8. No response message is sent to any of the other messages.


Framebuffer Update Request

Framebuffer Update

Figure 8: Framebuffer Update Phase



Framebuffer Update Request Client

Incremental

(as specified in RFB)

x-position

y-position

Width

Height

Framebuffer Update Server Number-of-rectangles

(as specified in RFB) x-position

- 24 -



y-position

Width

Height

encoding-type 0 (Raw)

Set Colour Map Entries Server

first color

(Message not implemented at Server)

number-of-colours

Red

Green

Blue

Key Event Client Down-flag

(as specified in RFB) Key

Pointer Event Client

Button-mask

(as specified in RFB) x-position

y-position

Client Cut Text Client Length

(as specified in RFB) Text

Bell Server No parameter (as specified in RFB)

Server Cut Text Server Length

(as specified in RFB) Text

Table 4: Requirements for Framebuffer Update and Event Phase Messages

The VNC Client can request two types of framebuffer updates, using the incremental flag in the FramebufferUpdateRequest message.

• A ‘0’ indicates the VNC server to provide a non-incremental update, i.e. the VNC server must provide the requested area or a superset of it, independent of whether it has changed or not.

• A ‘1’ indicates the VNC server to provide an incremental update, i.e. the VNC server must provide only the area(s) containing all framebuffer changes.

Requirement: The VNC Client must support Zero framebuffer update messages, i.e. Fra-mebuffer Update messages where the width and height equals zero.5

Requirement: The VNC Client must support framebuffer update messages of a bigger fra-mebuffer area, than the original requested one.6

Requirement: The VNC Client must support that the VNC Server may ignore framebuffer update request messages, if multiple are sent at a time. Multiple non-incremental framebuffer update request messages may be combined into one

5 Zero Framebuffer updates are not forbidden from the VNC specification specifically. Though some existing VNC clients, display warnings.

6 This occurs, if the VNC Client requests a non-incremental framebuffer update for a specific area and other areas have changed in the mean time as well.

- 25 -


framebuffer update response. It is recommended that the VNC Client sends only one Framebuffer Update Request message at a time.

Requirement: The VNC Client must support that the VNC Server will not respond imme-diately to an incremental framebuffer update request. The Server may wait for the next response until the framebuffer has changed on the Server side.

Requirement: The VNC Server must respond immediately to a non-incremental framebuf-fer update request.

Recommendation: It is recommended that the VNC Client has a copy of the server side frame-buffer locally available. Therefore it is recommended that the VNC Client re-quests incremental framebuffer updates.

- 26 -


5.3 VNC Terminal Mode Extension Messages

The existing RFB protocol specification is not sufficient to cover the mobile device – remote car display configuration space. Therefore extensions to the current protocol are specified in this chapter. Extensions are made in compliance with the RFB protocol, introducing a new Terminal Mode message type (128).

Under the Terminal Mode message type, a couple of additional messages are introduced to the VNC protocol. These can be distinguished via unique extension-types. All extension messages must use the Terminal Mode message type and must follow the following fundamental design principle.

# bytes Type Value Description

1 U8 128 Terminal Mode Message-type

1 U8 Extension-type

2 U16 N Payload length

N U8 array Message specific payload data

Table 5: VNC Extension Type Message Structure

Requirement: The VNC Server and Client must handle Terminal Mode extension messages with unknown extension types, by reading the whole message (body and payload) and ignoring it.

The Terminal Mode Message type defines the following set of new VNC messages given in Table 6. All extension messages are mandatory server-side, but the execution of some messages can be disabled within the Server or Client Event Configuration message.

Exten-sion-Type

Message Message Name Origin Support Disable Execu-tion

1 Display Configura-tion

ServerDisplayConfiguration Server Mandatory No

2 ClientDisplayConfiguration Client Mandatory No

3 Event Con-figuration

ServerEventConfiguration Server Mandatory No

4 ClientEventConfiguration Client Mandatory No

5 Event Map-ping

EventMapping Server Mandatory No

6 EventMappingRequest Client Optional No

7 Key Event Listing

KeyEventListing Server Mandatory Yes

8 KeyEventListingRequest Client Optional Yes

9 Virtual Key-board Trig-ger

VirtualKeyboardTrigger Server Mandatory Yes

10 VirtualKeyboardTriggerRequest Client Optional Yes

11 Device Sta-tus

DeviceStatus Server Mandatory No

12 DeviceStatusRequest Client Optional No

13 Content At-testation

AttestationResponse Server Mandatory No

14 AttestationRequest Client Optional No

16 Framebuffer Blocking FramebufferBlockingNotification Client Optional No

Table 6: New Extension Types for Terminal Mode Messages

- 27 -


Requirement: The Client must disable the key event listing and the virtual keyboard trigger support in the Client Event Configuration message, if it has not implemented the respective request messages.

- 28 -


5.3.1 Display Configuration Messages

The Server and Client Configuration message pair exchanges additional display information be-tween the client and the server. Based on the received information the client may change the pixel format, sending a Set Pixel Format message. The server will use this information to optionally provide higher resolution virtual framebuffer copies. The message flow is shown in Figure 9.


Server Display Configuration

Client Display Configuration

Set Encodings -523 (Terminal Mode)

Set Pixel Format

Figure 9: Server and Client Display Configuration

Requirement: The Server Display Configuration Message must be sent immediately in re-sponse to the Set Encoding message, indicating Terminal Mode (-523) sup-port, prior any other message.

Requirement: The Client Display Configuration Message must be sent immediately in re-sponse to the Server Display Configuration Message, prior any other mes-sage.

The Server Display Configuration message is given in Table 7. It will be responded from the Client with a Client Display Configuration message.


1 U8 128 Message-type

1 U8 1 Extension-type

2 U16 12 Payload length

1 U8 1 Terminal Mode Server Major Version

1 U8 0 Terminal Mode Server Minor Version

2 U16

Bit Framebuffer configuration (1 = yes, 0 = no)

0

Server-side framebuffer orientation switch available 1. The VNC server must start in default orientation, which

is given in the Server Init message (values width and height).

1 Server-side framebuffer rotation available

• The VNC server must start with no rotation.

2

Server-side framebuffer scaling available • The VNC server must be able to scale the framebuffer to

the client resolution (as provided from the VNC client in the Client Display Configuration message)

2 U16 Relative pixel width (set to zero, if relative width not known)

2 U16 Relative pixel height (set to zero, if relative height not known)

4 U32 Bit RGB Color conversion support (1 = yes, 0 = no)

0 32 bit ARGB 888 (mandatory support for VNC server)

- 29 -



7 All other 32 bit formats

8 16 bit RGB 565 (mandatory support for VNC server)

9 16 bit RGB 555

15 All other 16 bit formats

25 16 bit single color (grayscale)

• Client must use red_shift and red_mask to set gray range

26 8 bit single color (grayscale)

• Client must use red_shift and red_mask to set gray range

Table 7: Server Display Configuration Message

Recommendation: The relative pixel width and height should be used to compensate for differ-ent pixel aspect rotation on client and server side.

Requirement: The Server Display Configuration message must be sent only once.

The Client Display Configuration message is shown in the following table.





1 U8 1 Terminal Mode Client Major Version

1 U8 0 Terminal Mode Client Minor Version

2

U16

Bit Framebuffer Configuration (1 = yes, 0 = no)

0 Server-side framebuffer orientation switch used

• If enabled, the VNC client must use the Device Status Request message (section 5.3.6)

1 Server-side framebuffer rotation used

• If enabled, the VNC client must use the Device Status Request message (section 5.3.6)

2

Client-side framebuffer scaling available • If enabled, the VNC client must be able to scale the

server framebuffer (as provided in the Server Display Configuration message) to the client resolution7

2 U16 Client display width [pixel] (unknown value must be 0)

2 U16 Client display height [pixel] (unknown value must be 0)

2 U16 Client display width [mm] (unknown value must be 0)

2 U16 Client display height [mm] (unknown value must be 0)

2 U16 Distance driver – client display [mm] (unknown value must be 0)

Table 8: Client Display Configuration Message

7 According to the RFB specification, the client must support any server framebuffer resolution. A client must not expect the server to support scaling.

- 30 -


Requirement: The Client Display Configuration message must be sent only once.

Requirement: The VNC client must support Desktop Size Pseudo Encoding, if it enables bit 0 or 1 in the framebuffer configuration entry.

Requirement: If the VNC client does not support scaling (disabled bit 2 in the framebuffer configuration entry), the VNC server must provide the framebuffer content in the requested client display resolution in one of the following options shown in Figure 10.

Clipping

Scaling

Framing

Figure 10: VNC Server Options for non-scalable Clients

� Scaling, if the VNC server supports scaling (maintaining the server as-pect ratio – with (0, 0) offset; other client area remains unspecified), or

� Framing, if the VNC server does not support scaling and the server reso-lution is smaller than the client one (with (0, 0) offset; other client area remains unspecified), or

� Clipping, if the VNC server does not support scaling and the server reso-lution is bigger than the client one (framebuffer content aligned to the upper left corner).

Requirement: No pixel data must be transmitted for unspecified client area (shown in red in Figure 10)

Requirement: The VNC client must not provide a Terminal Mode minor and major version, which is higher than the VNC server supported version.

Requirement: VNC client and server must be backward compatible with regard to different Terminal Mode versions.

- 31 -


5.3.2 Event Configuration Messages

The Server and Client Event Configuration message pair provides additional information about the event handling, i.e. which key and pointer events are natively supported on the server and client. This information helps the server to map specific incoming client events to server events and helps the client to directly use specific server events. The message flow is shown in Figure 11.


Server Event configuration

Client Event configuration

Figure 11: Server and Client Event Configuration

Requirement: The Server Event Configuration Message must be sent immediately in re-sponse to the Set Encoding message, indicating Terminal Mode (-523) sup-port. The message may be delayed only until reception of the Client Display Configuration message.

Requirement: The Client Event Configuration Message must be sent immediately in re-sponse to the Server Event Configuration Message, prior any other message.

The Server Event Configuration message is given Table 9.


1 U8 128 Message-type 1 U8 3 Extension-type 2 U16 28 Payload length 2 U16 Keyboard layout – Language code (according ISO 639-1)

2 U16 Keyboard layout – Country code (according ISO 3166-1 al-pha-2)

2 U16 UI Language – Language code (according ISO 639-1)

2 U16 UI Language – Country code (according ISO 3166-1 alpha-2)

4 U32 Bit l

Knob shift & rotate configuration (Bitmask according Table 37)

• ‘1’: Server supports knob key events8 • ‘0’: Server does not support knob key events

4 U32 Bit m

Device keys (Bitmask according Table 38) • Bitmask defined in Table 38 • ‘1’: Server supports device key events • ‘0’: Server does not support the key events

4 U32 Bit n Multimedia keys (Bitmask according Table 38)

• Bitmask defined in Table 38 • ‘1’: Server supports multimedia key events

8 The Server can claim e.g. the device cursor keys plus a Device_Ok key to represent a simple knob, i.e. supporting shift up, down, left right and push knob events.

- 32 -



• ‘0’: Server does not support the multimedia key events

4 U32

Bit Key related (1 = support, 0 = no support)

0 ITU keypad (T9) events (‘0’, … ,’9’, ‘#’, ‘*”)

1 Virtual keyboard trigger support

2 Key event listing support

8 – 15 # additional soft and hard keys, the server requires

• Key events defined in Table 38 • Key events start with TM_Key 0, no subsequent gaps

4 U32

Bit Pointer related (1 = support, 0 = no support)

0 Single-touch events

1 Multi-touch events

8 – 15 Pointer event button mask (according RFB spec)

Table 9: Server Event Configuration Message

Requirement: Server event configuration must be sent only once.

The payload structure of the Client Event Configuration message is fully symmetrical with the Server Event Configuration message, as shown in Table 10.


1 U8 128 Message-type 1 U8 4 Extension-type 2 U16 28 Payload length

2 U16 Keyboard layout – Language code (according ISO 639-1)

2 U16 Keyboard layout – Country code (according ISO 3166-1 al-pha-2)

2 U16 UI Language – Language code (according ISO 639-1)

2 U16 UI Language – Country code (according ISO 3166-1 alpha-2)

4 U32 Bit l

Knob shift & rotate configuration (Bitmask according Table 37)

• ‘1’: Client supports knob key events • ‘0’: Client does not support knob key events

4 U32 Bit m Device keys (Bitmask according Table 38)

• ‘1’: Client supports device key events • ‘0’: Client does not support device key events

4 U32 Bit n Multimedia keys (Bitmask according Table 38)

• ‘1’: Client supports multimedia key events • ‘0’: Client does not support multimedia key events

4 U32

Bit Key related (1 = support, 0 = no support)

0 ITU keypad (T9) events (‘0’, … ,’9’, ‘#’, ‘*”)

1 Virtual keyboard trigger support (ignored at server)

2 Key event listing support (ignored at server)

8 – 15 # additional soft and hard keys, the client supports

• Key events defined in Table 38 • Key events start with TM_Key 0, no subsequent gaps

4 U32 Bit Pointer related (1 = support, 0 = no support)

- 33 -



0 Single-touch events

1 Multi-touch events

8 – 15 Pointer event button mask (according RFB spec)

Table 10: Client Event Configuration Message

Requirement: Client Event Configuration must be sent only once.

- 34 -


5.3.3 Event Mapping Messages

The Event Mapping and Event Mapping Request message pair provides the client with informa-tion about the server mapping of high key symbol values. The Client can send an Event Mapping Request message at any time, either requesting or setting a specific mapping within the server. The server must always send an Event Mapping message in response of an Event Mapping Re-quest message. If the server changes any event mapping locally, the server must inform the client via an Event Mapping message, if the client has requested the mapping at least once.

The message flow follows the following steps, as shown in Figure 12.


Event Mapping

Event Mapping Request

Event Mapping

Server changes Event Mapping

Figure 12: Example Event Mapping Message Flow

Requirement: The Server must respond to any Event Mapping Request message imme-diately.

The Event Mapping message is given in Table 11.





4 U32 Client Key Symbol Value

4 U32 Server Key Symbol Value (0 = client key value not support from server)

Table 11: Event Mapping Message

The Default Mapping Request message is given in Table 12.





4 U32 Client Key Symbol Value

4 U32 Server Key Symbol Value (0 = request value from Server)

Table 12: Event Mapping Request Message

Requirement: If the server locally changes any event mapping, it must send an Event Map-ping message with appropriate Client and Server Key Symbol Values. The

- 35 -


server must send the Event Mapping message only, if the Client has re-quested the Client Key Symbol Value previously using an Event Mapping Request message.

Requirement: If the server does not support a new mapping request according to the Event Mapping Request message, it must send an Event Mapping message, con-taining the existing mapping. The server key symbol value is set to zero if the server does not allow any mapping for the client symbol value.

- 36 -


5.3.4 Key Event Listing Message

The Key Event Listing and Key Event Listing Request message pair provides the client with in-formation about the next valid characters, while entering text information. The VNC Server sup-port is announced in the Server Event Configuration message. The Client may start the key event listing at any time. In case a text entry is required, the server will provide the initial list of valid keys, which is getting updated after each key event, either as incremental or full update. An ex-ample message flow is shown in Figure 13.


Key Event

Key Event ListingInitial List – Counter n

Key Event Listing RequestStart Information Flow

Key Event ListingList Update – Counter n+1

Key Event Listing RequestStop Information Flow

Figure 13: Key Event Listing Messages

Requirement: The VNC Server must send Key Event Listing messages only, if the VNC Client has enabled or requested them.

Requirement: The VNC Client must not assume, that the VNC server will send Key Event Listing messages even if it has indicated support (the current application may not be able to support this feature).

The Key Event Listing message is given in Table 13.




2 U16 4 + 4*n Payload length

1 U8

Bit Configuration

0 Update flag (0 = non-incremental, 1 = incremental)

1 Listing flag (0 = black list, 1 = white list)

2

Default event list flag (0 = normal list, 1 = default list ) • To reference to the default list, set this flag and set #

key events in list to zero. • To set the default event list, set this flag and add key

events to the KeySymValue list

3 Other key event listing follows (0 = no, 1 = yes)

• Next key event listing must follow immediately • Black lists and white lists can be mixed

1 U8 n # key events in list

2 U16 Key event counter

- 37 -



4 * n U32 array KeySymValue list used to define the next valid character

Table 13: Key Event Listing Message

The flow using incremental updates is shown in Figure 14. Here, a default layout must be defined once per VNC session. This list can be used as a reference point prior the incremental update process. During the incremental update, the white list contains all key symbols to be added to the key event list, where as black lists contains all key symbols to be removed from the key event list.

Default layout = 1# key events in list = 0

Update flag = 1Listing flag = 1

Default layout flag = 0

White listing?

Default layout = 1# key events in list != 0

Update flag = 1Listing flag = 0

Default layout flag = 0

Wait for Key Event

Last Char?

Wait for Text Event

Yes

No

NoYes

Other List?

Other List?

Yes Yes NoNo

Figure 14: Key Event Listing – Incremental Updates

In case of non-incremental (i.e. full) updates, the key event listing looks like in Figure 15. Incre-mental and non-incremental updates can be mixed at any time.

- 38 -


Default layout = 0Update flag = 0

# key events in list != 0

Wait for Key Event

Last Char?

Wait for Text Event

No Yes

Other List?

Yes

No

Figure 15: Key Event Listing – Non-Incremental Updates

Requirement: The valid key symbol list must not differentiate between upper and lower case characters

The Key Event Listing Request message is shown in Table 14.





4 U32

Bit Configuration (0 = Disable, 1 = Enable)

0 Server key event listing

1 Incremental updates

2 Reset key event counter

Table 14: Key Event Listing Request Message

The key event counter can be used to synchronize the server key event listing message to key events. The counter value must represent all received key events (key press events only). The counter must be set to zero on the reception of the Client Event Configuration message and if the specific bit is set in the Key Event Listing Request message. The counter must roll over to zero, once the maximum number is reached.

The VNC Client must not assume that the VNC Server will send a key event list update before the client sends the next key press event. This specifically can happen if the key events are entered faster than the Server can provide key event list updates. In such case, the client should use the default key event list instead.

- 39 -


5.3.5 Virtual Keyboard Trigger Messages

The Virtual Keyboard Trigger and Virtual Keyboard Trigger Request message pair informs the Client that a text input is expected. The Client can use this information, e.g. to bring up a virtual keyboard. In addition the message provides, if available, information about the current cursor po-sition and the text entry box. The client can start and stop this service at any time.

The message flow is given in Figure 16.


Virtual Keyboard Trigger Show keyboard

Virtual Keyboard Trigger RequestEnable Service

Virtual Keyboard Trigger RequestDisable Service

Virtual Keyboard Trigger Remove keyboard

Figure 16: Example Virtual Keyboard Trigger Message Flow

Requirement: The server must send Virtual Keyboard Trigger messages only, if it has set

the “Virtual keyboard trigger support” flag in the Server Event Configura-

tion message and the client has set the “Virtual keyboard trigger support” flag in the Client Event Configuration message.

Requirement: The server must automatically enable the virtual keyboard trigger at startup,

if the client has set the “Virtual keyboard trigger support” flag in the Client Event Configuration message.

The Virtual Keyboard Trigger message is given in Table 15.





4 U32

Bit Configuration (0 = no, 1 = yes)

0 Valid cursor position

1 Valid text entry dimension

2 Key Event listing follows

3 Trigger source (0 = internal, 1 = external)

4 Virtual keyboard control (0 = show keyboard, 1 = remove keyboard)

2 U16 Cursor – X Position

- 40 -



2 U16 Cursor – Y Position

2 U16 Text input area – X-Position

2 U16 Text input area – Y-Position

2 U16 Text input area – Width

2 U16 Text input area – Height

Table 15: Virtual Keyboard Trigger Message

The Virtual Keyboard Trigger Request message is given in Table 16.





4 U32

Bit Configuration (0 = no, 1 = yes)

0 Enable internal trigger

1 Enable external trigger

Table 16: Virtual Keyboard Trigger Request Message

The trigger can originate from two different sources

• Internal Trigger: The VNC server automatically checks for the availability of a cursor, in-dependent of any application. This triggering process may result in false positive and false negative events.

• External Trigger: The VNC server will trigger based on application level trigger. This can

be used only, if there is support from the application available.

Requirement: If application-level support is available, the server must disable internal trig-ger mechanisms, as long as the respective application is driving the UI and while external triggering is enabled.

- 41 -


5.3.6 Device Status Messages

The Device Status and Device Status Request message pair provides the client with status infor-mation of specific device settings at the server and the ability to set them. The server will inform the client about any status change, if it supports this feature. The message flow is shown in Figure 17.


Device Status

Device Status Request

Device Status

Server changes Device Information

Figure 17: Example Device Status Message Flow

Requirement: The VNC Server must immediately respond to a Device Status Request

The Device Status message is given in Table 17.





4 U32

Bit Status of Device Features (00 = unknown, 01 = not used, 10 = disabled, 11 = enabled)

0:1 Key-lock (Do not allow key and pointer event entry from serv-er)

2:3

Device-lock (In device-lock state, the CE device does not re-spond to local and remote key and pointer events) Note: User may need to enter PIN code to un-lock the device.

4:5 Screen saver (Do not show content on server display)

6:7 Night mode

8:9 Voice control input on Terminal Mode Server9

9 The terminal mode server may use these bits to indicate to the terminal mode client, that it ex-pects voice control input to be enabled or disabled.

- 42 -



10:11 Microphone input on Terminal Mode Client10

16:17 Driver Distraction

24:26 Framebuffer Rotation (clock-wise) (000 = unknown, 001, 010, 011 = not used 100 = 0º, 101 = 90º, 110 = 180º, 111 = 270º)

27:28 Framebuffer Orientation (00 = unknown, 01 = not used, 10 = Landscape, 11 = Portrait)

Table 17: Device Status Message

The Device Status Request message is given in Table 18.





4 U32

Bit Status of Device Features (00 = ignore, 01 = not used 10 = disable, 11 = enable) )

0:1 Key-lock (block key entry on the device)

2:3 Device lock (block key entry on the device and from Terminal Mode client)

4:5 Screen saver (power-down the device screen)

6:7 Night mode (run device in night mode)

8:9 Voice input (route the incoming audio stream to a voice recognition engine on the mobile device)11

10:11 Microphone input on Terminal Mode Client routed from microphone to the terminal mode server

16:17 Driver Distraction mechanisms enabled at Terminal Mode server

24:26 Framebuffer rotation (clock-wise) (000 = ignore, 001, 010, 011 = not used 100 = 0º, 101 = 90º, 110 = 180º, 111 = 270º)

27:28 Framebuffer orientation (00 = ignore, 01 = not used, 10 = Landscape, 11 = Portrait)

Table 18: Device Status Request Message

10 The terminal mode server may use these bits to indicate to the terminal mode client, that audio input is expected from the microphone (e.g. VoIP call). These bits must not be used to indicate end or start of phone call for BT HFP case.

11 The Terminal Mode client must use this flag only, if the voice command is streamed via RTP. In case an existing BT HFP connection is used, the Terminal Mode client must use the BT HFP voice activation mechanism (AT + BVRA command) instead.

- 43 -


The Server may ignore some or all device features as set in the Device Status Request message. The Client must be able to detect this from the received Device Status message, following the original Device Status Request message.

The default values settings for the Terminal Mode server are listed in the following Table 19.

Parameter Default Value

Key-lock Disabled (“10”)

Device-lock Disabled (“10”)

Screen saver Disabled (“10”)

Night mode Disabled (“10”)

Voice input Disabled (“10”)

Microphone input Disabled (“10”)

Driver distraction Disabled (“10”)

Framebuffer rotation No rotation, 0° (“100”)

Framebuffer orientation As provided in ServerInit message

Table 19: Terminal Mode Server Device Status Default Values

- 44 -


5.3.7 Content Attestation Messages

The Content Attestation Request and Response message pair allows the Terminal Mode client to securely verify the content stream received from the Terminal Mode server. This message pair al-lows the Terminal Mode client to detect the following security risks originating from a man-in-the-middle attack:

1. Modification of framebuffer content, with unwanted content 2. Modification of context information, with more favorable settings

To address those risks, the VNC Client can challenge at any time the VNC server to provide a signed response, containing framebuffer characteristics, allowing to identify and to minimize the risks.

The Contest Attestation message flow is shown in Figure 18.


Content Attestation Response

Content Attestation Request

Framebuffer Update Request

Framebuffer Update

Figure 18: Example Content Attestation Message Flow

Requirement: The VNC Server must immediately respond to a Content Attestation Re-quest, immediately after sending the next Framebuffer Update.

Requirement: The VNC Server must include full context information to the framebuffer update message, even if the context has not changed.

Requirement: The VNC client must send a Framebuffer Update Request message imme-diately after the Content Attestation Request Message.

The Content Attestation Response message is given in Table 20.




2 U16 4 + N + M Payload length

2

U16 Bit

Signature Flag Defines, what has been attested and included into the hash (‘1’ = include, ‘0’ = do not include) Note, that the Terminal Mode server may choose to attest different content than what was requested by the client, i.e. the Signature flag set in Content At-testation Response may be different from the one in Content Attestation Request. It is up to the client to

- 45 -


# bytes Type Value Description decide whether such attestation is acceptable.

0 Signature includes all context information pseudo encoded rectangles, as defined in Table 26, as pro-vided within the last framebuffer update

1 Signature includes the framebuffer content, as pro-vided with the last framebuffer update

2 Signature includes number of framebuffer update bytes sent since last device attestation response message

2

U16

Value Error code

0 Success – no change to signature flag

1 Success – with change to signature flag

2 Error – no session key

255 Error – other error

N Array of U8 SignedInfo

M Array of U8 Signature

Table 20: Content Attestation Response Message

The Signature element in Table 20 is a signature over SignedInfo element described below in Ta-ble 21. The used signature algorithm is defined in Content Attestation Request message.


4 U32 Nonce as provided by the Terminal Mode client in Content Attestation Request (Table 22)

2 U16 Signature flag that defines the attested content. The possible values are defined in Table 20

20 Array of U8 (Optional) SHA1 hash of Framebuffer context informa-tion data. Included if Signature flag has bit 0 set.

20 Array of U8 (Optional) SHA1 hash of Framebuffer content. In-cluded if Signature flag has bit 1 set.

4 U32 (Optional) Number of framebuffer bytes sent since previous attestation. 32-bit unsigned integer in network byte order. Included if Signature flag has bit 2 set.

Table 21: SignedInfo Element for HMAC-SH1 Signature Type

The Content Attestation Request message is given in Table 22.




2 U16 8 + N Payload length

4 U32 Random Nonce

2

U16

Bit Defines, what must be attested and included into the hash (‘1’ = include, ‘0’ = do not include)

0 Include all context information pseudo encoded rec-tangles, as defined in Table 26, and provided within the last framebuffer update message.

1 Include entire last framebuffer update

- 46 -



2

Include 32-bit unsigned integer (in network byte-order) that represents the number of bytes sent since previous attestation (consider framebuffer content only)

1 U8

Value Used signature type

0 No signature

1 The signature algorithm is HMAC-SHA1 signature. The signed data is defined in Table 21.

1 U8

Value Used session key

0 No session key included

1

Random 128-bit symmetric session key that is en-crypted using the application specific public key that was bound to attestation of VNC server (see Sec-tion 9.3). The encryption is done according to RSA-SHA1 PKCS v1.5 format. The session key is used from the Terminal Mode server in all subsequent Content Attestation Response messages, until a new key is provided in next Content Attestation Re-quest.

N Array of U8

Session key. The client must set session key in the beginning of each session so that the server does not have to remember previous session keys and mapping of these keys to different client devices.

Table 22: Content Attestation Request Message

- 47 -


5.3.8 Framebuffer Blocking Notification

The support of this message is optional.

The Framebuffer Blocking Notification message allows the Terminal Mode client to notify the Terminal Mode server, that certain framebuffer content delivered to it (as part of a Framebuffer Update message) has been blocked.

The original VNC protocol does not provide a mechanism to inform the Terminal Server about the blocking. If the server would be aware, it could take further actions. The Framebuffer Notifi-cation message will provide that feedback, as shown in the figure below.


Framebuffer Update

Framebuffer Blocking Notification

Figure 19: Example Framebuffer Blocking Notification Message Flow

The Terminal Mode server may utilize this information for e.g. further content updates, received events or application focus changes. Note: Whether and how the server uses this information is implementation dependent and not subject to this specification.

The Framebuffer Blocking Notification message is given below.




2 U16 8 + 4*N Payload length

2 U16 N Number of blocked rectangles

4*N Array of U16 List of blocked rectangles, according the structure defined in Table 24.

Table 23: Framebuffer Blocking Notification Message

A blocked rectangle is described in Table 24 below.


2 U16

Rectangle number being blocked from the Terminal Mode client. Number must correspond to the posi-tion of the rectangle within the last received Frame-buffer Update message. Allowed values are in [0; Number-of-rectangles - 1], whereas Number-of-rectangles is given in Table 4.

2 U16

Bit Reason for blocking (‘1’ = reason, ‘0’ no reason)

0 Not allowed content category

1 Not allowed application category

2 Not sufficient content trust level

3 Not sufficient application trust level

4 Content rules not followed

8 UI not in focus on remote display

- 48 -



9 UI not visible on remote display

Table 24: Blocked Rectangle from Framebuffer Update

Requirement: The Framebuffer Blocking Notification message must correspond to the last received Framebuffer Update message.

- 49 -


5.4 Additional Encodings and Pseudo Encodings

Extensions to the VNC protocols include additional encodings and pseudo encodings. All new encodings are made in compliance with the RFB protocol. The additional encodings are summa-rized in Table 25.

Type Value Description Functionality Support

Pseudo Encoding

-523 Terminal Mode Encoding

Advertise the support of Terminal Mode extension messages. Not used within Framebuffer Update messages.

Mandatory

Pseudo Encoding -524 Context Infor-

mation Indicate context information within a Framebuffer Update message Mandatory

Table 25: Additional VNC Encodings

The encodings are described in more detail in the following paragraphs.

5.4.1 Terminal Mode Pseudo Encoding

The Terminal Mode Pseudo Encoding is used within the Set Encoding message to inform the server about the client support of the Terminal Mode extension messages. The Client must use Terminal Mode Pseudo Encoding only within the Set Encoding message to indicate support for the Terminal Mode extension messages. This Pseudo Encoding must not be used within any Fra-mebuffer Update message.

Requirement: The support for Terminal Mode Pseudo Encoding is mandatory.

5.4.2 Context Information Pseudo Encoding

The Context Information Pseudo Encoding is added to the framebuffer encoding types to provide the client additional meta-information about the applications and content, copied via the frame-buffer update messages. The context information can originated from different sources, giving different level of trust in its correctness. If context information is available from different trust level, the server must provide the highest trust level to the client.

The context information can be used within the VNC client to make an informed decision, to what extend to bring the mobile device framebuffer content to the attention of the Terminal Mode client user. Dependent on legal considerations regarding driver distraction, part of the framebuf-fer content or all may not be shown. It is the responsibility of the VNC client to make such deci-sion. The VNC server must support this decision process, providing accurate context information.

General UI Framework

Menu0x0001 0002

Navigation Application 0x0005 0000

Phone Call Notification0x0001 0003

Figure 20: Context Information (Example)

- 50 -


Context information can be given either for the entire display, or for multiple, individual rectan-gular areas. Context information must valid, until the next context information pseudo encoding. The server must provide context information for the entire display. If multiple overlapping rec-tangles are given, the sequence of the rectangles defines the stacking order (last rectangle on top). If the Terminal Mode client requests a non-incremental framebuffer update, the Terminal Mode client must provide full context information at least for the request rectangle, even if the context information has not changed from previous transfer.

For some part of the display, the application category may not be available, but the terminal mode server may be able to provide more information about the framebuffer content type or vice versa.

The Context Information Pseudo Encoding is following the RFB protocol specification for encod-ings. The whole pseudo encoding rectangle is given in Table 26.


2 U16 X-position of rectangle (top left corner)

2 U16 Y-position of rectangle (top left corner)

2 U16 Width of rectangle

2 U16 Height of rectangle

4 U32 -524 Encoding type

4 U32

Unique application id. For application being adver-tised via UPnP, the unique application id must match the advertised uiID. This field may be left empty (i.e. zero value).

2 U16 Trust Level for Application Category (see Table 39)

2 U16 Trust Level for Content Category (see Table 39)

4 U32 Application Category (see Table 40)

4 U32 Content Category (see Table 41)

4

U32

Bit Content rules, which are followed to tackle Driver Distraction. RuleIds are defined in Table 42.

0 ‘1’. Rule with ruleId 0 supported. ‘0’ otherwise

1 ‘1’: Rule with ruleId 1 supported. ‘0’ otherwise

… …

31 ‘1’: Rule with ruleId 31 supported. ‘0’ otherwise

Table 26: Context Information Pseudo Encoding

Driver distraction rules, which should be followed from applications running on the Terminal Mode server, are provided from the Terminal Mode client using the UPnP TmClientProfile Set-ClientProfile() action, as specified in [20].

Requirement: If no context information has been given, the client must assume “Other ap-

plication” / “Unknown Application”.

Requirement: If no content information has been given, the client must assume

“Unknown” content.

- 51 -


6 AUDIO OUTPUT AND INPUT

The Terminal Mode architecture defines mechanisms using the Real-time Transport Protocol for streaming audio captured from the mobile device, to the terminal mode client. The audio output from the mobile device is streamed in an application agnostic manner so that it does not require re-design or modification of existing applications running on the device.


Device

Automotive Head UnitAudio / Voice

Audio Client

Audio Server

Audio Server

Speaker & Micro

Audio Client

Figure 21: Audio Setup

This specification covers both Audio Output from and Audio Input to the CE device. Unless oth-erwise stated, the specification applies for the Audio Server and the Audio Client in the same way.

• The Audio Output will be handled from the Audio Server on the terminal mode server and the Audio Client on the terminal mode client.

• The Audio Input will be handled from the Audio Client on the terminal mode server and the Audio Client on the terminal mode client.

In both cases, Audio must not be played via the mobile device speakers. The audio RTP server and client on the Terminal Mode server listen on pre-specified ports, which are advertised using UPnP mechanisms. They are started using the TmApplicationServer UPnP service. The Terminal Mode UPnP services are described in Chapter 7. Interoperability with Bluetooth is described in Chapter 6.5.

When the audio server captures audio data, it will encode the audio into RTP packets using the negotiated RTP Payload type and transmit the RTP packets over UDP/IP.

The audio client is fully responsible of receiving and decoding the data packets and restoring the packet order if they arrive in out of order sequence. More detailed information about the RTP packet structure can be found later in the document.

Requirement: The terminal mode server must support RTP audio streaming for uni-directional audio to the terminal mode client.

Recommendation: The terminal mode server may support RTP audio streaming for bi-directional audio.12

12 If bi-directional RTP streaming is not supported, telephony use cases work only if BT HFP is supported.

- 52 -


6.1 RTP Packet Structure and Header Definition

RTP packet contains the standard RTP message header and the payload. Usually each RTP packet audio payload contains predefined amount of audio data, but in special case of end of stream (M=1), payload can be zero length. Therefore, RTP client should not assume fixed payload length. Each RTP packet audio payload contains predefined amount of audio data. Audio samples are sent in sequential order (in sampling order, first sample first).

Each RTP packet contains the standard header as defined in IETF RFC3550. The header fields and their default values are described in the following section. The RTP packet structure is shown be-low.

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Ver P X CC M PT Sequence Number

Timestamp

SSRC

CSRC [0..15] :::

Table 27: RTP Packet Header Definition

The first twelve octets are present in every RTP packet, while the list of CSRC identifiers is present only when inserted by a mixer. The fields are following the RTP specification, unless oth-erwise mentioned:

• Version (V) : 2 bits

The RTP version defined by this specification is two (2).

• Padding (P) : 1 bit

If the padding bit is set, the packet contains one or more additional padding octets at the end which are not part of the payload.

• Extension (X) : 1 bit

If the extension bit is set, the fixed header MUST be followed by exactly one header ex-tension.

• CSRC count (CC) : 4 bits

The CSRC count contains the number of CSRC identifiers that follow the fixed header.

• Marker (M) : 1 bit The interpretation of the marker is defined (in reference to RFC 2190);

0: More packets will follow.

1: Current package carries the end of stream13

• Payload type (PT) : 7 bits

This field identifies the format of the RTP payload and determines its interpretation by the application.

• Sequence number : 16 bits

The sequence number increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence.

13 Audio client may use this information to e.g. empty any available receive buffer.

- 53 -


• Timestamp : 32 bits

The timestamp reflects the sampling instant of the first octet in the RTP data packet. The initial value of the timestamp is random.

• SSRC Synchronization source : 32 bits This field identifies the synchronization source.

If only one source contributes to the audio stream, the SSRC field contain the application category information (as defined in chapter 0) to identify the contributing application class. Otherwise the field is set to zero and the CSRC fields must be used.

• CSRC Contributing Source : 32 bits An array of 0 to 15 CSRC elements identifying the contributing sources for the payload contained in this packet. The number of identifiers is given by the CC field.

The CSRC fields must be used only, if multiple audio sources contributed to the then mixed audio stream. The fields then contain the application category information (as de-fined in chapter 0) to identify contributing application classes.

6.2 RTP Audio Payload Definition

RTP payload length and sampling frequency must be negotiated in beforehand. This must be done using UPnP mechanisms as defined in chapter 7.

The following paragraphs define the audio payload format for 8 and 16 bit audio samples.

Requirement: The server and client must support payload types 0 (8 bit, 8 kHz, mono)14 and 99 (16 bit, stereo, 48 kHz – CD Audio quality)

The Audio Server can optionally support other standardized RTP payload types as well.

6.2.1 16 Bit Audio Payload (Mono)

This payload type denotes uncompressed audio data samples, using 16-bit signed representation with 65,535 equally divided steps between minimum and maximum signal level, ranging from -32,768 to 32,767. The value is represented in two's complement notation and transmitted in net-work byte order (most significant byte first).

The audio data has the following properties:

• One audio channels (mono)

• 16 bits

• Frequency: 48 kHz

• Payload type: 98

Each audio sample is stored to the RTP payload using 32 bits. Each sample is stored to the payl-oad using the order it was taken (first sample first).

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

Sample #(n-1)

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

14 This is an RTP requirement.

- 54 -


Sample #n

Table 28: RTP Payload Format – 16 Bit Mono

6.2.2 16 Bit Audio Payload (Stereo)

This payload type denotes uncompressed audio data samples, using 16-bit signed representation with 65,535 equally divided steps between minimum and maximum signal level, ranging from -32,768 to 32,767. The value is represented in two's complement notation and transmitted in net-work byte order (most significant byte first).

The audio data has the following properties:

• Two audio channels (stereo).

• 32 bits (16 bits per channel).

• Frequency: 48 kHz

• Audio data for each channel interleaved.

• Payload type: 99

Each audio sample is stored to the RTP payload using 32 bits. Each sample is stored to the payl-oad using the order it was taken (first sample first). Audio sample’s left channel data (16 bits) is stored first and then the right channel data (16 bits). This process is applied for each of the audio samples. Audio payload contains always both the right and left channel data. Channel data is never divided amongst different RTP packets.

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

Left channel Sample #n

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

Right channel Sample #n

Table 29: RTP Payload Format – 16 Bit Stereo

6.3 Establishing the RTP Connection

RTP streaming in Terminal Mode is based on UDP, which is connectionless. Some level of initial connection is required for the Terminal Mode server to know the Terminal Mode client’s IP ad-dress and port number.

Therefore, when Terminal Mode server is taking on the role of audio server, the Terminal Mode client must send an UDP packet, containing 1 byte15 to the port number and IP address assigned for the Terminal Mode server’s audio server. On reception of that UDP packet, the audio server must determine the IP address and port number of the audio client.

After receiving that packet, the audio server within the Terminal Mode server will start RTP streaming to the RTP client using the port number, for the received 1 byte packet, when audio stream is available.

The message flow, as shown in Figure 22, consists of the following steps:

15 The content of the byte can be any arbitrary value.

- 55 -


1. RTP Server: Wait for a UDP packet with server’s start. 2. RTP Client: Send 1 byte UDP packet to the RTP server; the RTP server address is obtained

through the TmApplicationServer UPnP service. 3. RTP Client: Get ready for receiving RTP packets. 4. RTP Server: Determine client’s IP address and port number. 5. RTP Server: Begin RTP streaming, when audio data is available; client receives RTP pack-

ets.

RTP Server RTP Client

Send 1 byte UDP packet

RTP streaming

(1)(1)(1)(1)

(5)(5)(5)(5)

(4)(4)(4)(4)(3)(3)(3)(3)

(2)(2)(2)(2)

Figure 22 Sequence for RTP connection: RTP Server by TM Server

This procedure is not necessary when the Terminal Mode client is taking the RTP server role as the Terminal Mode client does know the port number and IP address of Terminal Mode server’s RTP client in advance (from the GetCompatibleUIs response).

6.4 Recommendation for Client and Server Implementation

The audio client must do local buffering of the incoming RTP packets and should start playing the audio, if the buffer is filled with a reasonable number of packets. This will allow compensat-ing for potential jittering.

The audio client must compensate potential frequency differences between the server and client audio sampling rate. The Audio client may need to provide streaming control to the Audio serv-er.16

The audio server is responsible for maintaining long term accuracy of audio clock. For example, if 48kHz audio stream is sent over 10 seconds, RTP server should make sure that all data are deli-vered around 10 seconds’ duration with minimal error. Note that each packet can arrive at differ-ent intervals depending on the network load, and reasonable amount of buffering should solve this problem. Audio client is expected to use its own audio clock to play the received audio stream, and it is audio client’s responsibility to compensate potential frequency differences be-tween the server and client audio clock. When audio server supplies audio stream in much faster frequency than audio client can handle, audio client can either drop some packets or adjust its playback frequency, and it is up to client to decide which method to choose.

It is expected that the Audio is played immediately, without any delays, besides the buffering de-lay. No specific synchronization between the audio stream and the VNC based display updates is provided.

16 The buffer control will need some further investigation. Conclusion expected for final specifica-tion.

- 56 -


6.5 Interoperability with Bluetooth

Interoperability (IOP) of the Terminal Mode RTP streaming audio framework with Bluetooth au-dio profiles must be given. Bluetooth can be used to replace missing or existing RTP audio streaming functionality as defined in this Terminal Mode specification.

Requirement: The terminal mode server must at least distinguish between speech (phone call) and application audio (media player, navigation directions, etc.).

Requirement: The terminal mode server shall provide context information for media sources; otherwise the audio must be treated as originating from "unknown application"

Recommendation: The terminal mode server shall provide BT HFP, if it supports phone call or voice control functionality.17

6.5.1 Bluetooth Profiles relevant for Terminal Mode

Bluetooth Handset Profile (BT HSP)

Bluetooth Headset profile (BT HSP) must not be used in Terminal Mode.

Bluetooth Hands-Free Profile (BT HFP)

If Bluetooth Hands-free profile (BT HFP) is used for voice call together with RTP stream-ing for application audio, it is the responsibility of the terminal mode server to take con-trol of audio link (SCO) when phone call is active: The terminal mode server will take care of both opening and closing the SCO audio link. The terminal mode client will accept the setup of the SCO audio link. 18 It is the terminal mode client’s responsibility to make sure that the SCO audio link is opened as soon as possible when requested from the ter-minal mode server. Except the explicit terminal mode server’s responsibility for the audio link control, Bluetooth Hands-Free Profile 1.5 specification will be followed.

Bluetooth Advanced Audio Distribution Profile (BT A2DP)

Bluetooth A2DP may optionally be used to stream application audio from the device to the terminal mode client. If RTP streaming is available, it is recommended not to use BT A2DP for application audio.

6.5.2 Interoperability States –Terminal Mode Server Perspective

Within the Terminal Mode context, the following Interoperability (IOP) states with Bluetooth (BT) can be distinguished, as shown in Figure 23 from the CE Terminal mode server, i.e. the CE device perspective.

• Mobile device has not established any Terminal Mode and no other Bluetooth connec-tions (State: None)

• Mobile device has established a Bluetooth connection to the vehicle head-unit, but no Terminal Mode connection (State: BT to HU)

17 The terminal mode server cannot assume the client to support bidirectional RTP streaming.

18 There is no difference for the head-unit, whether the audio link is controlled from a phone ap-plication (without terminal mode being active) or from the terminal mode server.

- 57 -


• Mobile device has established a Bluetooth connection to a head-set (State: BT to HS)

• Mobile device has established a Terminal Mode connection (State: TM to HU)

• Mobile device has established a Bluetooth connection to a head-set in addition to a Ter-minal Mode connection (State: TM to HU+ BT to HS)

None BT to HU

TM to HU

Disconnect TM

Connect TM

BT to HS

Connect HUConnect HS

Disconnect HS

TM to HU + BT to HS

Disconnect TM

Connect TM

Disconnect TM

Connect TM

Disconnect HU

Connect HS

Disconnect HS

Figure 23: Bluetooth / Terminal Mode IOP States – Terminal Mode Server Perspective

The transitions between the different IOP states must be followed from actions with regard to the head-set (HS) or head-unit (HU) Bluetooth profiles or the RTP streaming. These actions are given in Table 30. Transitions marked in dotted lines, are outside the scope of this specification and giv-en for completeness only.

Current State Action Next State

Immediate Activity for Terminal Mode Server Connectivity

HS BT HFP

HU BT HFP

HU BT A2DP

HU RTP stream

None Connect TM TM to HU - Connect (*)

Connect (**)

Connect (***)

Fallback if BT setup

fails Connect HS BT to HS Connect - - - Connect HU BT to HU - Connect Connect -

BT to HS

Connect TM

TM to HU + BT to HS TM to HU (if connec-tion is over-ridden)

Keep, but disconnect if overrid-

den

Connect if overridden, otherwise reject HU

connection request

Connect if overridden, otherwise reject HU

connection request

Connect (***)

Disconnect HS None Disconnect - - -

BT to HU Connect TM TM to HU -

Keep (*) - otherwise disconnect

Keep (**) - otherwise disconnect

Connect (***)

- 58 -



Immediate Activity for Terminal Mode Server Connectivity

HS BT HFP

HU BT HFP

HU BT A2DP

HU RTP stream

(****) (****)

Disconnect BT None

Connect if previously overridden

(*****)

Disconnect (*****)

Disconnect (*****) -

TM to HU

Disconnect TM

None - -19 -20 Disconnect BT to HU Keep Keep Disconnect

Connect HS TM to HU + BT to HS

Connect Disconnect Disconnect Keep

TM to HU + BT to HS

Disconnect HS TM to HU Disconnect Connect (*)

Connect (**) Keep

Disconnect TM

BT to HS Keep - - Disconnect

Table 30: IOP Transition (from Terminal Mode Server perspective)

(*) If BT HFP is the preference after the UPnP negotiation

(**) If BT A2DP is the preference after UPnP negotiation

(***) If RTP is the preference after the UPnP negotiation

(****) The terminal mode server should disconnect the respective BT profile only, if the terminal mode client has connected to the RTP server. This provides Audio functionality in case the Terminal Mode client does not utilize UPnP for audio link selection (as introduced in chapter 8).

(*****) The terminal mode server should restore previous BT connection if the previous connec-tion was overridden by HU’s BT connection.

6.5.3 Interoperability States –Terminal Mode Client Perspective

Within the Terminal Mode context, the following Interoperability (IOP) states with Bluetooth (BT) can be distinguished, as shown in Figure 24 from the terminal mode client, i.e. Head-Unit pers-pective.

• Head-unit has not established any Terminal Mode and no other Bluetooth connections (State: None)

• Head-unit has established a Bluetooth connection to a mobile device, but no Terminal Mode connection (State: BT to CE1)

• Head-unit has established a Bluetooth connection to a mobile device, different from CE1 (State: BT to CE2)

19 The TM connection did not incorporate a BT HFP connection.

20 The TM connection did not incorporate a BT A2DP connection

- 59 -


• Head-unit has established a Terminal Mode connection with CE1 (State: TM to CE1)

• Head-unit has established a Bluetooth connection to CE2 in addition to a Terminal Mode connection with CE1 (State: TM to CE1 + BT to CE2)

None BT to CE2

TM to CE1

Disconnect TM

TM to CE1+ BT to CE2

Connect TM

BT to CE1

Connect CE2Connect CE1

Disconnect CE1

Disconnect TM

Connect TM

Disconnect TM

Connect TM

Disconnect CE2

Connect CE2

Disconnect CE2

Figure 24: Bluetooth / Terminal Mode IOP States – Terminal Mode Client Perspective

The transitions between the different IOP states must be followed from actions with regard to the CE1 and CE2 Bluetooth profiles or the RTP streaming. These actions are given in Table 31. Transi-tions marked in dotted lines, are outside the scope of this specification and given for complete-ness only.


Immediate Activity for Terminal Mode Client Con-nectivity

CE2 BT HFP

CE1 BT HFP

CE1 BT A2DP

CE1 RTP stream

None Connect TM TM to CE1 - Connect (*) Connect

(**)

Connect (***)

Fallback if BT setup

fails Connect CE2 BT to CE2 Connect - - - Connect CE1 BT to CE1 - Connect Connect -

BT to CE2

Connect TM TM to CE1 + BT to CE2

Keep Reject CE1 connection

request

Reject CE1 connection

request

Connect (***)

Disconnect CE2 None Disconnect - - -

BT to CE1

Connect TM TM to CE1 - Keep (*) - otherwise disconnect

Keep (**) - otherwise disconnect

Connect (***)

- 60 -



Immediate Activity for Terminal Mode Client Con-nectivity

CE2 BT HFP

CE1 BT HFP

CE1 BT A2DP

CE1 RTP stream

Disconnect CE1 None - Disconnect Disconnect -

TM to CE1

Disconnect TM

None - -21 -22 Disconnect BT to CE1 Keep Keep Disconnect

Connect CE2 TM to CE1 + BT to CE2

Connect if CE1 not

connected via BT

- - Keep

TM to HU + BT to CE2

Disconnect CE2 TM to HU Disconnect Connect (*)

Connect (**) Keep

Disconnect TM

BT to CE2 Keep - - Disconnect

Table 31: IOP Transition (from Terminal Mode Client perspective)

(*) If BT HFP is the preference

(**) If BT A2DP is the preference

(***) If RTP is the preference

The selection of the Audio Link is specified within chapter 8.

21 The TM connection does not incorporate a BT HFP connection.

22 The TM connection does not incorporate a BT A2DP connection

- 61 -


7 SERVICE NEGOTIATION FRAMEWORK

7.1 UPnP Usage Models

The basic UPnP Device architecture consists of three basic instances, which are shown in Figure 25. The UPnP Device Server device, implementing the UPnP Device Server service(s), represents a data source that can provide data content. The UPnP Device Client, implementing the UPnP De-vice Client service(s), represents the data sink.

Both UPnP functionalities are controllable by an UPnP Device Control Point. This is an applica-tion listening to UPnP advertisements or being able to discover appropriate UPnP devices or ser-vices within an IP based network.

Terminal Mode Client DeviceTerminal Mode Server DeviceData

UPnP Actions

Terminal Mode Control Point

Figure 25: General UPnP Device Architecture

Combining the three functionalities within different devices, results in several usage models, which are described in the following subsection.

7.1.1 2-Box Pull Model

Within the 2-Box pull model, the Terminal Mode Server is discoverable and controllable, due to its embedded UPnP Device service. The Terminal Mode Client, implementing an UPnP Device Control Point, is able to discover the remote Terminal Mode Server and can invoke its services, as shown in Figure 26.

Terminal Mode Server


Server Device

Terminal Mode Server Device

DataClient Device

UPnP Actions

Terminal Mode Client

Figure 26: 2-Box Pull Model

From user’s point of view the data will be pulled from the remote Terminal Mode Server to the Terminal Mode Client.

Requirement: The 2-Box Pull Model is mandatory for Terminal Mode.

7.1.2 2-Box Push Model

Within the 2-Box push model the Terminal Mode Client is discoverable and controllable, due to its embedded UPnP device services. The Terminal Mode Server, implementing an UPnP Device Control Point, is able to find the remote Terminal Mode Client and can invoke its services, as shown in Figure 27.

- 62 -



Terminal Mode Client Device

Server Device


DataClient Device

UPnP Actions


Figure 27: 2-Box Push Model

From user’s point of view the data will be pushed from the Terminal Mode Server to the Terminal Mode Client.

The 2-Box push model is not supported in the Terminal Mode specification.

However, the specification does not prevent the use of a complementary 2-Box push model, which will require an additional Terminal Mode Client Device, specified in [21].

7.1.3 3-Box Model

Within the 3-Box model both the Terminal Mode Server and the Client are discoverable and con-trollable, due to their embedded UPnP Devices’ service functionalities. A third Terminal Mode Control device, implementing the UPnP Device Control Point, will discover the Terminal Mode Server and Clients and can invoke its services, as shown in Figure 28.


Terminal Mode Client Device

Server Device

Terminal Mode Server Device

DataClient Device

UPnP Actions


Terminal Mode Control-Unit


Figure 28: 3-Box Model

From user’s point of view the data will be transferred from the Terminal Mode Server to the Ter-minal Mode Client. The user will control services by a specific Control-Unit, which he is using.

The 3-Box model is not supported in the Terminal Mode specification.

However, the specification does not prevent the use of a complementary 3-Box model, which will require an additional Terminal Mode Client Device, specified in [21]. From the Terminal Mode Server perspective, there is no difference between the 2-Box pull and the 3-Box model.

- 63 -


7.2 UPnP Device Architecture

The Universal Plug and Play (UPnP) is used within Terminal Mode to enable basic service dis-covery, configuration and negotiation. In addition it provides capabilities for the Terminal Mode Control Point to remotely control applications, residing with the Terminal Mode server and re-ceiving its user interface.

Requirement: The Terminal Mode Server must implement either the UPnP 1.0 stack or the UPnP 1.1 stack. In either case, it should be able to operate with both UPnP 1.0 and UPnP 1.1 Control Points.

Requirement: The Terminal Mode Client must implement either an UPnP 1.0 control point or an UPnP 1.1 control point. In either case it should be able to operate with both UPnP 1.0 and UPnP 1.1 services residing on the Terminal Mode server.

7.3 TmServerDevice:1 Device

Terminal Mode is specifying a TmServerDevice containing two services as shown in Figure 25.

TmServerDevice:1

TmApplicationServer:1

TmClientProfile:1

Figure 29: Terminal Mode UPnP Service Architecture

The TmServerDevice:1 is specified as a separate device in [18]. It provides two services.

• TmApplicationServer:1 service, as specified in [19], allows to control remote applications and provides access to an out-of-band communication channel.

• TmClientProfile:1 service, as specified in [20], allows to provide client specific informa-tion updates.

The Terminal Mode specific use of the above described device and services are described in more detail in the following chapters.

Requirement: The Terminal Mode server must implement a TmServerDevice:1 Server.

Requirement: The Terminal Mode client must implement a TmServerDevice:1 Control Point.

7.3.1 TmApplicationServer:1 Service

Requirement: The Terminal Mode server must support TmApplicationServer:1 service.

Requirement: The Terminal Mode client must support TmApplicationServer:1 service.

With regard to this specification, the UPnP TmApplicationServer:1’s GetApplicationList action must meet the following requirements:

• The appCategory element (A_ARG_TYPE_String) will define the application category ac-cording Table 40.

Default value: “0x00000000” (unknown application category)

- 64 -


• The contentCategory element (A_ARG_TYPE_String) will define the content category ac-cording Table 41.

Default value: “0x00000000” (unknown content category)

• The contentRules element (A_ARG_TYPE_String) will define, which UI rules, the remote application is following. The bitmask is defined according Table 42.

Default value: “0x00000000” (no rules)

• The trustLevel element (A_ARG_TYPE_String) will define the trust level for the applica-tion and content categories acchording Table 39.

Default value: “0x00” (no trust)

• The format element (A_ARG_TYPE_String) in remomtingInfo will define the payload type used in case of RTP audio streaming, according to chapter 6.2.

Default value: 99 (16 bit, 48 kHz, stereo)

• The audioType element (A_ARG_TYPE_String) will define the audio type:

o “phone” – Phone call audio o “application” – Generic application audio o “all” – Phone and application audio o “none” – no Audio

Default value: “none”

7.3.2 TmClientProfile:1 Service

Requirement: The Terminal Mode server must support TmClientProfile:1 service.

Recommendation: The Terminal Mode client may support TmClientProfile:1 service.

With regard to this specification, the UPnP TmClientProfile:1’s SetClientProfile action must meet the following requirements:

• The contentRules element (A_ARG_TYPE_String) will define, which UI rules, the remote application is following. The bitmask is defined according Ta-ble 42 which lists rules for minimizing driver distraction. The default value is “0x00000000” (no rules)

7.4 TmClientDevice:1 Device

Terminal Mode is specifying a Terminal Mode Client Device, containing three services as shown in Figure 30.

TmClientDevice:1

TmApplicationCliet:1

TmClientProfile:1

TmConnectionManager:1

Figure 30: Terminal Mode Client Device Architecture

The TmClientDevice:1 is specified as a separate device in [21]. It provides three services.

- 65 -


• TmApplicationClient:1 service, as specified in [22], allows to inform the client about re-mote controllable applications of the server.

• TmClientProfile:1 service, as specified in[20], allows to provide client specific informa-tion updates.

• TmConnectionManager:1 service, as specified in [23], allows to remote control connec-tions between client and server devices.

Recommendation: The Terminal Mode client may implement a TmClientDevice:1 device.

Requirement: If the Terminal Mode client implements a TmClientDevice:1 device, this de-vice must not limit or interfere with any operations of the mandatory TmSer-verDevice:1 Control Point.

Recommendation: The Terminal Mode server may implement a TmClientDevice:1 Control Point.

Requirement: If the Terminal Mode server implements a TmClientDevice:1 Control Point, this Control Point must not limit or interfere with any operations of the man-datory TmServerDevice:1 device.

- 66 -


8 AUDIO LINK SELECTION

This chapter specifies, how the Terminal Mode client selects the transport media, the Terminal Server must use to route audio streams. The Terminal Mode server distinguishes between two audio streams, namely phone and application audio. It advertises available transport options, us-ing UPnP TmServerDevice:1 services. Audio streams are specified for the following <remotePro-tocol> options:

• RTP Real-Time Protocol

• BTA2DP Bluetooth Advanced Audio Distribution Profile

• BTHFP Bluetooth Hands Free Profile

It is the terminal client’s responsibility to select from the advertised audio transport mechanisms how the terminal mode server must stream the different audio sources. The Audio Link Selection is done according the following priorities (highest priority first):

1. Keep existing Bluetooth HFP or A2DP connection to another external device23 if overriding the resource assignment is not allowed.

2. Follow audio link selection using the mechanism described in this chapter. 3. Manual Bluetooth pairing (same behavior as in non-Terminal Mode use cases) 4. Phone speaker & microphone (same behavior as in non-Terminal Mode and non-Bluetooth

use cases)

Section 8.1 specifies over which transport mechanisms (i.e. RTP, BT A2DP, or BT HFP the audio (i.e. phone and application audio) based on the selections the terminal mode client has taken. Sec-tion 8.2 specifies how TmApplicationServer:1 service is used to select the audio link.

8.1 Audio Link Options

The outcome of the audio link selection from the UPnP negotiation phase is given in Table 32. The table lists the possible audio link configurations. Each available element, i.e. BT HFP, BT A2DP,

RTP Server and RTP client, is individually advertised from the Terminal Mode server. “Used”

or “Not used” these available elements, determine, which audio link is selected for streaming Phone and Media audio from the Terminal Mode server, as specified in Table 32. Mechanisms, described in section 8.2 are notifying the Terminal Server that the client will use (or not use) a specific configuration element. In addition, the table includes access to play list information.

The audio selection is following the a set of rules

• RTP for phone is selected only if RTP server and client is available

• BT HFP should be used only for the phone audio

• BT A2DP should be used for media (or application) audio.

Audio Link Configuration Audio Link Selection Media

Meta Data & Control

BT HFP BT A2DP RTP

Server RTP

Client

Phone audio

via

Media audio

out via

Media audio in

via Used Used Used Used Invalid configuration

Behavior is implementation dependent Used Used Used Not used Used Used Not used Used BT HFP BT A2DP RTP BT AVRCP

23 Which is not the Terminal Mode client

- 67 -


Audio Link Configuration Audio Link Selection Media

Meta Data & Control

BT HFP BT A2DP RTP

Server RTP

Client

Phone audio

via

Media audio

out via

Media audio in

via Used Used Not used Not used BT HFP BT A2DP Mic BT AVRCP Used Not used Used Used BT HFP RTP RTP MTP 1.0 Used Not used Used Not used BT HFP RTP Mic MTP 1.0 Used Not used Not used Used BT HFP Speaker RTP - Used Not used Not used Not used BT HFP Speaker Mic -

Not used Used Used Used Invalid configuration Behavior is implementation dependent Not used Used Used Not used

Not used Used Not used Used Speaker BT A2DP RTP BT AVRCP Not used Used Not used Not used Speaker BT A2DP Mic BT AVRCP Not used Not used Used Used RTP RTP RTP MTP 1.0 Not used Not used Used Not used Speaker RTP Mic MTP 1.0 Not used Not used Not used Used Speaker Speaker RTP - Not used Not used Not used Not used Speaker Speaker Mic -

Table 32: UPnP Negotiation for Audio Selection

If a connection fails, the Terminal Mode server and client must set the corresponding Audio link

entry to “Not used” (as the Terminal Mode client would have terminated the audio link) and reset the audio routing accordingly.

8.2 Audio Link Selection

The TmApplicationServer:1’s services can be used for controlling audio streams. Each type of au-dio source or sink on the Terminal Mode server is considered a remote application which can be remotely controlled by the Terminal Mode Control Point using TmApplicationServer:1 service.

An example remote application listing of audio links is included in the A_ARG_TYPE_AppList example in [19].

The Audio servers and clients can be started and terminated the same way as any other remote application, using the LaunchApplication() and TerminateApplication() SOAP actions respective-ly. The Audio server, optionally running as part of the Terminal Mode client, will provide audio input like voice control to the Terminal Mode server.

Next a description is provided of how TmApplicationServer:1’s SOAP actions are utilized to se-lect the audio links. Note that only the aspects specific to audio link selection are covered here and the reader is required to refer to the corresponding service specification for complete details of the TmApplicationServer:1.

8.2.1 LaunchApplication (AppID, ProfileID)

The LaunchApplication() action must be used to start the audio streaming on the Terminal Mode server side and therefore select the underlying audio link. The response received will be an URI to the audio streaming sources/sinks using the audio streaming protocol identifier. The URI must follow the A_ARG_TYPE_URI definition as specified in [19].

- 68 -


Audio Server Audio Client

TmApplicationServer:1A_ARG_TYPE_URI

TmApplicationServer:1LaunchApplication()

TmApplicationServer:1GetApplicationList()

TmApplicationServer:1A_ARG_TYPE_AppList

UPnP Server UPnP Control Point

(2)

(1)

(5)

(6)

(4)

(7)

(3)

Figure 31: Message Flow – Launch Audio Link

The message flow for selecting an Audio Link, using LaunchApplication(), as shown in Figure 31, consists of the following steps:

1. UPnP Control Point: Call GetApplicationList() SOAP action

2. UPnP Server: Return A_ARG_TYPE_AppList, which is including a list of available Audio Servers

3. Select the preferred Audio Server

a. BT only: Prepare for BT connection, if Terminal Mode server is expected to in-itiate the BT connection24

4. UPnP Control Point: Call LaunchApplication() SOAP action containing respective Audio Server application ID

5. Start selected Audio Server

a. Determine new audio link configuration, using Table 32 b. RTP only: Prepare RTP streaming25 c. BT only: Prepare for BT connection. d. BT only: Initiate BT connection if Terminal Mode server is expected to initiate the

BT connection.

1. UPnP Server: Return A_ARG_TYPE_URI representing the Audio Server URI

2. Start Audio Client

a. RTP only: Connect to RTP streaming port b. BT only: Initiate BT connection, if Terminal Mod client is expected to initiate the

BT connection.

24 The Terminal Mode client can indicate to the server via the SetClientProfile action that the serv-er must initiate the BT connection. By default, the Terminal Mode server will not start the BT con-nection.

25 RTP streaming is done over UDP.

- 69 -


Using LaunchApplication() will enable a specific audio link. This link is valid until TerminateAp-plication() action with the same AppID is called, or the Terminal Mode session is closed.

Requirement: If Bluetooth is already turned on, the Terminal Mode server must accept BT connection requests after responding to launching a specific Bluetooth pro-file.

Recommendation: If Bluetooth is turned off, it is recommended that the terminal mode server switches on Bluetooth before responding to launching a specific Bluetooth profile.

The Terminal Mode server must only initiate a Bluetooth connection, if the Terminal Mode client has specifically requested the server to do so, setting the startConnection tag in

A_ARG_TYPE_ClientProfile to “false”. To ensure automatic pairing, without user intervention, the terminal mode client should provide its Bluetooth MAC address (bdAddr) in A_ARG_TYPE_ClientProfile.

If the Terminal Mode server does not support Bluetooth connection initialization, i.e. it has specif-

ically set startConnection in the UPnP TmServerDevice:1 device XML to “false”, and the Ter-minal Mode client does not support Bluetooth connection initialization either, the Terminal Mode client must not use the LaunchApplication() to start a Bluetooth connection.

8.2.2 TerminateApplication (AppID, ProfileID)

The TerminateApplication() action must be used to stop the audio streaming (in either direction) on the Terminal Mode Server side and follows the specification [19]. Invoking the TerminateAp-plication action causes the corresponding audio link to be closed.

Audio Server Audio Client

TmApplicationServer:1TerminateApplication()

UPnP Server UPnP Control Point

TmApplicationServer:1A_ARG_TYPE_Bool

(2)

(1)

(4)(3)

Figure 32: Message Flow – Terminate Audio Link

The message flow for unselecting an Audio Link, using TerminateApplication(), as shown in Fig-ure 32, consists of the following steps:

1. UPnP Control Point: Call TerminateApplication() SOAP action containing respective Au-dio Server application ID

2. Stop Audio Server

a. Determine new audio link configuration, using Table 32 b. RTP only: Stop the RTP streaming c. BT only: Disconnect BT profile; optionally power down BT

3. UPnP Server: Return termination success response (true or false)

4. Stop Audio Client

a. RTP only: Disconnect from RTP streaming port b. BT only: Disconnect BT profile; optionally power down BT

- 70 -


In case the TerminateApplication() action is used to terminate the audio server residing in the Terminal Mode server device then the Terminal Mode client will stop receiving the audio stream from the Terminal Mode server.

In case the TerminateApplication() action is used to terminate the audio client residing in the Terminal Mode server then the Terminal Mode server will stop receiving the audio stream from the Terminal Mode client.

8.2.3 GetApplicationStatus (AppID, ProfileID)

The GetApplicationStatus() action provides the current status of an audio server or client running on the Terminal Mode server and is following [19].

The return values (of the type A_ARG_TYPE_AppStatus) specified for this SOAP action can be one of the following:

• Foreground: Audio link is launched.

• Background: Not used.

• Notrunning: Audio link is terminated / not launched.

- 71 -


9 DEVICE ATTESTATION

The term “device attestation” in this context refers to Terminal Mode client verifying that the Terminal Mode server is from compliant manufacturer and running approved software. The at-testation will be based on standard X.509 certificates [13] and attestation mechanisms standar-dized by Trusted Computing Group [14].

9.1 Device Attestation Launch

The Terminal Mode server advertises the Device Attestation Protocol within the UPnP

A_ARG_TYPE_AppList listing, using the <protocolID> “DAP”. An example listing is included in the A_ARG_TYPE_AppList example in [19].

The message flow to start the Device Attestation Protocol using Remote Application Control is given in Figure 33.

Device Attestation

Server

Device Attestation

Client

TmApplicationServer:1A_ARG_TYPE_URI

TmApplicationServer:1LaunchApplication()

TmApplicationServer:1GetApplicationList()

TmApplicationServer:1A_ARG_TYPE_AppList

UPnP Server

UPnP Control Point

(2)

(1)

(4)

(5)(6)(7)

(3)

Device Attestation ProtocolAttestationResponse()

Device Attestation ProtocolAttestationRequest()

(8)

Figure 33: Message Flow – Launch Device Attestation Protocol

The message flow, as shown in the previous figure, consists of the following steps:

1. UPnP Control Point: Call GetApplicationList() SOAP action

2. UPnP Server: Return A_ARG_TYPE_AppList, which includes the Device Attestation

Server (protocolID is set to “DAP”).

3. UPnP Control Point: Call LaunchApplication SOAP action containing respective Device Attestation Server application ID

4. UPnP Server: Start Device Attestation Server

5. UPnP Server: Return A_ARG_TYPE_URI containing the Device Attestation Server URI

6. UPnP Control Point: Start Device Attestation Client

7. Device Attestation Client: Send Attestation Request

8. Device Attestation Server: Send Attestation Response

- 72 -


9.2 Device Attestation Termination

The message flow to terminate the Device Attestation Protocol using Remote Application Control is shown in Figure 34

Device Attestation

Server

Device Attestation

Client

TmApplicationServer:1TerminateApplication()

UPnP Server

UPnP Control Point

(2)

(3)

(1)

Figure 34: Message Flow – Terminate Device Attestation Protocol

The message flow, as shown in the previous figure, consists of the following steps:

1. UPnP Control Point: Terminate Device Attestation Client.

2. UPnP Control Point: TerminateApplication() SOAP action containing respective Device Attestation Server application ID

3. UPnP Server: Terminate Device Attestation Server

In order to restart the device attestation, after previous termination, the terminal mode client must not follow the entire launch device attestation message flow. It can directly go to step (3).

9.3 Device Attestation Protocol

The prerequisite of successful Device Attestation Protocol run is that the terminal mode server has a X.509 device certificate for its device key pair from the server device manufacturer (see Fig-ure 35). Additionally, the server must have and one or more X.509 manufacturer certificates (client manufacturer has certified server manufacturer) from one or more client manufacturers. The server device key pair should be stored securely within the server device, preferably using hardware-based Mobile Trusted Module (MTM) [16] implementation or equivalent.

Figure 35: Device Attestation certification infrastructure

A more detailed overview of device and software attestation protocol is shown in Figure 36 be-

low. The protocol is two-flow protocol: terminal mode client sends attestationRequest mes-

- 73 -


sage and terminal mode server replies with attestationResponse message. Both of these messages are XML formatted.

Figure 36: Device and software attestation protocol overview

In the protocol terminal mode client first picks random nonce and sends that together with iden-tifier of the attested software component(s) to terminal mode server. A trusted component with the terminal mode server measures the requested software component(s). If the measurement matches expected value, a component identifier, URL that this component is using, and optional hash of public part of application key are extended into a platform configuration register (PCR) that is reserved for this use using TPM_Extend command [15]. (TDB: reserved PCR number).

Terminal mode server performs TPM_Quote operation [15] on the PCR using the nonce as an in-put. This operation signs the PCR structure using a certified device key. The old value of PCR, the resulting signature, IP address and port number, and device and device manufacturer certificates are sent to terminal mode client, which verifies the device manufacturer certificate (using pre-installed trust root), verifies device certificate using verified device manufacturer certificate, and finally verifies the attestation signature using the verified device certificate.

The elements of attestationRequest XML message are defined below:

Element Description Parent Availability attestationRe-quest Main container of attestation request none Optional

version Version information attestation Response Mandatory

majorVersion Major version number. Should be “1” version Mandatory minorVersion Minor version number. Should be “0” version Mandatory

trustRoot

Identifier of the trust root used by ter-minal mode client for attestation verifi-cation. 20-byte SHA1 hash of client manufacturer public key in Base64 en-coded format [12]. Based on this iden-tifier the server can send correct manufacturer certificate(s) to client.

attestation Request Mandatory

- 74 -


Element Description Parent Availability

nonce 16-byte random number Base64-encoded [12].

attestation Request

Mandatory

componentID

Identifier of the software component to be attested. Possible values are de-scribed in Table 34.

attestation Request Mandatory

Table 33: Device Attestation – attestationRequest elements

The software components, which can be attested, are listed below with their component IDs. The Protocol ID is used as part of the attestation’s response URL binding

componentID Description of what functionality is attested Protocol ID VNC-Server VNC server functionality as specified in chapter 5 VNC

UPnP-Server

UPnP TmServerDevice server services as specified in chapter 7. The attestation includes the service advertisements, using UDP broadcasts and the SOAP and eventing mechanisms based on HTTP.

UDP HTTP

UPnP-ControlPoint

UPnP TmClientDevice control point functionality as specified in chapter 7.4. The attestation includes the listening to service ad-vertisements, using UDP, and the SOAP and eventing mechan-isms based on HTTP.

UDP HTTP

RTP-Server RTP-Server as specified in chapter 6 RTP RTP-Client RTP-Client as specified in chapter 6 RTP

*

Wildcard. All components must be attested. In this case terminal mode server replies with multiple attesta-tionResponse messages. Each attestationResponse message includes the identifier of the attested component.

-

Table 34: Device Attestation – Component List

Below is an example of attestationRequest message:

<attestationRequest> <version> <majorVersion>1</majorVersion> <minorVersion>0</minorVersion> </version> <trustRoot>dbR5...dT5S3=</trustRoot> <nonce>7Thg34saHd5...4hkjd=</nonce> <componentID>VNC-Server</componentID> </attestationRequest>

The elements of attestationResponse XML message are defined below:

Element Description Parent Availability attestation Response Main container of attestation response None Optional

version Version information attestation Response Mandatory

major Version

Major version number. Should be “1” Version Mandatory

minor Version Minor version number. Should be “0” Version Mandatory

- 75 -


Element Description Parent Availability

result Indicates success/failure of attestation operation. Possible values are defined in Table 36.

attestation Response Optional

attestation* Contains attestation of one component. attestation Response

Mandatory26

componentID Identifier of the attested component. Possible val-ues are listed in Table 3. Details Mandatory

oldValue The old value of the PCR reserved for device and software attestation use. 20-byte binary value Base64-encoded [12].

attestation Mandatory

quote Signature

RSA PKCSv1.5 formatted signature produced by TPM_Quote command as specified in [18] (with exception that 1024-bit signatures are accepted in addition to 2048-bit signatures). 128-byte or 256-byte binary value Base64-encoded [12].


URL

URL that defines the Protocol ID (according Table 34), IP address and port number that the attested software component is currently holding, according the following format:

[ProtocolID]://[IP]:[Port]


application PublicKey

Public part of key pair that the attested application may later use to authenticate (e.g. sign) transferred data. (The private part of this key pair should be accessible only by the attested application. The mechanism used to protect the private key de-pends on the platform of server device.) The key pair should be 1024-bit or 2048 bit RSA key and the public part should be Base64 encoded [12] from X.509 SubjectPublicKeyInfo format [13].

attestation Optional

device Certificate

X.509v3 [15] certificate issued by the Terminal Mode server device manufacturers. The certificate contains the public part of the 1024-bit or 2048-bit RSA device key. The certificate may have variable length. The certificate is Base64 encoded from ASN.1 DER format [17].

attestation Response Mandatory27

manufacturer Certificate*

A (chain of) X.509v3 [15] certificate(s) issued for the Terminal Mode server manufacturer by the Terminal Mode client manufacturer. The certifi-cate(s) may have variable length. The certificate(s) are Base64 encoded from DER format.

attestation Response Mandatory28

Table 35: Device Attestation – attestationResponse Elements

The elements marked with an (*) can have multiple instances.

result Description of result value 0 Successful attestation (default)

26 Mandatory only in case of successful attestation (result == 0).



- 76 -


result Description of result value 1 Component not existing29 2 Error in attestation: Version not supported 3 Error in attestation: unknown trust root 4 Error in attestation: unknown component ID 5 Error in attestation: Attestation failed

Table 36: Device Attestation – Possible Attestation Result Values

Below is an example of attestationRequest message:

<attestationResponse> <version> <majorVersion>1</majorVersion> <minorVersion>0</minorVersion> </version> <result>0</result> <attestation> <componentID>VNC-Server</componentID> <oldvalue>jlFGh...kj=</oldValue> <quoteSignature>IL7h9j9...4J3234Kg=</quoteSignature> <URL>VNC://172.0.0.1:5500</URL> </attestation> <deviceCertificate>gTsd7d3...763rJKh=</deviceCertificate> <manufacturerCertificate>6sbk5..7d4dkH= </manufacturerCertificate> </attestationResponse>

To verify quoteSignature a terminal mode client should perform following steps:

1. Calculate hash H1 as SHA1(applicationPublicKey) if attestationResponse message in-cluded applicationPublicKey element

2. Calculate hash H2 as SHA1(oldValue || componentID || URL || H1). Include H1 if at-testationResponse message included applicationPublicKey element.

3. Create TPM_PCR_COMPOSITE structure C 4. Set C->select to TBD! 5. Set C->valueSize to 20 6. Set C->pcrValue to H2 7. Caluculate hash H3 as SHA1(C) 8. Construct TPM_QUOTE_INFO structure Q 9. Set Q->version to 1.1.0.0

10. Set Q->fixed to “QUOT” 11. Set Q->digestValue to H3 12. Set Q->externalData to nonce 13. Verify that quoteSignature is valid RSA-SHA1 PKCS v1.5 signature over Q using public

part of device key extracted from deviceCertificate

For the exact formats of the above mentioned structures, refer to [16].

Requirement: If device and software attestation is mandated by the Terminal Mode client, it should attest all (TDB) software components on the Terminal Mode server before presenting content received from these components to the user.

29 Component ID is known, but component is not implemented on the Terminal Mode server, e.g. an RTP client.

- 77 -


Requirement: When a software component on Terminal Mode server is attested, the client should check that it has active (TCP) connection with the attested software component that matches the attested IP address and port number. Once the active (TCP) connection breaks, the client should run the attestation protocol again for the same component (if mandated by the client).

Note: When device and software attestation protocol is run over networked con-nection, such as protected WiFi, an attacker within the same network may be able to masquerade as the attested device. Performing such an attack requires that the attacker is able to (1) spoof its IP address and (2) manipulate TCP connection parameters (such as sequence numbers). Both of these are possi-ble with right kind of equipment, but the user must have intentionally added the malicious device into the protected WiFi network. In such a case it is rec-ommended to use the application specific key (that was bound to attestation of each component) to authenticate (e.g. sign on server and verify on client) all or subset of the communication.

It is recommended to terminate the Device Attestation Protocol, using the appropriate Termi-nateApplicaton() SOAP action, after all components have been successfully attested. The Device Application Protocol can be launched again at any point in time.

- 78 -


10 REFERENCES

[1] IETF, RFC 3550, “RTP: A Transport Protocol for Real-Time Applications”, July 2003, http://tools.ietf.org/html/rfc3550

[2] IETF, RFC 3551, “RTP Profile for Audio and Video Conferences with Minimal

Control”, July 2003, http://www.ietf.org/rfc/rfc3551.txt

[3] USB CDC/ECM, “Universal Serial Bus Class Definitions for Communication Devices

Class Subclass Specifications for Ethernet Control Model Devices”, Revision 1.2, Feb-ruary 9, 2007.

[4] USB CDC/NCM, “Universal Serial Bus Communications Devices Class Subclass Spe-

cifications for Network Control Model Devices”, Revision 1.0, April 30, 2009.

[5] BT SIG, “Simple Pairing”, White Paper, Core Specification Working Group, Revision V10r00, August 3, 2006.

[6] BT Specification, “Hands-free Profile 1.5”, Car Working Group, Revision V10r00, November 25, 2005.

[7] BT Specification, “Handset Profile”, Car Working Group, Revision V12r00, December 18, 2008.

[8] UPnP Specification, “UPnP™ Device Architecture”, Version 1.1, Revision October 15, 2008

[9] Bluetooth Assigned Numbers, http://www.bluetooth.org/assigned-numbers.htm

[10] Bluetooth Specification “Audio/Video Distribution Transport Protocol (AVDTP)”,

Revision 1.00

[11] XML Signature Syntax and Processing, http://www.w3.org/TR/xmldsig-core/, W3C Recommendation 10 June 2008

[12] IETF, RFC 4648, The Base 16, Base 32 and Base 64 Data Encodings, October 2006. http://tools.ietf.org/html/rfc4648

[13] IETF, RFC 5280, Internet X.509 Public Key Infrastructure Certificate, May 2008. http://tools.ietf.org/html/rfc5280

[14] Trusted Computing Group. http://www.trustedcomputinggroup.org/

[15] Trusted Platform Module (TPM) specifications. http://www.trustedcomputinggroup.org/resources/tpm_main_specification

[16] Mobile Trusted Module (MTM) specifications. http://www.trustedcomputinggroup.org/resources/mobile_phone_work_group_mobile_trusted_module_specification_version_10

[17] ITU-T ASN.1 Encoding Rules. http://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf

[18] “TmServerDevice:1 Device Template”, Version 0.5, for UPnP™ Version 1.0, Proposed DCP, April 15, 2010

[19] “TmApplicationServer:1 Service Template”, Version 0.5, for UPnP™ Version 1.0, Pro-posed DCP, April 26, 2010

[20] “TmClientProfile:1 Service Template”, Version 0.9, for UPnP™ Version 1.0, Proposed DCP, April 26, 2010

- 79 -


[21] “TmClientDevice:1 Device Template”, Version 0.9, for UPnP™ Version 1.0, Proposed DCP, April 26, 2010

[22] “TmApplicationClient:1 Service Template”, Version 0.9, for UPnP™ Version 1.0, Pro-posed DCP, April 26, 2010

[23] “TmConnectionManager:1 Service Template”, Version 0.9, for UPnP™ Version 1.0, Proposed DCP, April 26, 2010

- 80 -


APPENDIX A – EVENT MAPPING

Knob Shift and Rotate Events

The shift and rotate operations of a 3D knob is given according to the coordinate system, given in Figure 37. The gray 2D area references the surface, the knob is mounted to..

x

y

z

RightLeft

Push

Up

Down

Pull

xx clockwise

y clockwise

z clockwise

z

y

Figure 37: Coordinate System for Knob Events

The knob shift & rotate configuration is shown in the following table.


4 U32

Bit Knob shift & rotate configuration (1 = support, 0 = no support)

0 x-axis shift

1 y-axis shift

2 z-axis shift

3 x & y-axis shift

4 x & z-axis shift

5 y & z-axis shift

6 x & y & z-axis shift

7 not defined

8 x-axis rotation

9 y-axis rotation

10 z-axis rotation

11 x & y-axis rotation

12 x & z-axis rotation

13 y & z-axis rotation

14 x & y & z-axis rotation

Table 37: Knob Shift and Rotate Configuration Settings

The Terminal Mode server may support long key press events, i.e. multiple key down press events, before the final key release event. The long key press does include knob rotation events. If

- 81 -


a knob provides haptic feedback, while rotating, it may give better user experience not to use long press events, but rather individual events (i.e. key press event, followed by key release event).

Key Event Mapping

The Key event mapping is shown in the following table.

Category Mnemonic KeySymValue Description

Car Head-unit Key

Reserved 0x3000 0000 -

Knob_3D_shift_right 0x3000 0001 Right shift

Knob_3D_shift_left 0x3000 0002 Left shift


Knob_3D_shift_up 0x3000 0004 Up shift

Knob_3D_shift_up_right 0x3000 0005 Up & right shift

Knob_3D_shift_up_left 0x3000 0006 Up & left shift


Knob_3D_shift_down 0x3000 0008 Down shift

Knob_3D_shift_down_right 0x3000 0009 Down & right shift

Knob_3D_shift_down_left 0x3000 000A Down & left shift

Reserved

0x3000 000B - …

0x3000 000F

Knob_3D_shift_pull 0x3000 0010 Pull

Knob_3D_shift_pull_right 0x3000 0011 Pull & right shift

Knob_3D_shift_pull_left 0x3000 0012 Pull & left shift


Knob_3D_shift_pull_up 0x3000 0014 Pull & up shift

Knob_3D_shift_pull_up_right 0x3000 0015 Pull & up & right shift

Knob_3D_shift_pull_up_left 0x3000 0016 Pull & up & left shift


Knob_3D_shift_pull_down 0x3000 0018 Pull & down shift

Knob_3D_shift_pull_down_right 0x3000 0019 Pull & down & right shift

Knob_3D_shift_pull_down_left 0x3000 001A Pull & down & left shift

Reserved

0x3000 001B

- …

0x3000 001F

Knob_3D_shift_push 0x3000 0020 Push

Knob_3D_shift_push_right 0x3000 0021 Push & right shift

Knob_3D_shift_push_left 0x3000 0022 Push & left shift


Knob_3D_shift_push_up 0x3000 0024 Push & up shift

Knob_3D_shift_push_up_right 0x3000 0025 Push & up & right shift

Knob_3D_shift_push_up_left 0x3000 0026 Push & up & left shift

- 82 -




Knob_3D_shift_push_down 0x3000 0028 Push & down shift

Knob_3D_shift_push_down_right 0x3000 0029 Push & down & right shift

Knob_3D_shift_push_down_left 0x3000 002A Push & down & left shift

Reserved

0x3000 002B - …

0x3000 0040

Knob_3D_rotate_x 0x3000 0041 X clockwise rotation

Knob_3D_rotate_X 0x3000 0042 X anti-clockwise rotation


Knob_3D_rotate_y 0x3000 0044 Y clockwise rotation

Knob_3D_rotate_yx 0x3000 0045 Y clockwise, x clockwise rota-tion

Knob_3D_rotate_yX 0x3000 0046 Y clockwise, x anti-clockwise rotation


Knob_3D_rotate_Y 0x3000 0048 Y anti-clockwise rotation

Knob_3D_rotate_Yx 0x3000 0049 Y anti-clockwise, x clockwise rotation

Knob_3D_rotate_YX 0x3000 004A Y anti-clockwise, x anti-clockwise rotation

Reserved

0x3000 004B

- …

0x3000 004F

Knob_3D_rotate_z 0x3000 0050 Z clockwise rotation

Knob_3D_rotate_zx 0x3000 0051 Z clockwise, X clockwise rota-tion

Knob_3D_rotate_zX 0x3000 0052 Z clockwise, X anti-clockwise rotation


Knob_3D_rotate_zy 0x3000 0054 Z clockwise, Y clockwise rota-tion

Knob_3D_rotate_zyx 0x3000 0055 Z clockwise, Y clockwise, x clockwise rotation

Knob_3D_rotate_zyX 0x3000 0056 Z clockwise, Y clockwise, x anti-clockwise rotation


Knob_3D_rotate_zY 0x3000 0058 Z clockwise, Y anti-clockwise rotation

Knob_3D_rotate_zYx 0x3000 0059 Z clockwise, Y anti-clockwise, x clockwise rotation

Knob_3D_rotate_zYX 0x3000 005A Z clockwise, Y anti-clockwise, x anti-clockwise rotation

Reserved

0x3000 005B

- …

0x3000 005F

- 83 -



Knob_3D_rotate_Z 0x3000 0060 Z anti-clockwise rotation

Knob_3D_rotate_Zx 0x3000 0061 Z anti-clockwise, X clockwise rotation

Knob_3D_rotate_ZX 0x3000 0062 Z anti-clockwise, X anti-clockwise rotation


Knob_3D_rotate_Zy 0x3000 0064 Z anti-clockwise, Y clockwise rotation

Knob_3D_rotate_Zyx 0x3000 0065 Z anti-clockwise, Y clockwise, x clockwise rotation

Knob_3D_rotate_ZyX 0x3000 0066 Z anti-clockwise, Y clockwise, x anti-clockwise rotation


Knob_3D_rotate_ZY 0x3000 0068 Z anti-clockwise, Y anti-clockwise rotation

Knob_3D_rotate_ZYx 0x3000 0069 Z anti-clockwise, Y anti-clockwise, x clockwise rotation

Knob_3D_rotate_ZYX 0x3000 006A Z anti-clockwise, Y anti-clockwise, x anti-clockwise ro-tation

Reserved

0x3000 006B

- …

0x3000 00FF

Mobile ITU

ITU_Key_0 0x3000 0100 0, ' '

ITU_Key_1 0x3000 0101 1, '.', ','

ITU_Key_2 0x3000 0102 2, a, b, c

ITU_Key_3 0x3000 0103 3, d, e, f

ITU_Key_4 0x3000 0104 4, g, h, i

ITU_Key_5 0x3000 0105 5, j, k, l

ITU_Key_6 0x3000 0106 6, m, n, 0

ITU_Key_7 0x3000 0107 7, p,q, r, s

ITU_Key_8 0x3000 0108 8, t, u, v

ITU_Key_9 0x3000 0109 9, w, x, y, z

ITU_Key_Asterix 0x3000 010A *, +

ITU_Key_Pound 0x3000 010B #, shift

Reserved 0x3000 010C

- …

0x3000 01FF

Mobile Function Keys

Device_Phone_call 0x3000 0200 Take a phone call / invoke call log

Device_Phone_end 0x3000 0201 End phone call / go to home screen

Device_Soft_left 0x3000 0202 Left soft key

Device_Soft_middle 0x3000 0203 Middle soft key

Device_Soft_right 0x3000 0204 Right soft key

- 84 -



Device_Application 0x3000 0205 Application list

Device_Ok 0x3000 0206 Ok

Device_Delete 0x3000 0207 Delete (Backspace)

Device_Zoom_in 0x3000 0208 Zoom in (e.g. into a map)

Device_Zoom_out 0x3000 0209 Zoom out (e.g. out of a map)

Device_Clear 0x3000 020A Clear

Device_Application_forward 0x3000 020B Application level forward

Device_Application_backward 0x3000 020C Application level backward

Reserved

0x3000 020D

Reserved …

0x3000 02FF

Terminal Mode Keys

TM_Key_0 0x3000 0300 Soft and hard keys available on the client and server user interface

TM_Key_1 0x3000 0301

… …

TM_Key_255 0x3000 03FF

Multimedia Device Keys

Multimedia_Play 0x3000 0400 Start media playing

Multimedia_Pause 0x3000 0401 Pause media playing

Multimedia_Stop 0x3000 0402 Stop media playing

Multimedia_Forward 0x3000 0403 Forward

Multimedia_Rewind 0x3000 0404 Rewind

Multimedia_Next 0x3000 0405 Go to next track in playlist

Multimedia_Previous 0x3000 0406 Go to previous track in playlist

Multimedia_Mute 0x3000 0407 Mute the audio stream at source

Multimedia_Unmute 0x3000 0408 Unmute the audio stream at source

Multimedia_Photo 0x3000 0409 Take a photo

Reserved

0x3000 040A

- …

0x3000 04FF

Reserved Reserved

0x3000 0500

- …

0x3000 FFFF

Table 38: Key Event Mapping

- 85 -


APPENDIX B – APPLICATION CONTEXT INFORMATION

Trust Level

Trust level used in the Context Information Pseudo Encoding is given in Table 39.

Value Description Trust Level

0x00 Unknown No Trust

0x40 User Configuration Trust the user

0x60 Self-Registered Application Trust the application

0x80 Registered Application Trust the VNC server

0xA0 Application certificate Trust a 3rd party certification entity

Table 39: Trust Level

Application Categories

The application categories and sub-categories are given in the following table. The table can be amended in future versions of the specifications. Application categories are used in the Context Information messages.

Application Cat-egory

Application Sub-category Description

0x0000 0x0000 Unknown Application

0x0001

0x0000 General UI Framework

0x0001 Application list, home screen

0x0002 Menu

0x0003 Notification

0x0002

0x0000 General Phone call application

0x0001 Contact list

0x0002 Call log

0x0003

0x0000 General Media applications

0x0001 Music

0x0002 Video

0x0003 Gaming

0x0004 Image

0x0004

0x0000 General Messaging applications

0x0001 SMS

0x0002 MMS

0x0003 Email

0x0005 0x0000 General Navigation

0x0006 0x0000 General Browser

0x0007

0x0000 General Productivity

0x0001 Document Viewer

0x0002 Document Editor

0xF000 0x0000 General UI less applications

- 86 -


Application Cat-egory

Application Sub-category Description

0x0001 Audio Server

0x0002 Audio Client

0xFFFF

0x0000 General System

0x0001 PIN input for Device unlock

0x0002 Bluetooth PIN code Input

0x000F Other password input

0x0010 Voice Command Confirmation

Table 40: Application Categories

Content Categories

The content categories are given in the following table. The table can be amended in future ver-sions of the specification. Content categories are used in the Context Information messages.

Content Category (Bit) Description

0 Text

1 Video

2 Image

3 Vector Graphics

4 3D Graphics

5 User Interface (e.g. Application menu)

16 Car Mode

24 Media Audio

25 Voice Audio

31 Misc. content

Table 41: Content Categories

If no Bit is set, the content must be treated as to be unknown.

Content Rules

The content rules, which should be followed from the Terminal Mode server’s User Interface, to minimize driver distraction are given in Table 42. Each rules, does have a unique rule identifier (ruleID). Some of the rules require an additional value (ruleValue),.

ruleID Description ruleValue

0

Minimum font size required. ruleValue is given as the minimum font size required in pixel divided by the target vertical screen resolution in pixel. Terminal Mode server assumes linear scaling of its provided framebuffer to the target resolu-tion. The format must be Q32, represented in hexadecimal format.

Mandatory

1 No video is shown. A video is defined as a sequence of motion images, representing a scene or a sequence of scenes.

Not used

2 No automatic scrolling text The rule does not apply to user controlled scrolling Not used

- 87 -


ruleID Description ruleValue

3 ruleValue gives maximum feedback time allowed after user input in [s]. The format must be 32 bit unsigned Integer (uint32_t), represented in hexadecimal format.

Mandatory

Table 42: Content Rules to tackle Driver Distraction

Terminal mode architecture

Technology

Transcript of Terminal mode architecture