Introducing Uber’s Open Source

Autonomous Visualization System

June 2019 | Xiaoji Chen | Sr Software Engineer, Uber Visualization

01 What is AVS02 Components & Features03 Applications built with AVS04 Next Steps

Agenda

● Overwhelming volume of data

● Emerging subdivisions and use cases

● Rapidly evolving requirements

● Flexibility and maintainability

What is AVS?

The Challenges

AVS is a new standard for describing and

visualizing autonomous vehicle perception,

motion, and planning data, offering a

powerful web-first toolkit to build

applications for exploring, interacting and,

most critically, making important

development decisions with that data.

https://eng.uber.com/avs-autonomous-vehicle-visualization/

What is AVS?

A Unified Standard

● Decouple presentation from platform

○ Time-based

○ Geometries, imageries, metrics, UI and data bindings

● Designed for best runtime performance

○ Binary optimized

○ Data format and styling aligned with GPUs

○ Object-data association

● Put data and control near the source

○ Server protocol

○ Multiple platform versions work in the same app

● Make visualization testable

○ Tools for verification and manipulation

Components & Features

XVIZ

The XVIZ Data Flow

● Web-based toolkit that makes consuming and working with XVIZ easy

● Handles the hard problems out-of-the-box

○ Data buffering, synchronization, coordinate transforms, dynamic

styling, 3D scene rendering, objects & camera interaction

● A foundation of UI components for building sophisticated apps

○ Video, chart, table, popover, settings, HUD, playback control

○ Highly reusable & customizable

● Performance first

Components & Features

Streetscape.gl

Components & Features

WebGL-Powered Open SourceVisualization Frameworks

react-vis

React components for d3

react-map-gl

React components for mapbox-glMap Tiles © Mapbox | Data Attributions

deck.gl

GPU powered large-scale

visualization

luma.gl

WebGL2 framework

https://vis.gl

CPU GPU

Population of New York City

Internally, Uber ATG’s autonomous vehicle

web platform enables log viewing, triaging,

simulation, debugging, to map editing,

and labeling of web-based applications.

We also work with external partners in the

industry to address their particular needs

and continuously improve AVS.

Production applications built with AVS at Uber and beyond

Applications

https://foresight.ai/blogs/foresightai-big-data-visualization.html

Towards a smarter transportation ecosystem for the future

What’s Nextv1.0 Release

● Stabilized XVIZ protocol

● Node.js builder pipeline

● Feature-complete visualizer and web

components

● Node.js server module

● ROS > XVIZ converter

● Interactive playground

SOON

● Point cloud compression

● New primitive types

● More examples and tutorials

On ROADMAP

● Protobuf support

● c++ builder & server

● Python builder & server

https://avs.auto

Thank you

Proprietary © 2018 Uber Technologies, Inc. All rights reserved. No part of thisdocument may be reproduced or utilized in any form or by any means,electronic or mechanical, including photocopying, recording, or by anyinformation storage or retrieval systems, without permission in writing fromUber. This document is intended only for the use of the individual or entity towhom it is addressed. All recipients of this document are notified that theinformation contained herein includes proprietary information of Uber, andrecipient may not make use of, disseminate, or in any way disclose thisdocument or any of the enclosed information to any person other thanemployees of addressee to the extent necessary for consultations withauthorized personnel of Uber.