Building a Cloud Platform for RobotsDhananjay Sathe, Lead rapyuta.io

Task Offloading

The Genesis

It worked!

Collaborative 3D Mapping Dynamic Mobile Manipulation

"containerTag" : "roombaClone"

RoboEarth Compute Environment

"addNodes" : [{ "containerTag" : "roombaClone", "nodeTag" : "positionRecorder", "pkg" : "testPkg", "exe" : "posRecorder.py" }],

RoboEarth Compute Environment

"addInterfaces" : [{ "endpointTag" : "roombaClone", "interfaceTag" : "pos", "interfaceType" : "PublisherInterface", "className" : "geometry_msgs/Pose2D", "addr" : "/posPub"}]

RoboEarth Compute Environment

* NAT + proprietary protocol or raw sockets over GRE-tunnels and Openvswitch* Websockets* Proprietary RCE Protocol

Networking in RoboEarth

Re-imagining the Platform

LXC - Container

Master Process(built-in scheduling)

Container ProcessContainer Process Robot Endpoint

Robot Endpoint

Env EP

JSON Protocol Websocket / CLIIaaS Hooks

User Workload Nodes

RoboEarth Architecture Overview

Master Process(built-in scheduling)

Robot EndpointRobot Endpoint

JSON Protocol Websocket / CLI

IaaS Hooks

Containerization

LXC - Container

Container ProcessContainer Process

Env EP

User Workload Nodes

● No standard for describing a container

● Distribution of tarballs

● Changing dependency trees

● Exposing the right interfaces

Master Process(built-in scheduling)

Robot EndpointRobot Endpoint

JSON Protocol Websocket / CLI

IaaS Hooks

Containerization

LXC - Container

Container ProcessContainer Process

Env EP

User Workload Nodes

● Dockerfile format

● Docker registry API

● Pods allow coupling related software at runtime

● Host isolation

● Pluggable runtime

User Workload Nodes

kubelet

kubelet

LXC - Container

Container ProcessContainer Process Robot Endpoint

Robot Endpoint

Env EP

User Workload Nodes

Control PlaneMaster Process

(built-in scheduling)JSON Protocol

Websocket / CLIIaaS Hooks

● Scalability

● Day-2 operations nightmare

● Code modularity and maintainability

● Vendor support

LXC - Container

Container ProcessContainer Process Robot Endpoint

Robot Endpoint

Env EP

User Workload Nodes

Control PlaneMaster Process

(built-in scheduling)JSON Protocol

Websocket / CLIIaaS Hooks

● Battle tested at sufficient scale

● Rich ecosystem and tooling

● Modular extendable codebase

● cloud-controller-manager

API Server

Kube API

cloud-controller-manager

Scheduler

rapyutaAPI

LXC - Container

Master Process(built-in scheduling)

Container ProcessContainer Process

Env EP

JSON Protocol Websocket / CLI

IaaS Hooks

User Workload Nodes

Networking

Robot EndpointRobot Endpoint

● Complex multi-host port mapping

● Flakey SDN implementation

● Reconfiguration on node failure

● Limited protocol extensibility

● Centralized Robot endpoint messaging

LXC - Container

Master Process(built-in scheduling)

Container ProcessContainer Process

Env EP

JSON Protocol Websocket / CLI

IaaS Hooks

User Workload Nodes

ROS Routing MeshRobot Protocol Routing Layer

Ingress / LoadBalancers

Multi Protocol DataChannel

● CNI - high performance L3 networking

● Full support for a variety of L4-L7 protocols

● Ingress + load-balancer for edge routing

● DNS based service discovery

● NetworkPolicy per robot bulkheading and

isolation

Networking

LXC - Container

Master Process(built-in scheduling)

Container ProcessContainer Process Robot Endpoint

Robot Endpoint

Env EP

JSON Protocol Websocket / CLI

IaaS Hooks

User Workload Nodes

The Onset

API Server

Kube API

Cloud Controller Manager

Scheduler

platformAPI

ROS Routing MeshRobot Protocol Routing Layer

User Workload Nodes

kubelet

kubelet Ingress / LoadBalancers

Env EPEnv EP

Multi Protocol DataChannel

KubeAPI

The Culmination

Breaking out of the Datacenter

The Anatomy of a Robotics Application

Heterogeneous Infrastructure

● Multiple architectures

● Weak network links

● Spatial and temporal variations

● Reproducibility and configuration

Binary Artifacts

rapyuta.ioPackage

Strategies + Architecture

Registry

Build Configs

Parse

Workflow CRD

MultiArch-Build Engine

● Unified declarative application description

● Uniform lifecycle management across the datacenter and devices

● Permit coupling of components and inter-dependencies

● Allow cross pollination of ideas, services and code

Unified Application Description and Runtime

ServiceClass Composition

ServiceClass Composition

The Platform Broker

Service Catalog

Platform Broker

Device Broker

Cloud Broker

IaaS Broker

Service Catalog

Cloud Controller

Render to Cluster (eg. Helm/Templates)

Service Catalog

Platform Broker

Device Broker

Cloud Broker

IaaS Broker

Service Catalog

The Platform Broker

Service Catalog

Platform Broker

Device Broker

Cloud Broker

IaaS Broker

Service Catalog

Control Channel

Device Controller

The Platform Broker

Training the Broker{ "apiVersion":"v1.0.0", "name":"cloud_pub_sub", "plans":[ { "name":"default", "components":[ { "name":"listener", "cloudInfra":{ "replicas":1}, "requiredRuntime":"cloud", "executables":[ { "docker":"rapyutians/listener “, cmd":["roslaunch listener listener.launch"]}], }, { "name":"talker", "ros":{ "topics":[ { "name":"/telemetry"}], "isROS":true }, "requiredRuntime":"device", "executables":[ { "git":"https://github.com/rapyuta/talker", " cmd":["roslaunch talker talker.launch"]}], }] } ]}

ManifestPlatform BrokerAdd Service Class

DataStore

Persist Service Class Objects

Training the Broker

Unlocking rapyuta.io Platform value

Dynamic multi-robot orchestration

● Federation

● Custom Resource Definitions

● KubeProxy

● VirtualKube

● TUF Conformance

● Opentracing Extended

Foreglimpse

Acknowledgements

Questions?