SYNTHETIC DATA / AI - Nvidia › video › ... · 7 SYNTHETIC DATA Real data is expensive,...
Transcript of SYNTHETIC DATA / AI - Nvidia › video › ... · 7 SYNTHETIC DATA Real data is expensive,...
Rev Lebaredian - Vice President, Simulation Technolgy
SYNTHETIC DATA / AI
2
NEURAL NETWORKS NEED DATA
Deep Learning is amazing!
Require huge amounts of quality data
Data needs labeling
For some problems, data + labeling is available
Good data doesn’t exists for most problems
And Labels!
Image-Net (http://www.image-net.org/) example
3
DOMINOS, ANYONE?Isaac at SIGGRAPH 2017
4
NO EXISTING DATASET
5
Labeled Data
1min per object
7
SYNTHETIC DATA
Real data is expensive, sometimes dangerous
Synthetic labels are automatic and accurate
Useful for validation, in addition to training
8
AV SYNTHETIC DATASETSVirtual KITTI
Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig: Virtual Worlds as Proxy for Multi-Object Tracking Analysis IEEE CVPR 2016
9
DOMAIN RANDOMIZATIONExplore the gap using random cars, textures, camera, distractors, etc
10
DOMAIN RANDOMIZATIONExample Scenes
11
STRUCTURED DOMAIN RANDOMIZATIONPutting it all together
12
SIMULATETRAIN MODELSCOLLECT DATA DRIVE
NVIDIA DRIVEEND-TO-END PLATFORM
Lanes Lights
Path
Signs
PedestriansCars
Lanes Lights
Path
Signs
PedestriansCars
13
14
15
AlphaZero OpenAI Five
OpenAI, 2018Deepmind, 2018
REINFORCEMENT LEARNING SUCCESSES
16
APPLICATIONS
Locomotion/Animation
Reinforcement Learning
Liang, Makoviychuk, Handa etc, 2018NVIDIA
17
APPLICATIONS
Sim2Real Robotics
Robotics
Chebotar, Handa, Makoviychuk, etc, 2018NVIDIA
18
EXAMPLESLocomotion
19
EXAMPLESLocomotion + Physics
20
EXAMPLESLocomotion + Physics
21
ISAAC GYMToolkit for Parallel AI Learning Experiments
22
PHYSICSPhysX
23
PHYSICSFleX
• Multi-physics
• Rigid and FEM soft bodies
• Cloth, ropes
• Liquids
• Two-way coupling and force propagation between different phases
24
CABLE ROBOT
25
ROBOTS
LEARN& PLAN
SEE, HEAR,TOUCH
ACT
26
27
28
BARTENDING ROBOTTrained in Isaac Gym
29
ISAAC SIM
30
31
32
33
Our Reality Virtual Reality
Reality
Human
ArtificialIn
telli
gen
ce