Rev Lebaredian - Vice President, Simulation Technolgy
SYNTHETIC DATA / AI Rev Lebaredian - Vice President, Simulation - - PowerPoint PPT Presentation
SYNTHETIC DATA / AI Rev Lebaredian - Vice President, Simulation - - PowerPoint PPT Presentation
SYNTHETIC DATA / AI Rev Lebaredian - Vice President, Simulation Technolgy NEURAL NETWORKS NEED DATA And Labels! Deep Learning is amazing! Require huge amounts of quality data Data needs labeling For some problems, data + labeling is available
2
NEURAL NETWORKS NEED DATA
Deep Learning is amazing! Require huge amounts of quality data Data needs labeling For some problems, data + labeling is available Good data doesn’t exists for most problems
And Labels!
Image-Net (http://www.image-net.org/) example
3
DOMINOS, ANYONE?
Isaac at SIGGRAPH 2017
4
NO EXISTING DATASET
5
Labeled Data
1min per object
7
SYNTHETIC DATA
Real data is expensive, sometimes dangerous Synthetic labels are automatic and accurate Useful for validation, in addition to training
8
AV SYNTHETIC DATASETS
Virtual KITTI
Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig: Virtual Worlds as Proxy for Multi-Object Tracking Analysis IEEE CVPR 2016
9
DOMAIN RANDOMIZATION
Explore the gap using random cars, textures, camera, distractors, etc
10
DOMAIN RANDOMIZATION
Example Scenes
11
STRUCTURED DOMAIN RANDOMIZATION
Putting it all together
12
SIMULATE TRAIN MODELS COLLECT DATA DRIVE
NVIDIA DRIVE END-TO-END PLATFORM
Lanes Lights
Path
Signs
Pedestrians Cars
Lanes Lights
Path
Signs
Pedestrians Cars
13
14
15
AlphaZero OpenAI Five OpenAI, 2018 Deepmind, 2018
REINFORCEMENT LEARNING SUCCESSES
16
APPLICATIONS
Locomotion/Animation
Reinforcement Learning
Liang, Makoviychuk, Handa etc, 2018 NVIDIA
17
APPLICATIONS
Sim2Real Robotics
Robotics
Chebotar, Handa, Makoviychuk, etc, 2018 NVIDIA
18
EXAMPLES
Locomotion
19
EXAMPLES
Locomotion + Physics
20
EXAMPLES
Locomotion + Physics
21
ISAAC GYM
Toolkit for Parallel AI Learning Experiments
22
PHYSICS
PhysX
23
PHYSICS
FleX
- Multi-physics
- Rigid and FEM soft bodies
- Cloth, ropes
- Liquids
- Two-way coupling and force
propagation between different phases
24
CABLE ROBOT
25
ROBOTS
LEARN & PLAN SEE, HEAR, TOUCH ACT
26
27
28
BARTENDING ROBOT
Trained in Isaac Gym
29
ISAAC SIM
30
31
32
33