Dhruv Batra Long-term Goal Physical agent Is there smoke in any - - PowerPoint PPT Presentation

▶
dhruv batra long term goal
SMART_READER_LITE
LIVE PREVIEW

Dhruv Batra Long-term Goal Physical agent Is there smoke in any - - PowerPoint PPT Presentation

Feb 13, 2023 135 likes •1.72k views

Habitat: A Platform for Embodied AI Research Dhruv Batra Long-term Goal Physical agent Is there smoke in any room capable of taking around you? actions in the world and talking to humans Yes, in one room in natural language Go there and

slide-1
SLIDE 1

Habitat: A Platform for Embodied AI Research

Dhruv Batra

slide-2
SLIDE 2
slide-3
SLIDE 3

3

Long-term Goal

Is there smoke in any room around you? Yes, in one room Go there and look for people …

Physical agent capable of taking actions in the world and talking to humans in natural language

Image Credit: Lockheed Martin; DARPA Robotics Challenge Slide credit: Abhishek Das

slide-4
SLIDE 4

…

Image Credit: Lockheed Martin; DARPA Robotics Challenge Slide credit: Abhishek Das 4

slide-5
SLIDE 5

…

Slide credit: Abhishek Das

Internet AI à Embodied AI

5 Image Credit: Image-Net Image Credit: Lockheed Martin; DARPA Robotics Challenge

slide-6
SLIDE 6

6 Image Credit: Image-Net, Video Credit: Lee et al., 2012

Egocentric vision

No access to well-composed, curated images

Slide credit: Abhishek Das

Internet AI à Embodied AI

slide-7
SLIDE 7

7 Video Credit: Lee et al., 2012

Active perception Action Observation

Agent controls incoming data distribution

Slide credit: Abhishek Das

Internet AI à Embodied AI

slide-8
SLIDE 8

8 Image Credit: Image-Net

Sparse rewards

Slide credit: Abhishek Das

Internet AI à Embodied AI

slide-9
SLIDE 9

9 Image Credit: Image-Net

Sparse rewards

Slide credit: Abhishek Das

Internet AI à Embodied AI

slide-10
SLIDE 10

10 Image Credit: Image-Net

+ —

{

Sparse rewards

Slide credit: Abhishek Das

Internet AI à Embodied AI

slide-11
SLIDE 11

11

Language understanding

asda nkslndjksan asdlaskmdlas

Slide credit: Abhishek Das

Internet AI à Embodied AI

slide-12
SLIDE 12

12

  • Slow
  • Dangerous
  • Expensive
  • Difficult to control
  • Not easy reproducible

Problems with reality

…

slide-13
SLIDE 13

13

  • Slow
  • Dangerous
  • Expensive
  • Difficult to control
  • Not easy reproducible

Our Approach: Sim2Real

slide-14
SLIDE 14

14

Resurrection of Embodied AI

slide-15
SLIDE 15

15

EmbodiedQA

SUNCG (Song et al., 2017)

Datasets Simulators Tasks

Matterport3D (Chang et al., 2017) AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) Stanford 2D-3D-S (Armeni et al., 2017) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Chaplot et al., 2017, Hermann & Hill et al., 2017) Visual Navigation (Zhu & Gordon et al., 2017, Savva et al., 2017, Wu et al., 2017) HoME (Brodeur et al., 2018) VirtualHome (Puig et al., 2018) AdobeIndoorNav (Mo et al., 2018) Matterport3DSim (Anderson et al., 2018)

>= 2017 (!)

Slide credit: Abhishek Das 15

slide-16
SLIDE 16

Abhishek Das (Georgia Tech) Samyak Datta (Georgia Tech) Devi Parikh (FAIR/Georgia Tech) Dhruv Batra (FAIR/Georgia Tech) Stefan Lee (Georgia Tech) Georgia Gkioxari (FAIR)

Embodied Question Answering

[CVPR ’18 Oral]

16

slide-17
SLIDE 17
slide-18
SLIDE 18

18

What is to the left of the shower? Cabinet

slide-19
SLIDE 19

EmbodiedQA

Slide credit: Abhishek Das

slide-20
SLIDE 20

EmbodiedQA

House3D (Wu et al., 2017)

Slide credit: Abhishek Das

slide-21
SLIDE 21

EmbodiedQA

SUNCG (Song et al., 2017) House3D (Wu et al., 2017)

Slide credit: Abhishek Das

slide-22
SLIDE 22

EmbodiedQA

SUNCG (Song et al., 2017) House3D (Wu et al., 2017)

Dataset

Slide credit: Abhishek Das

slide-23
SLIDE 23

EmbodiedQA

SUNCG (Song et al., 2017) House3D (Wu et al., 2017)

Dataset Simulator

Slide credit: Abhishek Das

slide-24
SLIDE 24

EmbodiedQA

SUNCG (Song et al., 2017) House3D (Wu et al., 2017)

Dataset Simulator Task

Slide credit: Abhishek Das

slide-25
SLIDE 25

EmbodiedQA

SUNCG (Song et al., 2017)

Datasets Simulators Tasks

Matterport3D (Chang et al., 2017) AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) Stanford 2D-3D-S (Armeni et al., 2017) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Chaplot et al., 2017, Hermann & Hill et al., 2017) Visual Navigation (Zhu & Gordon et al., 2017, Savva et al., 2017, Wu et al., 2017) HoME (Brodeur et al., 2018) VirtualHome (Puig et al., 2018) AdobeIndoorNav (Mo et al., 2018) Matterport3DSim (Anderson et al., 2018)

Slide credit: Abhishek Das

>= 2017 (!)

slide-26
SLIDE 26

EmbodiedQA

SUNCG (Song et al., 2017)

Datasets Simulators Tasks

Matterport3D (Chang et al., 2017) AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) Stanford 2D-3D-S (Armeni et al., 2017) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Chaplot et al., 2017, Hermann & Hill et al., 2017) Visual Navigation (Zhu & Gordon et al., 2017, Savva et al., 2017, Wu et al., 2017) HoME (Brodeur et al., 2018) VirtualHome (Puig et al., 2018) AdobeIndoorNav (Mo et al., 2018) Matterport3DSim (Anderson et al., 2018)

Slide credit: Abhishek Das

slide-27
SLIDE 27

Datasets: Matterport3D

slide-28
SLIDE 28

Datasets: Matterport3D

slide-29
SLIDE 29

Matterport3d dataset

[Chang 3DV 2017]

10,800 panoramic views 194,400 RGB-D images of 90 building-scale scenes

Datasets: Matterport3D Datasets: Matterport3D

slide-30
SLIDE 30

30

slide-31
SLIDE 31

EmbodiedQA

SUNCG (Song et al., 2017)

Datasets Simulators Tasks

Matterport3D (Chang et al., 2017) AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) Stanford 2D-3D-S (Armeni et al., 2017) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Chaplot et al., 2017, Hermann & Hill et al., 2017) Visual Navigation (Zhu & Gordon et al., 2017, Savva et al., 2017, Wu et al., 2017) HoME (Brodeur et al., 2018) VirtualHome (Puig et al., 2018) AdobeIndoorNav (Mo et al., 2018) Matterport3DSim (Anderson et al., 2018)

Slide credit: Abhishek Das

slide-32
SLIDE 32

Example: House3D

32

[Wu et al. 2017]

Slide credit: Manolis Savva

slide-33
SLIDE 33

Example: MINOS

33

[Savva et al. 2017]

Slide credit: Manolis Savva

slide-34
SLIDE 34

Gibson [Xia et al. 2018]

34 Slide credit: Manolis Savva

slide-35
SLIDE 35

AI2 THOR [Kolve et al. 2017]

35

slide-36
SLIDE 36

DeepMind Lab [Beattie et al. 2016]

36

slide-37
SLIDE 37

EmbodiedQA

SUNCG (Song et al., 2017)

Datasets Simulators Tasks

Matterport3D (Chang et al., 2017) AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) Stanford 2D-3D-S (Armeni et al., 2017) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Chaplot et al., 2017, Hermann & Hill et al., 2017) Visual Navigation (Zhu & Gordon et al., 2017, Savva et al., 2017, Wu et al., 2017) HoME (Brodeur et al., 2018) VirtualHome (Puig et al., 2018) AdobeIndoorNav (Mo et al., 2018) Matterport3DSim (Anderson et al., 2018)

Slide credit: Abhishek Das

slide-38
SLIDE 38

Vision Language Robotics / RL

Slide credit: Dhruv Batra 38

slide-39
SLIDE 39

Vision Language Robotics / RL

Visual Navigation

Slide credit: Dhruv Batra 39

slide-40
SLIDE 40

Vision Language Robotics / RL

V&L Navigation Embodied QA

Slide credit: Dhruv Batra 40

slide-41
SLIDE 41

Vision Language Robotics / RL

Language Grounding

Slide credit: Dhruv Batra 41

slide-42
SLIDE 42
  • Create the ImageNet/COCO/VQA of Embodied AI
  • Dataset à Simulator à Task à Benchmark Challenge

Our Vision

slide-43
SLIDE 43

SUNCG (Song et al., 2017)

Datasets

Matterport3D (Chang et al., 2017) 2D-3D-S (Armeni et al., 2017)

Simulators

AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) CHALET (Yan et al., 2018) House3D (Wu et al., 2017)

Habitat Sim

Generic Dataset Support

Habitat API Habitat Platform

EmbodiedQA (Das et al., 2018)

Tasks

Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Hill et al., 2017) Visual Navigation (Zhu et al., 2017, Gupta et al., 2017)

Standardizing the Embodied Agent Stack

slide-44
SLIDE 44

SUNCG (Song et al., 2017)

Datasets

Matterport3D (Chang et al., 2017) 2D-3D-S (Armeni et al., 2017)

Simulators

AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) EmbodiedQA (Das et al., 2018)

Tasks

Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Hill et al., 2017) Visual Navigation (Zhu et al., 2017, Gupta et al., 2017)

Standardizing the Embodied Agent Stack

Julian Straub (FRL) Richard Newcombe (FRL)

slide-45
SLIDE 45

Julian Straub (FRL) Richard Newcombe (FRL)

slide-46
SLIDE 46

The Replica Dataset: A Digital Replica of Indoor Spaces [Straub et al. 2019]

FRL Surreal Team: high quality 3D reconstructions

slide-47
SLIDE 47
slide-48
SLIDE 48
slide-49
SLIDE 49
slide-50
SLIDE 50
slide-51
SLIDE 51

SUNCG (Song et al., 2017)

Datasets

Matterport3D (Chang et al., 2017) 2D-3D-S (Armeni et al., 2017)

Simulators

AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) EmbodiedQA (Das et al., 2018)

Tasks

Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Hill et al., 2017) Visual Navigation (Zhu et al., 2017, Gupta et al., 2017)

Standardizing the Embodied Agent Stack

Manolis Savva (FAIR) Yili Zhao (FAIR)

slide-52
SLIDE 52

Challenge: human vs machine needs

54

1080p @ 60Hz 256x256 @ 1000+ Hz

…

Slide credit: Manolis Savva

slide-53
SLIDE 53
slide-54
SLIDE 54

Habitat-Sim

  • Photorealistic 3D simulator

(C++ with pybind11)

  • Generic 3D dataset support

(Replica, Gibson, MP3D, +more)

  • Fast: over 1,000 FPS single-threaded

10,000 FPS multi-process (single GPU)

slide-55
SLIDE 55

57

Habitat-Sim: Datasets agnostic!

slide-56
SLIDE 56

100 200 300 400 500 600 700 800 900 1000 1100 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx10

Frames Per Second

  • ver 2x faster
slide-57
SLIDE 57

100 200 300 400 500 600 700 800 900 1000 1100 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx10

Frames Per Second

slide-58
SLIDE 58

100 200 300 400 500 600 700 800 900 1000 1100 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx10

Frames Per Second

slide-59
SLIDE 59

100 200 300 400 500 600 700 800 900 1000 1100 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx10

Frames Per Second

slide-60
SLIDE 60

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx10

Frames Per Second

slide-61
SLIDE 61

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx5

Frames Per Second

  • ver 50x faster
slide-62
SLIDE 62

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx5

Frames Per Second

1.2 Million / 180 seconds = ~7111 FPS

slide-63
SLIDE 63

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx5

Frames Per Second

slide-64
SLIDE 64

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx5

Frames Per Second

slide-65
SLIDE 65

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx5

Frames Per Second

~22,000 FPS

slide-66
SLIDE 66

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Gibson AI2 Thor MINOS House3D Habitat-Sim Habitat-Simx5

Frames Per Second

~22,000 FPS

slide-67
SLIDE 67

Why does speed matter?

Because you can now run experiments you couldn’t before.

slide-68
SLIDE 68

PointGoal Navigation

slide-69
SLIDE 69
slide-70
SLIDE 70
slide-71
SLIDE 71
slide-72
SLIDE 72

Goal

slide-73
SLIDE 73
slide-74
SLIDE 74

Agent and Model Design

slide-75
SLIDE 75

Agent and Model Design

slide-76
SLIDE 76

Agent and Model Design

  • 1.25m tall cylinder with 0.1m radius
slide-77
SLIDE 77

Agent and Model Design

  • 1.25m tall cylinder with 0.1m radius
  • Actions:
  • <stop>: Indicates the agent

believes it has completed the task

  • <forward>: Moves 0.25m forward
  • <left>, <right>: Turn 10 degrees
slide-78
SLIDE 78

Agent and Model Design

slide-79
SLIDE 79

Agent and Model Design

slide-80
SLIDE 80

Agent and Model Design

slide-81
SLIDE 81

Agent and Model Design

slide-82
SLIDE 82

Agent and Model Design

slide-83
SLIDE 83

Agent and Model Design

slide-84
SLIDE 84

Agent and Model Design

slide-85
SLIDE 85

Agent and Model Design

slide-86
SLIDE 86

Agent and Model Design

  • How do we train this agent?
slide-87
SLIDE 87

Agent and Model Design

  • How do we train this agent?
  • Both actions (they are discrete) and

the simulation are non-differential-able

slide-88
SLIDE 88

Agent and Model Design

  • How do we train this agent?
  • Both actions (they are discrete) and

the simulation are non-differential-able

  • Use reinforcement learning!
slide-89
SLIDE 89

Learning vs SLAM

slide-90
SLIDE 90
slide-91
SLIDE 91

Depth Agent (RL)

slide-92
SLIDE 92

Blind Agent (RL)

slide-93
SLIDE 93

Depth Agent (RL)

slide-94
SLIDE 94
slide-95
SLIDE 95

The agent must decide between left, right, and straight at the end of the kitchen

slide-96
SLIDE 96

Goal Sensor (GPS+Compass) indicates straight

slide-97
SLIDE 97

However, it can see there is wall straight

slide-98
SLIDE 98

and a wall on the left

slide-99
SLIDE 99

It correctly predicts that right is the direction to pursue

slide-100
SLIDE 100
slide-101
SLIDE 101

Backtracking

slide-102
SLIDE 102
slide-103
SLIDE 103
slide-104
SLIDE 104
slide-105
SLIDE 105
slide-106
SLIDE 106
slide-107
SLIDE 107
slide-108
SLIDE 108
slide-109
SLIDE 109

SUNCG (Song et al., 2017)

Datasets

Matterport3D (Chang et al., 2017) 2D-3D-S (Armeni et al., 2017)

Simulators

AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) CHALET (Yan et al., 2018) House3D (Wu et al., 2017) EmbodiedQA (Das et al., 2018)

Tasks

Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Hill et al., 2017) Visual Navigation (Zhu et al., 2017, Gupta et al., 2017)

Standardizing the Embodied Agent Stack

Abhishek Kadian (FAIR) Oleksandr Maksymets (FAIR)

slide-110
SLIDE 110

Habitat-API

  • Modular high-level Python library
  • Easy to define virtual robot configurations
  • Multiple Embodied AI tasks
  • PointGoal, ObjectGoal, VLN, EmbodiedQA
  • Baselines: Classical Robotics (SLAM),

Imitation and Reinforcement Learning

slide-111
SLIDE 111

115

slide-112
SLIDE 112

117

slide-113
SLIDE 113

PointGoal Navigation: Go to (x,y)

Slide credit: Abhishek Kadian

Habitat Challenge and Workshop @ CVPR ‘19

slide-114
SLIDE 114

Agent

Habitat Challenge and Workshop @ CVPR ‘19

slide-115
SLIDE 115

Habitat Challenge and Workshop @ CVPR ‘19

Agent

slide-116
SLIDE 116

Agent

Habitat Challenge and Workshop @ CVPR ‘19

slide-117
SLIDE 117

Agent

Habitat Challenge and Workshop @ CVPR ‘19

slide-118
SLIDE 118

Agent

Habitat Challenge and Workshop @ CVPR ‘19

slide-119
SLIDE 119

Agent

Habitat Challenge and Workshop @ CVPR ‘19

slide-120
SLIDE 120

Agent

Habitat Challenge and Workshop @ CVPR ‘19

slide-121
SLIDE 121

Habitat Challenge and Workshop @ CVPR ‘19

126

slide-122
SLIDE 122

Replica (Straub et al., 2019)

Datasets

Matterport3D (Chang et al., 2017) 2D-3D-S (Armeni et al., 2017)

Simulators

AI2-THOR (Kolve et al., 2017) MINOS (Savva et al., 2017) Gibson (Zamir et al., 2018) CHALET (Yan et al., 2018) House3D (Wu et al., 2017)

Habitat Sim

Generic Dataset Support

Habitat API Habitat Platform

EmbodiedQA (Das et al., 2018)

Tasks

Interactive QA (Gordon et al., 2018) Vision-Language Navigation (Anderson et al., 2018) Language grounding (Hill et al., 2017) Visual Navigation (Zhu et al., 2017, Gupta et al., 2017)

Standardizing the Embodied AI Stack

127

slide-123
SLIDE 123
  • Create the ImageNet/COCO/VQA of Embodied AI
  • Dataset à Simulator à Task à Benchmark Challenge

Our Vision

slide-124
SLIDE 124

ICCV ‘19

[Best Paper Award Nominee]

slide-125
SLIDE 125

130

Decentralized Distributed PPO: Mastering PointGoal Navigation

S t e f a n L e e D e v i P a r i k h D h r u v B a t r a I r f a n E s s a E r i k W i j m a n s A b h i s h e k K a d i a n M a n

  • l

i s S a v v a A r i M

  • r

c

  • s
slide-126
SLIDE 126

Decentralized Distributed PPO

131

Gradient Worker (simulation + RL) Gradient Gradient Gradient Worker (simulation + RL) Worker (simulation + RL) Worker (simulation + RL)

slide-127
SLIDE 127

132

Decentralized Distributed PPO

256 196

slide-128
SLIDE 128

We utilize DD-PPO to train an agent for 2.5 Billion steps of experience

slide-129
SLIDE 129
  • ver 180 days of GPU-time training
slide-130
SLIDE 130

in under 3 days of wall-clock time

slide-131
SLIDE 131

Visual Turing Test

slide-132
SLIDE 132

Option 1 Option 2

slide-133
SLIDE 133

Option 1

slide-134
SLIDE 134

Option 1 Option 2

slide-135
SLIDE 135

Option 2

slide-136
SLIDE 136

Option 1 Option 2

slide-137
SLIDE 137

Learned Agent Shortest Path Oracle

slide-138
SLIDE 138
slide-139
SLIDE 139

Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation

Abhishek Kadian* Erik Wijmans Dhruv Batra Joanne Truong* Aaron Gokasalan Alex Clegg Manolis Savva Stefan Lee Sonia Chernova

slide-140
SLIDE 140

Does progress in simulation translate to progress on real robots?

(In the context of embodied navigation)

slide-141
SLIDE 141

146

slide-142
SLIDE 142

Georgia Tech CODA Building Scans

  • https://my.matterport.com/show/?m=yZVvKaJZghh

147

slide-143
SLIDE 143
slide-144
SLIDE 144

149

slide-145
SLIDE 145
slide-146
SLIDE 146

Sim-vs-Real Correlation Coefficient (SRCC)

slide-147
SLIDE 147

Sim-vs-Real Correlation Coefficient (SRCC)

slide-148
SLIDE 148

Cheating by Sliding

153

St St+1 0.28m 0.43m

SPL Path Sliding path

slide-149
SLIDE 149

Sim-vs-Real Correlation Coefficient (SRCC)

slide-150
SLIDE 150

155

slide-151
SLIDE 151

Import Objects

156

  • Why?
  • Egocentric CV
  • Domain

randomization

slide-152
SLIDE 152

Physics

157

  • Why?
  • Intuitive physics
  • Robotics,

sim2real

  • Egocentric CV
slide-153
SLIDE 153

Habitat in Browser

158

  • Why?
  • Grounded Dialog via

2-player data collection

  • Demo:
  • https://aihabitat.org/iccv2019-

demo/

slide-154
SLIDE 154

Plans

  • Full support for object interaction + physics
  • Physics is slow! Need to spend time optimizing.
  • Articulated robot integration (URDF)
  • Humans-as-agents (Web + VR)
  • CVPR20 Challenge
  • PointGoal Navigation w/ GPS+Compass
  • ObjectGoal Navigation

159

slide-155
SLIDE 155

Vladlen Koltun5 Abhishek Kadian1* Oleksandr Maksymets1* Jia Liu1 Manolis Savva1,4* Erik Wijmans1,2,3 Bhavana Jain1 Yili Zhao1 Julian Straub2 Jitendra Malik1,6 Devi Parikh1,3 Dhruv Batra 1,3 1 2 3 4 5 6 * denotes equal contribution

Habitat Core Team

slide-156
SLIDE 156

Marcus Rohrbach Georgia Gkioxari Xinlei Chen Amanpreeet Singh Saurabh Gupta Leo Guibas Or Litany Richard Newcombe Steven Lovegrove James Hillis Michael Shvartsman Naga Venkata Medathati

slide-157
SLIDE 157

In collaboration with Devi Parikh’s and my groups at Georgia Tech

slide-158
SLIDE 158

Internet AI à Embodied AI

Thank you