Cars at GM Efrat Rosenman, Ph.D. Head of Cognitive Driving Group - - PowerPoint PPT Presentation

Leveraging AI for Self-Driving Cars at GM

Efrat Rosenman, Ph.D. Head of Cognitive Driving Group General Motors – Advanced Technical Center, Israel

Agenda

• The vision
• From ADAS (Advance Driving Assistance Systems) to AV (Autonomous Vehicles)
• AI for Self-Driving cars
• ADAS, AV and in-between
• Summary

10/19/2017 General Motors 2

The Vision

3

?

• Mobility – one of the most significant revolutions of modern times
• Self-driving cars will take mobility to a completely new phase…

”Zero Crashes, Zero Emissions, Zero Congestion” (Mary Barra, GM CEO)

10/19/2017 General Motors

The Vision

4

Increase Mobility: anywhere, anytime Increase Car Sharing & Reduce Road Capacity and Parking needs

10/19/2017 General Motors

Increase Safety Increase Productivity

5

From ADAS to AV

10/19/2017 General Motors

L5:Full automation Level 4: High automation Level 3: Conditional automation Level 2: Partial automation Level 1: Driver assistance Level 0: Driver in full control

Info, warnings Cruise control, lane position Traffic jam assist Anywhere, anytime Fully autonomous specific scenarios Highway driving (driver takes control with notice)

6

From ADAS to AV

• Will incremental steps get us to the

top of this pyramid?

10/19/2017 General Motors

10/19/2017 General Motors 7

Sensing Mapping Perception Decision Making Control

Components of self driving cars

Components of self driving cars

AI AGENT serves as the “brain” of the car

10/19/2017 General Motors 8

Perception Decision Making Control

AI for Self-Driving Cars

9

AI in Perception

• Unsupervised learning
• Finding structure in point clouds
• Feature learning
• Supervised learning
• Object detection
• 2D object recognition (Classification)
• 3D scene understanding and modeling (3D objects

pose)

• Semantic segmentation (boundaries of objects, free

space)

10/19/2017 General Motors 11

AI in Perception - E2E trend

• Classification:
• Scene understanding:
• Perception:

10/19/2017 General Motors 12

Pixels Key Points Model SIFT features Labels Sensors 2D object detection Pose estimation Depth estimation 3D World state Pixels Segmentation Contextual relations Object detection Scene description

AI in Perception - E2E trend

• Classification:
• Scene understanding:
• Perception:

10/19/2017 General Motors 13

Pixels Key Points Model SIFT features Labels Sensors 2D object detection Pose estimation Depth estimation 3D World state Pixels Segmentation Contextual relations Object detection Scene description

DNN DNN DNN

Towards E2E: Sensors Fusion

10/19/2017 General Motors 14

• All sensors

contribute

• Enables learning
• f complex

dependencies “optimally”

• Sparse Vs. dense

sensors

• Larger models,

harder to learn

• Utilizes domain

knowledge

• Model is

explainable

• Based on tailored

rules

• Suboptimal

performance

Low Level: raw data combined in input stage High Level: tailored hierarchy between sensors

Towards E2E: Multi-Task Learning

• Most our outputs are inter related
• Objects, free space, lanes, etc.
• Cross regularization allows reaching a better local minima
• TPT
• Major parts of the Deep Net are used for multiple tasks
• Data Efficiency

10/19/2017 General Motors 15

Mask R-CNN Facebook AI Research (FAIR); Apr 2017

What about data?

16

Automatic Data Annotation

• Data is the key contributor to perception

accuracy – With no visible saturation

• How can we create annotated data
• Manual annotation – Expensive and inaccurate
• Automatically

10/19/2017 General Motors 17

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era, Google 2017

Automatic Data Annotation

• Technology
• High end sensors (Lidar, IMU, etc.)
• High accuracy detectors (on behalf of computation time)

10/19/2017 General Motors 18

Example – AGT for StixelNet

• StixelNet - Monocular obstacle detection
• Based on stixel representation
• Identify road free space
• Ground truthing is based on Lidar

10/19/2017 General Motors 19

Dan Levi, Noa Garnett, Ethan Fetaya. StixelNet : A Deep Convolutional Network for Obstacle Detection and Road Segmentation. In BMVC 2015. Lidar (Velodyne HDL32) is used to identify

• bstacle on each stixel in the image

[Badino, Franke, Pfeiffer 2009]

Compact, local representation

Is Perception “solved”?

• Challenge of Cost
• Sensors
• Mapping
• Computation
• Challenge of false positive & false negative
• Data uncertainty (noise)
• Model uncertainty (confidence)

10/19/2017 General Motors 20

Label: Cyclist RGB: Pedestrian (0.56)

Decision Making

10/19/2017 General Motors 21

Perception Decision Making Control

Learning Decision Making

Decision Making cannot learn from static examples Need interactive domain

• > Reinforcement Learning (RL)

RL has seen some major successes in the recent years:

10/19/2017 General Motors 22 Go

Poker

Autonomous Helicopter Flight

Atari

RL challenges in Self-Driving agents

• Learn to act in a very high dimensional space
• Plan sequences of driving actions
• Predict long term behaviors of other road users
• Few sec
• Complicated situations
• Negotiate with other road user
• Guarantee safety

10/19/2017 General Motors 23

Simulation

• Advanced simulations are required
• Multi-agent
• Various conditions
• Focus on “interesting miles”
• Drive billions of “virtual miles” (fuzzing)

“Any system that works for self driving cars will be a combination of more than 99 percent simulation.. plus some on-road testing.” [Huei Peng director of Mcity, the University of Michigan’s autonomous- and connected- vehicle lab]

10/19/2017 General Motors 24

Waymo simulation: https://www.engadget.com/2017/09/11/waymo-self- driving-car-simulator-intersection/

Safety Guarantees - From ADAS to AV

Will incremental steps get us to the top of this pyramid? The technological heart is different in kind

10/19/2017

What’s the difference?

• For ADAS – Safety guarantee is based on the driver
• For autonomous – Safety guarantee should come from the system itself

10/19/2017 General Motors 26

Example: Highway Driving in Super Cruise™

10/19/2017 General Motors 27

The 2018 Cadillac CT6 will feature Super Cruise™ - a hands-free driving technology for the highway It includes an Exclusive driver attention system to support safe operation

Safe Driving for level 4/5

• System should handle 100% of the cases
• Redundancy requires at all levels
• Sensing
• Algorithm
• Computing
• Control
• Fallback strategies
• Guarantee of Safety is a must to the acceptance of AV
• Statistical data-driven approach [miles-per-interrupts] requires driving billions of

miles to validate an agent

• Should be repeated with every SW version
• Need safety constrains (rule-based/model-based)

10/19/2017 General Motors 28

Summary

• Advances in AI are key to success of self-

driving cars

• AI-based features can bring ADAS to a

new level in terms of accidence avoidance, productivity gain and saving in human lives

• Level 4/5 AV should be a parallel effort

focus on redundancy and safety constrains

10/19/2017 General Motors 29

GM Advanced Technical Center in Israel (ATCI)

Thank you

10/19/2017 General Motors 31