Following Dirt Roads at Night-Time Sensors and Features for Lane - - PowerPoint PPT Presentation

▶
following dirt roads at night time
SMART_READER_LITE
LIVE PREVIEW

Following Dirt Roads at Night-Time Sensors and Features for Lane - - PowerPoint PPT Presentation

Oct 04, 2023 416 likes •718 views

Following Dirt Roads at Night-Time Sensors and Features for Lane Recognition and Tracking Sebastian F. X. Bayerl Thorsten Luettel Hans-Joachim Wuensche Autonomous Systems Technology (TAS) Department of Aerospace Engineering University of the

slide-1
SLIDE 1

Following Dirt Roads at Night-Time

Sebastian F. X. Bayerl Thorsten Luettel Hans-Joachim Wuensche

Autonomous Systems Technology (TAS) Department of Aerospace Engineering University of the Bundeswehr Munich

2015-09-28

Sensors and Features for Lane Recognition and Tracking

slide-2
SLIDE 2

Motivation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 2

  • Recognition of ego lane is prerequisite for many ADAS
  • Camera based methods usually
  • Most methods valid for well marked roads at night
  • Little work done for unmarked rural roads at night

Color Color gradient Color gradient Color

slide-3
SLIDE 3

Hardware

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 3

  • Stock color camera
  • Color of surface

Integration time: 30ms Integration time: 100ms

slide-4
SLIDE 4

Hardware

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 4

  • Stock color camera
  • Color Night Vision (CNV) camera
  • Color of Surface

Integration time: 50ms

slide-5
SLIDE 5

Hardware

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 5

  • Stock color camera
  • Color Night Vision (CNV) camera
  • Near Infrared (NIR) camera
  • Reflectivity

Paved road Forest road

slide-6
SLIDE 6

Hardware

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 6

  • Stock color camera
  • Color Night Vision (CNV) camera
  • Near Infrared (NIR) camera
  • Far Infrared (FIR) camera
  • Temperature

(12.5°C - 15.0°C)

slide-7
SLIDE 7

Hardware

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 7

  • Velodyne LiDAR
  • 3D measurements
  • NIR reflectivity
slide-8
SLIDE 8

Fusion and Accumulation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 8

  • Fusion and accumulation into a Local Terrain Map
  • Multiple layers
  • Heights
  • Slopes
  • Obstacles
  • NIR Reflectivity
  • Color
  • Temperature
  • …
  • 1. Update robot position
  • 2. Update Velodyne
  • 3. Update camera layers

update step

slide-9
SLIDE 9

Features

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 9

  • Color Features
  • Gradient at lane boundary
  • Saturation channel
  • Ratio of green color channel
  • …

Green color ratio g / ( r+g+b ) Color saturation Color with obstacles (red) Color gradient

slide-10
SLIDE 10

Features

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 10

  • Temperature
  • Transitions at lane boundary
  • Temperature back projection

Temperature gradient Temperature back projection Surface temperature 15°C – 17°C

slide-11
SLIDE 11

Features

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 11

  • 3D / LiDAR
  • Obstacle probability
  • Heights
  • Slopes

Heights Obstacles (red)

cross section of a valley cross section of a hill

slide-12
SLIDE 12

Features - Evaluation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 12

  • 10.000 positive and negative samples for each feature
  • Different road scenes: paved, unpaved, dirt, forest, …
  • Different seasons: summer, winter, …
  • Different weather conditions: sun, rain, snow
  • Different day times!
  • Receiver Operating Characteristic (ROC)

Color gradient

Color gradient

False positive rate True positive rate

slide-13
SLIDE 13

Features - Evaluation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 13

  • Color
  • Degradation of color from day to night!

Color green ratio

False positive rate

Color gradient

False positive rate True positive rate True positive rate

slide-14
SLIDE 14

Features - Evaluation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 14

  • Temperature
  • E.g. temperature gradient
  • Temperature more informative without illumination!
slide-15
SLIDE 15

Features - Evaluation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 15

  • Thermal limitations

road covered by leaves CNV Camera FIR Camera

False positive rate True positive rate ROC of temperature back projection

slide-16
SLIDE 16

Features - Evaluation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 16

  • 3D/LiDAR
  • e.g. obstacle probability
  • No dependency to illumination
  • No „stand alone“ feature

PDF ROC

slide-17
SLIDE 17

Features - Evaluation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 17

  • Benefit of night sensors:
  • Classifier with full feature capability (𝐷𝐵)
  • Classifier with reduced feature capability (𝑫𝑪)

False positive rate True positive rate

slide-18
SLIDE 18

Tracking

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 18

  • Geometry
  • Clothoid(s) for modelling road net

𝐲lane = d Ψ c0 c1 w T

d

𝛀

w

𝐲cross = p𝑦 p𝑧 𝐲brach1 𝐲brach2 …

T

𝐲branch = 𝛺 c0 c1 w T

slide-19
SLIDE 19

Tracking

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 19

  • Particle Filter
  • Correction step:

▫ Project particles into Local Terrain Map ▫ Calculate mean feature values 𝐺

𝑔 for all particles (state vector 𝑦𝑞)

▫ Naive Bayes Classification result as particle weight

𝑥𝑞 𝐺

1, … , 𝐺 𝑜 = 𝑔=1,…,𝑜

𝑞𝑔(𝐺

𝑔|𝑦𝑞)

▫ State Vector and Covariance from weighted mean

  • Prediction: model road as static object moving with inverse robot motion
slide-20
SLIDE 20

Movie

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 20

slide-21
SLIDE 21

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 21

Thank you for your attention! Questions?

slide-22
SLIDE 22

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 22

  • Motivation
  • Sensors
  • Hardware
  • Fusion and Accumulation
  • Features
  • Road Features
  • Evaluation
  • Perception
  • Particle Filter
  • Limitations
  • Movie
slide-23
SLIDE 23

Hardware

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 23

  • Robot: Mucar-3
slide-24
SLIDE 24

Features

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 24

  • Color Features
  • Edges at lane boundary
  • Saturation channel
  • Ratio of green color channel
  • …

Green color ratio Color saturation Color edge intensity Color with obstacles (red)

slide-25
SLIDE 25

Limitation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 25

  • Limitations
  • 1. Sun scene (14.5 °C – 17.0 °C)
  • 2. Rain scene (14.5 °C – 16.0 °C)
  • 3. Forest scene (14.5 °C – 17.0 °C)
  • 4. Forest scene (15.0 °C – 17.0 °C)
slide-26
SLIDE 26

Limitation

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 26

  • Limitations
  • 6. Snow scene (3.0 °C – 4.0 °C)
  • 5. Foggy winter scene (10.0 °C – 10.5 °C)
  • 7. Winter scene (8.0 °C – 10.0 °C)
slide-27
SLIDE 27

Tracking

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 27

  • Thermal limitations
  • Set of features provides robustness:
  • At least one significant feature necessary

ROC of thermal edge direction

slide-28
SLIDE 28

Tracking

2015-09-28 PPNIV 2015 - Sebastian Bayerl: Following Dirt Roads at Night Time 28

  • Geometry
  • Clothoid(s) for modelling road net

𝐲lane = d Ψ c0 c1 w T

  • Using rough information of road map to switch between road and crossroad

▫ Distance to crossroad ▫ Direction of outgoing branch

d

𝛀

w

𝐲cross = p𝑦 p𝑧 𝐲brach1 𝐲brach2 …

T

𝐲branch = 𝛺 c0 c1 w T