A 240x180 120dB 10mW 12us latency sparse output vision sensor for - - PowerPoint PPT Presentation

▶
a 240x180 120db 10mw 12us latency sparse output vision
SMART_READER_LITE
LIVE PREVIEW

A 240x180 120dB 10mW 12us latency sparse output vision sensor for - - PowerPoint PPT Presentation

Nov 23, 2022 22 likes •206 views

A 240x180 120dB 10mW 12us latency sparse output vision sensor for mobile applications Raphael Berner, Christian Brandli, Minhao Yang, Shih Chii Liu and Tobi Delbruck Inst. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland

slide-1
SLIDE 1

A 240x180 120dB 10mW 12us‐latency sparse

  • utput vision sensor for mobile applications

Raphael Berner, Christian Brandli, Minhao Yang, Shih‐Chii Liu and Tobi Delbruck

  • Inst. of Neuroinformatics,

University of Zurich and ETH Zurich, Switzerland

1

slide-2
SLIDE 2

Machine Vision for Mobile Applications

  • ENERGY matters because it is limited
  • LATENCY is crucial for real time interaction

With conventional imager

  • Low frame rate: low energy / high latency
  • High frame rate: low latency / high energy

Most of the time and energy spent on redundant information in and between frames WE NEED SMARTER WAYS OF SENSING & PROCESSING

2

slide-3
SLIDE 3

3

Simplified Dynamic Vision Sensor (DVS) pixel architecture

Lichtsteiner et al. JSSC 2009

slide-4
SLIDE 4

Example DVS application ‐ RoboGoalie

4

slide-5
SLIDE 5

Achieves 550fps (equivalent) at 4% CPU load

Example DVS application – RoboGoalie

Achieves 550Hz update rate at 4% laptop CPU load

5

Delbruck et al., 2009

slide-6
SLIDE 6

The apsDVS chip adds an intensity readout to the DVS

6

logI

photoreceptor

I

Intensity reset Intensity value

Event reset

ON OFF change amplifier (bipolar cells) comparators (ganglion cells) threshold

Change events

I

slide-7
SLIDE 7

apsDVS pixel

(aps=Active Pixel Sensor)

7

slide-8
SLIDE 8

The apsDVS chip SBRET10

8

  • 0.18um 1P6M CMOS
  • MiM capacitors
  • 240x180 pixels
  • 22% fill factor
  • 12mW power

Column APS readout Row APS readout

slide-9
SLIDE 9

9

slide-10
SLIDE 10

10

slide-11
SLIDE 11

11

slide-12
SLIDE 12

The apsDVS can enhance machine vision in several ways

  • Low APS frame rate  Low power
  • DVS output  Low latency, sparse data

Slide 12

Combined advantages:

  • 1. DVS delivers motion features for

free

  • 2. Continuous tracking solves

correspondence problem

  • 3. DVS can be used to control frame

rate or ROI

slide-13
SLIDE 13

The apsDVS in Comparison

This work (apsDVS) ATIS (Posch et al.) Intensity readout APS Time domain Image Dynamic Range 57 dB 125 dB Image FPN 1 % <0.25 % Pixel size [um2] 18.5 x 18.5 30 x 30 Fill factor 22% 15% Pixel complexity 44 FET, 2 C, 1 PD 77 FET, 2 C, 2 PD Fixed integration time YES NO Power Consumption ~10mW (240x180) ~100mW (304x240)

13

slide-14
SLIDE 14

Thank you

14

  • Prof. Tobi

Delbruck

  • Dr. habil.

Shih‐Chii Liu

  • Dr. Raphael

Berner Minhao Yang Funding Organizations University of Zurich ETH Zurich EU 7th framework program (SeeBetter) Swiss National Science Foundation (NCCR Robotics)

sensors.ini.uzh.ch

Christian Brandli

slide-15
SLIDE 15

SBRET10 pixel layout

18.5um

15

22% fill factor

slide-16
SLIDE 16

Technology

UMC 180 6M 1P

Chip size

5x5mm

APS-DVS array size

240x180

Pixel size

18.5 x18.5 um2

Supply voltage(s)

3.3V/1.8V

Pixel architecture

apsDVS, 44T+2C.

Fill factor

22%

Power consumption

7.4mW low activity 13.5mW high activity

DVS event threshold limit

12% contrast

DVS illumination operating range

120dB (down to 0.1lux)

DVS latency

12us

APS readout frame rate

40 FPS

APS FPN (midlevel)

1% of full signal swing

Readout Noise

<2mV

Output Voltage Range

1.1V

APS Dynamic Range

~55dB

Design Measured

16

slide-17
SLIDE 17

17

Simplified Dynamic Vision Sensor (DVS) pixel architecture

logI A random variation

Lichtsteiner et al. JSSC 2009

slide-18
SLIDE 18

18