Fault detection and mitigation from uninterpreted data of robotic - - PowerPoint PPT Presentation

1

Andrea Censi* Richard M. Murray

Fault detection and mitigation from uninterpreted data

• f robotic sensorimotor cascades

* 5th year graduate student, defending in a few weeks, currently looking for a job

Magnus Håkansson#

# spent a California summer down in the basement collecting data

LTH

• 1. Robust agents need to learn/verify

the models that they use.

• 2. Bootstrapping: learning low-level

models for robotic sensorimotor cascades

• 3. “Sensorimotor faults” can be defined

independently of a nominal model.

• 4. “Orthogonal” integration within

a traditional robot architecture Outline Caltech Caltech

2

miscalibration interference sabotage hardware failures

Sutton’s “Verification principle”: An intelligent system can create and maintain knowledge

• nly to the extent that it can verify that knowledge itself.
• Models used by robotic agents

are often implicit and never verified.

• How to predict the unpredictable?

3

p(bird) p(image|bird droppings) p(image|¬ bird droppings)

accuracy Bayesian filters max

accuracy * (robustness to assumptions) (design effort) * computation

a simpler approach based on low-level sensorimotor models max

• Fault detection is an estimation problem,

but what should be the design goal?

✓ the optimal solution ✓ fewer assumptions

that could be violated

✓ “surprising” effects

with reduced design effort

× sensitive to prior

assumptions

× needs lots of design effort × not optimal

4

• Bootstrapping = starting

from no prior information

• All robots << all systems
• Sensors are more similar

than what you would think.

• Models for low-level

sensorimotor learning

• 1. Robust agents need to learn/verify

the models that they use.

• 2. Bootstrapping: learning low-level

models for robotic sensorimotor cascades

• 3. “Sensorimotor faults” can be defined

independently of a nominal model.

• 4. “Orthogonal” integration within

a traditional robot architecture Outline

5

agent

physical environment

model

physical environment

• bservations

commands robot

• Usually agents start with a model

(implicit or explicit) of the robot and the physical environment.

“world” or “sensorimotor cascade”

• In the bootstrapping scenario,

the agent has no prior knowledge about its sensors, its actuators, and the external environment.

6

bootstrapping agent

unknown sensor(s)

external environment

unknown actuator(s)

uninterpreted

• bservations

uninterpreted commands

?

“sensels”: pixels, range readings, …

✓ zero assumptions

that can be violated

“world” or “sensorimotor cascade”

• The “set of all robots” is much smaller

than the set of all dynamical systems.

7

all robots all dynamical systems

• “Canonical robotic sensors”

have similar dynamics at the sensel level.

8

camera range-finder field-sampler

exactly bilinear bilinear, up to a nonlinearity bilinear, up to a hidden state y(s) = intensity at point s y(s) = distance in direction s y(s) = luminance in direction s [ICRA’11]

v (t), ω(t): kinematic velocities

9

• What are the simplest models

that we can use to detect faults?

too simple for discriminating faults keeps priors

• n bird migration

patterns instantaneous models

• f how the commands u(t)

determine y(t+dt) all robots all dynamical systems today more accurate simpler

10

simpler less assumptions more data required

• Several classes of models

for low-level sensorimotor data have been studied.

more complex more assumptions less data required

BGDS

Models bilinear flows of the observations space: [IROS’11]

BDS

Models a bilinear relation between ẏ, y, and u: [ICRA’11] Models diffeomorphisms of the observations space.

DDS

Paper ThA01.2, Thursday 08:45−09:00, Meeting Room 1 (Mini-sota) all robots all dynamical systems

11

• Traditional fault detection

requires a nominal model

• “Sensorimotor faults”:

“faulty”= uninformative = unpredictable

• Application to camera data
• 1. Robust agents need to learn/verify

the models that they use.

• 2. Bootstrapping: learning low-level

models for robotic sensorimotor cascades

• 3. “Sensorimotor faults” can be defined

independently of a nominal model.

• 4. “Orthogonal” integration within

a traditional robot architecture Outline

• 1. Acquire a nominal model

for the healthy system.

• 2. Check the observed data

against the model.

• 3. Do something if the data

does not support the model.

12

given identified / learned residuals likelihood ratios

• Traditional fault detection

requires a nominal model. Traditional fault detection shut off the reactor ...

13

... ua ya ub ... ... yb y u

“faulty” sensels “faulty” actuators

• bservations

commands

a dead pixel an occluded pixel a random pixel

• “Sensorimotor faults” can be defined

without reference to a nominal model. a “faulty” sensel yi does not give information about u. a “faulty” actuator uj does not give information about y.

DEF: examples:

14

=

correlation between the observed ẏ and the prediction of the learned model

assuming additive noise, for a limited class of models, and a particular distance

(“predictability”) ... ua ya ub ... ... yb y u

“faulty” sensels

• bservations

commands

“usefulness” of the i-th sensel distance between p(ut|y:t) and p(ut|y:t - {yi})

• “Sensorimotor faults” can be defined

without reference to a nominal model. a “faulty” sensel yi does not give information about u.

DEF:

15

• 1. Acquire a nominal model

for the healthy system.

• 2. Check the observed data

against the model.

• 3. ...

Traditional fault detection

• 1. Learn from the running system

from sensorimotor data (u(t), y(t)) (for some model class C).

• 2. Compute the sensel “usefulness”:

distance between p(ut|y:t, C) and p(ut|y:t - {yi}, C).

• 3. “Useless” sensels are marked

as “faulty”.

• 4. ...

“Sensorimotor faults” detection

16

a second camera looks through a mirror ER1

y(t)

• Differential-drive robot

equipped with two cameras Agent fits the model: [IROS’11]

• 1. Learn from the running system

from sensorimotor data (u(t), y(t)) (for some model class C).

• 2. Compute the sensel “usefulness”:

distance between p(ut|y:t, C) and p(ut|y:t - {yi}, C).

• 3. “Useless” sensels are marked

as “faulty”.

• 4. ...

“Sensorimotor faults” detection

u(t): linear/angular velocities

17

useless / unpredictable (“faulty”) useful / predictable

Usefulness / predictability

robot frame and wheels border between images

• “Faults” detected:
• self-occlusions
• border between images
• sampling limitations

(points at infinity)

✓ with no prior knowledge

about the robot

✓ with minimal computation

(learning/inference)

a second camera looks through a mirror ER1

y(t) u(t): linear/angular velocities

• No “faulty” actuators in this example,

but the instantaneous commands anomaly signal can detect:

18

• Violations of planarity assumption
• Delays in commands execution
• Synchronization issues

robot drives

• ver a bump

… and off the bump

time (s)

Instantaneous anomaly signal for u(t)

synchronization issues delays delays

✓ with no prior knowledge

about the robot

✓ with minimal computation

(learning/inference)

19

• How to integrate

these techniques into an existing architecture, with minimal changes?

• Application

to range-finder data

• 1. Robust agents need to learn/verify

the models that they use.

• 2. Bootstrapping: learning low-level

models for robotic sensorimotor cascades

• 3. “Sensorimotor faults” can be defined

independently of a nominal model.

• 4. “Orthogonal” integration within

a traditional robot architecture Outline

20

traditional agent

robot

sanitized y

y glue u u y robot

traditional agent

• A bootstrapping agent computes the sensel usefulness.
• A glue component marks useless sensels as invalid.
• ...with an adaptive threshold on the basis
• f a “panic” signal sent by the traditional agent (no tuning needed)
• How to integrate these techniques

in a traditional architecture with minimal changes?

y u bootstrapping agent

sensel usefulness panic signal

21

antenna antenna sensel #

panic mode normal operation

Sensel usefulness / predictability

• Landroid robot equipped with a

range finger, partially occluded by (vibrating) WiFi antennas threshold α sensel s sensels marked invalid

explorer

y’ y glue u Landroid y u

bootstrapping agent sensel usefulness panic signal

antennas range finder u(t) y(s, t) α

21

antenna antenna sensel #

panic mode normal operation

Sensel usefulness / predictability

• Landroid robot equipped with a

range finger, partially occluded by (vibrating) WiFi antennas threshold α sensel s sensels marked invalid

explorer

y’ y glue u Landroid y u

bootstrapping agent sensel usefulness panic signal

antennas range finder u(t) y(s, t) α

Summary

• “Sensorimotor faults” can be defined

with no reference to a nominal model.

• Many faults can be identified

with simple instantaneous sensorimotor models.

• “Orthogonal” integration

in a traditional architecture

22

sensel usefulness / predictability

• riginal

ROS graph modified ROS graph learned models world magic box logs

Future work: engineering issues

• Define proper semantics (e.g., “panic signal”)
• Can we automate the integration?

“robustified” system added components

antenna antenna

extra slides

23

• Prediction: given y and u,

predict the instantaneous change ẏ.

24

linear/angular velocities

agent

M

y(t) learned model

u(t) y(t)

predicted ẏ(t)

Rawseeds data

25

• Anomaly detection

Assuming a static world, failures of prediction are likely to be other agents

• r equally interesting things.
• bserved ẏ(t)

predicted ẏ(t) “anomaly”

changes due to self-motion are anticipated

• ther agents

and occlusions flicker