Mode Estimation of Probabilistic Hybrid Systems Michael Hofbaur - - PowerPoint PPT Presentation

Mode Estimation of Probabilistic Hybrid Systems

Michael Hofbaur1&2 & Brian C. Williams1

1) Artificial Intelligence & Space Systems Laboratories

MIT, USA

2) Department of Automatic Control,

TU-Graz, Austria

2

Motivation

• Highly complex artifact
• Long autonomous operation in a harsh environment
• Robust operation – fault tolerance

Monitoring and diagnosis capabilities are critical for building highly autonomous artifacts that can operate robustly in harsh environments of a long period of time. Advanced Life Support System

• BIO-Plex

3

Overview

• Probabilistic Hybrid Automata

– Model & Execution

• Concurrent Probabilistic Hybrid Automata
• Hybrid Estimation

– Overview – intuitively & filtering background – Problem Formulation – A* Formulation

• Example
• Discussion & Conclusion

4

why Hybrid Mode/State Estimation?

Monitoring and Diagnosis has to track the system’s behavior along both its continuous state changes and its discrete mode changes and their system-wide interaction.

• operational modes
• failure modes
• estimation and filtering of

continuously valued variables

600 700 800 900 1000 1100 1200 1300 1400 400 500 600 700 800 900 1000 1100 1200 time (minutes) CO 2 c oncent ration (p pm)

crew requests entry to plant growth chamber crew enters chamber lighting fault crew leaves chamber

5

Hybrid Model

A concurrent Probabilistic Hybrid Automata (cPHA) is a hidden Markov model, encoded as a set of components with modes that exhibit a continuously valued dynamical behavior that is expressed by difference / algebraic equations.

concurrent Probabilistic Hybrid Automata (cPHA)

mc7 mc2 mc4 mc5 mc8 mc1 tc1 tc2 tc3 tc5 tc6 tc9 mc3 tc4 mc6 tc7 tc8 tc10

chamber control servo valve

mr1 mr2 mr3 mr5 mr6 mr4 tr1 tr2 tr4 tr3

ml1 ml2 ml4 ml3 tl1 tl2

...

gas sensor

hybrid model

6

Probabilistic Hybrid Automata

servo valve

mr1 mr2 mr3 mr5 mr6 mr4 tr1 tr2 tr4 tr3

Probabilistic Hybrid Automata xd mode (discrete state) with domain Xd xc continuous state with domain ud discrete command with domain Ud uc continuous command with domain yc continuous output with domain F ................. discrete-time dynamics for each mode (sampling-period Ts) T ................. guarded probabilistic transitions between modes

, , , , , ,

d d s

F T X U T x w

...{ } ........

d c

x x x

• ...

...

d c c

w u u y

• n
• i

m

• m

7

Mode / State Transition

Discrete mode changes and continuously valued evolution of the state variables take place at two different rates: a) continuous evolution is captured at the sampling-rate b) probabilistic mode changes take place instantly

• ,( )

,( 1) ,( 1) ,( 1) ,( ) ,( ) ,( ) ,( ) ( ) ( 1)

, , ' , ( , , ),

c k c k c k d k c k c k c k d k k k s

f x g x t t T x x u y x u

• ,( )

,( )

( , ) ,( ) ,( ) ,( ) ,( )

, ' , '

c k d k

guard d k c k d k c k

x x

x u

x x

• m2

m1 m3 C11 Pt11 Pt12

8

Mode / State Transition

State transition: no transition is triggered (x’d,(k) = xd,(k+1)) and time proceeds for

• ne sampling period: t(k+1)= t(k) + Ts. . The evolution of the continuous state

x’c,(k) → xc,(k+1) is captured by the discrete-time dynamic model that holds for x’d,(k).

,( ) ,( )

' '

d k c k

x x

,( ) ,( ) d k c k

x x

Ti

,( 1) ,( 1) d k c k

x x

• t(k)

t’(k) t(k+1)

Fj

Mode Transition State Transition Mode transition: time proceeds only infinitesimally t’(k)= t(k)+ ε so that the evolution of the continuous state xc,(k) → x’c,(k) can be neglected: x’c,(k)= xc,(k)

9

concurrent Probabilistic Hybrid Automata

PHA1 PHA2 PHA3 PHA4 continuous input uci

• utput / observed

variable yci (cont.) PHA component internal variable discrete input udj

• concurrent PHA components are connected to inputs (continuous and discrete)

and outputs of the cPHA and interconnected by internal variables.

• observed variables = internal variable + additive Gaussian noise

10

PHA1 PHA2 PHA3 PHA4

Probabilistic Hybrid Automata

Concurrent Probabilistic Hybrid Automata A ................ set of PHAs continuous and discrete command variables yc ................. observed continuous variables vs, vo ............ state disturbances and sensor noise inputs characterized by Nx, Ny

, , , , , ,

c s

• x

y

A N N u y v v

... ...

d c

u u u

• 1

2 1 2 ,( ) ,( 1) ,( 1) ,( 1) ,( 1) ( ) ,( ) ,( ) ,( ) ,( )

... { , ,..., } , , ' ( , , )

c c c cl d d d dl c k c k c k d k s k k c k c k d k

• k

x x x f g x x x x x x x u x v y x u x v

11

Roadmap

• Probabilistic Hybrid Automata

– Model & Execution

• Concurrent Probabilistic Hybrid Automata
• Hybrid Estimation

– Overview – intuitively & filtering background – Problem Formulation – A* Formulation

• Example
• Discussion & Conclusion

12

Hybrid Mode / State Estimation

Task Overview:

Hybrid Estimation Problem: Given a cPHA model for a system, a sequence

• f observations and the history of the control inputs generate the leading set
• f most likely states at time-step k

PHA1 PHA2 PHA3 PHA4

continuous input uci

• utput / observed

variable yci (cont.) discrete input ucj

13

Background: Multi-Model Estimation

Hypothesis Selection and Data Fusion Continuous Estimators (e.g. Kalman Filter Bank)

estimated mode & state {xd , xc } sensor signals yc and control inputs uc

advantages: high fidelity estimation of continuous behaviors noise handling and incipient fault detection disadvantages: limited to tracking a small number of hypothesis (limited size of the filter bank) Hypothesis selection and Data Fusion: determines the most likely mode and continuous state for the system as well as provides the initialization for the filter bank. State Estimator: Static filter bank that maintains a trajectory estimate for every mode.

14

hybrid Mode / State Estimation

Hybrid Mode Estimator Concurrent PHA Model Continuous Estimators (e.g. Kalman Filter Bank)

estimated mode & state x = {xd ,xc} and it’s belief state h[x] sensor signals yc and control inputs uc , ud

Hybrid State Estimator Maintains the set of most likely hybrid state estimates as a set of trajectories. A Hidden Markov Model style belief state update is used to determine the likelihood for each traced trajectory Hybrid Mode estimator: Determines for each trajectory the possible transitions, and specifies (dynamically) the candidate trajectories to be tracked by the continuous state estimators.

15

hybrid Mode/State Estimation

At each time step k, we evaluate for each trajectory:

• ld estimate:

x(k-1)={xd,(k-1) , xc,(k-1)}, h(k-1)

16

hybrid Mode Estimation

At each time step k, we evaluate for each trajectory:

Pt mode transition: xd,(k-1) = mi → x’d,(k-1) = mj x’(k-1) = {x’d,(k-1) , xc,(k-1)}, h’ = Pt h(k-1)

• ld estimate:

x(k-1)={xd,(k-1) , xc,(k-1)}, h(k-1)

17

Transition Probability Po

C12 guards the transition to either m3 (nominal transition)

• r to m4 (failure transition):

12 2

: 580

CO

C c ppm

• 580

cCO2 mean of estimated CO2 concentration guard boundary probability PC

• f guard C12

transition probability = guard probability * thread probability

m3 m1 m2 m4 C12 C11 Pt13 Pt14 Pt12 Pt11

18

hybrid Mode Estimation

At each time step k, we evaluate for each trajectory:

new estimate x(k) = {xd,(k) , xc,(k)}, h(k) = Po h’ continuous behavior x’c,(k-1) → xc,(k) , xd,(k)= x’d,(k-1) Pt Po

• ld estimate:

x(k-1)={xd,(k-1) , xc,(k-1)}, h(k-1) mode transition: xd,(k-1) = mi → x’d,(k-1) = mj x’(k-1) = {x’d,(k-1) , xc,(k-1)}, h’ = Pt h(k-1)

19

Observation Probability Pt

We compare the sensor signal yc(k) with its estimation for mode mj using an extended Kalman filter.

→ one extended Kalman filter for each hypothesis

1 T

• P

e

−

−

=

r S r

• peration performed by an (extended) Kalman filter:
• state prediction:

xc,(k-1), P(k-1) , uc,(k-1) → x’c,(k), P’(k)

• residual calculation:

x’c,(k), P’(k), yc(k)

→

r(k) , S(k) , Po

• Kalman filter gain calculation:

P’(k)

→

k(k)

• state estimate refinement:

x’c,(k), P’(k), k(k) , r(k) → xc,(k), P(k)

20

mode transition: xd,(k-1) = mi → x’d,(k-1) = mj x’(k-1) = {x’d,(k-1) , xc,(k-1)}, h’ = Pt h(k-1)

• ld estimate:

x(k-1)={xd,(k-1) , xc,(k-1)}, h(k-1)

exponential Explosion

At each time step k, we evaluate for each trajectory:

The number of possible transitions at each time step can be very large: E.g. a model with 10 components, each of which can transition to 3 successor modes has 310 = 59049 possible successor modes for each trajectory at each time step! Po new estimate x(k) = {xd,(k) , xc,(k)}, h(k) = Po h’ continuous behavior x’c,(k-1) → xc,(k) , xd,(k)= x’d,(k-1)

21

Approach:

how do we tackle the exponential blowup?

• A mode transition of a component is guarded by C(xc,(k),ud,(k))

i.e. it depends on the continuous state, local to the component, and the component’s discrete command input.

• no discrete interconnection among the components

→ component transitions independent of each other → take mode transition of the cPHA component-wise

22

Approach:

formulate node expansion as: A* search:

Transition Expansion – expand component-wise:

x(k-1) x(k) PO ... PT1 PT2 PT3 PTl

component 2 component 1 component 3 component l transition expansion estimation

h(k-1) h(k)

23

Overall hybrid Mode Estimation Scheme:

formulate hybrid estimation as: beam search:

• t(k-3)
• t(k-2)
• t(k-1)
• t(k)

24

Bio-Plex / Plant Growth Chamber

Airlock Plant Growth Chamber Crew Chamber

CO2 tank lighting system chamber control flow regulator 2 pulse injection valves

CO2

flow regulator 1

FR1 FR2 PIV1 PIV2 PGC LS

ud3 ud1 ud2 uc1 yc1 yc3 yc2

25

Simulation Result

components: 6 ( FR1, FR2, PIV1, PIV2, LS, PGC) total no of modes: 9600 fringe size: 5, (400 estimation steps): average candidates: 24.3

• max. candidates:

236 filter calculations: 144 filter executions: 9733 average runtime: ~0.3 s/step (PII-400, 128mb)

850 900 950 1000 1050 1100 1150 1200 460 480 500 520 540 560 time [minutes] CO2 concentration [ppm] 850 900 950 1000 1050 1100 1150 1200 2 4 6 PGC time [minutes] mode number 850 900 950 1000 1050 1100 1150 1200 2 4 6 Lighting System time [minutes] mode number 850 900 950 1000 1050 1100 1150 1200 50 100 150 200 250 samples

26

Simulation Result

components: 6 ( FR1, FR2, PIV1, PIV2, LS, PGC), total no of modes: 9600 fringe size: 20, (400 estimation steps): average candidates: 90.2

• max. candidates:

856 filter calculations: 242 filter executions: 36050 average runtime: ~1 s/step (PII-400, 128mb)

850 900 950 1000 1050 1100 1150 1200 2 4 6 PGC time [minutes] mode number 850 900 950 1000 1050 1100 1150 1200 460 480 500 520 540 560 time [minutes] CO2 concentration [ppm] 850 900 950 1000 1050 1100 1150 1200 2 4 6 Lighting System time [minutes] mode number

27

Discussion & Conclusion

Summary

• cPHA - discrete-time dynamics, probabilistic transition
• additive Gaussian noise model - extended Kalman filter
• comparison to multi-model filtering

Future Research

• ther noise models - particle filters
• unknown mode - decomposition
• conflict directed search
• hME in the context of generalized state feedback control
• mode estimation - mode reconfiguration

• ptional slides

29

Guard Probability Pco multi-var. -case

e.g. 2nd order system

3 1 1 2

1 1 , 2 1 guard: 1 5157 0.344 15000

co

x x x P x P

• −2

2 4 −2 −1 1 2 3 4 5 6 x1 x2

30

Simulation Result

components: 4 (flow regulator, pulse valve, light system, chamber), total no of modes: 480 fringe size: 20, average candidates: 57, filter calculations (total for experiment): 60 average runtime 0.3 s/step (PII-400, 128mb), space used (total): 1MB (program & data)