Real Tim e TRON TRON TRON TRON Testing Testing using UPPAAL W - - PowerPoint PPT Presentation

▶
real tim e
SMART_READER_LITE
LIVE PREVIEW

Real Tim e TRON TRON TRON TRON Testing Testing using UPPAAL W - - PowerPoint PPT Presentation

Mar 10, 2023 807 likes •1.44k views

Real Tim e TRON TRON TRON TRON Testing Testing using UPPAAL W ith W ith Mariius Mikucionis, Brian Nielsen, Arne Skou, Anders Hessel, Paul Pettersson, Jacob I llum Rasm ussen Overview Introduction gi knolog Off-line Test

slide-1
SLIDE 1

Real Tim e Testing

TRON TRON TRON TRON

Testing

W ith

using UPPAAL

W ith Mariius Mikucionis, Brian Nielsen, Arne Skou, Anders Hessel, Paul Pettersson, Jacob I llum Rasm ussen

slide-2
SLIDE 2

Overview

gi

 Introduction

knolog

 Off-line Test Generation

ll bl d

CLASSI C CLASSI C CLASSI C CLASSI C CORA CORA CORA CORA

  • nstek
  • Controllable Timed Automata
  • Observable Timed Automata

CLASSI C CLASSI C CLASSI C CLASSI C CORA CORA CORA CORA TI GA TI GA TI GA TI GA

rmatio

 On-line Test Generation

TRON TRON TRON TRON

Infor

 Conclusion and Future Work

Kim G Larsen 2 Summer School on Informatics RIO 2012

slide-3
SLIDE 3

Testing

gi

Primary validation technique used in industry

  • In general avg. 10-20 errors per 1000 LOC

30 50 % f d l t ti d t i b dd d ft

knolog

  • 30-50 % of development time and cost in embedded software

To find errors

To determine risk of release Pa t of s stem de elopment life c cle

  • nstek

Part of system development life-cycle

Output

rmatio

p Input

Environ- m ent System Under Test

Infor

Expensive, error prone, time consuming (for Real-Time Systems)

UPPAAL model can be used to generate test specifications

Kim G Larsen 3 Summer School on Informatics RIO 2012

slide-4
SLIDE 4

Real-tim e Model-Based Testing

gi

Plant Controller Program

knolog

sensors

Continuous

g

Discrete

  • nstek

actuators

Conform s-to?

rmatio

a 1 2 4 3 1 2

inputs

Test generation (offline or

Infor

c b 4 3 a c b 1 2 4 3 4 3 1 2 a

  • utputs
  • nline) wrt.

Design Model

c b 3 4 3 c b

UPPAAL Model

Kim G Larsen 4 Summer School on Informatics RIO 2012

slide-5
SLIDE 5

Off-Line Test Generation

Controllable Tim ed Autom ata

slide-6
SLIDE 6

Model Based Conform ance Testing gi

DBLclick!

pass

Test

Model Test suite

knolog

Test Gene- rator

click? x:=0 x>=2 DBLclick!

fail

Test execution tool

Event

Test Generator tool

  • nstek

tool

click? x<2

a mapping Driver

tool Selection & ti i ti

rmatio

  • ptimization

Infor

Implementation Relation

Does the behavior of the (blackbox) ( ) implementation comply to that of the specification?

Kim G Larsen 6 Summer School on Informatics RIO 2012

slide-7
SLIDE 7

Controllable Tim ed Autom ata gi

I nput Enabled: all inputs can always be accepted. Assumption about model of SUT

knolog

p y p Output Urgent: enabled outputs will occur immediately model of SUT

  • nstek

enabled outputs will occur immediately. Determ inism : two transitions with same input/output leads to the

rmatio

two transitions with same input/output leads to the same state. I l t d O t t

Infor

I solated Outputs: if an output is enabled, no other output is enabled.

Kim G Larsen 7 Summer School on Informatics RIO 2012

slide-8
SLIDE 8

Test Generation

using Verification gi using Verification knolog

System model System model

myGearControl.xml

  • nstek

Uppaal Model- Checker Trace (witness) Trace (witness)

rmatio

Test purpose Property Test purpose Property Some Random Shortest

E<> Gear Gear5

Infor

Fastest

E<> Gear.Gear5 testGear5.trc

8 Summer School on Informatics RIO 2012

Use trace scenario as test case??!!

Kim G Larsen

slide-9
SLIDE 9

Exam ple Light Controller

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 9 Summer School on Informatics RIO 2012

slide-10
SLIDE 10

Test Purposes

gi

Test Purpose: A specific test objective (or observation) the tester wants to make on SUT

knolog

  • nstek

rmatio

TP: Check that the light can become bright: E<> L==10

Infor

  • ut(IGrasp);silence(500);in(OSetLevel,0);silence(1000);

in(OSetLevel,1);silence(1000);in(OSetLevel,2); silence(1000); in(OSetLevel,3);silence(1000);in(OSetLevel,4);silence(1000); in(OSetLevel 5);silence(1000);in(OSetLevel 6);silence(1000); 10 Summer School on Informatics RIO 2012 in(OSetLevel,5);silence(1000);in(OSetLevel,6);silence(1000); in(OSetLevel,7);silence(1000);in(OSetLevel,8);silence(1000); in(OSetLevel,9);silence(1000);in(OSetLevel,10);

  • ut(IRelease);

Kim G Larsen

slide-11
SLIDE 11

Coverage Based Test Generation

gi

 Multi purpose testing

knolog u t pu pose test g

 Cover measurement  Examples:

  • nstek

 Examples:

  • Location coverage,
  • Edge coverage,

rmatio

g g ,

  • Definition/use

pair coverage

Infor

11 Summer School on Informatics RIO 2012 Kim G Larsen

slide-12
SLIDE 12

Coverage Based Test Generation

gi

 Multi purpose testing

knolog u t pu pose test g

 Cover measurement  Examples:

  • nstek

 Examples:

  • Location coverage,
  • Edge coverage,

rmatio

g g ,

  • Definition/use

pair coverage

Infor

12 Summer School on Informatics RIO 2012 Kim G Larsen

slide-13
SLIDE 13

Coverage Based Test Generation

gi

 Multi purpose testing

knolog u t pu pose test g

 Cover measurement  Examples:

  • nstek

 Examples:

  • Location coverage,
  • Edge coverage,

rmatio

g g ,

  • Definition/use

pair coverage

Infor

13 Summer School on Informatics RIO 2012 Kim G Larsen

slide-14
SLIDE 14

Coverage Based Test Generation

gi

 Multi purpose testing

knolog u t pu pose test g

 Cover measurement  Examples:

  • nstek

 Examples:

  • Location coverage,
  • Edge coverage,

rmatio

g g ,

  • Definition/use

pair coverage

Infor

14 Summer School on Informatics RIO 2012 Kim G Larsen

slide-15
SLIDE 15

Edge Coverage

gi

 Test sequence traversing all edges  Encoding:

knolog

 Encoding:

  • Enumerate edges

e0,…,en

  • nstek
  • Add auxiliary variable

e[i] for each edge

  • Label each edge

rmatio

  • Label each edge

e[i]:=1

Infor

 Check:

E<>( e[0]=1   e[n]=1 )

15 Summer School on Informatics RIO 2012

E<>( e[0] 1  …  e[n] 1 )

slide-16
SLIDE 16

Fastest Edge Coverage

gi

Time=12600 ms

knolog

  • nstek

rmatio

  • ut(IGrasp); //touch:switch light on

silence(200);

  • ut(IRelease);

//13

  • ut(IGrasp); //@900 // Bring dimmer PassiveDn->ActiveDN->

silence(500);//hold // ActiveUP+increase to level 10 silence(1000); in(OSetLevel,1); silence(1000); in(OSetLevel,2); silence(1000); in(OSetLevel,3);

Infor

( ); in(OSetLevel,0);

  • ut(IGrasp); //@200 // touch: switch light off

silence(200);

  • ut(IRelease);//touch

in(OSetLevel,0); // ( ); ( , ); silence(1000); in(OSetLevel,4); silence(1000); in(OSetLevel,5); silence(1000); in(OSetLevel,6); silence(1000); in(OSetLevel,7); silence(1000); in(OSetLevel,8); silence(1000); in(OSetLevel,9); silence(1000); in(OSetLevel,10

16 Summer School on Informatics RIO 2012

//9

  • ut(IGrasp); //@400 //Bring dimmer from ActiveUp

silence(500); //hold //To Passive DN (level=0) in(OSetLevel,0);

  • ut(IRelease);

silence(1000); in(OSetLevel,9); //bring dimm State to ActiveDN

  • ut(IRelease); //check release->grasp is ignored
  • ut(IGrasp); //@12400
  • ut(IRelease);

silence(dfTolerance);

Page 1 Page 2

Kim G Larsen

slide-17
SLIDE 17

Pow er-Optim al Edge Coverage

gi

Cost=320 J

knolog

1 W 5 L W 5 L W

  • nstek

5 · L W 5 · L W 1 W

rmatio

  • ut(IGrasp); //touch:switch light on

silence(200);

  • ut(IRelease);

//13

  • ut(IGrasp); //@900 // Bring dimmer PassiveDn->ActiveDN->

silence(500);//hold // ActiveUP+increase to level 10 silence(1000); in(OSetLevel,1); silence(1000); in(OSetLevel,2); silence(1000); in(OSetLevel,3);

Infor

( ); in(OSetLevel,0);

  • ut(IGrasp); //@200 // touch: switch light off

silence(200);

  • ut(IRelease);//touch

in(OSetLevel,0); // ( ); ( , ); silence(1000); in(OSetLevel,4); silence(1000); in(OSetLevel,5); silence(1000); in(OSetLevel,6); silence(1000); in(OSetLevel,7); silence(1000); in(OSetLevel,8); silence(1000); in(OSetLevel,9); silence(1000); in(OSetLevel,10

17 Summer School on Informatics RIO 2012

//9

  • ut(IGrasp); //@400 //Bring dimmer from ActiveUp

silence(500); //hold //To Passive DN (level=0) in(OSetLevel,0);

  • ut(IRelease);

silence(1000); in(OSetLevel,9); //bring dimm State to ActiveDN

  • ut(IRelease); //check release->grasp is ignored
  • ut(IGrasp); //@12400
  • ut(IRelease);

silence(dfTolerance);

Page 1 Page 2

Kim G Larsen

slide-18
SLIDE 18

V-PLUS: Model-based GUI Testing for Automatic or Manual Execution gi

Requirement spec.:

Automatic or Manual Execution knolog

Requirement 2.1 When the user presses the button save, all user data is saved to the

  • database. The values stored in the

database are output to the screen Requirement 2.1 When the user presses the button save, all user data is saved to the database The values stored in the Requirement 2.1 When the user presses the button save all user data is saved to the Requirement 2.1 When the user presses the button

Create Models

  • nstek
  • database. The values stored in the

database are output to the screen save, all user data is saved to the

  • database. The values stored in the

database are output to the screen When the user presses the button save, all user data is saved to the

  • database. The values stored in the

database are output to the screen

Generate Covering Test Suite:

rmatio

Output:

Infor

Output:

Excel

TestDrive Gold Quality Center/

18 Summer School on Informatics RIO 2012

Excel

Gold QTP

Kim G Larsen

slide-19
SLIDE 19

Test Generation from UML Statecharts

UML Di

gi

UML Diagrams

knolog

  • nstek

Custom Scripts Uppaal Models

rmatio Infor

19 Summer School on Informatics RIO 2012 Kim G Larsen

slide-20
SLIDE 20

An I ndustrial Tool Chain…

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 20 Summer School on Informatics RIO 2012

slide-21
SLIDE 21

An I ndustrial Tool Chain..

gi

Build functional model as UML state m achines

w Jacob Illum Rasmussen, Arne Skou

knolog

as UML state m achines using Rational Systems Developer from IBM

Arne Skou Formal Methods 2009

  • nstek

Generate edge coverage test using Yggdrasil

XMI-to-TA XMI

rmatio

Output tests from Yggdrasil to JavaScript

GUI testing

Infor

Yggdrasil to JavaScript

(QTP, Selenium, Ruby)

GUI testing

Execute scripts

Kim G Larsen 21 Summer School on Informatics RIO 2012

slide-22
SLIDE 22

Off-Line Test Generation

Observable Tim ed Autom ata

slide-23
SLIDE 23

Observable Tim ed Autom ata

gi

Observable Tim ed Autom ata

 Determ inism :

knolog

two transitions with same input/output leads to the same state

 I nput Enabled:

  • nstek

 I nput Enabled:

all inputs can always be accepted

 Tim e Uncertainty of outputs:

rmatio

y p timing of outputs uncontrollable by tester

 Uncontrollable output:

IUT controls which enabled output will occur in

Infor

IUT controls which enabled output will occur in what order

Kim G Larsen 23 Summer School on Informatics RIO 2012

slide-24
SLIDE 24

Tim ed Gam es and Test Generation

gi

Tidle=20 Tsw=4

knolog

  • nstek

rmatio Infor

Off-line test-case generation = Compute winning strategy for reaching Bright Assign verdicts st. lost game means IUT not conforming

Kim G Larsen 24 Summer School on Informatics RIO 2012

slide-25
SLIDE 25

A trick light control

gi

Tidle=20 Tsw=4

knolog

Tsw 4

  • nstek

rmatio

How to test for Bright ?

Infor

How to test for Bright ?

E<> (control: A<> Bright)

  • r
  • r

<<c,u>> ♦(<<c>> ♦ Bright)

Kim G Larsen 25 Summer School on Informatics RIO 2012

slide-26
SLIDE 26

Cooperative Strategies

gi

Model Statespace

knolog

winning possibly winning

  • nstek

initial goal

rmatio

loosing

Infor

  • Play the game (execute test) while time available or game is lost
  • Play the game (execute test) while time available or game is lost
  • Possibly using ranomized online testing

Kim G Larsen 26 Summer School on Informatics RIO 2012

slide-27
SLIDE 27

O i i On-Line Testing

slide-28
SLIDE 28

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 28 Summer School on Informatics RIO 2012

slide-29
SLIDE 29

Autom ated Model Based Conform ance T ti gi

DBLclick!

Testing

pass

Test

Model Test suite

knolog

Test Gene- rator

click? x:=0 x>=2 DBLclick!

fail

Test execution tool

Ad t

Test Generator tool

  • nstek

tool

click? x<2

a Adaptor

tool Selection & ti i ti

rmatio

  • ptimization

Infor

Correctness Relation

Does the behavior of the (blackbox) ( ) implementation comply to that of the specification?

Kim G Larsen 29 Summer School on Informatics RIO 2012

slide-30
SLIDE 30

Online Testing

gi

DBLclick!

pass

Test

Model

knolog

Test Gene- rator

click? x:=0 x>=2 DBLclick!

input fail

Test execution tool

Ad t

Test Generator tool

  • utput

input input input

  • utput
  • utput
  • utput
  • nstek

tool

click? x<2

a Adaptor

tool

  • utput

Selection & ti i ti

  • utput
  • utput
  • utput

rmatio

  • ptimization

Infor

Correctness Relation

  • Test generated and executed event-

by event (randomly) by-event (randomly)

  • A.K.A on-the-fly testing

Kim G Larsen 30 Summer School on Informatics RIO 2012

slide-31
SLIDE 31

On-line Testing

Li ht C t ll

gi

Light Controller

knolog

mousePress

  • nstek

LightControllerGUI Test Fixture UPPAALTRON tcp/ip mousePress mouseRelease

rmatio

g Fixture UPPAALTRON Testing Host release grasp release grasp setLevel setLevel

  • Real-time
  • Simulated time

Infor

LightController

31 Summer School on Informatics RIO 2012

JavaVM+w2k/Linux

Kim G Larsen

slide-32
SLIDE 32

Mutants

gi

 Mutant: Non-conforming program version

with a seeded error knolog

  • M1 incorrectly implements switch

synchronized public void handleTouch() {

  • nstek

if(lightState==lightOff) { setLevel(oldLevel); lightState=lightOn; }

rmatio

} else { //was missing if(lightState==lightOn){

  • ldLevel=level;

Infor

  • M2 violates a deadline
  • ldLevel level;

setLevel(0); lightState=lightOff; }

32 Summer School on Informatics RIO 2012 Kim G Larsen

slide-33
SLIDE 33

l i h An Algorithm

slide-34
SLIDE 34

Algorithm I dea:

State set tracking

gi

State-set tracking

 Dynamically compute all potential states that the

knolog

 Dynamically compute all potential states that the

model M can reach after the timed trace 0,i0,1,o1,2,i2,o2,…

[Tripakis] Failure Diagnosis

  • nstek

 Z= M after (0,i0,1,o1,2,i2,o2)

rmatio

 If Z=  the IUT has made a computation not in model:

FAI L

Infor

 i is a relevant input in Env iff I ∈ EnvOutput(Z)

Kim G Larsen 34 Summer School on Informatics RIO 2012

slide-35
SLIDE 35

Online State Estim ation

gi

State-set explorer: i t i d l t f b li

Timed Automata S ifi ti

knolog

maintain and analyse a set of symbolic states in real time!

Specification

  • nstek

Z4 Z i!

rmatio

4

Z0 Z1 Z3 Z7 Z5 Z8 Z Z11 Z14 Z17 Z i! 2.75 O?

System Under Test

Infor

Z2

8

Z6 Z9 Z12 Z15 Z18 Z16 O?

Test

Kim G Larsen 35 Summer School on Informatics RIO 2012

slide-36
SLIDE 36

( Abstract) Online Algorithm

gi

Algorithm TestGenExe (S, E, IUT, T ) returns {pass, fail) Z := {(s0, e0)}. w hile Z   and ♯iterations ≤ T do either randomly:

knolog

w hile Z   and ♯iterations ≤ T do either randomly: 1. // offer an input if EnvOutput(Z)   randomly choose i∈ EnvOutput(Z) d i t IUT

  • nstek

send i to IUT Z := Z After i 2. // wait d for an output randomly choose d∈ Delays(Z)

rmatio

w ait (for d time units or output o at d′ ≤ d) if o occurred then Z := Z After d′ Z := Z After o // may become  (fail)

Infor

Z : Z After o // may become  (fail) else Z := Z After d // no output within d delay 3. restart: Z := {(s0 e0)} reset IUT //reset and restart Z := {(s0, e0)}, reset IUT //reset and restart if Z =  then return fail else return pass

Kim G Larsen 36 Summer School on Informatics RIO 2012

slide-37
SLIDE 37

( Abstract) Online Algorithm

gi

Algorithm TestGenExe (S, E, IUT, T ) returns {pass, fail) Z := {(s0, e0)}. w hile Z   ♯iterations ≤ T do either randomly:

knolog

w hile Z   ♯iterations ≤ T do either randomly: 1. // offer an input if EnvOutput(Z)   randomly choose i EnvOutput(Z) d i t IUT

  • nstek

send i to IUT Z := Z After i 2. // wait d for an output randomly choose d Delays(Z)

  • Sound
  • Complete (as T  ∞)

(Under some technical

rmatio

w ait (for d time units or output o at d′ ≤ d) if o occurred then Z := Z After d′ Z := Z After o // may become  (fail)

(Under some technical assumptions)

Infor

Z : Z After o // may become  (fail) else Z := Z After d // no output within d delay 3. restart: Z := {(s0 e0)} reset IUT //reset and restart Z := {(s0, e0)}, reset IUT //reset and restart if Z =  then return fail else return pass

Kim G Larsen 37 Summer School on Informatics RIO 2012

slide-38
SLIDE 38

State-set Operations

gi

Z after a: possible states after action a (and * ) Z after  :possible states after * and i , totaling a delay of 

knolog

   a

Z

   

Z

  • nstek

a

rmatio

time

 (5)

Infor

 Can be computed efficiently using the

symbolic data structures and algorithms i U l in Uppaal

Kim G Larsen 38 Summer School on Informatics RIO 2012

slide-39
SLIDE 39

Online Testing Exam ple

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 39 Summer School on Informatics RIO 2012

slide-40
SLIDE 40

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 40 Summer School on Informatics RIO 2012

slide-41
SLIDE 41

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 41 Summer School on Informatics RIO 2012

slide-42
SLIDE 42

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 42 Summer School on Informatics RIO 2012

slide-43
SLIDE 43

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 43 Summer School on Informatics RIO 2012

slide-44
SLIDE 44

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 44 Summer School on Informatics RIO 2012

slide-45
SLIDE 45

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 45 Summer School on Informatics RIO 2012

slide-46
SLIDE 46

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 46 Summer School on Informatics RIO 2012

slide-47
SLIDE 47

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 47 Summer School on Informatics RIO 2012

slide-48
SLIDE 48

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 48 Summer School on Informatics RIO 2012

slide-49
SLIDE 49

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 49 Summer School on Informatics RIO 2012

slide-50
SLIDE 50

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 50 Summer School on Informatics RIO 2012

slide-51
SLIDE 51

Online Testing

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 51 Summer School on Informatics RIO 2012

slide-52
SLIDE 52

I ndustrial Application:

gi

Danfoss Electronic Cooling Controller

Sensor I nput i t t

knolog

  • air temperature sensor
  • defrost temperature sensor
  • (door open sensor)

Keypad I nput

  • nstek

Output Relays yp p

  • 2 buttons (~40 user settable

parameters)

rmatio

  • compressor relay
  • defrost relay
  • alarm relay
  • (fan relay)

Infor

( y) Display Output

  • alarm / error indication
  • mode indication
  • current calculated temperature
  • current calculated temperature
  • Optional real-time clock or LON network module

Kim G Larsen 52 Summer School on Informatics RIO 2012

slide-53
SLIDE 53

I ndustrial Cooling Plants

gi knolog

  • nstek

rmatio Infor

Kim G Larsen 53 Summer School on Informatics RIO 2012

slide-54
SLIDE 54

I ndustrial Application:

gi

Danfoss Electronic Cooling Controller

Sensor I nput i t t

knolog

  • air temperature sensor
  • defrost temperature sensor
  • (door open sensor)

Keypad I nput

  • nstek

Output Relays yp p

  • 2 buttons (~40 user settable

parameters)

rmatio

  • compressor relay
  • defrost relay
  • alarm relay
  • (fan relay)

Infor

( y) Display Output

  • alarm / error indication
  • mode indication
  • current calculated temperature
  • current calculated temperature
  • Optional real-time clock or LON network module

Kim G Larsen 54 Summer School on Informatics RIO 2012

slide-55
SLIDE 55

Exam ple Test Run

gi

3700 3800

knolog

2900 3000 3100 3200 3300 3400 3500 3600 3700 setTemp modelTemp ekcTemp CON COFF

Outcom e 4 instances of discrepancy

  • nstek

2200 2300 2400 2500 2600 2700 2800 2900 COFF AON AOFF alarmRst HADOn HADOff DON

4 instances of discrepancy between model and actual behavior, also involving timing errors.

rmatio

1500 1600 1700 1800 1900 2000 2100 100000 200000 300000 400000 500000 600000 700000 800000 900000 DON DOFF manDefrostOn manDefrostOff

Infor

100000 200000 300000 400000 500000 600000 700000 800000 900000

defrostOff? alarm On! resetAlarm ? AOFF! HighAlarm DisplayOff! / / defrost com plete DOFF! CON! alarm On! alarm DisplayOn! m anualDefrostOn? COFF! DON! com pressorOn!

Kim G Larsen 55 Summer School on Informatics RIO 2012

slide-56
SLIDE 56

Model-based Testing

  • f

Real Tim e System s

Conclusions

slide-57
SLIDE 57

Advantages of MBT

gi

 Engineer focus on w hat to test at a high

level of abstraction knolog level of abstraction

 Avoids cost of making scripts

  • As much test code as production code
  • nstek
  • As much test code as production code
  • Maintenance nightmare

 Heard of, but is still considered an

rmatio

 Heard of, but is still considered an

advanced technique by industry

 Industry is very motivated, MB A&T will

Infor y y , give

  • 1 0 % cost reduction
  • 2 0 % quality im provem ent

Kim G Larsen 57 Summer School on Informatics RIO 2012

slide-58
SLIDE 58

Verification & Testing

gi Testing

 Checks the actual

Verification

 Abstract models

knolog

 Checks the actual

implementation

 Only few  Abstract models  Exhaustive “proof”

  • nstek

 Only few

executions checked

 But is the most  Limited size and

expressivity rmatio

 But is the most

direct method Infor How to effectively com bine the different verification and testing techniques?

Kim G Larsen 58 Summer School on Informatics RIO 2012

slide-59
SLIDE 59

Conclusions

gi

 Testing real-time systems is theoretically

and practically challenging knolog

 Promising techniques and tools  Explicit environment modeling

R li d idi

  • nstek
  • Realism and guiding
  • Separation of concerns
  • Modularity

rmatio

  • Modularity
  • Creative tool uses
  • Theoretical properties

Infor

 Real-time online testing from timed

automata is feasible, but

  • Many open research issues
  • Many open research issues

Kim G Larsen 59 Summer School on Informatics RIO 2012

slide-60
SLIDE 60

Research Problem s

gi

 Testing Theory  Timed games with partial observability

knolog

g p y

 Hybrid extensions  Other Quantitative Properties

  • nstek

 Probabilistic Extensions, Performance testing  Efficient data structures and algorithms for state

set computation

rmatio

set computation

 Diagnosis & Debugging  Guiding and Coverage Measurement

Infor

Guiding and Coverage Measurement

 Real-Time execution of TRON  Adaptor Abstraction, IUT clock synchronization  Further Industrial Cases

Kim G Larsen 60 Summer School on Informatics RIO 2012

slide-61
SLIDE 61

Related W ork

gi

 Formal Testing Frameworks

  • [Brinksma Tretmans]

knolog

  • [Brinksma, Tretmans]

 Real-Time Implementation Relations

  • [Khoumsi’03, Briones’04, Krichen’04]
  • nstek

 Symbolic Reachability analysis of Timed

Automata

  • [Dill’89 Larsen’97

]

rmatio

  • [Dill 89, Larsen 97,…]

 Online state-set computation

  • [Tripakis’02]

Infor

 Online Testing

  • [Tretmans’99, Peleska’02, Krichen’04]

Kim G Larsen 61 Summer School on Informatics RIO 2012