Synthesis of On-line Planning Tester for Non-deterministic EFSM - - PowerPoint PPT Presentation

▶

Dec 26, 2022 267 likes •446 views

Synthesis of On-line Planning Tester for Non-deterministic EFSM Models Marko Kramees Jri Vain Kullo Raiend Tallinn University of Technology Eliko Competence Centre Elvior Overview Scope and main idea of the work Workflow of

SLIDE 1

Synthesis of On-line Planning Tester for Non-deterministic EFSM Models

Marko Kääramees Jüri Vain Kullo Raiend Tallinn University of Technology Eliko Competence Centre Elvior

SLIDE 2

Overview

Scope and main idea of the work Workflow of testing Off-line preparation algorithm and example On-line testing algorithm and example Implementation and complexity issues Conclusions

SLIDE 3

Scope of the work

Black box model based testing

tests are generated from the model

Model is non-deterministic

n-line testing needed
utput observability

assumed

Several test goals are

tackled at the same time

minimizing the amount

and length of the tests

SLIDE 4

Testing non-deterministic models

Test cases cannot be prepared beforehand Tester must decide inputs during the test based

n observed outputs and active goals

Test planning is costly and not feasible on-line

Proposed solution

Model is analysed off-line Result is expressed as a set of data constraints

for each test goal

Data instance generation is done on-line

SLIDE 5

Model of SUT

Model is given as EFSM

input/ output, guard, update input parameter t [ temp] and variable d [ delay]

Requirements

fridge must switch off when t is 4..5 fridge must switch on when t is 6..7 and it has

been off 20..39 seconds (tick every 10 seconds)

d: = 0 t ≤ 7 ∨ d ≤ 3 d: = d+ 1 tick(t)/ off tick(t)/ off t ≤ 5 d: = 0 tick(t)/ off t ≥ 6 ∧ d ≥ 2 tick(t)/ on t ≥ 4 tick(t)/ on

SLIDE 6

Modeling of test goals

Test goals are expressed by traps

trap is a pair < transition,predicate> expressed as update of trap variable in model

Can express

transition coverage transition sequence repeated pass using auxiliary variable

d: = 0 t ≤ 7 ∨ d ≤ 3 d: = d+ 1 tick(t)/ off tick(t)/ off t ≤ 5 d: = 0 tick(t)/ off t ≥ 6 ∧ d ≥ 2 tick(t)/ on t ≥ 4 tick(t)/ on trap1 := ( d= 3 ) trap2 := true

SLIDE 7

Workflow

model (EFSM)

ff-line

test generator test goals (traps) testing data (constraints)

n-line

tester SUT test verdict Adapter

SLIDE 8

Constraints

A set of constraints is generated for every trap

help to guard the on-line tester towards the trap

Constraints for states

Minimal path constraint C s

condition for the shortest paths to trap tr from state s

Maximal path constraint C* s

condition for all paths to trap tr from state s that extend the constraint

Constraints for transitions

Minimal Ct and maximal C* t as for states Guarding constraint Cg t

if the shortest path to the trap starts with the transition

Path lengths L s , L*

s , L t and L* t are recorded also

SLIDE 9

Offline algorithm for trap tr

initialise C to false, L to 0

C*

t = guard t ∧

conditiontr while fixpoint or search depth is reached for each state s on the depth level do

C*

s = simplify(C*’ s ∨

∃I:.C*

) // ti - t leaving from s; I - input if SAT(¬(C*

⇒ C*’

)) // C*

s changed

L*

s = depth

if not C s // minimal constraint

Cs = C*

s ; Ls = L* s

for each transition t coming to s

C*

t = simplify(C*’ t ∨

guard t ∧ wp(updatet , C*

)) record L*

s , C t , L s if needed

Cg

= simplify(Cg’

t ∨

(∃I:C*

t ∧ ¬C* source(t)

) )

SLIDE 10

Off-line constraint generation

d: = 0 t ≤ 7 ∨ d ≤ 3 d: = d+ 1 tick(t)/ off tick(t)/ off t ≤ 5 d: = 0 tick(t)/ off t ≥ 6 ∧ d ≥ 2 tick(t)/ on t ≥ 4 tick(t)/ on trap1: = (d= 3) trap2: = true C| 2 : d= 2 C| 1 : d= 3 C| 5 : true C| 6 : true

Constraints C| L give the condition and length

for the shortest path

C* | 4 : d ≤ 2 C* | 6 : true C* | 5 : true C* | 6 : true

Constraints C* | L* give the condition and length

for all paths up to fixpoint (or search depth)

Cg: d ≤ 2 Cg: d ≥ 3 Cg: true Cg: false

Constraints Cg give the condition for choosing the

next transition depending on the values of variables

Constraints for trap1:

SLIDE 11

On-line algorithm (greedy)

while exist uncovered traps //at state s select nearest reachable trap tr // using SAT() select transition with Cg

t satisfiable

// using SAT() select input parameters valuation by solving C t or C*

// constraint solving communicate the inputs to SUT if the output does not conform to the model // using SAT() stop(test_failed) move to the next state end while stop(test_passed)

SLIDE 12

Example (on-line)

tick(true):

ff, d= 0

tick(true):

ff, d= 1

tick(true):

ff, d= 2

tick(t < 6):

ff, d= 3

tick(t ≥ 6):

n, d= 3 trap1☺
ff, d= 4

tick(t > 7):

n, d= 4

tick(t > 5):

n, d= 4 trap2☺

tick(t < 4):

ff, d= 0
ff
n

d: = 0 t ≤ 7 ∨ d ≤ 3 d: = d+ 1 tick(t)/ off tick(t)/ off t ≤ 5 d: = 0 tick(t)/ off t ≥ 6 ∧ d ≥ 2 tick(t)/ on t ≥ 4 tick(t)/ on trap1: = (d= 3) trap2: = true C| 2 : d= 2 C| 1 : d= 3 C| 5 : true C| 6 : true C* | 4 : d ≤ 2 C* | 6 : true C* | 5 : true C* | 6 : true Cg: d ≤ 2 Cg: d ≥ 3 Cg: true Cg: false

SLIDE 13

Implementation issues

UPPAAL used for modelling (Uppsala & Aalborg U) Z3 SMT solver suite (Microsoft Research)

simplification of constraints quantifier elimination SAT solver constraint solving (model generation)

Python scripts for parsing and constraining

generation algorithm implementation

TestCast - TTCN3 toolset (Elvior)

running generated TTCN3 scripts

SLIDE 14

Complexity issues

Constraints limited to decidable theories

linear arithmetic (+ others supported by solver)

Theoretical limits

SAT problem is NP-complete decision procedures and simplification of Presburger

arithmetic is double-exponential

Practical aspects

number of constraints is in O(traps* transitions) Z3 does a good job in SAT and simplification

Search depth

complexity of the constraints depends on the

structure of the model and search depth

search depth can be constrained off-line when the

time for the SAT check needed on-line exceeds the predefined limit

SLIDE 15

Constrained search

trap depth 8