UKPEW 2005 Invited Talk The Future is Collaborative Performance - - PowerPoint PPT Presentation
UKPEW 2005 Invited Talk The Future is Collaborative Performance Engineering! Jeremy Bradley Email: jb@doc.ic.ac.uk Department of Computing, Imperial College London Produced with prosper and L A T EX JTB [14/07/2005] p.1/29 Drowning!
Jeremy Bradley
Email: jb@doc.ic.ac.uk
Department of Computing, Imperial College London
Produced with prosper and L
A
T EX
JTB [14/07/2005] – p.1/29
We are drowning in modelling techniques and formalisms: SPA (Stochastic Process Algebras) SAN (Stochastic Automata Networks) SPN (Stochastic Petri Nets) LQN (Layered Queueing Networks) ...
JTB [14/07/2005] – p.2/29
We have underlying models: CTMC (Continuous Time Markov Chains) DTMC (Discrete Time Markov Chains) MDP (Markov Decision Processes) MRP (Markov Reward Processes) SMP (Semi Markov Processes) GSMP (Generalised Semi Markov Processes) ...
JTB [14/07/2005] – p.3/29
Not to mention analysis techniques: steady-state analysis transient analysis passage time analysis entropy maximisation simulation continuous approximation ...
JTB [14/07/2005] – p.4/29
...performance query formalisms CSL/eCSL/aCSL pCTL stochastic probes NICE automata ...
JTB [14/07/2005] – p.5/29
...and tools: ETMCC PRISM ipc/DNAmaca PEPA workbench DSPNexpress SMARTA Two Towers GreatSPN Möbius ... and many others...
JTB [14/07/2005] – p.6/29
Do all the tools produce the same results for the same questions and models? Are some tools/formalisms/performance models better at capturing/modelling certain systems better than others? Which tools run faster on which type of model? Is this because they use BDDs/out-of-core solvers/MDDs/hypergraph partitioning? Are we reinventing the wheel?... in terms of tools and models?
JTB [14/07/2005] – p.7/29
The answer is another tool: Perform-DB! Not quite! More an integrated collaborative performance engineering environment Encourage a performance engineering lifecycle
JTB [14/07/2005] – p.8/29
Fully deterministic system:
S ∼ det(1) T ∼ det(1)
JTB [14/07/2005] – p.9/29
π00(t) :probability of being in state 0 having started in
state 0
0.2 0.4 0.6 0.8 1 1.2 2 4 6 8 10 12 Probability, p Time, t pi_00(t)
JTB [14/07/2005] – p.10/29
Semi deterministic system:
S ∼ exp(1) T ∼ det(1)
JTB [14/07/2005] – p.11/29
π00(t) :probability of being in state 0 having started in
state 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 2 4 6 8 10 12 Probability Time, t Analytic solution, pi_00(t) 1 iteration 2 iterations 4 iterations 6 iterations 8 iterations
JTB [14/07/2005] – p.12/29
PEPA syntax:
P ::= (a, λ).P P + P P ✄
L P
P/L A
JTB [14/07/2005] – p.13/29
PEPA syntax:
P ::= (a, λ).P P + P P ✄
L P
P/L A
Action prefix: (a, λ).P
JTB [14/07/2005] – p.13/29
PEPA syntax:
P ::= (a, λ).P P + P P ✄
L P
P/L A
Action prefix: (a, λ).P Competitive choice: P1 + P2
JTB [14/07/2005] – p.13/29
PEPA syntax:
P ::= (a, λ).P P + P P ✄
L P
P/L A
Action prefix: (a, λ).P Competitive choice: P1 + P2 Cooperation: P1 ✄
L P2
JTB [14/07/2005] – p.13/29
PEPA syntax:
P ::= (a, λ).P P + P P ✄
L P
P/L A
Action prefix: (a, λ).P Competitive choice: P1 + P2 Cooperation: P1 ✄
L P2
Action hiding: P/L
JTB [14/07/2005] – p.13/29
PEPA syntax:
P ::= (a, λ).P P + P P ✄
L P
P/L A
Action prefix: (a, λ).P Competitive choice: P1 + P2 Cooperation: P1 ✄
L P2
Action hiding: P/L Constant label: A
JTB [14/07/2005] – p.13/29
Sys
def
= (AA ✄
{run} A1) ✄
{alert} (BB ✄
{run} B1)
JTB [14/07/2005] – p.14/29
Sys
def
= (AA ✄
{run} A1) ✄
{alert} (BB ✄
{run} B1)
AA
def
= (run, ⊤).(alert, r5).AA
JTB [14/07/2005] – p.14/29
Sys
def
= (AA ✄
{run} A1) ✄
{alert} (BB ✄
{run} B1)
AA
def
= (run, ⊤).(alert, r5).AA A1
def
= (start, r1).A2 + (pause, r2).A3 A2
def
= (run, r3).A1 + (fail, r4).A3 A3
def
= (recover, r1).A1
JTB [14/07/2005] – p.14/29
Sys
def
= (AA ✄
{run} A1) ✄
{alert} (BB ✄
{run} B1)
AA
def
= (run, ⊤).(alert, r5).AA A1
def
= (start, r1).A2 + (pause, r2).A3 A2
def
= (run, r3).A1 + (fail, r4).A3 A3
def
= (recover, r1).A1 BB
def
= (run, ⊤).(alert, r5).BB
JTB [14/07/2005] – p.14/29
Sys
def
= (AA ✄
{run} A1) ✄
{alert} (BB ✄
{run} B1)
AA
def
= (run, ⊤).(alert, r5).AA A1
def
= (start, r1).A2 + (pause, r2).A3 A2
def
= (run, r3).A1 + (fail, r4).A3 A3
def
= (recover, r1).A1 BB
def
= (run, ⊤).(alert, r5).BB B1
def
= (start, r1).B2 + (pause, r2).B1 B2
def
= (run, r3).B1
JTB [14/07/2005] – p.14/29
Steady-state and transient analysis in PEPA:
A1
def
= (start, r1 ).A2 + (pause, r2 ).A3 A2
def
= (run, r3 ).A1 + (fail, r4 ).A3 A3
def
= (recover, r1 ).A1 AA
def
= (run, ⊤).(alert, r5 ).AA Sys
def
= AA ✄
{run} A1
JTB [14/07/2005] – p.15/29
Steady-state and transient analysis in PEPA:
A1
def
= (start, r1 ).A2 + (pause, r2 ).A3 A2
def
= (run, r3 ).A1 + (fail, r4 ).A3 A3
def
= (recover, r1 ).A1 AA
def
= (run, ⊤).(alert, r5 ).AA Sys
def
= AA ✄
{run} A1
JTB [14/07/2005] – p.15/29
Steady-state and transient analysis in PEPA:
A1
def
= (start, r1 ).A2 + (pause, r2 ).A3 A2
def
= (run, r3 ).A1 + (fail, r4 ).A3 A3
def
= (recover, r1 ).A1 AA
def
= (run, ⊤).(alert, r5 ).AA Sys
def
= AA ✄
{run} A1
JTB [14/07/2005] – p.15/29
Steady-state and transient analysis in PEPA:
A1
def
= (start, r1 ).A2 + (pause, r2 ).A3 A2
def
= (run, r3 ).A1 + (fail, r4 ).A3 A3
def
= (recover, r1 ).A1 AA
def
= (run, ⊤).(alert, r5 ).AA Sys
def
= AA ✄
{run} A1
JTB [14/07/2005] – p.15/29
Steady-state and transient analysis in PEPA:
A1
def
= (start, r1 ).A2 + (pause, r2 ).A3 A2
def
= (run, r3 ).A1 + (fail, r4 ).A3 A3
def
= (recover, r1 ).A1 AA
def
= (run, ⊤).(alert, r5 ).AA Sys
def
= AA ✄
{run} A1
JTB [14/07/2005] – p.15/29
Extract a passage-time density from a PEPA model:
A1
def
= (start, r1 ).A2 + (pause, r2 ).A3 A2
def
= (run, r3 ).A1 + (fail, r4 ).A3 A3
def
= (recover, r1 ).A1 AA
def
= (run, ⊤).(alert, r5 ).AA Sys
def
= AA ✄
{run} A1
JTB [14/07/2005] – p.16/29
✲ ✲ ✏✏✏✏ ✶ PPPP q
PEPA
ipc
HYDRA
hydra
Passage-time PDF Passage-time CDF Transient
For state spaces of O(109) Very precise probabilistic results
JTB [14/07/2005] – p.17/29
✲ ✏✏✏✏ ✶ PPPP q
PEPA
ipc/pwb
Rate equations
Dizzy
Stochastic simulation Numerical ODE solution
For very large state spaces, e.g. 101000+ states Aggregate deterministic results
JTB [14/07/2005] – p.18/29
Keeping track of: model version documentation of model version type/version of query parameter sets in model tool version numbers ...is non-trivial
JTB [14/07/2005] – p.19/29
Allow modellers to link to/import/alter! other peoples’ models Keep track of model-result trail model → query → result Allow modellers to compare results from: different queries different parameters different models!
JTB [14/07/2005] – p.20/29
Model Rate parameter sweep Results PEPA net
A ✄
✁
L B[B]
A def = (up, ρ).A′ + · · ·
JTB [14/07/2005] – p.21/29
Model Rate parameter sweep Results PEPA net
A ✄
✁
L B[B]
A def = (up, ρ).A′ + · · ·
ρ = 0.8
✲ ✲
JTB [14/07/2005] – p.21/29
Model Rate parameter sweep Results PEPA net
A ✄
✁
L B[B]
A def = (up, ρ).A′ + · · ·
ρ = 0.8
✲ ✲
ρ = 0.9
❍❍❍❍❍ ❍ ❥ ✲
JTB [14/07/2005] – p.21/29
Model Rate parameter sweep Results PEPA net
A ✄
✁
L B[B]
A def = (up, ρ).A′ + · · ·
ρ = 0.8
✲ ✲
ρ = 0.9
❍❍❍❍❍ ❍ ❥ ✲
ρ = 1.0
❅ ❅ ❅ ❅ ❅ ❅ ❘ ✲
JTB [14/07/2005] – p.21/29
Model description Performance query Performance results GSPN
... \transition{t1}{ \condition{cpu>0} \action{ cpu = cpu - 1; id = id + 1; } \rate{mu} } ...
PRISM m
... module queue q : [0..N]; [] q < N -> mu:(q’=q+1); [] q = N -> mu:(q’=q); [serve] q > 0 -> lambda:(q’=q-1); endmodule ...
JTB [14/07/2005] – p.22/29
Model description Performance query Performance results GSPN
... \transition{t1}{ \condition{cpu>0} \action{ cpu = cpu - 1; id = id + 1; } \rate{mu} } ...
PRISM m
... module queue q : [0..N]; [] q < N -> mu:(q’=q+1); [] q = N -> mu:(q’=q); [serve] q > 0 -> lambda:(q’=q-1); endmodule ...
✲ ✲
Response time
\passage{ \source{cpu==1} \target{id==1} \t start{0.1} \t stop{10.0} }
JTB [14/07/2005] – p.22/29
Model description Performance query Performance results GSPN
... \transition{t1}{ \condition{cpu>0} \action{ cpu = cpu - 1; id = id + 1; } \rate{mu} } ...
PRISM m
... module queue q : [0..N]; [] q < N -> mu:(q’=q+1); [] q = N -> mu:(q’=q); [serve] q > 0 -> lambda:(q’=q-1); endmodule ...
✲ ✲
Response time
\passage{ \source{cpu==1} \target{id==1} \t start{0.1} \t stop{10.0} }
❍❍❍❍❍❍ ❥ ✲
Transient dist.
\transient{ \source{id==1} \target{ex==1} \t start{0.1} \t stop{5.0} }
JTB [14/07/2005] – p.22/29
Model description Performance query Performance results GSPN
... \transition{t1}{ \condition{cpu>0} \action{ cpu = cpu - 1; id = id + 1; } \rate{mu} } ...
PRISM m
... module queue q : [0..N]; [] q < N -> mu:(q’=q+1); [] q = N -> mu:(q’=q); [serve] q > 0 -> lambda:(q’=q-1); endmodule ...
✲ ✲
Response time
\passage{ \source{cpu==1} \target{id==1} \t start{0.1} \t stop{10.0} }
❍❍❍❍❍❍ ❥ ✲
Transient dist.
\transient{ \source{id==1} \target{ex==1} \t start{0.1} \t stop{5.0} }
✲ ✲
CSL query
serve ⇒ P(ar U ≤t fi)
JTB [14/07/2005] – p.22/29
When can I reuse the results of someone else’s model as well as the model itself separable solution steady-state → RCAT, product-forms passage-time/transient → ?
JTB [14/07/2005] – p.23/29
use your tool/formalism of choice allow modellers to write system components in the formalism that best suits the component
JTB [14/07/2005] – p.24/29
How can a stochastic Petri net model synchronise with stochastic process algebra component? ...or a PEPA component with a PRISM module? ...or an IMC component with a Stochastic automata network?
JTB [14/07/2005] – p.25/29
t1 p1 p2
❄ ❄ ❄ ❅ ❅ ❘
A def = (a, ν).A′ A′ def = (b, ⊤).A B def = (c, λ).B′ B′ def = (b, µ).B Sys def = A ✄
{b} B
?
JTB [14/07/2005] – p.26/29
t1 p1 p2
❄ ❄ ❄ ❅ ❅ ❘
A def = (a, ν).A′ A′ def = (b, ⊤).A B def = (c, λ).B′ B′ def = (b, µ).B Sys def = A ✄
{b} B
{t1/a}
JTB [14/07/2005] – p.27/29
My idea of how a DSPN should communicate with a SAN will differ from another modeller user-definable cooperation semantics user-definable translation semantics
JTB [14/07/2005] – p.28/29
Integrate with some parallelised toolsets: e.g. ETMCC, PRISM, DNAmaca, PWB Attach to a large cluster computer Allow jobs to be received/sent to other Perform-DB enabled clusters Hopefully has potential!
JTB [14/07/2005] – p.29/29