Developing Correctly Replicated Databases Using Formal Tools Nicolas - - PowerPoint PPT Presentation

▶
developing correctly replicated databases using formal
SMART_READER_LITE
LIVE PREVIEW

Developing Correctly Replicated Databases Using Formal Tools Nicolas - - PowerPoint PPT Presentation

Aug 02, 2023 447 likes •814 views

Developing Correctly Replicated Databases Using Formal Tools Nicolas Schiper, Vincent Rahli , Robbert Van Renesse, Mark Bickford, and Robert L. Constable May 30, 2017 Vincent Rahli May 30, 2017 1/35 PRL & System Groups PRL group Mark

slide-1
SLIDE 1

Developing Correctly Replicated Databases Using Formal Tools

Nicolas Schiper, Vincent Rahli, Robbert Van Renesse, Mark Bickford, and Robert L. Constable May 30, 2017

Vincent Rahli May 30, 2017 1/35

slide-2
SLIDE 2

PRL & System Groups

PRL group Mark Bickford Robert L. Constable Richard Eaton Vincent Rahli System group Robbert van Renesse Nicolas Schiper Vincent Rahli May 30, 2017 2/35

slide-3
SLIDE 3

Goals

What we strive for: A platform to develop provably correct programs. Our current interest: Specify, verify, and generate distributed systems using formal

  • tools. (As part of the CRASH project funded by DARPA.)
{ Today applications are distributed over many machines. { Even critical applications used by governments, banks,

armies, etc.

Vincent Rahli May 30, 2017 3/35

slide-4
SLIDE 4

Goals

Correctness? How can we make sure that these applications are correct? Distributed programs are hard to specify, implement, and reason about.

{ We need to tolerate failures. { It is hard to test all possible scenarios. { State space explosion using model checking. { Model checking often done on abstractions of the code

rather than on the code itself. We use a proof assistant (Nuprl) that implements a constructive type theory.

Vincent Rahli May 30, 2017 4/35

slide-5
SLIDE 5

Achievements

{ A logic of events implemented in Nuprl. { Specified, verified, and generated consensus protocols

(e.g., Paxos).

{ Aneris: a total ordered broadcast service [RSR+12]. { ShadowDB: a replicated database with 2 parametrizable

replication protocols (PBR & SMR) built on top of Aneris [SRR+12].

{ Improved performance without introducing bugs [RBA13]. { We get decent performance.

Vincent Rahli May 30, 2017 5/35

slide-6
SLIDE 6

Table of contents

ShadowDB Aneris: a provably correct ordered broadcast service Evaluation Conclusion

Vincent Rahli May 30, 2017 6/35

slide-7
SLIDE 7

The Big Picture

Vincent Rahli May 30, 2017 7/35

slide-8
SLIDE 8

Primary-Backup Replication

Vincent Rahli May 30, 2017 8/35

slide-9
SLIDE 9

Primary-Backup Replication

Vincent Rahli May 30, 2017 9/35

slide-10
SLIDE 10

Primary-Backup Replication

Vincent Rahli May 30, 2017 10/35

slide-11
SLIDE 11

State Machine Replication

Vincent Rahli May 30, 2017 11/35

slide-12
SLIDE 12

Aneris

A synthesized and verified ordered broadcast service. ensures among other things (properties of atomic broadcast):

◮ agreement: for any slot s, if decisions (r1, s) and (r2, s)

get delivered then r1 = r2.

◮ validity: if decision (r, s) is delivered then r was

requested.

Vincent Rahli May 30, 2017 12/35

slide-13
SLIDE 13

Methodology

Vincent Rahli May 30, 2017 13/35

slide-14
SLIDE 14

Methodology

Vincent Rahli May 30, 2017 14/35

slide-15
SLIDE 15

Methodology

Vincent Rahli May 30, 2017 15/35

slide-16
SLIDE 16

Methodology

Vincent Rahli May 30, 2017 16/35

slide-17
SLIDE 17

Methodology

Vincent Rahli May 30, 2017 17/35

slide-18
SLIDE 18

Methodology

Vincent Rahli May 30, 2017 18/35

slide-19
SLIDE 19

Methodology

Vincent Rahli May 30, 2017 19/35

slide-20
SLIDE 20

EML, LoE, and GPM

In LoE [BC08, Bic09, BCR12], we specify distributed programs by combining event handlers (similar to Orc) which are all implementable by simple processes [BCG10]:

{ base: { parallel composition: A || B

λe.A(e) ∪ B(e)

Vincent Rahli May 30, 2017 20/35

slide-21
SLIDE 21

EML, LoE, and GPM

{ application: { buffer: { delegation:

Vincent Rahli May 30, 2017 21/35

slide-22
SLIDE 22

EventML

2/3-Consensus:

. . c l a s s TT Replica = NewVoters > >= Voter ; ; main TT Replica @ l o c s

Paxos Synod:

. . . c l a s s Leader = SpawnFirstSc out | | (( LeaderPropose | | LeaderAdopted ) > >= Commander ) | | ( LeaderPreempted > >= Scout ) ; ; main Leader @ l d r s | | Acceptor @ ac c pts

Aneris replicas:

. . . c l a s s R e p l i c a S t a t e = State (\ . ( i n i t s t a t e ,{}) ,

  • u t t r

p r o p o s e i n l , swap’base ,

  • u t t r

p r o p o s e i n r , b c a s t ’ b a s e ,

  • u t t r
  • n d e c i s i o n ,

d e c i s i o n ’ b a s e ) ; ; c l a s s R e p l i c a = (\ . snd ) o R e p l i c a S t a t e ; ; main R e p l i c a @ r e p s Vincent Rahli May 30, 2017 22/35

slide-23
SLIDE 23

Code Synthesis

Optimized version of the Aneris process:

aneris_main-program-opt(Cid;Op;clients;eq_Cid;pax_procs;reps;tt_procs) == λi.case bag-deq-member(λa,b.if a=2 b then inl · else (inr · );i;reps)

  • f inl() =>

fix((λmk-hdf,s. (inl (λv.let x,y = v in case name_eq(x;[swap]) ∧

b ...
  • f inl(x1) =>

let v1 ← ... aneris_propose_inl(Cid;Op;...;...;...;...;...) ... in let x,y = v1 in let v2 ← y @ [] in <mk-hdf <x, y>, v2> | inr(y1) => case name_eq(x;[bcast]) ∧

b ...
  • f inl(x1) =>

let v1 ← ... aneris_propose_inr(Cid;Op;...;...;...;...;...) ... in let x,y = v1 in let v2 ← y @ [] in <mk-hdf <x, y>, v2> | inr(y1) => case name_eq(x;[decision]) ∧

b ...
  • f inl(x1) =>

let v1 ← ... aneris_on_decision(Cid;Op;...;...;...;...;...;...;...) ... in let x,y = v1 in let v2 ← y @ [] in <mk-hdf <x, y>, v2> | inr(y1) => let v1 ← s in let x,y = v1 in let v2 ← y @ [] in <mk-hdf <x, y>, v2>) ))) <aneris_init_state(Cid;Op), []> | inr() => inr · Vincent Rahli May 30, 2017 23/35

slide-24
SLIDE 24

Verification

We use causal induction and inductive logical forms (ILFs).

Vincent Rahli May 30, 2017 24/35

slide-25
SLIDE 25

Verification

E.g., logical explanation of why decisions are made by Paxos:

∀[Cmd:{T:Type| valueall-type(T)} ]. ∀[accpts,ldrs:bag(Id)]. ∀[ldrs_uid:Id → Z]. ∀[reps:bag(Id)]. ∀[es:EO’]. ∀[e:E]. ∀[i:Id]. ∀[p:Proposal]. (decision’send(Cmd) i p ∈ pax_mb_main(Cmd;accpts;ldrs;ldrs_uid;reps)(e) ⇐ ⇒ loc(e) ∈ ldrs ∧ (header(e) = ‘‘pax_mb p2b‘‘) ∧ (msgtype(e) = P2b) ∧ i ∈ reps ∧ (∃e’:{e’:E| e’ ≤loc e } ∃z:PValue ((((header(e’) = [propose]) ∧ (msgtype(e’) = Proposal) ∧ ((↑ (proposal_slot (proposal_cmd LeaderStateFun(e’)))) ∧ (¬↑ (in_domain (proposal_slot msgval(e’)) (proposal_cmd (proposal_cmd LeaderStateFun(e’)))))) ∧ (z = (mk_pvalue (proposal_slot LeaderStateFun(e’)) msgval(e’)))) ∨ ((header(e’) = ‘‘pax_mb adopted‘‘) ∧ (msgtype(e’) = pax_mb_AState(Cmd)) ∧ ((astate_ballot msgval(e’)) = (proposal_slot LeaderStateFun(e’))) ∧ z ∈ map(λsp.(mk_pvalue (astate_ballot msgval(e’)) sp); update_proposals (proposal_cmd (proposal_cmd LeaderStateFun(e’))) (pmax(ldrs_uid) (astate_pvals msgval(e’)))))) ∧ (no commander_output(accpts;reps) z@Loc

  • (Loc,p2b’base(), CommanderState(accpts) (pval_ballot z) (proposal_slot (pval_proposal z)))

between e’ and e) ∧ ((pval_ballot z) = (bl_ballot (p2b_bl msgval(e)))) ∧ ((proposal_slot (pval_proposal z)) = (p2b_slot msgval(e))) ∧ ((pval_ballot z) = (p2b_ballot msgval(e))) ∧ (#(CommanderStateFun(pval_ballot z;proposal_slot (pval_proposal z);es.e’;e)) < threshold(accpts)) ∧ (p = (pval_proposal z))))) decision of p sent to i at e e happens at a leader location the decision is triggered by a p2b message the recipient of the decision message is a replica proposal p is extracted from a pvalue z either pvalue z is made from a proposal and current ballot

  • r either pvalue z received in an adopted message or in leader state

this decision is the first output of the commander the acceptor that sent the p2b message has accepted pvalue z the commander has received a p2b messages from a majority of acceptors

Vincent Rahli May 30, 2017 25/35

slide-26
SLIDE 26

Verification

EventML LoE GPM opt. GPM correctness correctness spec. spec. prog. prog. properties proofs CLK 79N (1H) 590N 452N 249N 73N (1H) 1A/3M (2H) 2/3 Consensus 646N (4H) 1398N 1343N 1752N 122N (1H) 8A/6M (3D) Paxos-Synod 1729N (2D) 2673N 2625N 3165N 97N (1H) 24A/75M (3W) Aneris 820N (2D) 1434N 1352N 1245N 418N (1H) 0A/22M (1W)

That was possible thanks:

◮ to Nuprl’s large library of definitions and facts, ◮ to the powerful logic of events theory developed in Nuprl

by Mark Bickford and Robert Constable over the past few years (especially to the delegation combinator), and

◮ to the collaboration between the PRL and system groups

at Cornell.

Vincent Rahli May 30, 2017 26/35

slide-27
SLIDE 27

Table of Contents

ShadowDB Aneris: a provably correct ordered broadcast service Evaluation Conclusion

Vincent Rahli May 30, 2017 27/35

slide-28
SLIDE 28

Evaluation

Setup:

◮ Quad-core 3.6 Ghz Xeons with 4GB running RH 5.8 ◮ Gigabit switch ◮ Various embedded and in-memory DBs

We evaluate:

◮ Aneris (the broadcast service) ◮ ShadowDB

◮ Micro-benchmark (1 table, single-row update) ◮ TPC-C (9 tables, 5 transaction types, 92% updates) Vincent Rahli May 30, 2017 28/35

slide-29
SLIDE 29

Evaluation - Aneris

1 10 100 1000 1 10 100 1000 10000

Latency (ms) Delivered messages per second Interpreted –+– Inter.-Opt. – – Compiled –×–

Vincent Rahli May 30, 2017 29/35

slide-30
SLIDE 30

Evaluation - ShadowDB - Micro-benchmark

0.1 1 10 100 2K 4K 6K 8K

Latency (ms) Committed transactions per second ShadowDB-PBR –+– ShadowDB-SMR – – H2-repl. – – MySQL-repl. – – H2-stdalone –•–

Vincent Rahli May 30, 2017 30/35

slide-31
SLIDE 31

Evaluation - ShadowDB - TPC-C

1 10 100 200 400 600 800 1000

Latency (ms) Committed TPC-C transactions per second ShadowDB-PBR –+– ShadowDB-SMR – –

MySQL-repl. – –

H2-stdalone –•–

Vincent Rahli May 30, 2017 31/35

slide-32
SLIDE 32

Table of Contents

ShadowDB Aneris: a provably correct ordered broadcast service Evaluation Conclusion

Vincent Rahli May 30, 2017 32/35

slide-33
SLIDE 33

Even More Trustworthy Distributed Systems

Vincent Rahli May 30, 2017 33/35

slide-34
SLIDE 34

Summary

{ Provably correct distributed protocols. { Aneris in used by the replicated database ShadowDB that

itself will be used by Nuprl.

{ Decent performance. { Example that our methodology to specify (using small

human manageable components) and verify (ILFs + causal induction) protocols works.

Vincent Rahli May 30, 2017 34/35

slide-35
SLIDE 35

References I

Mark Bickford and Robert L. Constable. Formal foundations of computer security. In NATO Science for Peace and Security Series, D: Information and Communication Security, volume 14, pages 29–52. 2008. Mark Bickford, Robert Constable, and David Guaspari. Generating event logics with higher-order processes as realizers. Technical report, Cornell University, 2010. Mark Bickford, Robert L. Constable, and Vincent Rahli. Logic of events, a framework to reason about distributed systems. In Languages for Distributed Algorithms Workshop, 2012. Mark Bickford. Component specification using event classes. In Component-Based Software Engineering, 12th Int’l Symp., volume 5582 of LNCS, pages 140–155. Springer, 2009. Vincent Rahli, Mark Bickford, and Abhishek Anand. Formal program optimization in Nuprl using computational equivalence and partial types. In ITP’13, volume 7998 of LNCS, pages 261–278. Springer, 2013. Vincent Rahli, Nicolas Schiper, Robbert Van Renesse, Mark Bickford, and Robert L. Constable. A diversified and correct-by-construction broadcast service. In The 2nd Int’l Workshop on Rigorous Protocol Engineering (WRiPE), October 2012. Nicolas Schiper, Vincent Rahli, Robbert Van Renesse, Mark Bickford, and Robert L. Constable. ShadowDB: A replicated database on a synthesized consensus core. In Eighth Workshop on Hot Topics in System Dependability, HotDep’12, 2012. Vincent Rahli May 30, 2017 35/35