Growing Solver-Aided Languages with ROSETTE Emina Torlak & - - PowerPoint PPT Presentation

Emina Torlak & Rastislav Bodik

U.C. Berkeley

Growing Solver-Aided Languages with ROSETTE

solver-aided domain-specific language

2

Solver-aided DSL (SDSL) Noun

• 1. A high-level language in which

partially implemented programs can be executed, verified, debugged and synthesized with the aid of a constraint solver.

programming …

3

specification assume pre(x) P(x) { … } assert post(P(x))

programming …

3

assume pre(x) P(x) { … } assert post(P(x))

formula, input/

• utput pairs,

traces, another program, …

programming with a solver

4

assume pre(x) P(x) { … } assert post(P(x)) assert post(P(x)) assume pre(x)

?

SAT/SMT solver translate(…)

programming with a solver: code checking

5

assume pre(x) P(x) { … } assert post(P(x)) SAT/SMT solver Is there a valid input x for which P(x) violates the spec?

∃x . pre(x) ⋀

¬post(P(x))

CBMC [Oxford], Dafny [MSR], Jahob [EPFL], Miniatur / MemSAT [IBM], etc.

programming with a solver: code checking

5

assume pre(x) P(x) { … } assert post(P(x)) SAT/SMT solver Is there a valid input x for which P(x) violates the spec?

∃x . pre(x) ⋀

¬post(P(x))

CBMC [Oxford], Dafny [MSR], Jahob [EPFL], Miniatur / MemSAT [IBM], etc.

model x = 42 counterexample

programming with a solver: localizing faults

6

SAT/SMT solver Given x and x′, what subset of P is responsible for P(x) ≠ x′? pre(x) ⋀ post(x′) ∧ x′ = P(x) assume pre(x) P(x) { v = x + 2 … } assert post(P(x))

BugAssist [UCLA / MPI-SWS]

programming with a solver: localizing faults

6

SAT/SMT solver MIN CORE / MAXSAT Given x and x′, what subset of P is responsible for P(x) ≠ x′? pre(x) ⋀ post(x′) ∧ x′ = P(x) repair candidates assume pre(x) P(x) { v = x + 2 … } assert post(P(x))

BugAssist [UCLA / MPI-SWS]

programming with a solver: angelic execution

7

SAT/SMT solver Given x, choose v at runtime so that P(x, v) satisfies the spec.

∃v . pre(x) ⋀

post(P(x, v)) assume pre(x) P(x) { v = choose() … } assert post(P(x))

Kaplan [EPFL], PBnJ [UCLA], Skalch [Berkeley], Squander [MIT], etc.

programming with a solver: angelic execution

7

SAT/SMT solver Given x, choose v at runtime so that P(x, v) satisfies the spec.

∃v . pre(x) ⋀

post(P(x, v)) assume pre(x) P(x) { v = choose() … } assert post(P(x)) model v = 0, … trace

Kaplan [EPFL], PBnJ [UCLA], Skalch [Berkeley], Squander [MIT], etc.

programming with a solver: synthesis

8

SAT/SMT solver Replace ?? with expression e so that Pe(x) satisfies the spec on all valid inputs. assume pre(x) P(x) { v = ?? … } assert post(P(x))

∃e . ∀x . pre(x) ⇒

post(Pe(x))

Comfusy [EPFL], Sketch [Berkeley / MIT]

programming with a solver: synthesis

8

SAT/SMT solver Replace ?? with expression e so that Pe(x) satisfies the spec on all valid inputs. assume pre(x) P(x) { v = ?? … } assert post(P(x)) model expressions

∃e . ∀x . pre(x) ⇒

post(Pe(x))

Comfusy [EPFL], Sketch [Berkeley / MIT]

x − 2

but building solver-aided languages is hard …

9

P(x) { … } SAT/SMT solver translate(…)

?? ?

(Q x …) R(x) : … translate(…) translate(…)

Each new SDSL created by careful custom compilation to constraints, requiring years of training and experience.

a solver-aided framework for building SDSLs

10

P(x) { … }

?? ?

(Q x …) R(x) : …

ROSETTE

interpret(…) interpret(…) API(…)

Implement a library or an interpreter for your SDSL, and get a synthesizer, verifier, debugger and angelic oracle for programs in that SDSL.

a tiny solver-aided extension of racket …

11

Racket

(define-symbolic id expr) (define-symbolic* id expr) (assert expr) (solve expr) (verify expr) (debug [expr ...+] expr) (synthesize #:forall expr #:guarantee expr)

ROSETTE

… with a symbolic evaluator and compiler

12

ROSETTE

racket SDSL + program synthesize debug verify solve solver

transform, evaluate & compile to constraints

KODKOD

… with a symbolic evaluator and compiler

12

ROSETTE

racket SDSL + program synthesize debug verify solve solver

map solution to program level

KODKOD

… with a symbolic evaluator and compiler

12

ROSETTE

racket SDSL + program synthesize debug verify solve solver

map solution to program level

KODKOD

rosette by example: an SDSL for circuits

13

F T T T F T T T spec impl

Why a circuit language?

• A teaching aid
• An oracle for

testing circuit transformations in SAT

• based

solvers Booln → Bool Booln → Bool

b c d b c d a a

rosette by example: an SDSL for circuits

13

F T T T F T T T spec impl

∀ a, b, c, d .

impl(a, b, c, d) ≡ spec(a, b, c, d)

verify Why a circuit language?

• A teaching aid
• An oracle for

testing circuit transformations in SAT

• based

solvers

b c d b c d a a

rosette by example: an SDSL for circuits

13

F T T T F T T T spec impl

debug Why a circuit language?

• A teaching aid
• An oracle for

testing circuit transformations in SAT

• based

solvers

F T

rosette by example: an SDSL for circuits

13

F T T T F T T T spec impl

synthesize Why a circuit language?

• A teaching aid
• An oracle for

testing circuit transformations in SAT

• based

solvers

b c d b c d a a

∀ a, b, c, d .

impl(a, b, c, d) ≡ spec(a, b, c, d)

verify

b c d b c d a a

a tiny circuit language (tcl) in racket

14

Warm up A classic DSL for testing and verification of circuits.

spec impl

a sample tcl program

15

A circuit is a procedure that works

• n boolean values.

(define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) #lang s-exp tcl (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity)

a sample tcl program

15

> (RBC-parity #f #f #t #f) #t > (AIG-parity #f #f #t #f) #t (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) #lang s-exp tcl (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity)

a sample tcl program

15

(define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) #lang s-exp tcl (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity)

And Inverter Graph (¬, ∧) Reduced Boolean Circuit (¬, ⇔)

a sample tcl program

15

Verifies equivalence of two

n-ary circuit functions.

(define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) #lang s-exp tcl (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity)

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

a shallow embedding of tcl in racket

16

(define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d))))

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

a shallow embedding of tcl in racket

16

(define (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d))))

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

a shallow embedding of tcl in racket

16

(define (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d))))

7

(define (! a) (if a #f #t))

8

(define (&& a b) (if a b #f))

9

(define (|| a b) (if a #t b))

10

(define (<=> a b) (if a b (! b)))

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

a shallow embedding of tcl in racket

16

Naive exhaustive verifier.

7

(define (! a) (if a #f #t))

8

(define (&& a b) (if a b #f))

9

(define (|| a b) (if a #t b))

10

(define (<=> a b) (if a b (! b)))

12

(define (verify-circuit impl spec)

13

(define n (procedure-arity spec))

14

(for ([i (expt 2 n)])

15

(define bits (for/list ([j n]) (bitwise-bit-set? i j)))

16

(unless (eq? (apply impl bits) (apply spec bits))

17

(error "failed on" bits))))

19

(provide

20

! && || <=> define-circuit verify-circuit

21

#%datum #%app #%module-begin #%top-interaction)

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

a shallow embedding of tcl in racket

16

#lang s-exp tcl

7

(define (! a) (if a #f #t))

8

(define (&& a b) (if a b #f))

9

(define (|| a b) (if a #t b))

10

(define (<=> a b) (if a b (! b)))

12

(define (verify-circuit impl spec)

13

(define n (procedure-arity spec))

14

(for ([i (expt 2 n)])

15

(define bits (for/list ([j n]) (bitwise-bit-set? i j)))

16

(unless (eq? (apply impl bits) (apply spec bits))

17

(error "failed on" bits))))

verifying circuits with tcl

17

#lang s-exp tcl (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity)

j failed on (#f #f #f #f)

> (RBC-parity #f #f #f #f) #f > (AIG-parity #f #f #f #f) #t

verifying circuits with tcl

17

#lang s-exp tcl (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity)

j failed on (#f #f #f #f)

> (RBC-parity #f #f #f #f) #f > (AIG-parity #f #f #f #f) #t

Where is the bug? How to fix it? Verify circuits with many inputs?

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

7

(define (! a) (if a #f #t))

8

(define (&& a b) (if a b #f))

9

(define (|| a b) (if a #t b))

10

(define (<=> a b) (if a b (! b)))

12

(define (verify-circuit impl spec)

13

(define n (procedure-arity spec))

14

(for ([i (expt 2 n)])

15

(define bits (for/list ([j n]) (bitwise-bit-set? i j)))

16

(unless (eq? (apply impl bits) (apply spec bits))

17

(error "failed on" bits))))

19

(provide

20

! && || <=> define-circuit verify-circuit

21

#%datum #%app #%module-begin #%top-interaction)

porting tcl to rosette

18

1

#lang racket

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

7

(define (! a) (if a #f #t))

8

(define (&& a b) (if a b #f))

9

(define (|| a b) (if a #t b))

10

(define (<=> a b) (if a b (! b)))

12

(define (verify-circuit impl spec)

13

(define n (procedure-arity spec))

14

(for ([i (expt 2 n)])

15

(define bits (for/list ([j n]) (bitwise-bit-set? i j)))

16

(unless (eq? (apply impl bits) (apply spec bits))

17

(error "failed on" bits))))

19

(provide

20

! && || <=> define-circuit verify-circuit

21

#%datum #%app #%module-begin #%top-interaction)

porting tcl to rosette

18

s-exp rosette

1

#lang s-exp rosette

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

7

(define (verify-circuit impl spec)

8

(define input (symbolic-input spec))

9

(evaluate input (verify (correct impl spec input))))

11

(define (symbolic-input spec)

12

(for/list ([i (procedure-arity spec)])

13

(define-symbolic* b boolean?)

14

b))

16

(define (correct impl spec input)

17

(assert (eq? (apply impl input) (apply spec input))))

19

(provide

20

! && || <=> define-circuit verify-circuit

21

#%datum #%app #%module-begin #%top-interaction)

tcl with solver-aided verification (tcl+)

19

Creates a list of fresh symbolic boolean constants to use as input to the circuits.

1

#lang s-exp rosette

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

7

(define (verify-circuit impl spec)

8

(define input (symbolic-input spec))

9

(evaluate input (verify (correct impl spec input))))

11

(define (symbolic-input spec)

12

(for/list ([i (procedure-arity spec)])

13

(define-symbolic* b boolean?)

14

b))

16

(define (correct impl spec input)

17

(assert (eq? (apply impl input) (apply spec input))))

19

(provide

20

! && || <=> define-circuit verify-circuit

21

#%datum #%app #%module-begin #%top-interaction)

tcl with solver-aided verification (tcl+)

19

(verify expr) searches for an interpretation of symbolic constants that causes expr to fail.

1

#lang s-exp rosette

3

(define-syntax-rule

4

(define-circuit (id in ...) expr)

5

(define (id in ...) expr))

7

(define (verify-circuit impl spec)

8

(define input (symbolic-input spec))

9

(evaluate input (verify (correct impl spec input))))

11

(define (symbolic-input spec)

12

(for/list ([i (procedure-arity spec)])

13

(define-symbolic* b boolean?)

14

b))

16

(define (correct impl spec input)

17

(assert (eq? (apply impl input) (apply spec input))))

19

(provide

20

! && || <=> define-circuit verify-circuit

21

#%datum #%app #%module-begin #%top-interaction)

tcl with solver-aided verification (tcl+)

19

#lang s-exp tcl+ (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (verify-circuit AIG-parity RBC-parity) ’(#f #t #t #t) > (RBC-parity #f #t #t #t) #t > (AIG-parity #f #t #t #t) #f

verifying circuits with tcl+

20

#lang s-exp tcl+ (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define/debug (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (debug-circuit AIG-parity RBC-parity ’(#f #t #t #t))

debugging circuits with tcl+ (desiderata)

21

implementation Why does the implementation differ from the specification

• n this input?

specification

22

(require

23

rosette/lang/debug

24

rosette/lib/tools/render)

26

(define (debug-circuit impl spec input)

27

(render

28

(debug [boolean?]

29

(correct impl spec input))))

31

(provide debug-circuit define/debug quote)

(debug [t] expr) computes a minimal set

• f expressions of type t

that are responsible for the failure in expr.

extending tcl+ with solver-aided debugging

22

#lang s-exp tcl+ (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define/debug (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (&& (! a) (! b)))) (! (&& (&& (! c) (! d)) (! (&& c d)))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (debug-circuit AIG-parity RBC-parity ’(#f #t #t #t))

debugging circuits with tcl+ (a minimal core)

23

Non-core (grayed-out) expressions are irrelevant to the failure: there is no way to fix the behavior of AIG- parity on the sample input by editing these expressions.

#lang s-exp tcl+ (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define-circuit (AIG-parity a b c d) (&& (Circuit [! &&] a b c d #:depth 3) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (synthesize-circuit AIG-parity RBC-parity)

synthesizing circuits with tcl+ (desiderata)

24

A sketch of the desired repair. Complete the sketch so that the given circuits are equivalent on all inputs.

(synthesize #:forall V #:guarantee expr[V, H]) searches for an interpretation of the symbolic constants H that guarantees expr will succeed for all interpretations of V.

32

(require rosette/lib/meta/meta)

34

(define (synthesize-circuit impl spec)

35

(define input (symbolic-input spec))

36

(generate-forms

37

(synthesize #:forall input

38

#:guarantee (correct impl spec input))))

40

(define-synthax (Circuit [op1 op2 ...] expr ... #:depth d)

41

#:assert (>= d 0)

42

([choose op1 identity]

43

[choose

44

expr ...

45

([choose op2 ...]

46

(Circuit [op1 op2 ...] expr ... #:depth (- d 1))

47

(Circuit [op1 op2 ...] expr ... #:depth (- d 1)))]))

49

(provide synthesize-circuit Circuit (for-syntax #%datum))

extending tcl+ with solver-aided synthesis

25

32

(require rosette/lib/meta/meta)

34

(define (synthesize-circuit impl spec)

35

(define input (symbolic-input spec))

36

(generate-forms

37

(synthesize #:forall input

38

#:guarantee (correct impl spec input))))

40

(define-synthax (Circuit [op1 op2 ...] expr ... #:depth d)

41

#:assert (>= d 0)

42

([choose op1 identity]

43

[choose

44

expr ...

45

([choose op2 ...]

46

(Circuit [op1 op2 ...] expr ... #:depth (- d 1))

47

(Circuit [op1 op2 ...] expr ... #:depth (- d 1)))]))

49

(provide synthesize-circuit Circuit (for-syntax #%datum))

extending tcl+ with solver-aided synthesis

25

To construct a circuit, choose a unary

• perator (or identity)

and apply it either to

• ne of the provided

terminal expressions or to a circuit constructed using one of the given binary operators.

#lang s-exp tcl+ (define-circuit (RBC-parity a b c d) (! (<=> (<=> a b) (<=> c d)))) (define-circuit (AIG-parity a b c d) (&& (! (&& (! (&& (! (&& a b)) (! (&& (! a) (! b))))) (! (&& (! (&& c d)) (! (&& (! c) (! d))))))) (! (&& (&& (! (&& a b)) (! (&& (! a) (! b)))) (&& (! (&& (! c) (! d))) (! (&& c d))))))) (synthesize-circuit AIG-parity RBC-parity)

synthesizing circuits with tcl+ (the repair)

26

(Circuit [! &&] a b c d #:depth 3) (! (&& (&& (! (&& a b)) (! (&& (!

stacking interpreters to synthesize a compiler

27

TTL+ RBC AIG Z3 rax

. RBC . RBC ≡ rax(RBC)

∃ ∀

stacking interpreters to synthesize a compiler

27

TTL+ RBC AIG Z3 rax

. RBC . RBC ≡ rax(RBC)

∃ ∀

Synthesized a rewriter from RBCs to AIGs that works for all RBCs of bounded size. Deep SDSL embedding

web, spatial programming, superoptimization

28

websynth: web scraping by demonstration

29

Websynth synthesizes a scraping script given a web page and a few examples

• f desired data, using a

declarative SDSL (XPaths). Implemented by two undergraduate students in a few weeks.

Sail, AWOLNATION Diamonds, Rihanna

websynth: web scraping by demonstration

29

Sail, AWOLNATION Diamonds, Rihanna 10 20 30 2 4 8 16 synthesis time (sec) number of examples iTunes IMBDb AlAnon

partitioning code & data for a low power chip

30

Figure by Per Ljung

Instructions/Second vs Power

~100x

GreenArrays GA144 Processor

• Stack-based 18-bit architecture
• 32 instructions
• 8 x 18 array of asynchronous cores
• No shared resources (cache, memory)
• Limited communication, neighbors only
• < 300 byte memory per core

partitioning code & data for a low power chip

30

Figure by Per Ljung

Instructions/Second vs Power

~100x

GreenArrays GA144 Processor

• Stack-based 18-bit architecture
• 32 instructions
• 8 x 18 array of asynchronous cores
• No shared resources (cache, memory)
• Limited communication, neighbors only
• < 300 byte memory per core

Manual function partitioning: break functions up into a pipeline with a few operations per core.

Drawing by Mangpo Phothilimthana

partitioning code & data for a low power chip

30

Figure by Per Ljung

Instructions/Second vs Power

~100x

GreenArrays GA144 Processor

partitioning code & data for a low power chip

31

partitioner code generator

high-level program per-core high-level programs per-core optimized machine code new programming model new approach using synthesis

Mangpo Phothilimthana Nishant Totla

partitioning code & data for a low power chip

31

partitioner code generator

high-level program per-core high-level programs per-core optimized machine code new programming model new approach using synthesis

Mangpo Phothilimthana Nishant Totla

partitioning code & data for a low power chip

32

Ri

256-byte mem per core initial data placement specified

106 6

K

103 3 102 104 4 105 5 2

F

M R M K

F

<<< <<<

high low

• ne iteration of MD5 hash
• ptimal code partitioning

(synthesized in 5 min)

Ki Mi

(define (fast-max x y) (let* ([t1 (bvge x y]) [t2 (bvneg t1]) [t3 (bvxor y x]) [t4 (bvand t3 t2]) [t5 (bvxor y t4)]) t5)) (define-fragment (fast-max x y) #:ensures (lambda (x y result) (= result (if (> x y) x y))) #:library (bvlib [{bvneg bvge bvand} 1] [{bvxor} 2]))

component-based synthesis of loop free code

33

Gulwani et al. PLDI’11 1 10 100 1000 synthesis time (sec) Hacker’s Delight benchmarks completed timeout

34

34

ROSETTE

{emina, bodik}@eecs.berkeley.edu