SMT Unsat Core Minimization OFER GUTHMANN, OFER STRICHMAN, ANNA TRO - - PowerPoint PPT Presentation

β–Ά
smt unsat core minimization
SMART_READER_LITE
LIVE PREVIEW

SMT Unsat Core Minimization OFER GUTHMANN, OFER STRICHMAN, ANNA TRO - - PowerPoint PPT Presentation

Jan 19, 2023 121 likes β€’394 views

SMT Unsat Core Minimization OFER GUTHMANN, OFER STRICHMAN, ANNA TRO STANETSKI FMCAD2016 1 SMT MUCS Satisfiability Modulo Theories Satisfiability Modulo Theories (SMT): decides satisfiability of formulas over first order theories, by

slide-1
SLIDE 1

SMT Unsat Core Minimization

OFER GUTHMANN, OFER STRICHMAN, ANNA TRO STANETSKI FMCAD2016

1

SMT MUCS

slide-2
SLIDE 2

Satisfiability Modulo Theories

Satisfiability Modulo Theories (SMT): decides satisfiability of formulas over first order theories, by combining

  • a SAT solver, and
  • decision procedures for conjunctions of first order literals.

2

SMT MUCS

slide-3
SLIDE 3

SMT solvers use Boolean Abstraction

Let πœ’ be an SMT formula πœ’β€™s Boolean Abstraction, 𝑓 πœ’ , assigns a Boolean variable to every theory literal in πœ’. Example:

  • πœ’ =

𝑦 = 0 ∧ 𝑦 = 1 ∨ Β¬ 𝑦 = 2

  • 𝑓 πœ’ = 𝑓1 ∧ 𝑓2 ∨ ¬𝑓3
  • Boolean structure unchanged.

Decoding: 𝑒 𝑓1 ≔ 𝑦 = 0 , 𝑒 𝑓2 ≔ 𝑦 = 1 , etc.

3

𝑓1 𝑓2 𝑓3

SMT MUCS

slide-4
SLIDE 4

The Minimal Unsat Core Problem (MUC)

Let πœ’ be an unsat SMT formula (in CNF). Find a minimal (i.e., irreducible) unsat core of πœ’β€™s clauses. πœ’ = 𝑏 ∧ ¬𝑏 ∨ 𝑐 ∧ ¬𝑏 ∨ ¬𝑐 ∧ 𝑐 ∨ 𝑑 𝐷 = 𝑏, ¬𝑏 ∨ 𝑐 , ¬𝑏 ∨ ¬𝑐 𝐷 is a minimal unsat core. Many applications may benefit from finding a MUC:

  • Abstraction refinement.
  • Formal equivalence verification.
  • Decision procedures.
  • Etc.

We know of no SMT MUC extractors in the public domain

4

SMT MUCS

slide-5
SLIDE 5

Deletion-based MUC Extraction (propositional case)

5

π‘Ίπ’‡π’π’‘π’˜π’‡ 𝒗𝒐𝒏𝒃𝒔𝒍𝒇𝒆 π’…π’Žπ’ƒπ’—π’•π’‡ 𝒅 ∈ 𝑫 𝑫 𝑺𝒇𝒖𝒗𝒔𝒐 𝑫 𝒁𝒇𝒕 𝑡𝒃𝒔𝒍 𝒅, 𝒃𝒐𝒆 𝒃𝒆𝒆 𝒋𝒖 𝒄𝒃𝒅𝒍 𝒖𝒑 𝑫 π‘©π’Žπ’Ž π’…π’Žπ’ƒπ’—π’•π’‡π’• 𝒏𝒃𝒔𝒍𝒇𝒆? 𝑻𝑩𝑼 𝑫 ? 𝒁𝒇𝒕 𝑢𝒑 𝑢𝒑 𝑫 ← 𝒅𝒑𝒔𝒇

SMT MUCS

slide-6
SLIDE 6

Z3 and Cores

Z3 is an open-source competitive SMT solver:

  • Developed by Microsoft Research.
  • Emits an unsat core (set of clauses used in proof).
  • Uses high-level proof rules

*Diagram taken from L. Zhang and S. Malik: Validating SAT Solvers Using an Independent Resolution-Based Checker: Practical Implementations and Other Applications. 2003.

SMT MUCS

6

Unsat Core

slide-7
SLIDE 7

HSmtMuc A Deletion-based SMT MUC Extractor

7

π‘Ίπ’‡π’π’‘π’˜π’‡ 𝑽𝒐𝒏𝒃𝒔𝒍𝒇𝒆 π’…π’Žπ’ƒπ’—π’•π’‡ 𝒅 ∈ 𝑫 𝑫 𝑺𝒇𝒖𝒗𝒔𝒐 𝑫 𝑡𝒃𝒔𝒍 𝒅, 𝒃𝒐𝒆 𝒃𝒆𝒆 𝒋𝒖 𝒄𝒃𝒅𝒍 𝒖𝒑 𝑫 π‘©π’Žπ’Ž π’…π’Žπ’ƒπ’—π’•π’‡π’• 𝒏𝒃𝒔𝒍𝒇𝒆? 𝑻𝑩𝑼 𝑫 ? 𝒁𝒇𝒕 𝑢𝒑 𝒁𝒇𝒕 𝑢𝒑 𝑫 ← 𝒅𝒑𝒔𝒇 π’‚πŸ’ 𝑫 ?

SMT MUCS

slide-8
SLIDE 8

Optimization: Rotation

* A. Belov and J. Marques-Silva. Accelerating MUS extraction with recursive model

  • rotation. 2011.

Let 𝑑 be a marked clause.

  • πœ’ βˆ– 𝑑 is satisfiable.
  • 𝛽 ⊨ πœ’ βˆ– 𝑑 .

Rotate(c, Ξ±)

  • Find Ξ±β€² β‰  Ξ± and cβ€² β‰  c, s.t. Ξ±β€² ⊨ Ο† βˆ– cβ€²
  • By flipping variables in Ξ± that appear in c.
  • If such cβ€² was found:
  • Mark cβ€²
  • Rotate(cβ€², Ξ±β€²)

8

SMT MUCS

slide-9
SLIDE 9

Now in SMT: Theory Rotation

Let 𝑑 be a marked clause.

  • πœ’ βˆ– 𝑑 is satisfiable.
  • 𝛽 ⊨ 𝑓 πœ’ βˆ– 𝑑

.

Rotate(c, Ξ±)

  • Find Ξ±β€² β‰  Ξ± and cβ€² β‰  c, s.t. Ξ±β€² ⊨ e Ο† βˆ– cβ€²

:

  • By flipping variables in Ξ± that appear in c.
  • If such cβ€² was found:
  • Mark cβ€²
  • Rotate(cβ€², Ξ±β€²)

The problem: the new assignment may not be T-consistent

9

Recall: 𝑓 applies Boolean abstraction Recall: 𝑓 applies boolean abstraction

SMT MUCS

slide-10
SLIDE 10

Theory Rotation – Contradiction Example

πœ’ = 𝑦 = 0

𝑑

∧ Β¬ 𝑦 = 0 ∨ 𝑦 = 1 ∧ Β¬ 𝑦 = 0 ∨ 𝑦 = 2 𝑓 πœ’ = 𝑓1

𝑓 𝑑

∧ ¬𝑓1 ∨ 𝑓2 ∧ ¬𝑓1 ∨ 𝑓3

For a model\interpretation where 𝑦 ⟼ 1 we have: 𝛽 ≔ 𝑓1, 𝑓3 ⟼ 𝐺, 𝑓2 ⟼ π‘ˆ

10

SMT MUCS

slide-11
SLIDE 11

Theory Rotation – Contradiction Example

πœ’ = 𝑦 = 0

𝑑

∧ Β¬ 𝑦 = 0 ∨ 𝑦 = 1 ∧ Β¬ 𝑦 = 0 ∨ 𝑦 = 2 𝑓 πœ’ = 𝑓1

𝑓 𝑑

∧ ¬𝑓1 ∨ 𝑓2 ∧ ¬𝑓1 ∨ 𝑓3

For a model\interpretation where 𝑦 ⟼ 1 we have: 𝛽 ≔ 𝑓1, 𝑓3 ⟼ 𝐺, 𝑓2 ⟼ π‘ˆ 𝛽 ⊨ 𝑓 πœ’ βˆ– 𝑑 Flipping 𝑓1 in 𝛽 results in a Tβˆ’contradiction.

  • both e1 β†’ 𝑦 = 0 and e2 β†’ (𝑦 = 1) now hold.

11

SMT MUCS

slide-12
SLIDE 12

Theory Rotation - Solution

After finding (c’, 𝛽′), check if 𝛽′ is T-consistent. If it is T-consistent use Rotate (c’, 𝛽′) as before. If it’s not...

  • One possibility is to give up and stop the recursion.
  • Let’s try and do better.

12

SMT MUCS

slide-13
SLIDE 13

Theory Rotation – Fixing a T-Contradiction

Try and find more variables to flip in 𝛽′. Variables to flip: choose from 𝑑𝑝𝑠𝑓 𝛽′ .

  • If resulting 𝛽′′ still contradictory, recursively flip more vars.
  • Recursion depth is determined heuristically.

𝛽′′ ⊨ πœ’ βˆ– 𝑑′′ and is T-consistent β‡’

  • mark 𝑑′′, and
  • Rotate (𝑑′′, 𝛽′′).

13

SMT MUCS

slide-14
SLIDE 14

Adaptive Activation of Theory Rotation

Failed Theory Rotation can be costly. Determine at runtime whether rotations is be continued: First option:

  • Fail Bound: stop after 𝑦 consecutive failures.
  • Failure: no clauses were marked.

Observation: Rotation success-rate declines through time.

14

SMT MUCS

slide-15
SLIDE 15

Adaptive Activation of Theory Rotation

Another option

  • Dynamic Measurement: estimate 𝑒𝑑𝑛𝑒 <

𝑒𝑠 π‘œπ‘  to stop rotation.

  • Problem: measurement is non-monotonic.

15

SMT MUCS 100 200 300 400 500 600 200 400 600 800 1000 1200 1400 1600 1800 2000

Time (ms) Iteration

Time cost per clause marking

SMT SAT check time 100 200 300 400 500 600 200 400 600 800 1000 1200 1400 1600 1800 2000

Time (ms) Iteration

Time cost per clause marking

SMT SAT check time Rotation time

slide-16
SLIDE 16

Adaptive Activation of Theory Rotation

Exponential smoothing: Given a stream of measurements 𝑒𝑑𝑛𝑒

𝑗

, 𝑒𝑠𝑝𝑒

𝑗

, π‘œπ‘ π‘π‘’

𝑗 𝑗=1 π‘œ

define: π‘ˆπ‘‘π‘›π‘’ = 𝑒𝑑𝑛𝑒 π‘ˆπ‘‘π‘›π‘’

𝑗

= 𝛽 β‹… 𝑒𝑑𝑛𝑒

𝑗

+ 1 βˆ’ 𝛽 β‹… π‘ˆπ‘‘π‘›π‘’

π‘—βˆ’1,

0 ≀ 𝛽 ≀ 1

  • Do the same for π‘ˆπ‘ π‘π‘’

𝑗

and 𝑂𝑠𝑝𝑒

𝑗

Stop rotation when π‘ˆπ‘‘π‘›π‘’

𝑗

<

π‘ˆ

𝑠𝑝𝑒 𝑗

𝑂𝑠𝑝𝑒

𝑗

holds. 𝛽 chosen heuristically.

SMT MUCS

16

slide-17
SLIDE 17

Adaptive Activation of Theory Rotation

Back to the example, now with exponential smoothing:

SMT MUCS

17

0.5 1 1.5 2 2.5 3 3.5 4 100 200 300 400 500 600 700 800 900 1000 1 21 33 44 58 67 76 88 99 107 118 127 136 147 156 165 171 180 189 198 207 216 223 235 245 251 257 264 271 278 284 294 303 309 316 325 333 341 348 354 361 369

Number of marked clauses Time (ms) Iteration

Time cost per clause marking (Uses exp. smoothing w. alpha = 0.1)

smt call time rotation call time (ratio) #clauses marked in rotation

slide-18
SLIDE 18

Experimental Results – Avg. core size reduction

561 unsat SMT-LIB instances*

  • Avg. core size:
  • Z3: 820 clauses.
  • Min:454 clauses.

*Same instances seleScted in A. Cimatti, A. Griggio, and R. Sebastiani: Computing small unsatisfiable cores in satisfiability modulo theories. 2011.

SMT MUCS

18

slide-19
SLIDE 19

Experimental Results – Theory Rotation

Reduces the number of (deletion) iterations.

19

SMT MUCS

slide-20
SLIDE 20

Experimental Results – Theory Rotation

Translates to a modest run-time improvement (~6%-10%) Can be attributed to time spent on failed rotations, T-contradiction checks and additional var. flipping. Best configuration is for Theory Rotation w. fail bound = 5

SMT MUCS

20

π·π‘π‘œπ‘”π‘—π‘•. Time (sec.) T-check Time (sec.) T-Conflicts Resolved (base) 30.5 0.0 0.0 T-Rotate 29.7 1.4 20.8 T-Rotate b 5 28.9 1.0 10.2 T-Rotate b 7 29.2 1.2 12.3 T-Rotate exp 29.6 1.2 11.2

slide-21
SLIDE 21

And now... Small Unsatisfiable Core (SUC)

[1] suggested an algorithm that finds a small (not necessarily minimal) SMT core

  • Based on MathSat and the propos. MUC extractor Muser2

We re-implemented [1] based on Z3 + HaifaMuc We also tested a hybrid approach in which we find a small core and then minimize it with HSmtMuc

HAIFA SMT MUCS

21

[1] A. Cimatti, A. Griggio, and R. Sebastiani. Computing small unsatisfiable cores in satisfiability modulo theories (2011).

slide-22
SLIDE 22

Small Unsatisfiable Core (SUC)

Our re-implementation with Z3 and HaifaMUC:

  • Requires proof logging (slows Z3 a lot).
  • Requires a propositional encoding of Z3’s proof objects.
  • Produces much larger proofs on avg. comparing to MathSat.
  • Turned-out to be slower

SMT MUCS

22

MathSat Muser2 Extract 𝑒(𝑠𝑝𝑝𝑒𝑑′) Proof 𝑓(𝑠𝑝𝑝𝑒𝑑 + π‘šπ‘“π‘›π‘›π‘π‘‘) 𝑓(𝑠𝑝𝑝𝑒𝑑’) SUC

slide-23
SLIDE 23

We also tried a hybrid approach

MathSat-based SUC + minimization with HSmtMuc.

  • Result is minimal.

The overall winner. Less time-outs (HSmtMuc alone: 171 vs. Hybrid: 138).

  • (but higher runtime than HSmtMuc on instances that

completed, HSmtMuc: 22.9 sec. vs. Hybrid: 27.9 sec.).

23

SMT MUCS

MathSat + Muser2 HSmtMuc πœ’ 𝑇𝑉𝐷 𝑁𝑉𝐷

slide-24
SLIDE 24

Summary

HSmtMuc is the first SMT-MUC extractor in the public domain.

  • Based on Z3.

Best observed results: MUC: the Hybrid algorithm

  • MathSat SUC extraction, followed by HSmtMuc.

SUC:

  • MathSat SUC extraction.

More information & our implementation is available at http://strichman.net.technion.ac.il/

24

SMT MUCS

slide-25
SLIDE 25

Questions?

25

SMT MUCS

slide-26
SLIDE 26

Thank you!

  • ofers@ie.technion.ac.il
  • ofer.guthmann@cs.technion.aci.il
  • annat@cs.technion.ac.il

26

SMT MUCS