Empirical Software Metrics for Benchmarking of Verification Tools - - PowerPoint PPT Presentation

β–Ά
empirical software metrics for benchmarking of
SMART_READER_LITE
LIVE PREVIEW

Empirical Software Metrics for Benchmarking of Verification Tools - - PowerPoint PPT Presentation

Dec 27, 2022 187 likes β€’331 views

Empirical Software Metrics for Benchmarking of Verification Tools Yulia Demyanova, Thomas Pani , Helmut Veith, Florian Zuleger TU Wien FMCAD17 Student Forum October 3, 2017 Fu Fully-au autom omated Soft oftwar are Verific ification

slide-1
SLIDE 1

Empirical Software Metrics for Benchmarking of Verification Tools

Yulia Demyanova, Thomas Pani, Helmut Veith, Florian Zuleger TU Wien FMCAD’17 Student Forum Β· October 3, 2017

slide-2
SLIDE 2

Fu Fully-au autom

  • mated Soft
  • ftwar

are Verific ification ion

.c 𝝌

Property

Input Output

slide-3
SLIDE 3

Fu Fully-au autom

  • mated Soft
  • ftwar

are Verific ification ion

.c 𝝌

Property

Input

SV Tool

Output

slide-4
SLIDE 4

Fu Fully-au autom

  • mated Soft
  • ftwar

are Verific ification ion

.c 𝝌

Property

Input

SV Tool

β€œπŒ holds.” β€œπŒ does not hold.” β€œI don’t know.” Output

slide-5
SLIDE 5

Fu Fully-au autom

  • mated Soft
  • ftwar

are Verific ification ion

.c 𝝌

Property

Input

SV Tool

β€œπŒ holds.” β€œπŒ does not hold.” β€œI don’t know.” Output

slide-6
SLIDE 6

Fu Fully-au autom

  • mated Soft
  • ftwar

are Verific ification ion

.c 𝝌

Property

Input

SV Tool

β€œπŒ holds.” β€œπŒ does not hold.” β€œI don’t know.” Output

slide-7
SLIDE 7

Pick cking a verification tool

.c 𝝌

Property

Input β€œπŒ holds.” β€œπŒ does not hold.” β€œI don’t know.” Output

CBMC CPAchecker SMACK BLAST

…

?

slide-8
SLIDE 8

Pick cking a verification tool

.c 𝝌

Property

Input β€œπŒ holds.” β€œπŒ does not hold.” β€œI don’t know.” Output

CBMC CPAchecker SMACK BLAST

…

?

largely complementary:

  • specific application areas (device drivers, …)
  • restricted program models (integer programs, …)
  • unsound, incomplete, buggy…
slide-9
SLIDE 9

Pick cking a verification tool

.c 𝝌

Property

Input Output

tool1: BLAST tool2: CBMC tool3: CPAchecker tooln: SMACK

…

β€œbest” tool

?

slide-10
SLIDE 10

Pick cking a verification tool

.c 𝝌

Property

Input Output

tool1: BLAST tool2: CBMC tool3: CPAchecker tooln: SMACK

…

VeriFolio (ML model) β€œbest” tool

slide-11
SLIDE 11

Co Contr tributi tions

.c 𝝌

Property

Input Output

tool1: BLAST tool2: CBMC tool3: CPAchecker tooln: SMACK

…

VeriFolio (ML model) β€œbest” tool

slide-12
SLIDE 12

Ex Expe peri riments – SV SV-CO COMP’16

Score (Overall) Medals

slide-13
SLIDE 13

Ex Expe peri riments – SV SV-CO COMP’16

VeriFolio 3269

UltimateAutomizer

1965

CPA-Seq

1907

  • sim. for SV-COMP’14 & β€˜15

Score (Overall) Medals

slide-14
SLIDE 14

[DPVZ13] On the concept of variable roles and its use in software analysis. FMCAD, 2013 [PVZ15] Loop patterns in C programs. ECEASST 72, 2015 [DPVZ15] Empirical Software Metrics for Benchmarking of Verification Tools. CAV, 2015 [DPVZ17] Empirical Software Metrics for Benchmarking of Verification Tools. FMSD 50(2-3), 2017 [DRZ17] Systematic predicate abstraction using variable roles. NFM, 2017 Implementation http://forsyte.at/software/verifolio/

Th Than anks!