Specification-less Semantic Bug Detection Zhendong Su ETH Zurich - - PowerPoint PPT Presentation

Specification-less Semantic Bug Detection

Zhendong Su

ETH Zurich

What is the key mission of Computer Science?

To help people turn creative ideas into working systems

Software research is central to this mission

A lot of progress to celebrate for!

To help people turn creative ideas into working systems

P ⊨ " ?

P ⊨ " ?

but … where is " ?

A dilemma

Need ! to show P ⊨ !

A dilemma

Need ! to show P ⊨ ! … but ! is not available

A dilemma

Need ! to show P ⊨ ! … but ! is not available One of the greatest challenges!

A dilemma

Need ! to show P ⊨ ! … but ! is not available One of the greatest challenges!

practical & technical

3 mitigation examples

qValidating compilers qValidating database engines qValidating object detection systems

3 mitigation examples

qValidating compilers qValidating database engines qValidating object detection systems

All critical infrastructures & specification-less validation

Validate Production Compilers

Compiler complexity

4 15 19 2 4 6 8 10 12 14 16 18 20

LLVM GCC Linux

LoC (million)

LLVM bug 14972

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

Developer comment

“... very, very concerning when I got to the root cause, and very annoying to fix …”

http://llvm.org/bugs/show_bug.cgi?id=14972

Vision P ≡

P1 P2 P3 Pk Pn …

Key challenges

qGeneration

u How to generate different, yet equivalent tests?

qValidation

u How to check that tests are indeed equivalent?

qBoth are long-standing hard issues

• Equiv. modulo inputs

qRelax equiv. wrt a given input i

uMust: P(i) = Pk(i) on input i uOkay: P(j) ≠ Pk(j) on all input j ≠ i

qExploit close interplay between

uDynamic program execution on some input uStatic compilation for all input

Profile

program P

• utput O

input I

executed unexecuted

Mutate

…..

O I

O I O I O I

EMI

…..

O I

O I O I O I

equivalent modulo I

Find bugs

…..

O I

O’ ¹ O

I

Revisit challenges

qGeneration (easy)

u How to generate different, yet equivalent tests?

qValidation (easy)

u How to check that tests are indeed equivalent?

LLVM bug 14972

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

Seed file

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out

Seed file

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out

unexecuted

Transformed file

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

Reduced file

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

LLVM bug autopsy

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

GVN: load struct using 32-bit load

LLVM bug autopsy

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

GVN: load struct using 32-bit load SRoA: read past the struct’s end

è

undefined behavior

LLVM bug autopsy

$ clang –m32 –O0 test.c ; ./a.out $ clang –m32 –O1 test.c ; ./a.out Aborted (core dumped)

GVN: load struct using 32-bit load

remove

SRoA: read past the struct’s end

è

undefined behavior

GCC bug 58731

$ gcc –O0 test.c ; ./a.out $ gcc –O3 test.c ; ./a.out ^C

GCC bug autopsy

$ gcc –O0 test.c ; ./a.out $ gcc –O3 test.c ; ./a.out ^C PRE: loop invariant

GCC bug autopsy

LIM

$ gcc –O0 test.c ; ./a.out $ gcc –O3 test.c ; ./a.out ^C

GCC bug autopsy

integer overflow

$ gcc –O0 test.c ; ./a.out $ gcc –O3 test.c ; ./a.out ^C

Seed program

$ gcc –O0 test.c ; ./a.out $ gcc –O3 test.c ; ./a.out

no longer a loop invariant

Athena (OOPSLA’15)

Prune & inject dead code

Hermes (OOPSLA’16)

Mutate live code

Orion (PLDI’14)

Prune dead code

Bug counts

GCC LLVM TOTAL Reported 841 781 1,622 Fixed 612 419 1,031

• ISSTA’15: Stress-testing link-time optimization
• ICSE’16: Analyzing compilers’ diagnostic support
• PLDI’17: Skeletal program enumeration (SPE)

LLVM 3.9 & 4.0 Release Notes

“… thanks to Zhendong Su and his team whose fuzz testing prevented many bugs going into the release …”

GCC’s list of contributors

“Zhendong Su … for reporting numerous bugs” “Chengnian Sun … for reporting numerous bugs” “Qirun Zhang … for reporting numerous bugs”

https://gcc.gnu.org/onlinedocs//gcc/Contributors.html

Validate Database Engines

Database engines (DBMS)

PostgreSQL

Who has heard about/used these Database Management Systems?

DBMS widely used

“SQLite is the most used database engine in the world. SQLite is built into all mobile phones and most computers and comes bundled inside countless other applications that people use every day.”

https://www.sqlite.org

animal description picture Cat A cute toast cat Dog Cute dog pic Cat Cat plants (cute!)

animal_pictures

Relational Data Model

animal description picture Cat A cute toast cat Dog Cute dog pic Cat Cat plants (cute!)

animal_pictures

A database schema describes the tables (relations) in the database

Relational Data Model

animal_pictures

Structured Query Language (SQL) is a declarative DSL to query and manipulate data

SELECT picture, description FROM animal_pictures WHERE animal = 'Cat' AND description LIKE '%cute%'

Relational Data Model

DBMS

Database Database Management System (DBMS)

SELECT * FROM <table> WHERE <cond>

Client Application

ü

row1 <cond> row2 <cond> row3

¬<cond>

row1 <cond> row2 <cond>

Goal

Aim: Detect logic bugs in DBMS

DBMS

Database Database Management System (DBMS)

SELECT * FROM <table> WHERE <cond>

Client Application

ü

row1 <cond> row2 <cond> row3

¬<cond>

row1 <cond> row2 <cond>

DBMS

Database Database Management System (DBMS) Client Application

û

row1 <cond> row2 <cond> row3

¬<cond>

SELECT * FROM <table> WHERE <cond>

row1 <cond> row3 ¬<cond>

Example SQLite bug

CREATE TABLE t1(c1, c2, c3, c4, PRIMARY KEY (c4, c3)); INSERT INTO t1(c3) VALUES (0), (0), (0), (0), (0), (0), (0), (0), (0), (0), (NULL), (1), (0); UPDATE t1 SET c2 = 0; INSERT INTO t1(c1) VALUES (0), (0), (NULL), (0), (0); ANALYZE t1; UPDATE t1 SET c3 = 1; SELECT DISTINCT * FROM t1 WHERE t1.c3 = 1;

ANALYZE gathers statistics about tables, which are then used for query planning

Example SQLite bug

CREATE TABLE t1(c1, c2, c3, c4, PRIMARY KEY (c4, c3)); INSERT INTO t1(c3) VALUES (0), (0), (0), (0), (0), (0), (0), (0), (0), (0), (NULL), (1), (0); UPDATE t1 SET c2 = 0; INSERT INTO t1(c1) VALUES (0), (0), (NULL), (0), (0); ANALYZE t1; UPDATE t1 SET c3 = 1; SELECT DISTINCT * FROM t1 WHERE t1.c3 = 1;

A bug in the skip-scan

• ptimization caused

this logic bug

c1 c2 c3 c4 NULL 1 NULL NULL 1 NULL NULL NULL 1 NULL

Expected result set

c1 c2 c3 c4 NULL 1 NULL

Actual result set

DBMS (very) well-tested

SQLite (~150,000 LOC) has 662 times as much test code as source code SQLite is extensively fuzzed (e.g., by Google’s OS-Fuzz Project) SQLite’s test cases achieve 100% branch test coverage SQLite’s performs anomaly testing (out-

• f-memory, I/O error, power failures)

https://www.sqlite.org/testing.html

• Small. Fast. Reliable. Choose any three.

Pivoted Query Synthesis (PQS)

>100 bugs in widely used DBMS

PQS idea

Column0 Column1 Column2 … … …

Valuei,0 Valuei,1 Valuei,2

… … …

Pivot Row

Database generation

Randomly Generate Database

To explore “all possible database states” we randomly create databases

Pivot row selection

Randomly Generate Database Select Pivot Row

Query generation

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row animal description picture Cat Cat plants (cute!)

SELECT picture, description FROM animal_pictures WHERE animal = 'Cat' AND description LIKE '%cute%'

Verifying the result

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row Verify that the Pivot Row is contained DBMS

ü

SELECT picture, description FROM animal_pictures WHERE animal = 'Cat' AND description LIKE '%cute%'

result set pivot row

pivot row ∈ result set

Verifying the result

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row Verify that the Pivot Row is contained DBMS

SELECT picture, description FROM animal_pictures WHERE animal = 'Cat' AND description LIKE '%cute%'

result set pivot row

û

pivot row ∉ result set

Approach

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row Verify that the Pivot Row is contained

Approach

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row Verify that the Pivot Row is contained

Approach

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row Verify that the Pivot Row is contained

Approach

Randomly Generate Database Select Pivot Row Generate Query for the Pivot Row Verify that the Pivot Row is contained

How do we generate this query?

How to generate queries?

Generate an expression that yields TRUE for the pivot row SELECT picture, description FROM animal_pictures WHERE

How to generate queries?

Randomly Generate Expression Evaluate Expression

• n Pivot Row

Modify expression to yield TRUE Use in WHERE clause

Random exp. generation

animal description picture

https://www.sqlite.org/syntax/expr.html

animal_pictures

We first generate a random expression

Random exp. generation

animal = 'Cat' AND description LIKE '%cute%'

AND = LIKE animal 'Cat' descri ption '%cute%'

Random exp. generation

animal = 'Cat' AND description LIKE '%cute%'

Evaluate the tree based

• n the pivot row

AND = LIKE animal 'Cat' descri ption '%cute%'

Random exp. evaluation

AND = LIKE animal 'Cat' descri ption 'Cat' '%cute%'

Constant nodes return their assigned literal values

'%cute%'

Random exp. evaluation

AND = LIKE animal 'Cat' descri ption 'Cat plants (cute!)' 'Cat' 'Cat' '%cute%'

Column references return the values from the pivot row

'%cute%'

Random exp. evaluation

AND = LIKE animal 'Cat' descri ption 'Cat plants (cute!)' 'Cat' TRUE TRUE 'Cat' '%cute%'

Compound nodes compute their result based on their children

'%cute%'

Random exp. evaluation

AND = LIKE animal 'Cat' descri ption 'Cat plants (cute!)' 'Cat' TRUE TRUE TRUE 'Cat' '%cute%' '%cute%'

SELECT picture, description FROM animal_pictures WHERE animal = 'Cat' AND description LIKE '%cute%'

Query synthesis

Random exp. evaluation

AND = LIKE animal 'Cat' descri ption 'Cat plants (cute!)' 'Cat' TRUE TRUE TRUE 'Cat' '%cute%'

What about when the expression does not evaluate to TRUE?

'%cute%'

Random exp. evaluation

= animal 'Dog' 'Cat' FALSE 'Dog'

What about when the expression does not evaluate to TRUE?

animal = 'Dog'

Random exp. rectification

switch (result) { case TRUE: result = randexpr; case FALSE: result = NOT randexpr; case NULL: result = randexpr ISNULL; }

Random exp. rectification

switch (result) { case TRUE: result = randexpr; case FALSE: result = NOT randexpr; case NULL: result = randexpr ISNULL; } animal = 'Dog'

FALSE

Random exp. rectification

switch (result) { case TRUE: result = randexpr; case FALSE: result = NOT randexpr; case NULL: result = randexpr ISNULL; } NOT(animal = 'Dog')

TRUE

How to generate queries?

SELECT picture, description FROM animal_pictures WHERE NOT(animal = 'Dog')

Tested DBMS

PostgreSQL

We tested these (and other DBMS) in a period of 3-4 months

DBMS

DBMS

DBMS

Bugs overview

DBMS Fixed Verified SQLite 65 MySQL 15 10 PostgreSQL 5 4 Sum 85 14

99 real bugs: code fixes or verified as bugs

Real Bugs

Bugs overview

The SQLite developers quickly responded to all

• ur bug reports à we focused on this DBMS

DBMS Fixed Verified SQLite 65 MySQL 15 10 PostgreSQL 5 4 Sum 85 14 Real Bugs

Bugs overview

DBMS Fixed Verified SQLite 65 MySQL 15 10 PostgreSQL 5 4 Sum 85 14

All MySQL bug reports were verified quickly

Real Bugs

Bugs overview

DBMS Fixed Verified SQLite 65 MySQL 15 10 PostgreSQL 5 4 Sum 85 14

MySQL’s trunk is unavailable, and it has a long release cycle

Real Bugs

Bugs overview

DBMS Fixed Verified SQLite 65 MySQL 15 10 PostgreSQL 5 4 Sum 85 14

We found the fewest bugs in PostgreSQL and not all could be easily addressed

Real Bugs

Oracles

DBMS Containment Error SEGFAULT SQLite 46 17 2 MySQL 14 10 1 PostgreSQL 1 7 1 Sum 61 34 4

Real Bugs

Oracles

DBMS Containment Error SEGFAULT SQLite 46 17 2 MySQL 14 10 1 PostgreSQL 1 7 1 Sum 61 34 4

Containment Oracle

Our Containment oracle allowed us to detect most errors

Real Bugs

Result: bug in SQLite3

CREATE TABLE t0(c1 TEXT PRIMARY KEY) WITHOUT ROWID; CREATE INDEX i0 ON t0(c1 COLLATE NOCASE); INSERT INTO t0(c1) VALUES ('A'); INSERT INTO t0(c1) VALUES ('a');

An index is an auxiliary data structure that should not affect the query’s result

Real Bugs Containment Oracle

Result: bug in SQLite3

CREATE TABLE t0(c1 TEXT PRIMARY KEY) WITHOUT ROWID; CREATE INDEX i0 ON t0(c1 COLLATE NOCASE); INSERT INTO t0(c1) VALUES ('A'); INSERT INTO t0(c1) VALUES ('a');

c1

'A' 'a'

Real Bugs Containment Oracle

Result: bug in SQLite3

CREATE TABLE t0(c1 TEXT PRIMARY KEY) WITHOUT ROWID; CREATE INDEX i0 ON t0(c1 COLLATE NOCASE); INSERT INTO t0(c1) VALUES ('A'); INSERT INTO t0(c1) VALUES ('a');

c1

'A' 'a'

SELECT * FROM t0;

c1

'A' û

SQLite failed to fetch 'a'!

Real Bugs Containment Oracle

CREATE TABLE t0(c0 INT PRIMARY KEY, c1 INT); CREATE TABLE t1(c0 INT) INHERITS (t0);

c0 c1 c0 c1

t0 t1

Real Bugs Containment Oracle

Result: bug in PostgreSQL

CREATE TABLE t0(c0 INT PRIMARY KEY, c1 INT); CREATE TABLE t1(c0 INT) INHERITS (t0); INSERT INTO t0(c0, c1) VALUES(0, 0);

c0 c1 c0 c1

t0 t1

Real Bugs Containment Oracle

Result: bug in PostgreSQL

CREATE TABLE t0(c0 INT PRIMARY KEY, c1 INT); CREATE TABLE t1(c0 INT) INHERITS (t0); INSERT INTO t0(c0, c1) VALUES(0, 0); INSERT INTO t1(c0, c1) VALUES(0, 1);

c0 c1 1 c0 c1 1

t0 t1

The inheritance relationship causes the row to be inserted both in t0 and t1

Real Bugs Containment Oracle

Result: bug in PostgreSQL

CREATE TABLE t0(c0 INT PRIMARY KEY, c1 INT); CREATE TABLE t1(c0 INT) INHERITS (t0); INSERT INTO t0(c0, c1) VALUES(0, 0); INSERT INTO t1(c0, c1) VALUES(0, 1);

c0 c1 1 c0 c1 1

t0 t1 SELECT c0, c1 FROM t0 GROUP BY c0, c1;

c0 c1

PostgreSQL failed to fetch the row 0 | 1

û

Real Bugs Containment Oracle

Result: bug in PostgreSQL

Result: bug in MySQL

CREATE TABLE t0(c0 TINYINT); INSERT INTO t0(c0) VALUES(NULL);

c0 NULL

t0

Real Bugs Containment Oracle

Result: bug in MySQL

CREATE TABLE t0(c0 TINYINT); INSERT INTO t0(c0) VALUES(NULL);

c0 NULL

t0

SELECT * FROM t0 WHERE NOT(t0.c0 <=> 2035382037);

c0

û

The MySQL-specific equality operator <=> malfunctioned for large numbers

FALSE Real Bugs Containment Oracle

Oracles

DBMS Containment Error SEGFAULT SQLite 46 17 2 MySQL 14 10 1 PostgreSQL 1 7 1 Sum 61 34 4

We also found many bugs that trigger DB errors

Real Bugs Error Oracle

SQLite3 bug

CREATE TABLE t1 (c0, c1 REAL PRIMARY KEY); INSERT INTO t1(c0, c1) VALUES (TRUE, 9223372036854775807), (TRUE, 0); UPDATE t1 SET c0 = NULL; UPDATE OR REPLACE t1 SET c1 = 1; SELECT DISTINCT * FROM t1 WHERE (t1.c0 IS NULL);

The INSERT and UPDATE statements corrupted the database

Database disk image is malformed

Real Bugs Error Oracle

Oracles

DBMS Containment Error SEGFAULT SQLite 46 17 2 MySQL 14 10 1 PostgreSQL 1 7 1 Sum 61 34 4

We found only a low number of crash bugs, likely because DBMS are fuzzed extensively

Real Bugs SEGFAULTs

Average #statements

Half of all bugs can be reproduced with only 4 SQL statements

Real Bugs

Bug importance

SQLite developers assigned severity levels

Severity Level # Critical 14 Severe 8 Important 14

Validate Object Detectors

DL-based object detectors

Auto-Driving Systems Surveillance Camera Surveillance Camera Medical Image Processing

“autonomous driving system failed to recognize a white truck against a bright sky” believed to be due to its failure to recognize the pedestrian in dark clothing

Typical image analysis tasks

Object Detection

elephant; elephant

Instance Segmentation

elephant; elephant

Semantics Segmentation

grass; elephant; tree Multiple Objects No objects; just pixels

Entire Image elephant

Image Classification

Object detection in automated driving systems

Object Detection (localization + classification)

Typical image analysis tasks

Object Detection

elephant; elephant

Instance Segmentation

elephant; elephant

Semantics Segmentation

grass; elephant; tree Multiple Objects No objects; just pixels

Entire Image elephant

Image Classification

Object detection in automated driving systems

Object Detection (localization + classification)

Focused on by existing DNN testing work

Typical image analysis tasks

Object Detection

elephant; elephant

Instance Segmentation

elephant; elephant

Semantics Segmentation

grass; elephant; tree Multiple Objects No objects; just pixels

Entire Image elephant

Image Classification

Object detection in automated driving systems

Object Detection (localization + classification)

6.5 USD for this figure Focused on by existing DNN testing work

Basic idea

Basic idea

Examples

Workflow

Instance Segmentation what to insert where to insert

Imagek Objects of “Bird” Cluster1 Cluster1 Cluster1 Clusterm Objects of “Person” Cluster1 Cluster1 Cluster1 Clustern Object Pool

…

+

Selection

Object Image Refinement & Selection Object Insertion

Image Dataset pick one background image Paste

Birdm Personn Object Refinement & Selection Object Pool Objects of “bird” Birdm Objects of “Person” Personn

…

• Extract objects from 1000 images from the COCO dataset
• Take 500 images as the “background” to generate test cases

Evaluation subjects

Result summary

Other domains

qSMT solvers (Z3 & CVC4) qSMC tools (CPAchecker, CBMC & Seahorn) qAndroid mobile apps (going beyond crashes) qMachine translation systems qSmart contracts q…

Finding a/the right balance between human & machine collaboration

Thank you!

Finding a/the right balance between human & machine collaboration