Ankou: Guiding Grey-box Fuzzing towards Combinatorial Difference - - PowerPoint PPT Presentation

Ankou: Guiding Grey-box Fuzzing towards Combinatorial Difference

Valentin Manès1, Soomin Kim2, Sang Kil Cha2

1CSRC, KAIST 2KAIST

The Success of Grey-box Fuzzing

2

“OSS-Fuzz has found over 20,000 bugs in 300 open source projects.”

github.com/mrash/afl-cve è Many CVEs

$$$$

Why one more ?

Grey-box, How?

3

Program

Test Case Output

Fuzzer

Seed Pool Test Case A Test Case B Test Case C

Fitness Function: if(“interesting”): Add to seed pool

Test Case

Which Feedback?

4

Cost

N

• f

e e d b a c k s t d

• u

t / s t d e r r B r a n c h C

• v

e r a g e B r a n c h H i t C

• u

n t T a i n t A n a l y s i s

• Vuzzer
• Angora

…

• AFL
• LibFuzzer

…

• zzuf
• BFF

…

Coverage has proved a good tradeoff between cost and benefits.

Ankou: Opportunity to improve?

Test Case A B C D Value “A” “BB” “AB” “ABC” Branch 1 X X X Branch 2 X X X Branch 3 X

Coverage-Based Fuzzing

5

Fuzzer

Seed Pool Test Case A Test Case B Test Case D Test Case A B C Value “A” “BB” “AB” Branch 1 X X Branch 2 X X Branch 3 Test Case A B Value “A” “BB” Branch 1 X Branch 2 X Branch 3

Program

Test Case Outputs int combinedBranches(char *data) { int bits = 0; if (data[0] == 'A') bits |= 1; if (data[1] == 'B') bits |= 2; if (data[2] == 'C') bits |= 4; if (bits == 7) printf("BINGO\n"); return 0; }

Fitness Function: if(new branch): Add to seed pool

A more informative Fitness Function is needed!

Informative Fitness with Combination

Ankou goal: developing a fitness function taking into account combinations.

• 1. Quantify the difference between program executions.
• 2. Make fitness computation fast.
• 3. Make the fitness adaptive to the program.

6

Point Representation

7 1 2 3 1 2 3 4 5 6

Branch 2

Branch 1

8 1 2 3 1 2 3 4 5 6

Branch 2

Branch 1

Euclidean Distance

Distance between Executions

Distance between Executions

9 1 2 3 1 2 3 4 5 6

Branch 2

Branch 1

Detects Combinatorial Difference!

Distance-based Fitness Function

10 1 2 3 4 5 6 7 1 2 3 4 5 6 7 8

Branch 2

Branch 1 Seed Pool ? Point-to-Pool ?

Distance-based Fitness Function

11 1 2 3 4 5 6 7 1 2 3 4 5 6 7 8

Branch 2

Branch 1 Seed Pool Point-to-Pool = Minimum Point-to-Point

Cost Sensitivity

12 5 10 15 20 25 30 35 5 10 15 20 25 30

Branch 2

Branch 1 Seed Pool

The fitness function is computed for every test case.

Problem: Slow Computation

13

Euclidean Distance = 𝒫(#branch)

Cost Reduction

14

Euclidean Distance = 𝒫(#branch) Euclidean Distance = 𝒫(#“reprentative branch”)

Dimensionality Reduction

See paper for details on the Dynamic PCA.

Coverage-based fitness function: if(new branch): Add test to seed pool Ankou fitness function: if(new branch): if(Point-to-Pool distance ??): Add test to seed pool

Ankou Adaptive Fitness Function

15

Ankou Adaptive Fitness Function

16 1 2 3 4 5 1 2 3 4 5 6 7

Branch 2 Branch 1

𝜄!"#

Ankou fitness function: if(new branch): if(Point-to-Pool distance > 𝜄!"#): Add test to seed pool 𝜄!"# ← Minimum inter-seed distance

Benchmark

• Use 24 packages used by CollAFL1.
• All experiments are 6x24 hours runs.
• In total: our experiments constitute 2,682 CPU days.

17

in Proceedings of the IEEE Symposium on Security and Privacy, 2018, pp. 660–677.

Q: Is the New Fitness Function Effective?

18

Ankou with and without Distance-based

19

0.25 1.00 4.00 0.25 1.00 4.00

Subjects Crash ratio (in log) Throughput ratio (in log)

# Crashes Throughput

Distance-based finds 44% more crashes.

Q: How does Ankou compare to other grey-box fuzzers?

20

Ankou vs. AFL

0.01 0.1 1 10 100

Subjects Crash / Coverage ratio (in log) # Crashes Coverage

21

Ankou finds 41% more unique crashes.

Ankou vs. AFL: Speed

22

0.1 1.0 10.0

Subjects Throughput ratio (in log)

Ankou is 35% slower than AFL.

Conclusion

• 1. Coverage-based fuzzers ignore combinations of branches.
• 2. Ankou distance-based fitness function quantify combinatorial

difference while being fast and adaptive to programs.

• 3. While being 35% slower than AFL, Ankou finds 41% more crashes.

23

Question?

24