Software Security Dynamic analysis and fuzz testing Bhargava - - PowerPoint PPT Presentation

Software Security

Dynamic analysis and fuzz testing Bhargava Shastry

• Prof. Jean-Pierre Seifert

Security in Telecommunications TU Berlin

SoSe 2016

1 / 21

Introduction

What is this lecture about? How do we find bugs in programs?

... by executing an open-source program ... applies to binary code in principle

Let’s recap

Last lecture, we discussed static analysis → Find bugs without program execution How does it compare with Dynamic analysis? Pros and cons?

2 / 21

Motivation

Why do this? Find an input to the program that leads to undefined/insecure behavior

Typically, we are looking for an input that crashes a program Program crashes provide an entry point to craft exploits → Not all program crashes are exploitable

Technically, this is just another way to find bugs in programs

3 / 21

Outline

Fuzz testing Black-box vs. White-box testing Dynamic analysis Limitations Conclusion

4 / 21

Example 1

Which input make this program crash?

1 #i n c l u d e <s t d i o . h> 2 #i n c l u d e <u n i s t d . h> 3 #i n c l u d e <s t d l i b . h> 4 #i n c l u d e <s t r i n g . h> 5 6 i n t main ( i n t argc , char ∗argv [ ] ) { 7 8 char buf [ 2 0 ] ; 9 10 while ( AFL LOOP (1000) ) { 11 memset ( buf , 0 , 20) ; 12 i f ( read (0 , buf , 19) < 0) { 13 p e r r o r ( ” read ” ) ; 14 return 1 ; 15 } 16 17 i f ( buf [ 0 ] != ’ p ’ ) 18 p r i n t f ( ” f i r s t l e t t e r i s not p\n” ) ; 19 e l s e i f ( buf [ 1 ] != ’w ’ ) 20 p r i n t f ( ” second l e t t e r i s not w\n” ) ; 21 e l s e i f ( buf [ 2 ] != ’ n ’ ) 22 p r i n t f ( ” t h i r d l e t t e r i s not n\n” ) ; 23 e l s e 24 abort () ; 25 26 p r i n t f ( ” buf c o n t a i n s %s\n” , buf ) ; 27 } 28 return 0 ; 29 } 5 / 21

1 #i n c l u d e <s t d i o . h> 2 #i n c l u d e <u n i s t d . h> 3 #i n c l u d e <s t d l i b . h> 4 #i n c l u d e <s t r i n g . h> 5 6 i n t main ( i n t argc , char ∗argv [ ] ) { 7 8 char buf [ 2 0 ] ; 9 10 while ( AFL LOOP (1000) ) { 11 memset ( buf , 0 , 20) ; 12 i f ( read (0 , buf , 19) < 0) { 13 p e r r o r ( ” read ” ) ; 14 return 1 ; 15 } 16 17 i f ( buf [ 0 ] != ’ p ’ ) 18 p r i n t f ( ” f i r s t l e t t e r i s not p\n” ) ; 19 e l s e i f ( buf [ 1 ] != ’w ’ ) 20 p r i n t f ( ” second l e t t e r i s not w\n” ) ; 21 e l s e i f ( buf [ 2 ] != ’ n ’ ) 22 p r i n t f ( ” t h i r d l e t t e r i s not n\n” ) ; 23 e l s e 24 abort () ; 25 26 p r i n t f ( ” buf c o n t a i n s %s\n” , buf ) ; 27 } 28 return 0 ; 29 }

pwn1

1Pwn is a leetspeak slang term derived from the verb own, as meaning to appropriate or to conquer to gain

• wnership. - Wikipedia

6 / 21

Fuzz testing aka Fuzzing

Idea attributed to Prof.Barton Miller (Miller, Fredriksen, & So, 1990)

. . . i n the F a l l

• f

1988 , t h e r e was a w i l d midwest thunderstorm . . . With the heavy rain , t h e r e was n o i s e

• n

the ( d i a l −up ) l i n e and that n o i s e was i n t e r f e r i n g with my a b i l i t y to type s e n s i b l e commands to the s h e l l . − Prof . M i l l e r

Feed random inputs to a program → No knowledge of program internals Idea bore very good initial results → crash or hang between 25-33%

• f the Unix utility programs

7 / 21

Let’s fuzz!

8 / 21

Black-box testing

Black-box testing treats the program under test as a black-box (opaque to tester) This means the testing framework feeds random inputs to the program However, random inputs alone are unlikely to trigger bugs

x = 1000/(input + 100000), how long will it take the fuzzer to feed input = −100000?

Fuzzing, in its initial form, was conceived as a black-box testing technique

Ever since, it has evolved into a white box testing technique State of the art fuzzers like afl (Zalewski, n.d.) “learn” from program behavior

9 / 21

White-box testing

While-box testing treats the PUT as a white-box (transparent to tester) Program internals can be used to reduce input search space

Techniques: Program coverage guided fuzzing, Concolic execution etc. However, these need a model that abstracts program behavior for generating inputs Uses program instrumentation to realize the model = ⇒ Access to program binary or source code Instrumentation may be static (compile-time) or dynamic (run-time)

10 / 21

Coverage guided fuzzing

Fuzz inputs based on program coverage information For instance, retain only those input mutations that lead to new coverage

11 / 21

Example

Each colour indicates a different program path

1 #i n c l u d e <s t d i o . h> 2 #i n c l u d e <u n i s t d . h> 3 #i n c l u d e <s t d l i b . h> 4 #i n c l u d e <s t r i n g . h> 5 6 i n t main ( i n t argc , char ∗argv [ ] ) { 7 8 . . . 9 . . . 10 11 i f ( buf [ 0 ] != ’ p ’ ) 12 // buf [ 0 ] != ’ p ’ 13 p r i n t f ( ” f i r s t l e t t e r i s not p\n” ) ; 14 e l s e i f ( buf [ 1 ] != ’w ’ ) 15 // buf [ 0 ] == ’ p ’ && buf [ 1 ] != ’w ’ 16 p r i n t f ( ” second l e t t e r i s not w\n” ) ; 17 e l s e i f ( buf [ 2 ] != ’ n ’ ) 18 // buf [ 0 ] == ’ p ’ && buf [ 1 ] == ’w ’ && buf [ 2 ] != ’ n ’ 19 p r i n t f ( ” t h i r d l e t t e r i s not n\n” ) ; 20 e l s e 21 // buf [ 0 ] == ’ p ’ && buf [ 1 ] == ’w ’ && buf [ 2 ] == ’ n ’ 22 abort () ; 23 24 . . . 25 . . . 26 } 12 / 21

Instrumentation

But how does the fuzzer know which program path was taken? Answer: Instrument the program What does the instrumentation look like? cur_location = random-number; shared_mem[cur_location ˆ prev_location]++; prev_location = cur_location >> 1; Each branch point indexes a byte in global state State captures number of times branch point taken Comparing state changes per input, it is possible to filter seemingly redundant inputs

13 / 21

Thought experiment Is the state captured by the instrumentation good enough? In other words, can you think of an example where the instrumentation might discard potentially crashing inputs Only a heuristic → But, turns out to be very good in practice

14 / 21

Type of instrumentation

Program instrumentation can be done at compile-time or at run-time Compile-time instrumentation incurs low performance overhead

Checking logic can be optimized before-hand Slow down only because of additional instructions and associated I/O But, hard to encode checking logic for dynamic language features in general

Run-time instrumentations incurs relatively high overhead

Checking logic needs to be inserted at program run time Typically implemented using binary translation = ⇒ very slow Since instrumentation occurs at run time, dynamism (e.g., indirect branches) can be handled relatively easily

Rest of the lecture looks at dynamic analyses that use compile-time instrumentation

15 / 21

Dynamic analysis

Why bother if a good fuzzer gives us crashing inputs?

A normal program crash happens only when there is a serious problem Example: Segmentation fault, assertion failure, stack canary failures etc. But, it does not uncover latent faults Example: Heap corruption, stack corruption but canary intact etc.

Dynamic analysis can help uncover latent faults Can be used in conjunction with a fuzzer for proactive security audits

16 / 21

AddressSanitizer

AddressSanitizer (Serebryany, Bruening, Potapenko, & Vyukov, 2012) is an instrumentation framework for detecting multiple classes of memory corruption faults

Heap/stack buffer overflows, use-after-free bugs

Idea is similar to coverage guided fuzzing i.e., insert bounds checks by instrumenting the program Each word of addressable heap maps to a byte of shadow memory Uses custom memory allocation functions to set shadow byte during allocation Uses shadow lookups during load and store instructions to detect faults e.g. loading from/storing to unaddressable memory location

17 / 21

ASan Instrumentation

Every 8-byte aligned word on x86 64 has 9 states

First n bytes 0 ≤ n ≤ 8 are addressable, rest not = ⇒ 1 shadow byte for 8 bytes of application memory

Instrumentation looks like char *shadow = MemToShadow(addr); if (*shadow && *shadow <= func(addr)) ReportError(addr) ...

18 / 21

Limitations

Fuzz testing and dynamic analysis covered in the lecture are program-input centric This means, if a fuzzed input cannot uncover a program path, bugs in that program path cannot be found Data on false negatives (bugs missed) by fuzzers is hard to come by

Exception: Recent research on bug insertion (Dolan-Gavitt et al., 2016) Preliminary results show that only 10% of inserted bugs found by a SotA fuzzer

Future: Use symbolic execution to maximize path coverage? Awaiting a public study of efficacy

19 / 21

Conclusions

Dynamic analysis attempts to uncover bugs during program execution We discussed fuzz testing and program instrumentation Discussed techniques and tools remain relevant for real-world security audits

Anecdotal evidence shows that over 90% of C/C++ bugs found using a white-box fuzzer

Recent data on bugs missed by fuzzing tells us that there is room for improvement

20 / 21

References I

Dolan-Gavitt, B., Hulin, P., Kirda, E., Leek, T., Mambretti, A., Robertson, W., . . . Whelan, R. (2016). Lava: Large-scale automated vulnerability addition. Miller, B. P., Fredriksen, L., & So, B. (1990). An empirical study of the reliability of unix utilities. Communications of the ACM, 33(12), 32–44. Serebryany, K., Bruening, D., Potapenko, A., & Vyukov, D. (2012). Addresssanitizer: A fast address sanity checker. In Presented as part

• f the 2012 usenix annual technical conference (usenix atc 12) (pp.

309–318). Zalewski, M. (n.d.). American fuzzy lop. http://lcamtuf.coredump.cx/afl/.

21 / 21