through Symbolic Execution Driller: Augmenting...
Transcript of through Symbolic Execution Driller: Augmenting...
![Page 1: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/1.jpg)
Driller: Augmenting Fuzzing through Symbolic Execution
Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili,
Christopher Kruegel, Giovanni Vigna
![Page 2: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/2.jpg)
Motivation
- Large number of memory corruption bugs
- Problems with testcase generation techniques- Fuzzing- Symbolic Execution
![Page 3: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/3.jpg)
Fuzzing
![Page 4: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/4.jpg)
x = int(input())if x > 10:
if x < 100:print "You win!"
else:print "You lose!"
else:print "You lose!"
Let's fuzz it!
1 ⇒ "You lose!"
593 ⇒ "You lose!"
183 ⇒ "You lose!"
4 ⇒ "You lose!"
498 ⇒ "You lose!"
4
48 ⇒ "You win!"
![Page 5: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/5.jpg)
Catching Bugs
- Monitors program for crashes
![Page 6: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/6.jpg)
x = int(input())if x > 10:
if x^2 == 152399025:print "You win!"
else:print "You lose!"
else:print "You lose!"
Let's fuzz it!
1 ⇒ "You lose!"
593 ⇒ "You lose!"
183 ⇒ "You lose!"
4 ⇒ "You lose!"
498 ⇒ "You lose!"
42 ⇒ "You lose!"
3 ⇒ "You lose!"
6
……….
57 ⇒ "You lose!"
![Page 7: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/7.jpg)
Symbolic Execution
![Page 8: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/8.jpg)
x = input()if x >= 10:
if x % 1337 == 0:print "You win!"
else:print "You lose!"
else:print "You lose!"
???
x < 10 x >= 10
x >= 10x % 1337 != 0
x >= 10x % 1337 == 0
![Page 9: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/9.jpg)
x = input()if x >= 10:
if x % 1337 == 0:print "You win!"
else:print "You lose!"
else:print "You lose!"
???
x < 10 x >= 10
x >= 10x % 1337 != 0
x >= 10x % 1337 == 0
1337
![Page 10: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/10.jpg)
Catching Bugs
- Checks each state for safety violations- symbolic program counter- writes/reads from symbolic address
![Page 11: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/11.jpg)
x = input()
def recurse(x, depth): if depth == 2000 return 0 else { r = 0; if x[depth] == “B”: r = 1 return r + recurse(x[depth], depth)
if recurse(x, 0) == 1: print “You win!”
???
x[d] == “B”x[d] != “B”
![Page 12: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/12.jpg)
Different Approaches
Fuzzing- Good at finding solutions
for general conditions
- Bad at finding solutions for specific conditions
Symbolic Execution- Good at finding solutions
for specific conditions
- Spends too much time iterating over general conditions
![Page 13: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/13.jpg)
Fuzzing vs. Symbolic Execution
Fuzzing Wins Symbolic Execution Wins
x = input()
def recurse(x, depth): if depth == 2000 return 0 else { r = 0; if x[depth] == “B”: r = 1 return r + recurse(x[depth], depth)
if recurse(x, 0) == 1: print “You win!”
x = int(input())if x >= 10:
if x^2 == 152399025:print "You win!"
else:print "You lose!"
else:print "You lose!"
![Page 14: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/14.jpg)
Fuzzing
good at finding solutions for general input
Symbolic Execution
good at find solutions for specific input
![Page 15: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/15.jpg)
American Fuzzy Lop + angr
AFL- state-of-the-art
instrumented fuzzer
- path uniqueness tracking
- genetic mutations
- open source
angr- binary analysis platform
- implements symbolic execution engine
- influenced by Mayhem
- works on binary code
- available on github
![Page 16: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/16.jpg)
Combining the Two (High-level)Test Cases
Control Flow Graph
![Page 17: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/17.jpg)
Combining the Two
“Y”
“X”
Test Cases
“Cheap” fuzzing coverage
Control Flow Graph
![Page 18: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/18.jpg)
Combining the Two
“Y”
“X”
Test Cases
“Cheap” fuzzing coverage
Tracing via Symbolic Execution
!
Control Flow Graph
Reachable?
![Page 19: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/19.jpg)
Combining the Two
“Y”
“X”
Test Cases
“Cheap” fuzzing coverage
Tracing via Symbolic Execution
“MAGIC”
New test cases generated
Control Flow Graph
Synthesized!
![Page 20: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/20.jpg)
Combining the Two
“Y”
“X”
Test Cases
“Cheap” fuzzing coverage
Tracing via Symbolic Execution
“MAGIC”
New test cases generated“MAGICY”
Control Flow Graph
Towards completer code coverage!
![Page 21: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/21.jpg)
AFL’s Path Selection
- Tracks state-transitions on each program run- Basic Block A -> Basic Block B
- Path uniqueness = Set of state-trans uniqueness
- Input generation is still primitive mutations
![Page 22: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/22.jpg)
Improving Path Selection with angrTest Cases
AFL
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
![Page 23: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/23.jpg)
Improving Path Selection with angrTest Cases
“X”AFL
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
![Page 24: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/24.jpg)
Improving Path Selection with angrTest Cases
“X”
“Y”
AFL
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
![Page 25: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/25.jpg)
Improving Path Selection with angrTest Cases
“X”
“Y”
AFL
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
“Z”
![Page 26: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/26.jpg)
Improving Path Selection with angrTest Cases
“X”
“Y”
AFL
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
![Page 27: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/27.jpg)
Improving Path Selection with angrTest Cases
“X”
“Y”
angr
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
?
![Page 28: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/28.jpg)
Improving Path Selection with angrTest Cases
“X”
“Y”
angr
strcmp(input, "MAGIC")
input[0] == 'X' ...
... ...
“MAGIC”
New state transition,
synthesize!
![Page 29: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/29.jpg)
Improving Path Selection with angr
...
... ...
......
...
...
Continue following “X”’s original path until completion, deviating when possible.
angr
![Page 30: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/30.jpg)
State Space Reduction
- Symbolic Execution’s state-space is reduced to AFL’s
- Reduces path explosion
![Page 31: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/31.jpg)
Binary Crashes per Technique
Symbolic Execution (angr) - 16 total
Fuzzing (AFL) - 68 total
68
16S & F Shared - 13 total
71 / 128 binaries
![Page 32: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/32.jpg)
Binary Crashes per Technique
Symbolic Execution (angr) - 16
Fuzzing (AFL) - 68
55
S & F Shared - 13 total
Driller - 77
77
16
68
77 / 128 binaries
![Page 33: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/33.jpg)
symbolic executionfuzzing
Distribution of Transitions Found as Iterations of Symbolic Execution and Fuzzing
![Page 34: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/34.jpg)
Limitationsint main(void) {
char data[100];char *computed_hash;char hash[16];
read(0, data, sizeof data);
computed_hash = hash(data);
read(0, hash, sizeof hash);
if (memcmp(hash, computed_hash, 16) != 0) {// `data` processed here// code susceptible to fuzzing
}}
Fuzzing beyond the hash is still problematic!
![Page 35: through Symbolic Execution Driller: Augmenting Fuzzingwp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/07... · Driller: Augmenting Fuzzing through Symbolic Execution](https://reader033.fdocuments.us/reader033/viewer/2022051601/5abfcb7d7f8b9a3a428eae93/html5/thumbnails/35.jpg)
Conclusion
- Driller is greater than the sum of its parts
- Offers a >10% increase in crashes over pure AFL
- Driller curbs path explosion