FUZZIFICATION: Anti-Fuzzing Techniques · snippet 1 pop rbp pop r15 ret Code snippet 2 pop rbp pop...
Transcript of FUZZIFICATION: Anti-Fuzzing Techniques · snippet 1 pop rbp pop r15 ret Code snippet 2 pop rbp pop...
FUZZIFICATION: Anti-Fuzzing Techniques
Jinho Jung, Hong Hu, David Solodukhin, Daniel Pagan,
Kyu Hyung Lee*, Taesoo Kim
*
1
Fuzzing Discovers Many Vulnerabilities
2
Fuzzing Discovers Many Vulnerabilities
3
Testers Find Bugs with Fuzzing
Source
Compilation
Released
binary
Normal users
Testers
Detected
bugs
Compilation Distribution Fuzzing
4
But Attackers Also Find Bugs
Source
Compilation
Released
binary
Testers
Detected
bugs
Compilation Distribution Fuzzing
5
Attackers
Normal users
Our work: Make the Fuzzing Only Effective to the Testers
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
6
Normal users
?
Threat Model
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
7
Normal users
Threat Model
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
8
Normal users
Adversaries try to find vulnerabilities from fuzzing
Threat Model
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
9
Normal users
Adversaries only have a copy of fortified binary
Threat Model
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
10
Normal users
Adversaries know Fuzzification and try to nullify
Research Goals
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
11
Normal users
Research Goals
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
12
Normal users
Hinder Fuzzing Reduce the number of detected bugs
Research Goals
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
13
Normal users
Generic Affect most of the fuzzers
AFL
HonggFuzz
VUzzer
QSym
…
Research Goals
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
14
Normal users
Low overhead to normal user
High overhead to attackersOverhead
Research Goals
Source
Fuzzification
Compilation
Fortified
binary
Binary Testers
Detected
bugs
Attackers
Compilation Distribution Fuzzing
15
Normal users
Fortified
binary
Resilient to the adversarial analysisResiliency
Why Existing Methods Are Not Applicable?
MethodGeneric to
most fuzzers
Low
overhead
Resilient to
adversary
Packing or obfuscation O X O
16
Why Existing Methods Are Not Applicable?
MethodGeneric to
most fuzzers
Low
overhead
Resilient to
adversary
Packing or obfuscation O X O
Bug injection O O X
17
Why Existing Methods Are Not Applicable?
MethodGeneric to
most fuzzers
Low
overhead
Resilient to
adversary
Packing or obfuscation O X O
Bug injection O O X
Fuzzer detection X O X
18
Why Existing Methods Are Not Applicable?
MethodGeneric to
most fuzzers
Low
overhead
Resilient to
adversary
Packing or obfuscation O X O
Bug injection O O X
Fuzzer detection X O X
Emulator detection X O X
19
Why Existing Methods Are Not Applicable?
MethodGeneric to
most fuzzers
Low
overhead
Resilient to
adversary
Packing or obfuscation O X O
Bug injection O O X
Fuzzer detection X O X
Emulator detection X O X
Fuzzification O O O
20
Fuzzification Hinders Advanced Features
21
Parallel execution
Fork
server
• Fast execution
• Coverage-guidance
• Hybrid approach
H/W
feature
Fuzzification Hinders Advanced Features
22
• Fast execution
• Coverage-guidance
• Hybrid approach
Parallel execution
Fork
server
H/W
feature
SpeedBump
Fuzzification Hinders Advanced Features
23
Parallel execution
Fork
server
Coverage
• Fast execution
• Coverage-guidance
• Hybrid approach
H/W
feature
Fuzzification Hinders Advanced Features
24
• Fast execution
• Coverage-guidance
• Hybrid approach
Parallel execution
Fork
server
Coverage
H/W
feature
BranchTrap
Fuzzification Hinders Advanced Features
25
Symbolic
execution
Dynamic
taint
analysis
Queue
Parallel execution
Fork
server
Coverage
• Fast execution
• Coverage-guidance
• Hybrid approach
H/W
feature
Fuzzification Hinders Advanced Features
26
• Fast execution
• Coverage-guidance
• Hybrid approach Symbolic
execution
Dynamic
taint
analysis
Queue
Parallel execution
Fork
server
Coverage
H/W
feature Anti-Hybrid
SpeedBump: Selective Delay Injection
27
Basic block
SpeedBump: Selective Delay Injection
28
Basic block
Rarely visited path
Frequently visited path
• Identify frequently and rarely visited paths
SpeedBump: Selective Delay Injection
29
1
2
Basic block
Rarely visited path
Frequently visited path
• Identify frequently and rarely visited paths
• Inject delays from the most rarely visited edges
SpeedBump: Selective Delay Injection
30
Basic block
Rarely visited path
Frequently visited path
• Why this is effective?
▫ User: follows common paths
▫ Attacker: searches for new paths
➔ Impact of delay is more significant to attackers
1
2
SpeedBump: How to delay?
• Strawman: using sleep()
➔ trivially removed by adversary
31
SpeedBump: How to delay?
• Strawman: using sleep()
➔ trivially removed by adversary
• Counter to advanced adversary
▫ Use randomly generated code ➔ avoid static-pattern
32
SpeedBump: How to delay?
• Strawman: using sleep()
➔ trivially removed by adversary
• Counter to advanced adversary
▫ Use randomly generated code ➔ avoid static-pattern
▫ Impose control-flow and data-flow dependency➔ avoid automated analysis
33
SpeedBump: Selective Delay Injection
34
int rarely_executed_code (){
return 0;}
SpeedBump: Selective Delay Injection
35
int rarely_executed_code (){
return 0;}
//define global variablesint global1 = 1;int global2 = 2;
int rarely_executed_code (){
//inject delay functionint pass = 20;global2 = func(pass);return 0;
}
SpeedBump: Selective Delay Injection
36
int func(int p6) {int local1[10];
// affect global1 variableglobal1 = 45; int local2 = global1;for (int i = 0; i < 1000; i++)// affect local1 variablelocal1[i] = p6 + local2 + i;
// affect global2 variablereturn local1[5];
}
int rarely_executed_code (){
return 0;}
//define global variablesint global1 = 1;int global2 = 2;
int rarely_executed_code (){
//inject delay functionint pass = 20;global2 = func(pass);return 0;
}
BranchTrap Hinders Coverage Management
37
• Fast execution
• Coverage-guidance
• Hybrid approach Symbolic
execution
Dynamic
taint
analysis
Queue
Parallel execution
Fork
server
Coverage
H/W
feature
BranchTrap#1: Fabricates Input-sensitive Paths
38
1
3
2
“AAAA”
Coverage #1
BranchTrap#1: Fabricates Input-sensitive Paths
39
1
3
2
“AAAA” “AAAB”
Coverage #1 Coverage #2
BranchTrap#1: Fabricates Input-sensitive Paths
40
1
3
2
“AAAA” “AAAB”
Coverage #1 Coverage #2
1
3
2
“AAAA”
Coverage #1
BranchTrap
BranchTrap#1: Fabricates Input-sensitive Paths
41
1
3
2
“AAAA” “AAAB”
Coverage #1 Coverage #2
1
3
2
“AAAA”
Coverage #1
“AAAB”
Coverage #2
BranchTrap
Func1 (arg1, arg2)
call Func1
next inst
Caller
Original
epilogue
pop rbp
pop r15
ret
42
BranchTrap#1: ROP-based Fake Paths Generation
BranchTrap#1: ROP-based Fake Paths Generation
Code
snippet 1
pop rbp
pop r15
ret
Code
snippet 2
pop rbp
pop r15
ret
…
call Func1
next inst
43
Original
epilogue
pop rbp
pop r15
retCode
snippet N
…
Func1 (arg1, arg2)
Caller
BranchTrap#1: ROP-based Fake Paths Generation
index = arg1 ^ arg2
Code
snippet 1
pop rbp
pop r15
ret
Code
snippet 2
pop rbp
pop r15
ret
Code
snippet N
…
…
call Func1
next inst
①②
44
Original
epilogue
pop rbp
pop r15
ret
Func1 (arg1, arg2)
Caller
BranchTrap#1: ROP-based Fake Paths Generation
index = arg1 ^ arg2
Code
snippet 1
pop rbp
pop r15
ret
Code
snippet 2
pop rbp
pop r15
ret
…
jmp table [index]call Func1
next inst
①②
③ ④
45
Original
epilogue
pop rbp
pop r15
retCode
snippet N
…
Func1 (arg1, arg2)
Caller
Code
snippet 1
pop rbp
pop r15
ret
Code
snippet 2
pop rbp
pop r15
ret
…
Code
snippet N
…
BranchTrap#1: ROP-based Fake Paths Generation
index = arg1 ^ arg2
jmp table [index]call Func1
next inst
①②
③ ④
⑤
46
Original
epilogue
pop rbp
pop r15
ret
Func1 (arg1, arg2)
Caller
BranchTrap#2: Saturate Feedback State
47
1
3
2
• One-time visit makes effect
• BranchTrap:
▫ Saturates bitmap data
▫ Prevents coverage recording
AntiHybrid Hinders Hybrid Fuzzing
48
• Fast execution
• Coverage-guidance
• Hybrid approach Symbolic
execution
Dynamic
taint
analysis
Queue
Parallel execution
Fork
server
Coverage
H/W
feature
Challenge of Hybrid Fuzzing
49
• Dynamic taint analysis
▫ Expensive implicit flow
Transform explicit data-flow ➔ implicit data-flow
Challenge of Hybrid Fuzzing
50
• Dynamic taint analysis
▫ Expensive implicit flow
• Symbolic execution
▫ Path explosion
Transform explicit data-flow ➔ implicit data-flow
Introduce an arbitrary path explosions
AntiHybrid Avoids Dynamic Taint Analysis
• Transform explicit data-flow to implicit data-flow
51
char input = ‘a’;
char anti_dta;if (input == 97)
anti_dta = ‘a’;
if (!strcmp(anti_dta, ‘a’)) { … }
char input = ‘a’;
if (!strcmp(input, ‘a’)) { … }
Unable to
taint
input
anti_dta
AntiHybrid Incurs Path Explosions
52
• Inject hash calculations into branches
if(a == 30) { … }
if(Hash(a) == 0x300df11) { … }
Path Explosion
Fuzzification Work-flow
Profile
Binary
Source
①RunValid/invlid
inputs
53
Fuzzification Work-flow
Profile
Binary
Source
② Inject
component
LLVM
IR
SpeedBump
BranchTrap
AntiHybrid
①RunValid/invlid
inputs
54
Fuzzification Work-flow
Profile
Binary
Source
② Inject
component
LLVM
IR
SpeedBump
BranchTrap
AntiHybrid Test run
①RunValid/invlid
inputs
55③Measure Overhead & Inject More Component
Fuzzification Work-flow
Profile
Binary
Source
② Inject
component
LLVM
IR
SpeedBump
BranchTrap
AntiHybrid Test run④Release
fortified
binary
①RunValid/invlid
inputs
56③Measure Overhead & Inject More Component
Evaluation Summary
• Implementation
▫ 6,599 lines of Python and 758 lines of C++
• Evaluation questions:
▫ Effective in “Reducing discovered paths and bugs?”
▫ Effective on “Various fuzzers?
▫ Impose “Low overhead” to the normal user?
57
Reduced the Discovered Coverage By 71%
BranchTrap
58
No Fuzzification
All Fuzzifications
AntiHybrid
SpeedBump
Dis
covere
d P
ath
s
* Fuzzing result on AFL-QEMU
objdump (binutils)
Reduced the Discovered Coverage By 71%
59* Fuzzing result on AFL-QEMU
Other binaries
Fuzzification is Effective on Various Fuzzers
Fuzzer Result
AFL (QEMU) 74%
HonggFuzz (PT) 61%
QSym (AFL-QEMU) 80%
Average 71%
60
Reduced code coverage
Reduced the Discovered Bugs
Fuzzer Result
AFL (QEMU) 88%
HonggFuzz (PT) 98%
QSym (AFL-QEMU) 94%
Average 93%
61
Fuzzer Result
Vuzzer 56%
QSym (AFL-QEMU) 78%
Average 67%
binutils v2.3.0 LAVA-M dataset
File size & CPU Overheads
Overhead Result
File Size 1.4MB (62.1%)
CPU Overhead 3.7%
* Both overheads are configurable
62
binutils v2.3.0
Overhead Result
File Size 1.3MB (5.4%)
CPU Overhead 0.73%
Real-world applications (e.g., GUI)
Discussion
• Best-effort protections against adversarial analysis
• Complementary to other defense techniques
▫ Not hiding all vulnerabilities
▫ But introducing significant cost on attacker
63
Comparison: Fuzzification vs. AntiFuzz
Component Fuzzification AntiFuzz
Delay execution ● (+ cold path) ●
Fake coverage ● (randomized return) ● (fake code)
Saturate coverage ● ○
Prevent crash ○ ●
Anti-hybrid ● (+ anti-DTA) ●
Countermeasures ◐ ○64
Conclusion
• SpeedBump: Inject delays and only affects attackers
• BranchTrap: Insert input-sensitive branches
• AntiHybrid: Hinder hybrid fuzzing techniques
65
Make the fuzzing only effective to the testers
https://github.com/sslab-gatech/fuzzification