AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job...
Transcript of AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job...
![Page 1: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/1.jpg)
AN OBSERVATIONAL INVESTIGATION OF REVERSE ENGINEERS’ PROCESSES
AND MENTAL MODELSDaniel Votipka, Seth Rabin, Kristopher Micinski, Michelle
L. Mazurek, and Jeffrey S. Foster
![Page 2: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/2.jpg)
SOFTWARE REVERSE ENGINEERING
2
Looking at someone else’s code to figure out how it works!
![Page 3: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/3.jpg)
SOFTWARE REVERSE ENGINEERING
2
Looking at someone else’s code to figure out how it works!
• Vulnerability Discovery
![Page 4: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/4.jpg)
SOFTWARE REVERSE ENGINEERING
2
Looking at someone else’s code to figure out how it works!
• Vulnerability Discovery
• Malware Analysis
![Page 5: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/5.jpg)
SOFTWARE REVERSE ENGINEERING
2
Looking at someone else’s code to figure out how it works!
• Vulnerability Discovery
30 students and RE professionals took ~39 minutes on average to reverse engineer decompiled code snippets with <150 lines.
Yakdan et al. 2016
• Malware Analysis
![Page 6: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/6.jpg)
3
CURRENT TOOLS
![Page 7: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/7.jpg)
3
CURRENT TOOLS
Many sophisticated tools developed by academia, industry, and practitioners
![Page 8: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/8.jpg)
3
CURRENT TOOLS
Many sophisticated tools developed by academia, industry, and practitioners
• Ad-hoc, based on the developers’ intuition
![Page 9: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/9.jpg)
3
CURRENT TOOLS
Many sophisticated tools developed by academia, industry, and practitioners
• Ad-hoc, based on the developers’ intuition
Limited theoretical model of RE process• Actions, habits, and mental models
![Page 10: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/10.jpg)
3
CURRENT TOOLS
Many sophisticated tools developed by academia, industry, and practitioners
• Ad-hoc, based on the developers’ intuition
Limited theoretical model of RE process• Actions, habits, and mental models
Goal: Develop a refined RE process model to
guide future tool development.
![Page 11: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/11.jpg)
4
PROGRAM COMPREHENSIONHow developers process unfamiliar code during modification, maintenance, and debugging tasks
• Hypotheses/Questions - approach unfamiliar programs from a non-linear, fact-finding perspective
• Beacons - Patterns that allow the reader to quickly infer program behavior
• Simulation methods - Any process for parsing the program to determine its function
![Page 12: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/12.jpg)
4
PROGRAM COMPREHENSIONHow developers process unfamiliar code during modification, maintenance, and debugging tasks
• Hypotheses/Questions - approach unfamiliar programs from a non-linear, fact-finding perspective
• Beacons - Patterns that allow the reader to quickly infer program behavior
• Simulation methods - Any process for parsing the program to determine its function
Is reverse engineering different?•No access to source code, developers, or documentation•Adversarial environment
![Page 13: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/13.jpg)
RESEARCH QUESTIONS
5
What high-level process do REs follow? • Steps of the process • Mental models
What technical approaches do REs use?
Are the RE and Program Comprehension processes different?
![Page 14: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/14.jpg)
RESEARCH QUESTIONS
6
What high-level process do REs follow? • Steps of the process • Mental models
What technical approaches do REs use?
Are the RE and Program Comprehension processes different?
![Page 15: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/15.jpg)
7
OBSERVATIONAL INTERVIEWSModified Critical Decision Method Protocol:
• Participants demonstrated how they reverse engineered a recent program
• Noted and asked further questions regarding items of interest: • Beacons • Hypotheses/Questions • Simulation Methods
• Decisions • Resources
Klein et al. 1989
![Page 16: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/16.jpg)
PARTICIPANTS
8
Participants 16
Gender 14 Male, 1 Female
Age (Median) 18-29
Location 7 US states, 5 countries
Education (Median) B.S.
Yrs. Experience 8.94 years
Skill 4.06 (Advanced)
Job 12* Vulnerability Discovery, 5* Malware Analysis,
*One participant performed both malware analysis and vulnerability discovery for employment
![Page 17: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/17.jpg)
PARTICIPANTS
8
Participants 16
Gender 14 Male, 1 Female
Age (Median) 18-29
Location 7 US states, 5 countries
Education (Median) B.S.
Yrs. Experience 8.94 years
Skill 4.06 (Advanced)
Job 12* Vulnerability Discovery, 5* Malware Analysis,
*One participant performed both malware analysis and vulnerability discovery for employment
![Page 18: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/18.jpg)
PARTICIPANTS
8
Participants 16
Gender 14 Male, 1 Female
Age (Median) 18-29
Location 7 US states, 5 countries
Education (Median) B.S.
Yrs. Experience 8.94 years
Skill 4.06 (Advanced)
Job 12* Vulnerability Discovery, 5* Malware Analysis,
*One participant performed both malware analysis and vulnerability discovery for employment
![Page 19: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/19.jpg)
THREE PHASE RE MODEL
9
Focused Experimentation
Sub-component Scanning
Overview
![Page 20: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/20.jpg)
OVERVIEW
10
Focused Experimentation
Sub-component Scanning
Identify specific functions and code segments to focus on
Full program
• List strings and APIs • Run the program • Review metadata
Overview
![Page 21: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/21.jpg)
11
Focused Experimentation
Sub-component Scanning• Scan beacons
Overview
Specific hypotheses/questions that require concrete information
Program slices
SUB-COMPONENT SCANNING
![Page 22: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/22.jpg)
12
![Page 23: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/23.jpg)
12
“it’s just trying to make a connection to each of
those [websites].”
![Page 24: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/24.jpg)
12
“if it’s able to make a connection, it’s going to return a non-zero value.”
![Page 25: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/25.jpg)
12
“usually you see this activity if [malware] is trying to see if it has connectivity.”
![Page 26: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/26.jpg)
SUB-COMPONENT SCANNING
13
Focused Experimentation
Sub-component Scanning• Scan beacons • Data flow/control flow paths
Overview
Specific hypotheses/questions that require concrete information
Program slices
![Page 27: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/27.jpg)
14
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 z = 1 val = a a += 1
} if val == -1 {
id_free(x) } else {
safe() }
![Page 28: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/28.jpg)
14
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 z = 1 val = a a += 1
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 29: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/29.jpg)
15
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 z = 1 val = a a += 1
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 30: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/30.jpg)
15
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 z = 1 val = a a += 1
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 31: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/31.jpg)
16
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 z = 1 val = a a += 1
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 32: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/32.jpg)
16
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 z = 1 val = a a += 1
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 33: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/33.jpg)
17
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 … val = a …
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 34: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/34.jpg)
17
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 … val = a …
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 35: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/35.jpg)
17
val = 0 if y == 1 {
x = y val = 1
} else{
a = -1 … val = a …
} if val == -1 {
id_free(x) } else {
safe() }
“Can I free undefined memory?”
![Page 36: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/36.jpg)
SUB-COMPONENT SCANNING
18
Focused Experimentation
Sub-component Scanning• Scan beacons • Data flow/control flow paths
Overview
Specific hypotheses/questions that require concrete information
Program slices
![Page 37: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/37.jpg)
FOCUSED EXPERIMENTATION
19
• Execute under inspection • Compare to reference function • Read line-by-line
Overview
Sub-component Scanning
Execution traces or a few lines of code
Focused Experimentation
Test hypotheses and produce concrete answers
![Page 38: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/38.jpg)
FOCUSED EXPERIMENTATION
19
• Execute under inspection • Compare to reference function • Read line-by-line
Overview
Sub-component Scanning
Execution traces or a few lines of code
Focused Experimentation
No more than 50 lines reviewed by any participants
Test hypotheses and produce concrete answers
![Page 39: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/39.jpg)
CROSS-PHASE TRENDS
20
Focused Experimentation
Sub-component Scanning
Overview
![Page 40: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/40.jpg)
CROSS-PHASE TRENDS
20
Static
Dynamic
Methods
Focused Experimentation
Sub-component Scanning
Overview
![Page 41: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/41.jpg)
CROSS-PHASE TRENDS
20
Static
Dynamic
MethodsRole of
ExperienceChoose focus areas
Recognize behaviors/vulnerabilities
Choose methodFocused Experimentation
Sub-component Scanning
Overview
![Page 42: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/42.jpg)
CROSS-PHASE TRENDS
20
Static
Dynamic
MethodsRole of
ExperienceChoose focus areas
Recognize behaviors/vulnerabilities
Choose methodFocused Experimentation
Sub-component Scanning
Overview
Preferred tools improve readability
![Page 43: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/43.jpg)
DISCUSSION
21
![Page 44: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/44.jpg)
DISCUSSION
21
• Guidelines for usable tool design
![Page 45: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/45.jpg)
DISCUSSION
21
• Guidelines for usable tool design• Framework for tool evaluation
![Page 46: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/46.jpg)
DISCUSSION
21
• Guidelines for usable tool design• Framework for tool evaluation• Insights for RE automation
![Page 47: AN OBSERVATIONAL INVESTIGATION OF REVERSE …Yrs. Experience 8.94 years Skill 4.06 (Advanced) Job 12* Vulnerability Discovery, 5* Malware Analysis, *One participant performed both](https://reader035.fdocuments.us/reader035/viewer/2022071420/6119fa6ad77d58264702c91d/html5/thumbnails/47.jpg)
SUMMARY
22
Three Phase Model: • Overview • Sub-component Scanning • Focused Experimentation
Takeaways: • Guidelines for usable tool design • Framework for tool evaluation • Insights for RE automation
[email protected] sec-professionals.cs.umd.edu
Questions:
Cross-phase trends: • Begin with static methods
and finish with dynamic • Experience guides where
to look