Art into Science 2017 - Investigation Theory: A Cognitive Approach
-
Upload
chrissanders88 -
Category
Technology
-
view
226 -
download
3
Transcript of Art into Science 2017 - Investigation Theory: A Cognitive Approach
![Page 1: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/1.jpg)
Investigation Theory A Cognitive Approach
Chris Sanders
![Page 2: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/2.jpg)
Chris Sanders (@chrissanders88)
Analyst @ FireEye Founder @ Rural Tech Fund PhD Researcher GSE # 64 BBQ Pit Master Author:
Practical Packet Analysis Applied NSM Investigation Theory Course
![Page 3: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/3.jpg)
Symptoms of a Cognitive Crisis1. Demand for expertise greatly
outweights supply2. Most information cannot be trusted or
validated3. Inability to mobilize and tackle big
systemic issues
![Page 4: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/4.jpg)
Ethnography of the SOC
“An analyst’s job is highly dynamic and requires dealing with constantly evolving threats. Doing the job is more art than science. Ad hoc, on-the-job training for new analysts is the norm."
Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
![Page 5: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/5.jpg)
Ethnography of the SOC
“The profession [security] is so nascent that the how-tos have not been fully realized even by the people who have the knowledge…the process required to connect the dots is unclear even to analysts.
Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
![Page 6: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/6.jpg)
Symptoms of a Cognitive Crisis1. Demand for expertise greatly
outweights supply2. Most information cannot be trusted or
validated3. Inability to mobilize and tackle big
systemic issues
![Page 7: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/7.jpg)
The Cognitive Revolution1. Understand the
processes used to draw conclusions
2. Develop repeatable methods and techniques
3. Build and advocate training that teaches practitioners how to think
![Page 8: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/8.jpg)
What separates novice and
expert analysts?
![Page 9: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/9.jpg)
Mapping the Investigation Sample:
Novice and expert analysts Methodology:
30+ case studies Stimulated recall interviews Focus on individual investigations of
varying types Perform key phrase analysis – analyze
results
![Page 10: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/10.jpg)
Key Phrase Mapping Dual Process Theory
Intuition: Implicit, unconscious, fast Reflection: Explicit, controlled, slow
IntuitionExperimentation
RestructuringImaginationIncubation
MetacognitionEvaluation
Goal SettingMaking Plans
ReflectionAnalytically
Viewing DataRule-Based Reasoning
Considering Alternatives
![Page 11: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/11.jpg)
Results
Expe
rimen
tation
Restruc
turing
Imag
inatio
n
Incub
ation
Evalu
ation
Goal S
etting
Making
Plan
s
Viewing
Data
Consid
ering
Alter
nativ
es
Novices Experts
Intuition Metacognition Reflection
![Page 12: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/12.jpg)
Analyzing the Flow of
the Investigation
![Page 13: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/13.jpg)
Investigations as Mental Labyrinths
The investigation is the core construct of information security.
How do we study them when everyone has a different toolset? Follow the Data!
Alert
OSINTReputation
File Hash
Sandbox Behaviors
AV Detections
(VT)
Imphash More File Hashes
Friendly Host
Network PCAP
Host
Windows Logs
Security Log
System Log
App LogRegistry
File SystemHostile
Host NetworkPCAP
Flow
![Page 14: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/14.jpg)
Studying the Investigation Process
![Page 15: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/15.jpg)
Studying the Investigation Process
![Page 16: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/16.jpg)
What data did analysts look at first?
72%16%
12%
Observed
PCAP FlowOSINT
Data Suggests: Analysts prefer a higher context data set…
…even if other data sets are available …even if lower context data sets can lead to a resolution.
![Page 17: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/17.jpg)
Did the first move affect analysis speed?
Data Suggests: While PCAP provides richer context, it may slow down the
investigation if that’s where you start Starting with a lower context data source can increase
speed when working with higher context data
PCAP Flow OSINT
16
10 9
Avg Time to Close
![Page 18: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/18.jpg)
What happens when Bro data replaces PCAP?
46%
25%
29%
Observed (Bro)
Bro Flow OSINT
72%
16%
12%
Observed (PCAP)
PCAP Flow OSINT
![Page 19: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/19.jpg)
What happens when Bro data replaces PCAP?
PCAP Flow OSINT
16
10 9
Avg Time to Close (PCAP)
Bro Flow OSINT
10 10 11
Avg Time to Close (Bro)
Data Suggests: Better organization of high context data
sources can yield improvements in analysts performance
![Page 20: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/20.jpg)
What data sources were viewed most and least frequently?
Data Suggests: Network data is used more frequently than host data…
…even when host data can be used exclusively to resolve. …even when easy access is provided to host sources.
Revisting data is more prevalent on higher context data sources
PCAP Flo
wOSIN
T
Host FS
OS Log
s
Memory
Data Sources Viewed Data Sources Revisited
PCAP84%
Flow11%
OSINT5%
![Page 21: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/21.jpg)
How many steps were taken to make a disposition judgement?
Data Suggests: At some point, the number of data sources you
investigate impacts the speed of the investigation Understanding where data exists and when to use it can
impact analysis speed
6-10 11-15 16-20 21-250
5
10
15
6
129
3
Number of Steps
6-10 11-15 16-20 21-2505
1015202530
9 12 14
24
Avg Time to Close
![Page 22: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/22.jpg)
Did analysts investigate friendly or hostile systems first?
9%
91%
Observed
Friendly Hostile
Data Suggests: Analysts are more compelled to investigate unknown external
threats than internal systems Analysts don’t fully understand their own techniques
41%59%
Friendly
Friendly Hostile
![Page 23: Art into Science 2017 - Investigation Theory: A Cognitive Approach](https://reader035.fdocuments.us/reader035/viewer/2022070512/589aa7d71a28abfc1a8b67a3/html5/thumbnails/23.jpg)
Thank You!
Mail: [email protected]: @chrissanders88
Blog: chrissanders.orgTraining:
chrissanders.org/training