From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be...
Transcript of From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be...
![Page 1: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/1.jpg)
© 2011 Carnegie Mellon University
From Bandwidth to Beacon Detection, Prism and Touchpoints
George JonesPaul KrystosekSid FaberSEI CERT NetSA
![Page 2: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/2.jpg)
2From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
NO WARRANTY
THIS MATERIAL OF CARNEGIE MELLON UNIVERSITY AND ITS SOFTWARE ENGINEERING INSTITUTE IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the trademark holder.
This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected].
This work was created in the performance of Federal Government Contract Number FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to the copyright license under the clause at 252.227-7013.
![Page 3: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/3.jpg)
3From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Introduction
New projects are
magical
You never know where
they will lead you
Keep an open mind
and be prepared to
act
![Page 4: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/4.jpg)
4From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Key
Pipeline
Analytical Process
Report
Tool
![Page 5: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/5.jpg)
5From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
The starting pointBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 6: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/6.jpg)
6From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
The Bandwidth StudyOnce upon a time…
There was a network that everyone thought was dirty.
They planned to get some sensors in place…but all they had for now was flow.What could be done to keep them safe until
sensors were deployed?
This is where our story starts
From there it meanders hither and yon
![Page 7: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/7.jpg)
7From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 8: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/8.jpg)
8From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
An iterative Process
Find a large usage
category, e.g. Web traffic
Split it off and look at the
rest
Repeat…
Wherever you stop there are probably flow
records left over
![Page 9: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/9.jpg)
9From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 10: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/10.jpg)
10From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
The Trend Script is born
A configuration file defines bins with enough
detail for SiLK rwfilter command
Primarily ports and protocols define bins
Run every hour from cron• Get flows• Calculate bin volumes• Append a record to a flat file
Visualize from the flat file for the desired time
![Page 11: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/11.jpg)
11From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Sample Trend Script Configuration
[bin]name: http-clienttitle: Client Webfilter: --protocol=6out-filter: --dport=80,443,8080in-filter: --sport=80,443,8080
![Page 12: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/12.jpg)
12From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
![Page 13: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/13.jpg)
13From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
![Page 14: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/14.jpg)
14From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 15: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/15.jpg)
15From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Traffic ActivityThe activity in the bins is fairly well knownThe “left over” flows, less soWhat is happening “at the edge”?Looking at flows by hand is tediousIt’s hard to program looking for the unknownThat means, it’s time for…
![Page 16: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/16.jpg)
16From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Flow Activity VisualizationWe want to find “interesting activity”But interesting means different things to different people
• “May you live in interesting times.” Chinese Curse
• “Only accurate rifles are interesting.” Colonel Townsend Whelen
• “The only interesting answers are those that destroy the questions.”Susan Sontag
![Page 17: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/17.jpg)
17From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Flow Activity VisualizationGoal: produce a self-maintaining network profile• Categorize and display activity
– Stuff we know about: Email, Web, DNS…– And everything else
• Need a mechanism to permit the analyst to examine “everything else” aka leftovers
• Too bad about the “self-maintaining” part
![Page 18: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/18.jpg)
18From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 19: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/19.jpg)
19From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
StripPlot “enables the eyeball”Get a good idea of what a particular IP addressing is doingSee how a port is usedStreaming video and audio are immediately apparentMake Beacons stand outFor more information on StripPlot see:• http://www.cert.org/flocon/2010/presentations/Faber_StripPlots.pdf
![Page 20: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/20.jpg)
20From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
The StripPlot Process
StripPlot Graphic
Plot them with
Gnuplot
Process the flows
Create “interesting” configuration
file
![Page 21: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/21.jpg)
21From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
How to Interpret StripPlot
![Page 22: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/22.jpg)
22From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Sample StripPlot
![Page 23: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/23.jpg)
23From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 24: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/24.jpg)
24From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Finding Malicious ActivityMalware Team to NetSA Analysis Team:• “You might find this interesting”
The visualization in StripPlot made it easy to spot the interesting behavior
![Page 25: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/25.jpg)
25From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 26: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/26.jpg)
26From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Spin off the Network Touchpoints Project
Find network indicators in malware
Find the indicators in Flow
Characterize and Report
![Page 27: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/27.jpg)
27From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 28: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/28.jpg)
28From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Beacon DetectionStripPlot “enabled the eyeball” to see botnet nodes phoning homeWe even saw a handoff from one C2 host to anotherBeacon detection attempts to “replace the eyeball”
![Page 29: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/29.jpg)
29From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Beacon DetectionSo… if we can find beacons we can find botnets, Right?Yes, if you can distinguish a beacon from other regular behaviorWhich is hard
![Page 30: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/30.jpg)
30From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 31: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/31.jpg)
31From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Paul’s Beacon DetectorBeacons exhibit regular behavior• A series of connections or connection attempts• Between the same two IP addresses• At regular time intervals
Implemented a Finite State Machine to find• X or more flows (5 flows)• At regular interval of Y (Y>= 5 minutes)• With a tolerance of Z percent (5%)
![Page 32: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/32.jpg)
32From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Beacon Detection
![Page 33: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/33.jpg)
33From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Did it work?Did it find regular behavior?• Yes, rather a lot of it
Did it find botnet beacons?• Probably but hard to distinguish from all the other stuff
What other stuff?• NTP, News updates, email updates, DNS…
![Page 34: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/34.jpg)
34From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Can it be made better?Three ways that we know of• Find more regular behavior
– Missing flows
• Additional information– Actual botnet beacon characteristics– Any other information that can be used with flow analysis
• Extreme whitelisting– Keep track of everything that beacons, and ignore it– Only look for new stuff
• Keep track of the beaconing addresses for the last 30 days• Whitelist them
![Page 35: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/35.jpg)
35From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 36: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/36.jpg)
36From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Get results soonerTraditional SiLK commands find flows in the repositoryTo get the most recent, set the search time and run it in cron, but how often• Run cron too often and one doesn’t finish before the next one
starts• Run it less often and you wait longer than necessary
We want to look at flows as soon as they are available
![Page 37: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/37.jpg)
37From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Pipeline fills that rolePipeline runs continuously and processes SiLK files as they are writtenPipeline has its own unique filtering strategyPaul’s Finite State Machine was implemented in PipelineIt will alert as beacons (instances of regular behavior) are found
![Page 38: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/38.jpg)
38From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 39: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/39.jpg)
39From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Eight Different Beacon Detectors?Motivation• Beacon detection is either very useful or a very shiny object: I
know of at least 8 implementations, 9 if you count stripplot.• Saw beaconing in strip plots of RAT• Recognized utility of finding beacons to detect certain RATs• Concluded that "eye charts don't scale”• Determined to explore algorithmic approaches
![Page 40: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/40.jpg)
40From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
YABD[1] – Yet Another Beacon Detector?Activities• Explored different algorithms, implemented several• Performed analysis of running time• Identified common sources of false positives• Generated RAT traffic in lab for testing• Explored live data
[1] Biologists use YABD as an index of the health of deer in relation to carrying capacity.
![Page 41: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/41.jpg)
41From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
From Eye Charts to…Outcomes• Two first generation beacon detectors • One second generation detector in pipeline• Tools delivered to different analyst communities with mixed
levels of adoption.
![Page 42: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/42.jpg)
42From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Lessons Learned• Your first thought on algorithms may not be right• You need need a large sample of ground truth to test against• Algorithms that work on a few samples may not work in the
wild.• It's hard to generate realistic background data.• False positives are common.• Need to socialize more with analyst community.• Adoption is tied to perceived utility of the tool, ownership the
analysts feel of it (homegrown tools win), and their trust in the person/organization providing the tool to meet their specific needs.
![Page 43: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/43.jpg)
43From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 44: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/44.jpg)
44From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
The Rayon Viz LibrarySeveral analytics had visualization requirements in commonStripPlot pushed Gnuplot to its limitsIt was time to move away from “Analyst Code”
![Page 45: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/45.jpg)
45From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Why didn’t he call it Yet Another Graphics Package
Phil Groce of the NetSA Development team • gathered requirements• wrote a set of “flow aware” graphics primitives• wrote several applications using the primitives• released it to the world as Rayon
– http://tools.netsa.cert.org/rayon/index.html
• ask us later if you don’t get the play on words
![Page 46: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/46.jpg)
46From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
OverviewBandwidth Study
Volume Visualization Traffic Activity Visualization
Beacon Detection
Trend Script
Prism
StripPlot
Malware Identification
Network Touchpoints
BeaconDetector
Kynk
Network Profiling
Profile ReportRayon
Pipeline
Key
Analytical Process
Report
Tool
![Page 47: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/47.jpg)
47From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
PrismThere was a renewed interest in Trend ScriptBut it is an analyst’s tool for specific tasksA continuous volume display has other requirementsPrism is a re-write of the Trend Script by NetSA Development Team member John Prevost
![Page 48: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/48.jpg)
48From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Prism vs Trend Script
Trend script uses a flat file
Prism uses a database
![Page 49: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/49.jpg)
49From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Prism vs Trend Script
One offs in Trend Script
are easy
There is no such thing
in Prism
![Page 50: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/50.jpg)
50From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Prism vs Trend Script
Search Trend Script with grep, vi or
emacs
SQL query for Prism
![Page 51: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/51.jpg)
51From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
ConclusionsOne thing leads to another“If we knew what we were doing, it wouldn’t be called research, would it?” A. EinsteinDon’t be afraid to scrap something and start over
![Page 52: From Bandwidth to Beacon Detection, Prism and Touchpoints€¦ · This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic](https://reader033.fdocuments.us/reader033/viewer/2022050422/5f913f8efbc95d18f9600c3e/html5/thumbnails/52.jpg)
52From Bandwidth to Beacon Detection…Jones, Krystosek, Faber, January 2012© 2011 Carnegie Mellon University
Paul Krystosek [email protected] Jones [email protected] Faber [email protected] Situational Awareness GroupCERT Software Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA