MOLOCH: Search for Full Packet Capture (OA Cyber Summit)
-
Upload
open-analytics -
Category
Technology
-
view
1.266 -
download
8
Transcript of MOLOCH: Search for Full Packet Capture (OA Cyber Summit)
![Page 1: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/1.jpg)
MOLOCH:SEARCH FOR FULL PACKET
CAPTURE
![Page 2: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/2.jpg)
It is a Great Horned Owl
Project Logo
![Page 3: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/3.jpg)
WHY THE OWL?
Owls are silent hunters that go after RATs. We think that’s pretty
cool.
3
![Page 4: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/4.jpg)
HISTORYLESSONWHY AOL BUILT MOLOCH
![Page 5: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/5.jpg)
WHAT IS MOLOCH?
5
Moloch is an open source, scalable IPv4 packet capture indexing and database system, built using open source technologies.
• A simple web GUI is provided for browsing,
searching, viewing and exporting PCAP data.
• Web APIs are accessible if you wish to design your
own GUI or directly grab PCAP with various
command line tools for further analysis or
processing.
• Find it on AOL’s GitHub page:
https://github.com/aol/moloch
It’s like AOL Search for PCAP repositories!
![Page 6: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/6.jpg)
WHAT IS MOLOCH NOT?
6
NOT IDS: NO ALERTS
NOT IPV6 (Today)
NOT SLOW
NOT CLOSED
NOT EXPENSIVE
![Page 7: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/7.jpg)
WHY USE MOLOCH?
7
Real-time capture of network traffic for forensic and investigative purposes
• Combine the power of Moloch with other indicators (intelligence
feeds, alerting from IDS/anti-virus) to empower your analysts to
quickly and effectively review actions on the network to
determine the validity/threat.
• Review past network traffic for post compromise investigations.
Static PCAP repository
• Import large collections of PCAP that were created by malware.
• Import collections of PCAP from Capture The Flag events.
• Custom tagging of data at time of import.
![Page 8: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/8.jpg)
THE PIECES OF MOLOCH
8
CAPTURE
• A C application that sniffs the network interface, parses the traffic, and creates the Session Profile Information (SPI data) and writes it to disk.
DATABASE
• Elasticsearch is used for storing and searching through the SPI data generated by the capture component.
VIEWER
• A web interface that allows for GUI and API access from remote hosts to browse or query SPI data and retrieve stored PCAP.
![Page 9: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/9.jpg)
THE PIECES OF MOLOCH:CAPTURE
9
Libnids based daemon written in C
Can be used to sniff network interface for live capture
Can be called from CLI to do manual imports
Parses layers 3-7 to create SPI data
• Spits them out to the Elasticsearch cluster. A lot like making owl pellets!
![Page 10: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/10.jpg)
![Page 11: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/11.jpg)
THE PIECES OF MOLOCH:DATABASE
11
Elasticsearch (http://www.elasticsearch.org)
• Powered by Apache Lucene (http://lucene.apache.org)
• Requests over HTTP(s)
• Results returned in JSON
Nosql
• Network traffic doesn’t fit the mold for relational DBs.
Documented oriented
• Great for lots and lots of network sessions.
Automatic sharding across multiple hosts
• At the time, we skipped SOLR because it couldn’t run distributed.
Fast, scalable, all that goodness
![Page 12: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/12.jpg)
THE PIECES OF MOLOCH:VIEWER
12
Node.js based application
• Event driven server side JavaScript platform.
• Based on Chrome’s JavaScript runtime.
• Comes with its own HTTP server and easy JSON for communication.
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP.
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs, consoles, and command line tools is easy.
• Easy automation to retrieve PCAP or sessions of interest.
![Page 13: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/13.jpg)
THE PIECES OF MOLOCH:VIEWER
13
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
![Page 14: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/14.jpg)
THE PIECES OF MOLOCH:VIEWER
14
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
![Page 15: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/15.jpg)
ARCHITECTURE OF MOLOCH:DATA FLOW
15
![Page 16: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/16.jpg)
ARCHITECTURE OF MOLOCH:MULTINODE WITH CLUSTER
16
![Page 17: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/17.jpg)
ARCHITECTURE OF MOLOCH:SCALE
17
Packets Captured Kilobytes Saved Sessions Saved
Example: Moloch Capture
Documents Disk Storage (MB)
Example: Elasticsearch
![Page 18: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/18.jpg)
MOLOCH: SPI-DATATYPESSESSION PROFILE INFORMATION
18
IP
• Source
• Destination
• Ports
• Protocol
HTTP
• Method
• Status Codes
• Headers
• Content Type
DNS
• IP Address
• Hostnames
![Page 19: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/19.jpg)
MOLOCH: SPI-DATATYPESSESSION PROFILE INFORMATION
19
SSL/TLS
• Cert Elements:
• Common Name
• Serial Number
• Alt Names
SSH
• Client Name
• Public Key
• Port
IRC
• Channel Name
• Hostname
![Page 20: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/20.jpg)
MOLOCH: CAPTURECREATING SPI DATA
20
![Page 21: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/21.jpg)
MOLOCH: CAPTURECREATING SPI DATA
21
![Page 22: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/22.jpg)
MOLOCH: CAPTURECREATING SPI DATA
22
![Page 23: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/23.jpg)
MOLOCH: CAPTURECREATING SPI DATA
23
![Page 24: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/24.jpg)
MOLOCH: CAPTURECREATING SPI DATA
24
![Page 25: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/25.jpg)
MOLOCH: DEMO
25
![Page 26: MOLOCH: Search for Full Packet Capture (OA Cyber Summit)](https://reader030.fdocuments.us/reader030/viewer/2022020218/55941a2a1a28ab032c8b45a1/html5/thumbnails/26.jpg)
MOLOCH: QUESTIONS?
26