aleph - Malware analysis pipelining for the masses
-
Upload
jan-seidl -
Category
Technology
-
view
706 -
download
4
Transcript of aleph - Malware analysis pipelining for the masses
alephMalware analysis pipe-lining for the
masses
Who we are?
Jan Seidl @jseidl
Aleph Project Lead Developer
*NIX/BSD freak
Digital tools blacksmith / python & C lover
Lousy guitar player
Coffee dependent
Hates printers, doesn't likes social networks anything
Selectively-social
Who we are?
Actually we are a bunch of people...
Malware
Definition
'Malware' is an umbrella term used to refer to a variety of
forms of hostile or intrusive software, including computer
viruses, worms, trojan horses, ransomware,
spyware, adware, scareware, and other malicious
programs.
It can take the form of executable code, scripts, active
content, and other software
Wikipedia
Malware growth
Detecting malware
Detecting malware● Signature-based
● Sample must be previously-known and flagged as malicious
● Heuristics-based● Can trigger loads of false-positives
● Behavior-based● Can trigger loads of false-positives
Detection is not enoughYou need to understand the malware
Understanding malware
● Features extraction
● Which characteristics this file has?
● Feature correlation
● Make sense of features combinations / disposition
● Sample correlation & Family classification
● Identify common features between different samples
Understanding malware
● Enables you to identify families
● Enables you to identify acting groups
● Enables you to identify techniques
● Enables you to identify trends
Manual approachEveryday workhorse method
Manual approach
● Use lots of separate tools to extract data from sample (each in its own format)
● Correlate output from the tools using spreadsheets, word files, napkins, tears
Manual approach
● Find out new samples embedded into original sample
● Rinse, repeat, get more whiskey/coffee
Automated approachPipeline method
Automated approach
● Insert sample into one end● Wait until processing is done● Get report on the other end● Get emotional about hours of work saved● Focus on most important evidences
Commercial playersGiving you some free analysis while you
feed their database for free
● Akana (Android files)● Anubis● BitBlaze Malware Analysis Service● Comodo Automated Analysis System
and Valkyrie● EUREKA Malware Analysis Internet
Service● Joe Sandbox Document Analyzer (PDF,
RTF and MS Office files)● Malwr● MASTIFF● VxStream Sandbox (Hybrid Analysis)● VirusTotal*● ThreatExpert● ThreatTrack● ViCheck● VisualThreat (Android files)● XecScan (PDF and MS Office files from
targeted attacks)
Bringing it to the massesGiving you the ability to spin up your
own commercial-grade malware research infrastructure
Components
Aleph ProcessFrom fetching samples to report building
Main Features
● Cross-platform (tested on: Windows, Linux, OS X)Almost all modules are pure-python
● Scalable + Easily Extensible
● Web Interface for browsing reports
From fetching samples to report building
Aleph Process
Sample
Sample data
Aleph Process
Sample Sample data
Aleph Process
Collector Sample Manager
Sample Queue
Sample Plugins
Data
Aleph Process: Collection
●Detect new file on medium (filesystem, email account etc)
●Check if meets predefined criteria (min/max size)
Aleph Process: Triage
●Detect file type (mimetype)
●Calculate hashes
●Add sample to process queue
Aleph Process: Processing
●Enumerate plugins suitable for sample mimetype
●Run plugins and extract features
●Save features as structured data into database
Aleph Process: Reporting
●Fetch sample information from database
●Generate report based on retrieved data
Currently supported files●Windows Portable Executable (PE) (exe,
cpl & dll)
Coming up support for:● Android APK
● PDF Documents
● Linux ELF
● iOS Apps
● URLs & Emails
● Apple Mach-O
● MS Office Documents
● SWF & Much more!
Aleph: Web Interface
Aleph: Web Interface
Aleph: Web Interface
Aleph: Web Interface
Aleph: Web Interface
Shifting the paradigmNew exciting features coming up
DecouplingDividing the functionality into
standalone components
Decoupling
Collector
Sample
WebInterface
AlephProcessor
RDBMSSample
DatastoreStorage
StorageConnector
Web UI Info
HTTP API, SSH...
TransportConnector
Transport
Executable, Email, URL,PDF, APK etc
Local File, FTP, Email etc Redis, RabbitMQ,Memcached
SQLite, MySQL,PostgreSQL, SQL Server
Filesystem, VXVault, Amazon S3, Azure etc
ElasticSearch
Sample + Initial Meta
Sample + Initial Meta
Send & Retrieve
Download
Sample + Initial Meta
Adapters & ConnectorsAbstracting functionality so you use
what you have, or what you're familiar with...
Adapters & Connectors
CollectorManager
Sample File Collector
E-Mail Collector
FTP Collector
Sample
Sample
Transport
TransportConnector
Redis, RabbitMQ,Memcached
Scoring SystemThe path to the Evil Index
Data Correlation PluginsAutomatic linking and enrichment of
gathered data
Data Visualization PluginsBecause everyone loves graphs and scatterplots
Online DisassemblyDisassembled code & memory dumps will be saved as a new sample/artifact
Tons of more cool plugins
●VxCage●Viper● Java Decompiler●Maltrieve
CollaborationWhat got us here in the first place
Collaboration
The power of many
Deployment TypesFrom small to huge scaling
Deployment Types
Deployed in a single host containing all the required services.
3rd Party Software Aleph Components
Redis Local Filesystem Elasticsearch SQLite
Collector Processor Web Interface
Deployment Types
Deployed across multiple hosts in order to achieve HP and HA.
Datastore Host GroupElasticsearch Cluster
Nodes
Transport Host GroupRabbitMQ Cluster Nodes
Processing Host GroupAleph Cluster Nodes
Web Interface Host Group
NGinx Cluster Nodes
Collection Host GroupAleph Cluster Nodes
Storage Host GroupDFS Cluster Nodes
Use, fork and contribute!
https://github.com/trendmicro/Aleph
Questions?We can try to answer some….
Thank you!
Jan Seidl @jseidl <[email protected]>
Slides: http://slideshare.net/jseidl Codes: https://github.com/jseidl