1
MAD: A Middleware Framework for Multi-Step Attack Detection
Panagiotis Papadopoulos, Thanasis Petsas, Giorgos Christou
and Giorgos Vasiliadis
2
Network Attacks• Automated attacks (e.g. worms and viruses) easy to detect by a
signature-based NIDS. • But there are also more sophisticated targeted attacks out there:
e.g. Advanced Persistent Threats (APT)Traditional Attacks APT Attacks
Attacker Mostly single person Highly organized, sophisticated, determined and well-resourced group
Target Unspecified, mostly individual Systems
Specific organizations, governmental institutions, commercial enterprises
Purpose Financial benefits, demonstrating abilities
Competitive advantages, strategic benefits
Approach Single-run, “smash and grab", short period
Repeated attempts, stays low and slow, adapts to resist defenses, long term
3
Multi-step attacksdesigned for political or economic espionage or sabotage and are fired
against governments, organizations, highly competitive companies, political activists etc.
require coordinated human involvement rather than an automated malicious script.
follow long-term steps of actions consist of multiple correlated steps to reach a specific target combine several attack methodologies:
• e.g. drive-by downloads, SQL injections, malware, spyware, phishing, spam emails etc. and tools:
• including zero-day vulnerability exploits, viruses, worms, and rootkits).
4
Attacks’ main characteristics (1/2)1. Persistence
• location of data, security controls deployed, existed vulnerabilities etc. not known a-priori. • attacker must overcome various security measures to finally gain access to
privileged hosts
2. Evasiveness• designed to evade common security mechanisms,• deliver threats through commonly used protocols (HTTP, SMTP, POP etc.)• To be stealthy custom malwares need to be used• Encryption techniques may be used to avoid firewalls while exfiltrating data out of
the target network.
5
Attacks’ main characteristics (2/2)3. Complexity
combination of attack vectors targeting as many vulnerabilities as possible (e.g. social engineering, Remote Access Trojans (RATs), or other custom malicious software)
very difficult to provide defense on all these different attack vectors
6
The attack phases1. Host Reconnaissance
collect useful information by scanning and studying the victim.2. Persistent Incursion
take advantage of possible host’s vulnerabilities and launch “low-and-slow” attack to avoid detection.
3. Control, Discover, Update, Spread map the network topology and the organization defenses from the inside, update tool chest, spread the infection to other nodes of the target network
4. Capture and Exfiltrationtotal control of a number of hosts and extraction of valuable data off the target network to be analyzed.
7
Multi-step attack example
8
Multi-step attack example
9
Multi-step attack example
10
Multi-step attack example
11
State-of-the-art countermeasureNetwork Intrusion Detection Systems (NIDS)
• The presence of a NIDS is a cornerstone in any modern security architecture e.g. Suricata, Snort, Bro.• captures the network traffic at ingress and egress points in the network• performs the required analysis and processing. • detects and stops malicious attacks or unwanted actions.
• Signature-based: a set of pre-defined signatures is matched against the live captured traffic.
12
NIDS Vs Multi-step attacks• When NIDS relies only on live network traffic accuracy decreases
significantly.
Solution:• archive the raw contents of the network traffic stream to disk• enable later inspection of activity
13
Data, data, data…• increasing network traffic and capacity make the collection and
archiving very challenging.
• E.g. 10 GbE network packet arrivals can be as short as 1.25 μs for a 1.5KB MTU storing full packet traces even for 2 hours can result to thousands of GB of data.
14
Challenges• Storage: wholesale recording and retention of entire data streams is
infeasible• A Gigabit network several TB per day• network trace with full packet content can provide much information for
investigating security incidents
• Data selection: only a very small subset of the traffic is relevant for later analysis• How to decide beforehand what data will be crucial?
• Analysis: data retrieval is like finding needle in a haystack
15
Our approach? Get MAD…
16
MAD: A middleware framework for Multi-step Attack Detection1. coupled with a network monitoring application enhances its functionalities
2. enables IDS to analyze and correlate multiple security incidents that may belong to the same attack pattern.
3. significantly reduces the rate of NIDS’s false alarms
4. post-mortem incident analysis in terms of forensic analysis asses the given damage
5. Includes different mechanisms to store the captured network traffic
17
Coupling MAD middleware framework with NIDS• Broadening the analysis context
Analyses traffic from past
• NIDS recovers from Packet DropsNIDS may incur measurement drops under heavy loadcan query MAD for connections that are missing packets and reprocess them
• PrioritizationNIDS can assign priorities to flows letting the rest of the traffic be processed
during idle times.
18
High Level Overview(1/2)• Packet Capturer
responsible for tapping the network link, monitor the traffic and filter the received packets.
• Query Engineresponsible for the responding to the IDS’s GET-requests.
19
High Level Overview(2/2)• Correlation Engine
component to correlate the attack steps by linking NIDS alerts.
• Size Controllerstorage capacity is not inexhaustible, appropriate actions are applied to achieve the highest possible storage size reduction
Snort
20
The Storage component
Storage
1. receives the raw packets, separates headers from payload, responds to the Query Engine
2. stores the fields of their header in a RDBS.3. packets payloads are stored serialized and
grouped by flow.
21
Handling the ever increasing size of datathe more “knowledge” MAD maintains, the more accurate the attack detection will become.
• ”knowledge”= the information residing in the archived historical traffic.
• traffic includes several events steps to a sequence, able to end up to a multi-step targeted attack
• maintaining such network history knowledge is a point of paramount significance
results in storing large amounts of network traces.
22
Controlling the size of the archived traffic• In Size Controller component we adopt several mechanisms to both:
reduce the size of data and at the same time protect the important information that might later be needed.
• These mechanisms include:- Compression - Deduplication - The Cutoff
Heuristic- Classification - Aggregation & Sampling
23
Mechanisms to control the stored data volume (1/3)• Classification
• classify the traffic according to the content of the packets• if we are interested in detecting multi-step attacks that target a specific
application, we can discard the rest non-suspicious traffic.
• Compression• the most efficient and fast method to reduce the required size of a large volume
of data. • flow-based algorithms for trace compression can result in 25% reduction of the
required storage size.• frequent IDS queries, may face increased response latency due to decompression.
24
Mechanisms to control the stored data volume (2/3)• Deduplication
• Reducing duplicates and redundant data better storage utilization.• Packet-level elimination techniques can reduce resource utilization by 10-50%• Inline: the data is processed immediately as its ingested (takes time).
Post-process: the data are deduplicated after they hit the disk (needs capacity).
• Aggregation & Sampling• Both significantly reduce the needed size.• Aggregation requires the traffic features of interest to be known in advance. Not
useful for signature-based checks since much of the payload’s information gets discarded• Random packet sampling decreases the detection accuracy
25
Mechanisms to control the stored data volume (3/3)• The Cutoff Heuristic• selective packet discarding technique. i.e., by discarding the less important
packets of a trace or a flow.
• most of the attacks are detected in the first few packets of a flow. (97% of the alerts are triggered within the first 100 packets of the flows)
• an attacker can evade detection, by transmitting data until the cutoff value has passed.
• a single network connection may exchange large amounts of data passing the cutoff value.
26
In summary…Compression Deduplication Classification Aggregation & Sampling Cutoff Heuristic
Up to 25% size reduction
10%-50% reduced resource utilization
Only specific flows will get archived
Significantly reduces the required storage size.
97% of the alerts in the first 100 packets
Decompression may increase response time in case of frequent queries
Inline takes time,
Post-process needs capacity
Specification of sensitive channels is needed.
Aggregation: the traffic features of interest has to be known in advance.
Random packet sampling may decrease the detection accuracy
Attacker can evade detection (transmit data until the cutoff value has passed.)
27
Future Work• This work is currently in progress
• Our further work would include: extensive evaluation:• in terms of performance, in comparison to other existing solutions • in terms of effectiveness by measuring the produced false positive alert
ratio. a case study to measure the detection percentage against several
known multi-step attacks.
28
Conclusions• Network attacks become more sophisticated and diversified. Several actions
may individually look harmless but when combined can constitute a serious threat.
• NIDS are inadequate countermeasures when rely only on live network traffic.
• We propose MAD to improve the accuracy of NIDS by archiving historical traffic providing knowledge regarding the previous steps.
• We examine several mechanisms to reduce the storage size needs and archive only important information.
Top Related