Multi-Agent System for APT Detection

15
Multi-Agent System for APT Detection Wim Mees & Thibault Debatty 2nd IEEE International Workshop on Reliability and Security Data Analysis (RSDA 2014)

Transcript of Multi-Agent System for APT Detection

Page 1: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection

Wim Mees & Thibault Debatty

2nd IEEE International Workshop on Reliability and Security Data Analysis (RSDA 2014)

Page 2: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 2

APTs : MiniDuke …

Page 3: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 3

APTs : … and all others!

● MiniDuke– Targeted PDF + 0-day exploit– June 2012 → February 2013

● Belgacom– Fake, targeted, LinkedIn pages – 2010 → June 2013

● …

Page 4: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 4

APTs : … and all others!

● MiniDuke– Targeted PDF + 0-day exploit– June 2012 → February 2013

● Belgacom– Fake, targeted, LinkedIn pages – 2010 → June 2013

● … Attackers WILL succeed

Achieve early detection

Page 5: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 5

The approach

Analysis at choke point

Page 6: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 6

The approach

Multi-agent

Page 7: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 7

The approach

Human analyst

Page 8: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 8

Agents

● Frequency analysis● Time-domain impulse● Upload● Domain name fan-in, fan-out● Geographic outlier● Domain name age● URL reputation● …

Page 9: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 9

Aggregation

● Ordered Weighted Averaging (Yager)

– E.g. : 0.2, 0.3, 0.5, 0.0● Agent activation logic

– Run “light” agents on all clients– Activate “heavy” agent on suspicious clients

Client honeypot, IDS, long time analysis, …

Page 10: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 10

First results

● Synthetic network traffic● Simulated APT traces (from literature)

Page 11: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 11

First results

● Real network traffic (anonymized)● Simulated APT traces (from literature)

Page 12: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 12

First results

● Real network traffic (anonymized)● Simulated APT traces (from literature)● Human interaction:

whitelisting

Page 13: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 13

Conclusions & future work …

● Promising approach (modular design)

● Current work:– Additional agents (SMTP, IDS, client honeypot, …)– Choose OWA coefficients to maximize area under ROC curve– Test on real network traffic and APT traces

● Future work:– Human feedback integration– Time behavior– Integration with SIEM– Comparison with other tools

Page 14: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 14

Thank you!

Page 15: Multi-Agent System for APT Detection

Multi-Agent System for APT Detection 15

Contact

Thibault DebattyRoyal Military Academy

Brussels, Belgium

www.rma.ac.be

[email protected]