User and entity behavior analytics: building an effective solution
-
Upload
yolanta-beresna -
Category
Data & Analytics
-
view
75 -
download
2
Transcript of User and entity behavior analytics: building an effective solution
![Page 1: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/1.jpg)
User and Entity Behaviour AnalysisBuilding an Effective SolutionYolanta Beresna
Research Manager, Threat Detection and Remediation, Software Defined Cloud Group
10 November 2016
![Page 2: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/2.jpg)
Outline
Overview of UEBA space Key components of an Effective Solution
– Threat Use cases– Data Sources– Analytics– Pluggable Analytics Modules
![Page 3: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/3.jpg)
UEBA: Overview
![Page 4: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/4.jpg)
4
User and Entity Behaviour AnalyticsThe Definition
User and entity behavior analytics is bringing profiling and anomaly detection based on machine learning to security, to detect malicious and abusive activity that otherwise goes unnoticed. Profile and baseline the activity of users, peer groups and
other entities such as endpoints, applications and networks.
Form peer groups based upon common user activities, using directory groupings and human resources information only as a starting point.
Correlate user and other entity activities and behaviors.
Detect anomalies using statistical models, machine learning and/or rules that compare activity to profiles.
Source: Gartner (September 2015)
![Page 5: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/5.jpg)
UEBA across IT systemsUsers-accounts
• Mapping: user-account-hostname• Behaviour: account usage across
applications and domains• Suspicious behaviour:
Changes in behaviour for highly privileged users and core systems
Changes in access and account usage behaviour
Peer group comparison• Data: active directory, LDAP, system
and application account usage
Users-entities
• Mapping: user-hostname-ipaddress
• Behaviour: network traffic patterns
• Suspicious behaviour Historical changes in behaviour Outliers based on peer group
comparison Specific threat patterns: malware
infections, tunnelling traffic, beaconing• Data: DNS, HTTP, Netflow, VPN
Entities-servers
• Behaviour: network traffic patterns• Data: DNS, HTTP, Netflow, system logs
Connections
Linked information between: user-accounts user-entities entities-servers
![Page 6: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/6.jpg)
Features of UEBA Solution
An effective UEBA the solution has at least the following properties:
Effective data collection and data representation layer
Correlation of entities identifiers to users and user accounts to users
Abnormal behaviour detectionSpecific threat detectionDiscovery of core systems and privileged users as
well as peer groups or communitiesLinking together of multiple detection results into a
coherent threat view across enterprise
Suspicious Entity and User Detection Analytics
In addition it is essential to have capabilities to add new analytics and reconfigure existing ones: play (by developing new analytics) and plug (for automated results) framework
![Page 7: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/7.jpg)
7
Creating an Effective Solution
![Page 8: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/8.jpg)
8
Core Components
The effectiveness of an UEBA greatly depends on these core components:
1. Focused threat scenarios and use cases
2. Availability of relevant data sources and variables
3. Appropriate analytics algorithms
![Page 9: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/9.jpg)
9
Anatomy of Attacks
![Page 10: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/10.jpg)
Threat Use Cases
Threat Actor
External Internal
Goal
Theft
Attack Story 1: A hacker organisation gains
access to the system over the Internet and
steals user credentials and business data.
Attack Story 2: An employee uses their
access to the system to steal business data.
Sabotage
Attack Story 3: Ransomware attack:
Business data shared on the internal network is
encrypted by ransomware running on
a client machine.
Attack Story 4: An employee reconfigures
the machines in the network to render their services unavailable to
legitimate users.
• Attack stories describe concrete attacks• What is happening?• In which order?• When?• Where?
• Goal/Actor Matrix to develop stories:• Goal: What do the attackers want to achieve?• Actor: Who are the attackers?
• Attack Story Steps:1. Gain access2. Get means to achieve goal3. Reconnaissance and lateral movement4. Achieve goal
![Page 11: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/11.jpg)
11
Attack Story 1: Data Exfiltration by External Actor
Stage Analytics Features Data Outcomes + Context
Gain Access/Initial Infection 1. Detect malicious web
communication from hosts to external web sites involving blacklisted/TI sites
2. Detect unusual/DGA DNS traffic with resolving domains
3. Identify user(s) with privileged access to those hosts and/or roles (e.g. AD administrator)
4. Analytic 1 AND/OR 2 triggers on at least an entity AND Analytic 3 identified a misused privilege user/account
- ENTITY: Requests of DGA Domains
- ENTITY: Access to blacklisted/TI domains
- ENTITY: DNS/HTTP traffic volume
- ENTITY: DNS NXDOMAIN rate and Resolving traffic rate
- USER: at least 1 user with privileged rights accessing that resource (phished/stolen credential)
- …
- Web proxy data- User-IP mapping
data- DNS data
- List of Privileged/Admin Users
- List of Critical Resources/Servers
- Timestamp- Suspicious entity- Suspicious user- Context:
INITIAL_INFECTION
![Page 12: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/12.jpg)
12
Attack Story 4: Revenge by Disgruntled Employee
Stage Analytics Features Data Outcomes + Context
Reconnaissance and lateral movements 1. Detect abnormal sequence of
privileged & system commands on a system by local user/account (sudo, system file changes, etc.)
2. Detect changes of cron tables listing new, unrecognised programs. Detect command to install these programs.
3. Detect unusual traffic towards other networked systems with unusual success/failure rates
4. User belongs to a list of admin users
4. Analytic 1,2,3,4 triggers on at least a user and a device
- USER: use of privileged command activities
- USER: installation of new programs
- USER: modification of critical system files, such as crons
- ENTITY: number of netflow connections towards different systems
- …
- User commands- System commands- Netflow data
- List of Privileged/Admin Users
- Timestamp- Suspicious entity- Suspicious user- Context:
RECONNAISSANCE LATERAL MOVEMENTS
![Page 13: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/13.jpg)
13
Data Sets
![Page 14: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/14.jpg)
14
Data Sets for Analytics
Core Data– Netflow
– HTTP traffic or Web proxy Logs
– DNS traffic or DNS Logs
– AD Logs
System Data– Windows system logs from critical servers
– Linux audit and system logs
– Other server/app logs: DB, git, web server
User-Hostname-IP Mapping– DHCP
– VPN
– AD Logs
– Aruba Clearpass
Data Enrichment– GeoIP
– ASN
– Threat Intel
![Page 15: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/15.jpg)
15
Scale of Core Data Sets Volume and Size within HPE worldwide network
Data Type # Events/day(after filtering)
TB/day Avg Event Size
Netflow 34 Billion(3 collection points)
3.40 TB 100 B
DNS 150 Million(4 collection points)
0.15 TB 1 KB
HTTP 65 Million(central collection)
0.13 TB 2 KB
AD 153 Million
TOTAL ~ 35 Billion/day ~ 3.7 TB/day
![Page 16: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/16.jpg)
Analytics
![Page 17: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/17.jpg)
17
Combination of Analytics
Abnormal Behaviour Detection
1. Inconsistent/abnormal behaviour Comparing to OthersOutliers by comparing to assumed “normal” behaviour across others or in peer community
2. Historical Changes in User-Entity Behaviour PatternsTemporal changes in an individual entity network patternsAbnormal user activity and account usage
Empirical Rules and Patterns
1. Specific malware infectionsDGA domains, malicious web traffic
2. Command & Control communicationsBeaconing + threat intelligence
3. Data ExfiltrationHigh volumes of data sent via DNS or HTTP
Graph Analytics
1. Using graph features to profile entities and detect abnormal behaviour
2. Enabling graph based queries on the already collected data sets: e.g. network activity
![Page 18: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/18.jpg)
Anomaly Detection
![Page 19: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/19.jpg)
Entity Profiling
Domain-name Server (DNS)
Web-Proxy Server (HTTP)
Internal Traffic (Netflow)
Threat Intelligence
Package analysis
Anti-virus logs
…
Events Sources
Users
Host machines
Domain Names
IP addresses
Port Numbers
Sites
…
Entities Profiles
𝑡 0 𝑡1 𝑡 2
𝑡 0 𝑡1 𝑡 2
![Page 20: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/20.jpg)
Peer and Temporal Comparison
Entity type
Profiles
𝑡 0 𝑡1 𝑡 2
Peer comparison
analysis
Temporal analysis
Most anomalous entities returned as an outcome
![Page 21: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/21.jpg)
Pattern-Based Analytics
![Page 22: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/22.jpg)
Empirical Rules: Pattern-based Anomaly Detection
Initial Infection / Gain Access
Command & Control / Means to
Achieve Attack
Lateral Movement
Exfiltration / Damages
Analytics based on deep knowledge of security attack patterns and infiltration processes
Could be applied across all attack phases:
• Devices with DGA infections • Abnormal device communications to external sites • Detection of privilege escalation• Abnormal execution of privileged/admin commands• Abnormal creation/usage of admin accounts or AD domains at unusual times and locations• Abnormal number and types of accesses to a device from remote locations
• Beaconing traffic to suspicious external sites • New device communication and traffic patterns based on historical data and threat intelligence• Unusual number of failed connections from a device to external sites
• Port scanning detection• Abnormal volume of traffic or types of connections from a device towards critical servers (e.g. AD, …) or the way around • Unusually large number of clients• successfully connecting to other clients• Abnormal number of connection failures from devices to network services or specific service ports (e.g. SSH)
• Abnormal volume of traffic from a device towards unknown/suspicious external sites• Abnormal content in queries issued to a set of unknown domains• Abnormal external download of content from organisation’s external facing servers (e.g. web site)• Abnormal activities/patterns on specific servers (e.g. file encryption on file servers)• Abnormal traffic/uploading towards an external web site/Dropbox/etc.
User AccountCompromise
• Abnormal Login Failure/Success Rate• Abnormal set of privileged commands • Abnormal command sequences• Creation of privileged account coupled with one or more above anomalies• Abnormal time of logins and activities
![Page 23: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/23.jpg)
23
Graph Analytics
![Page 24: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/24.jpg)
24
Graphs for Security
Graph Visualisation– Assist security experts by flexibly visualizing linked data
(topology + features)
Graph Database– Allow to query the data more naturally when thought of as a
graph
Graph Analytics– Data representation and tools to support compute on the
entire data– Centrality– Graph Clustering– Similar pattern recognition
1
2
centrality
pattern matchingsub-graph search
![Page 25: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/25.jpg)
25
Pluggable Analytics
![Page 26: User and entity behavior analytics: building an effective solution](https://reader035.fdocuments.us/reader035/viewer/2022062503/588896d81a28ab3e658b7da5/html5/thumbnails/26.jpg)
Security Analytics Marketplace
Browse Analytics:- Threat Scenario- Use Case- Attack Stage- Analytics Type
End-User
DownloadAnalyticsModule(s)
AnalyticsModule(s)
Analytics Engine(s)
AnalyticsOrchestration
Visualization Configuration
Threat Findings
New Alert Types
Threat Links
Visual Widgets
AnalyticsResults
New LinkCorrelations
NewWidget
Analytics StoreLegal/Privacy
Audit
Software Deployment