Outsourcing Security Analysis with Anonymized Logs
Jianqing Zhang, Nikita Borisov, William Yurcik
2nd International Workshop on the Value of Security through CollaborationFriday, September 1, 2006
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
2
Motivation Managed Security Service Providers: Security
outsourcing is a trend Security monitoring is getting more complicated and
sophisticated Economical: assemble skilled security professionals Effective: shared security infrastructure across
organizational boundaries
Challenges Sensitive data is shared
Data protected by privacy laws Valuable information to competitors Useful information to adversaries
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
3
Managed Security Service Provider
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
4
Problem Statement
What are the criteria for log anonymization that sufficiently protect privacy and guarantee MSSP’s efficiency?
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
5
Contributions Case studies of common attack types based on classic logs
Derive a common set of anonymization criteria Retain time interval dependence between records Pseudonymize the external IP addresses re-identifiably Pseudonymize the internal IP addresses re-identifiably an
d preserve some network topology information
First step for privacy-preserving MSSPs
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
6
NetFlows and Syslogs NetFlows: network-based log
Timestamps IP address pairs (source/destination) Port pairs (source/destination) …
Syslog: host-based log Application level critical events
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
7
Which Data is Sensitive? Identity information
External (source) IP Partner, common guest and adversary
Internal (destination) IP Internal user
System privacy & security Timestamp
When the transactions happen Destination port number
Services and applications hosted on the system Subnet number
Internal network structure Records number
Overall resource usage
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
8
Log Anonymization Mechanisms Timestamp anonymization
Time unit annihilation Random time shifts Enumeration
IP address anonymization Truncation Random permutation Prefix-preserving pseudonymization
Port number anonymization Bilateral Classification Black Marker Anonymization Random permutation
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
9
Traffic Traces Logs: Port Scan
Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts18:56:23.916 130.241.53.23 902 128.146.38.15 4138 6 1
18:56:23.924 130.241.53.23 900 128.146.38.15 4139 6 1
18:56:23.936 130.241.53.23 893 128.146.38.15 4140 6 1
18:56:23.944 130.241.53.23 891 128.146.38.15 4141 6 1
…
Scan all ports of a single host: Source: same address, different port numbers Destination:
Same addresses Different ports (sequentially)
In a short time
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
10
Traffic Traces Logs: DoS/DDoS SYN Flood
Source: same addresses, same (or different) port numbers Destination:
Same addresses Same port (intended to a particular protocol or application)
Protocol / Packets/ Packet size In a short time
Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts B/Pk
21:47:11.670 165.132.86.201 514 128.146.97.7 80 6 1 40
21:47:11.854 165.132.86.201 514 128.146.97.7 80 6 1 40
21:47:12.198 165.132.86.201 514 128.146.97.7 80 6 1 40
21:47:12.338 165.132.86.201 514 128.146.97.7 80 6 1 40
…
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
11
Anonymization Constraints on Traffic Traces Logs Timestamp (Start Time)
Events interval and time dependence should be retained
Anonymization Time unit annihilation Random time shifts Enumeration
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
12
Anonymization Constraints on Traffic Traces Logs (cont.) Source/Destination IP address
Anonymized and re-identifiable Retain virtual network topology (dest.)
Anonymization Truncation Random permutation (pseudonyms)
Source (external) IP address Prefix-preserving pseudonymization
Destination (internal) IP address
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
13
Anonymization Constraints on Traffic Traces Logs (cont.) Source/Destination port number
Contain sensitive information More efficient if retained
Anonymization Bilateral Classification Black Marker Anonymization Random permutation
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
14
Active Operating System Fingerprinting
Syslog
Syslog + Tcplog
Time Stamp Host Name (IP) MessageSource Port Dest. Port
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
15
Anonymization Constraints on Syslog
Attributes List Anonymization Constraints Recommended Anonymization
Start TimeRetain events interval and time dependence
Random Time Shifts
Source IP Address Anonymized and Re-identifiable Pseudonyms
Source Port More efficient if retained Pseudonyms
Dest. IP Address Retain virtual network topology Re-identifiable if anonymized
Pseudonyms + Prefix-preserving
Dest. Port More efficient if retained Re-identifiable if anonymized Pseudonyms
Msg. Retained --
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
16
Sensitive Data After Anonymization Traffic volumes
Batched upload Aggregate volumes
Dummy log records Sacrifice the efficiency at MSSP False positives and false negatives
Size of customer base; customer retention Change the pseudonym mappings periodically
Structure of the internal network Simple pseudonyms Periodic rotation of pseudonyms
Policy dependent
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
17
Conclusion Sensitive data should be anonymized for
security monitoring
Constraints on log anonymization
Sensitive data leakage after anonymization and countermeasures
Privacy and efficiency is a trade-off
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
18
Future Work
Analyze other attacks Anonymization strategies for wide range of a
ttacks Patterns of attack detection and general prin
ciples Study other log formats and types Analyze correlation of different logs acro
ss different organizations
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
19
Q & A
Jianqing [email protected]
Nikita [email protected]
William [email protected]
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
20
Anonymization Constraints on Traffic Traces Logs
Attributes List
Anonymization Constraints Recommended Anonymization
Start TimeRetain events interval and time dependence
Random Time Shifts
Source IP Address Anonymized and Re-identifiable Pseudonyms
Source Port More efficient if retained Pseudonyms
Dest. IP Address Retain virtual network topology Anonymized and Re-identifiable
Pseudonyms + Prefix-preserving
Dest. Port More efficient if retained --
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
21
Port Scan (cont.)
Portmap scan: Source: same address, different port numbers Destination: various addresses, same port (portmap
daemon) In a short time
Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts10:53:42.54 165.132.86.201 9781 128.146.0.16 111 6 1
10:53:42.54 165.132.86.201 9788 128.146.0.71 111 6 1
10:53:42.54 165.132.86.201 9791 128.146.0.11 111 6 1
10:53:42.54 165.132.86.201 9381 128.146.0.51 111 6 1
…
Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs
22
DoS/DDoS (cont.) Distributed SYN Flood
Source: different addresses, different port numbers Destination:
Same addresses Same ports (intended for a particular protocol)
Protocol / Packets/ Packet size In a short time
Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts B/Pk
19:08:40.492 192.1.6.69 77 194.20.2.2 1308 6 1 40
19:08:40.532 192.1.6.222 1243 194.20.2.2 1774 6 1 40
19:08:40.720 192.1.6.108 114 194.20.2.2 1869 6 1 40
19:08:40.764 192.1.6.159 804 194.20.2.2 1050 6 1 40
…
Top Related