Tutorial Overviewguerin/presentations/T1...BGP Hijacking DNS Poisoning Compromised Infrastrucure...
Transcript of Tutorial Overviewguerin/presentations/T1...BGP Hijacking DNS Poisoning Compromised Infrastrucure...
1
Backbone Attack Detection and Mitigation Methodologies
SECTION IOverview/Background
Danny McPherson <[email protected]>Craig Labovitz <[email protected]>
Farnam Jahanian <[email protected]>
2Sigcomm 2005
Tutorial Overview
Focus on bridging the gap between academic research and current practices and security challenges in tier1/2 backbones:§ Section I – Overview§ What do we mean by tier1/tier2 backbone security?§ Overview current design of tier1/tier2 backbone infrastructure
and security layers§ Section II -- Backbone Security (6 Phases) § Network instrumentation and preparation§ Attack detection, identification and classification§ Attack mitigation mechanisms and methodologies
§ Section III -- Ongoing and open research areas
2
3Sigcomm 2005
Tier1/2 Backbone Security
§ Focus on five primary backbone vulnerabilities1) DDoS/Worms2) DNS3) BGP4) Infrastructure compromise5) Change control/Upgrades
§ Will not focus on LAN or host security (e.g. buffer overflows, phishing, database security, sniffing)
§ We make no endorsement, recommendation of commercial router or security products discussed.
4Sigcomm 2005
What are Providers Concerned About?
0 5 10 15 20 25 30
DDoS
Worms
BGP Hijacking
DNS Poisoning
Compromised Infrastrucure
Other
Number of Surveyed ISPs
§ Recent UM/Arbor survey of 40 tier1/tier2 providers§ DDoS remains most significant issue
* Other included phishing schemes, and unspecified.
3
5Sigcomm 2005
And Threats Evolving
6Sigcomm 2005
Escalation of Threats..
§ For example:§ Code Red: DDoS against one IP
§ Changed IP/Null routed previous IP§ Blaster: DDoS against hostname
§ Repeated DNS Shifts§ Eventual NXDOMAINing of windowsupdate.com record
§ Deloder: Arbitrary DDoS toolkit§ Hrmm…?
§ Backdoors escalated from remote control (e.g., BO, NetBus) to harvesters and far more complicated.§ NetBus was originally written in March of 1998 and only had
Swedish UI. In November of 1998 it was translated to English and it’s use continues to grow even today!
§ Control channels include IRC commonly and other, encrypted mechanisms more and more.
4
7Sigcomm 2005
Emerging Trends in Security Threats
Globally scoped, respecting no geographic or topological boundaries. At peak, 5 Billion infection attempts per day during Nimda including significant numbers of sources from Korea, China, Germany, Taiwan, and the US.[Arbor Networks, Sep. 2001]
Exceptionally virulent, propagating to the entire vulnerable population in the Internet in a matter of minutes. During Slammer, 75K hosts infected in 30 min. [Moore et al, NANOG February, 2003]
Zero-day threats, exploiting vulnerabilities for which no signature or patch has been developed. In Witty, "victims were compromised via their firewall software the day after a vulnerability in that software was publicized”
Profound transformation underway: from attacks designed to disrupt to attack that take control. Over 900,000 infected bots as phishing attacks are growing at 28% per month [Anti-Phishing Working Group 2005]
8Sigcomm 2005
Largest Cause of Outages
§ Not hackers, backhoes nor DDoS -- change control and upgrades remain largest source of user visible outages
§ Many internal studies with same findings…
1999 FTCS Study of backbone outages in regional network
5
9Sigcomm 2005
DNS Security Issues
§ Not cryptographic§ Small ID space vulnerable to guessing§ Poor PRNG random number implementation in
some Bind versions§ Birthday paradox§ Man-in-middle attacks
§ Software bugs/issues§ DNSSEC not widely deployed (though large
deployment effort under way)
10Sigcomm 2005
DNS Attacks
§ Cache Poisoning§ Response Spoofing § ID Guessing§ DOS attacks§ Domain registration
§ All actively happening…
6
11Sigcomm 2005
DNS Cache Poisoning
§ Force DNS server to recursively query attack server
§ Attack server provides additional “poisoned” information
Graphics from http://www.securesphere.net/download/papers/dnsspoof.htm
12Sigcomm 2005
DNS ID Guessing Attacks
§ Flood nameserver with queries for domain§ Flood nameserver with forged responses for domain§ If forged ID of response matches, forged answers
wins
Attacker
DNS Server
www.acm.org A?
A www.acm.org a.b.c.d ID 1A www.acm.org a.b.c.d ID 2
…A www.acm.org a.b.c.d ID N
www.acm.org A?
(recursive query)
7
13Sigcomm 2005
BGP Security Issues
§ Verification of ASN ownership§ Verification of address space § Route and address advertisement
authorization§ Route withdrawal authorization§ Integrity and authenticity of all BGP traffic
on the wire§ Timeliness of BGP traffic
14Sigcomm 2005
Impact of BGP Security
§ Ability to alter forwarding table enables:§ Denial of service through dropping traffic to
host/network§ Misdirecting traffic for man-in-the-middle inspection
(e.g. credit cards)§ Misdirecting traffic to third-party end systems
§ No universally deployed database or filtering of routing announcements
8
15Sigcomm 2005
BGP Address Hijacking
199.222.0.0/16Sprint
MCI
ACM
Merit
Chicago IXP Small Peer
§ Though providers filter customer BGP announcements, few filter peers§ Memory, line-card limitations§ Maintenance problem
§ More specific announcements wins§ Injection attack requires
compromised commercial or PC-based router § man-in-middle session attacks rare
199.222.229.0/24
16Sigcomm 2005
BGP Security Commercial Overview
§ Packet Design: RouteExplorerTM monitors and visualize IGP (OSPF/ISIS) and BGP updates, BGP root cause analysis
§ IPSum: RouteDynamicsTM Identify BGP topology changes§ Arbor: PeakflowSPTM monitors and alerts BGP
local and remote changes, correlates with traffic§ Renesys : GradusTM remote BGP monitoring
service
9
17Sigcomm 2005
BGP Commercial Alarming
§ Commercial remote BGP monitoring services (RenesysTM)
18Sigcomm 2005
Current Directions
§ soBGP: protect origin & SBGP: protect path§ Very complicated§ Likely requires hardware upgrades§ How much of this can be accomplished simple with
IRRs and the right configuration management tool sets?
§ BTSH/GTSH: protect peering sessions§ We’ll talk about this some more in later sections
10
19Sigcomm 2005
Infrastructure Security
§ Most of the major US carriers have had compromised (“owned”) backbone routers§ Hundreds of distributed PC management boxes§ Asset management§ Employee churn leads to out of date passwords§ Automation tools use fixed password, even if
token-based authentication is used for user login§ Large European provider had internal tiger
team successfully phish security/authentication information from NOC
20Sigcomm 2005
Overview of DDoS Today
§ Moderate attacks impacting individual customers/links a daily occurrence in most backbones
§ Major attacks impacting backbone infrastructure less frequent (monthly)
§ Most attacks still brute -force flooding (TCP SYN or UDP)
§ Some belief in trend towards bot armies (discussed later)
§ Transition from hobbiest to commercial activity (extortion)
§ Largest issue facing ISPs and focus of this tutorial
11
21Sigcomm 2005
Overview of DDoS Today§Moderate attacks common
§Survey of 40 tier1/tier2 ISPs reports average of 40 attacks per month
Average Number Customer Impacting Attacks per Month
0%10%20%30%40%50%60%70%80%
100-500 10-100 Less than 10 None
22Sigcomm 2005
“In a shift from previous years, both virus attacks and denial of service outpaced the former top cost, theft of proprietary information… it may be due to last year’s rise in the degree to which virus threats were entwined with denial of service attacks.” -CSI/FBI 2004 Computer Crime & Security Survey
High Cost of DoS & Worms
12
23Sigcomm 2005
0%
20%
40%
60%
80%
100%
Per
cent S
urv
ery
Res
ponse
s
10+ Gbps
1-10 Gbps
500Mbps - 1Gbps
< 500 Mbps
Overview DDoS Today: Size of Attacks
§Survey of 40 tier1/tier2 providers§Attack size over last 6 months
24Sigcomm 2005
DDoS Overview: What is Under Attack?UMich/Arbor Survey of 40 tier1/tier2 ISPs
§ Most frequently attacked sites include:§ IRC servers§ Gambling, especially offshore§ Porn sites
§ Additional survey reports included:§ Residential users§ Web hosting§ The Chinese§ RIAA related sites
13
25Sigcomm 2005
DDoS Commercial Overview
§ Variety of commercial approaches§ Widely distribute content (e.g. Akamai). Used by
most fortunate 500.§ Funnel web traffic through scrubbing farm and back
to secret IP address (Prolexic)§ In-cloud provider-based detection based on flow and
packet export (Arbor, Adlex)§ In-cloud provider based mitigation via ALCs, BGP,
and intelligent filtering/scrubbing (Arbor, Cisco, Cloudshield)
§ CPE/Enterprise based filtering and traffic shaping (TopLayer, Mazu, Packeteer)
26Sigcomm 2005
DDoS Overview: Bots
§ A bot is a servant process on a compromised system§ Usually installed by a Trojan, though worms have
evolved to install bots as well (e.g., deloder)§ Communicates with a handler or controller, typically
via IRC, often running on public IRC servers or other compromised systems
§ Almost always unbeknownst to the systems owner -‘got bot?’
§ A botmaster or botherder commands bots to perform any of an number of different functions
§ System of bots and controller(s) is referred to as a botnet or zombie network
14
27Sigcomm 2005
What’s a botnet used for?
§ Bots are used for one or more of the following:§ Install key loggers and capture passwords, account information, etc..§ Gain access to local LANs or internal systems§ Phishing§ Spam relay§ Some bots harvest email addresses for spammers§ ID Theft§ Open proxies§ DOS Attacks§ Distributed cracking systems (e.g., Brute Force SSH activity)§ New Rbot capabilities include using webcams to capture video and still
images(!)§ An entire economy is evolving around bot ownership§ Sell and trade of bots ($0.10 for “generic bot”, $40 USD or more for an
“interesting bot; e.g., a .mil bot)§ Bots are a commodity - no significant resource constraints
28Sigcomm 2005
InternetBackbone
The Botnet
B
UK Broadband
US Corp US Broadband
B
JP CorpProviderB B
ThePeacefulVillage
15
29Sigcomm 2005
InternetBackbone
The Botnet
B
UK Broadband
US Corp US Broadband
B
JP CorpProviderB B
SystemsBecomeInfected
30Sigcomm 2005
InternetBackbone
The Botnet
UK Broadband
US Corp US Broadband
JP CorpProviderB
B
B
B
Bots forman overlay= botnet
P
16
31Sigcomm 2005
InternetBackbone
The Botnet
UK Broadband
US Corp US Broadband
JP CorpProviderB
B
B
B
ControllerConnects
C
P
32Sigcomm 2005
InternetBackbone
The Botnet
UK Broadband
US Corp US Broadband
JP CorpProvider
C
PB
B
B
B
AttackCommand
17
33Sigcomm 2005
Botnets
§ Many of the compromised hosts reside on internal networks or belong to customers - it’s their responsibility…
§ And if that’s not enough?§ The sheer size of these attacks and available firepower not only
thoroughly neutralize the target, they also yield a considerableamount of collateral damage on the infrastructure - I.e., their network!
§ Consider the fact that an OC-3 (155 Mbps) connected enterprise connection could be effectively rendered useless by a botnet comprised of only 200 home PCs, each with an average connection bandwidth of only 1 Mbps
§ Now, consider frequency of recruitment into botnets and couple that with proliferation of residential broadband access capacities
§ …and couple that with the evolving convergence architectures in IP networks today (e.g., VoIP, Video, Internet, VPN) and overlay ever more stringent services availability requirements
34Sigcomm 2005
How big is the problem?
§ As many as 172,000 new bots recruited every day according to a recent report by CipherTrust! (that’s >5M/mo.)
§ Symantec’s latest Internet Security Threat Report reports that bot observations currently average 30,000 a day
§ A single botnet comprised of more than 140,000 hosts was observed “in the wild” over 3 years ago
§ Botnet driven attacks have been responsible for single DDOS attack flows of more than 10Gbps aggregate capacity
§ A study conducted by the University of Michigan showed that an out of the box Windows 2000 PC was recruited into 3 discrete botnets upon being connected to the Internet for just 48 hours -Numerous studies reinforce similar infection rates/frequencies
18
35Sigcomm 2005
Backbone Design and Security Overview
§ Most tier1/tier2 backbones now have similar structure§ Peering edge§ Core (MPLS)§ Security/Aggregation§ Numerous IP POPs§ DIA, Layer 2, Layer 3 and IPSEC VPN
§ Backbones implement security/policies implemented in layers§ Policy and different administrative control§ Different hardware/software capabilities at each layer§ Largely reliant upon capital “hand-me-downs”
§ Significant security deployment issues associated with limitations of each layer
36Sigcomm 2005
Overview Backbone Design
Core BackboneRouters
POPInterconnect
Medium
NeighboringPOP
Dedicated Access PSTN/ISDN
Core 1 Core 2
SW 1 SW 2
Access 1 Access 2 NAS 1 NAS 2
External BGP Peering
NeighboringPOP
19
37Sigcomm 2005
CPE/Enterprise Firewall and VPN
3Com Security Switch 6200
Juniper Networks NetScreen
5400 & M7i
Cisco ISR
38Sigcomm 2005
CPE/Enterprise Security Devices
§ Most enterprises have firewall, IPS and IDS deployed at edge § New trend for IPS deployed for internal security and network
segmentation§ Trends towards single integrated device§ Evolution IDS -> IPS -> NBAD
§ Pros and Cons⌧Until recently IDS mostly PC based⌧ Low port density and low speed backplane support packet inspection
at less than 1Gbps⌧Point solutions (difficult to correlate and coordinate)þ Expressive filtering language and reporting within limits…þ Cheap
20
39Sigcomm 2005
Challenges of IDS/IPS
§ Flood of low quality information§ Too many alerts§ Too many false positives§ No “context” - only looks for signature of event§ Not adaptive to new threats (signature-based is fine IF you have
signatures) § Just reporting
§ As a result§ IDS mainly serve forensic purpose (not used in real time)§ Emergence of Security Event Management market (SEM)§ netForensic, ArcSight, GuardedNet
§ Emergence IPS (think IDS with a gun)
40Sigcomm 2005
Aggregation/Distribution Layer
Cisco Catalyst 6500 Series
Cisco GSRCisco GSR
Juniper Mi7
21
41Sigcomm 2005
Aggregation Distribution Devices
§ Trend towards providers offering on-net security services (AT&T, MCI, BT)
§ High port density
§ Limited packet inspection capabilities - high performance packet forwarding processors with limited reporting and analysis capabilities (*Flow, port span)
§ Some support blades with IDS (e.g. Cisco CAT, Juniper AS PIC)
§ Services require using revenue generating slots
42Sigcomm 2005
Core Routers
Juniper T seriesAvici
Cisco CSR
22
43Sigcomm 2005
Core Routers
§ Limited security options§ Limited data/statistics export§ Driven by traditional focus on availability and packet
forwarding performance - bolt on security§ With MPLS core, you get very little (limited ACL
support, limited analysis and reporting capabilities)§ Limited feature sets causes issues when they’re
moved towards the aggregation layers of the network§ Interconnection w/point-to-point links expensive model
but introduction of Layer 2 switches adds routing and management overhead
44Sigcomm 2005
Overview of Router/Switch Architectures
§ Initially, routers we’re traditional type 1 switches; all control, data and management plane functions were performed by single central processor
§ Forwarding functions were then offloaded to secondary CPU on primary “control card”
§ Hardware was expressly designed for packet forwarding functions and offloaded to dedicated blades (e.g., Cisco 7k SSE)
§ More distributed architectures began to emerge with dedicated forwarding hardware on each line card, some allowing reuse of existing chassis (e.g., VIPs with PAs on Cisco 75XX platforms)
§ Control and management functions still performed by central processor (e.g., Juniper RE, Cisco GSR GRP)
23
45Sigcomm 2005
Router Architecture: Cisco GSR
§ Central control processor§ Proprietary, real-time OS (IOS) running on RISC CPUs§ All processes share same space (no memory protection
between processes)
§ Variety of “engines” on line cards§ Micro-code running on the LCs next to the “main” IOS§ 0, 1, 2 core cards (limited filtering, NetFlow -- optimized for
forwarding, not for edge features)§ Slow Path” via CPU for line card types 2, 3, 4 and 4+§ ACLs, IP options, ICMP, multicast
§ Features vary, even in chassis, configuration of feature on an interface may require input line cards to support (e.g.,. Outbound ACL)
46Sigcomm 2005
Router Architecture: Juniper M40
Source: Juniper Networks
§ Central processor running BSD-ish OS
§ ASIC forwarding§ Limited bandwidth between
cards and CPU (constrains Cflowd export)
24
47Sigcomm 2005
Some Final Router Background Thoughts…
§ Adding a “simple” algorithm to router ASIC is not simple§ 2-3 years for ASIC development§ 4-7 years for deployment§ Business case, business case, business case
§ Tier12/ networks are BIG§ 600 PoPs, dozens of core routers, thousands of edge routers
§ OC192 cores, OC48 access
§ In choice between simple/incomplete and complex solutions, simple wins. Simple is hard.
Backbone Attack Detection and Mitigation Methodologies
SECTION IIa6 Phases of Infrastructure Security(Preparing the Network -- Best Common Practices)
Danny McPherson <[email protected]>Craig Labovitz <[email protected]>
Farnam Jahanian <[email protected]>
25
49Sigcomm 2005 49
Six Phases of Infrastructure Security
Preparation
§Prep the network§Create tools§Test tools§Prep procedures§Train team§Practice
Identification
§How do you know about the attack?§What tools can you use?§What’s your process for communication?
Classification
§What kind of attack is it?Traceback
§Where is the attack coming from?§Where and how is it affecting the network?
Reaction
§What options do you have to remedy?§Which option is the best under the circumstances?
Post Mortem
§What was done?§Can anything be done to prevent it?§How can it be less painful in the future?
Preparation
50Sigcomm 2005
Best Common Security Practices
1) Management Plane§ Data/statistics collection (SNMP, Flow Data, Syslog, etc.)§ Out of Band Access § Remote Access
2) Control Plane§ Routing and Signaling protocols (IGPs, BGP, MPLS/LDP)
3) Data Plane§ Packet forwarding (IP, MPLS, IPv6)§ Filtering, rate-limiting, QoS & discard functions§ Accounting functions for management plane
Preparation
26
51Sigcomm 2005
Key Challenges In Preparing Network
§ Hardware/software equipment limitations and differences§ Data collection (*flow, packet traces)§ Enforcement (limited ACL/rate-limiting) and minimize collateral
damage§ Vendors focus on “speed and feeds”. Management and security
typically a secondary priority.
§ Management/Operationalize§ 10,000s of lines of configuration per device§ Each device slightly different § Each version of software slightly different
§ Commercialize service (economics, productize)§ Can you make money with DDoS, secure VPN service?
Preparation
52Sigcomm 2005
BCP Management Plane
§ Out of band access§ Terminal server and remote power§ Migration from dialup -> frame relay -> MPLS VPN 2547 BIS§ Migration from telnet -> ssh (tacacs/radius)
§ Radius password often passed back in clear text
§ Configuration/OSS management § No standard commercial tools§ Wide variation across providers
§ Data collection/monitoring§ SNMP polled by MRTG and commercial tools (HPOpenView/NetCool)§ SNMP traps to commercial tools§ NO SNMP sets§ Flow data export to commercial collectors§ Syslog to central collector§ Significant hardware limitations
Preparation
27
Sigcomm 2005
BCP Management Plane
Out of Band Access
§ Separate Networks may be required to dump the data. §Accounting§Network Metrics
§ Interconnection for management, security, and accounting services§Netflow Devices—FlowCollector§Syslog collector for all network devices§SNMP collector (PC Based UNIX)§Security Auditing Tools (NetSonar)
POS
POS and ATM for Core Backbone
GSRGSR
75077507
Customer and Services
Managementand
Accounting
Preparation
54Sigcomm 2005
BCP Management Plane
Configuration
§ How to audit/monitor configuration changes§ State of art fairly static§ Rancid/RCS/tool§ More advanced IRR/templates§ No commercial solutions that work§ Provisioning does have OSS integration
§ Change control source of most failures§ Dynamic state forwarding table changes
(OSPF, BGP)
Preparation
28
55Sigcomm 2005
BCP Management Plane
Router Config Access
§ Most providers use TACACS+/RADIUS§ Unfortunately, RADIUS & TACACS still cleartext!§ OTP and token-based passwords required
§ Migration from telnet to sshd for router access§ Router CPUs give low priority to user functions (e.g., screen scraping
and config downloads) –§ Takes large amount of time to download/upload element
configurations
§ Configuration challenges§ Sanity Checking extremely difficult§ Configuration bits change with different software revisions§ Routing/forwarding state not part of configuration§ Secure access for automated tools
Preparation
56Sigcomm 2005
BCP Management Plane
Data Collection
§ DPI/RMON only appropriate edge§ Cisco and Juniper offer specialized cards/software, but
limitations…§ NBAR – dedicated hardware, poll model§ Juniper AS-PIC for basic DOS detection and additional flow-
export capabilities. Requires OC48 port.§ Specialized cards burning revenue generating slots.
§ SNMP v1-v3§ Not used for configuration (no ‘set’/RW enabled)§ SNMP does not provide insight into Layer 3 - 7 packet
attributes (src/dst, protocols or ports)
§ NetFlow/sFlow/JFlow/Nflow/IPFIX
Preparation
29
57Sigcomm 2005
BCP Management Plane
Flow Collection
§ Flow is router/switch:§ ASIC based packet header export mechanism§ Exported to one or more data collection servers via UDP§ Usually sampled (1/1000 in most GSR/M-Series routers)§ Usually V5 or IPFIX (V9/V10)
§ Common misconceptions about Flowý NetFlow part of fast switching path. § No longer true (modulo CAT6k) -- purely statistics mechanism today.
ý Flows are a useful aggregation mechanism§ Flows do not exist! Notion of flow is anachronism in most carrier networks.§ At OC48/192, all flows reduced to single packet (i.e., sampling MUST be
employed).ý Flow not deployable due to high router CPU usage§ Modulo Cisco engine 0/2 cards, implemented in ASIC
Preparation
58Sigcomm 2005
BCP Management Plane
Flow Export
Core Network
Enable NetFlow
Traffic
CollectorNFC, cflowd, flow-tools, Arbor
UDP NetFlowExport Packets
Application GUIArbor, FlowScan
PE
Export Packets• Approximately 1500 bytes• Typically contain 20-50 flow
records• Sent more frequently if
traffic increases on NetFlow-enabled interfaces
Preparation
30
59Sigcomm 2005
BCP Management Plane
Flow Planning Deployment Issues
§ Impact of sampling on flow accuracy § Most deployments use 1/100 -1/1000 sampling
§ Backhaul versus local collection § Bandwidth usually 1% of offered traffic
§ What is being measured? § DDoS or customer accounting? (impacts sampling rate)
§ Vendor sampling algorithms/knobs vary
§ Wide range of capabilities across router, engine cards and IOS/JUNOS/*OS version§ Generally available in ASIC on most modern Cisco cards § Some support on Juniper RE w/additional capabilities from AS PIC, support by other
vendors vary as well
Preparation
60Sigcomm 2005
BCP Management Plane
Flow Accuracy versus SNMP Octets
Preparation
§ Flow does not include link layer header (only IP)
§ Ground truth sometimes difficult -- significant implementation issues and inaccuracies with SNMP
31
61Sigcomm 2005
BCP Management Plane
Flow Accuracy at Low Traffic RatesMeasuring CBR 15Kbps on OC12 Link via tcpdump and 1/100 sampled netflow
Preparation
Source: 2004 Arbor Networks Technical Report
62Sigcomm 2005
BCP Control Plane
§ Infrastructure ACLs – protect routers§ Receive ACLs – protect route processor§ Routing security§ Filter customer BGP announcements§ Filter peer BGP announcements (rare)§ BGP max-prefix limit to constrain size of routing table§ BGP (BTSH/GSTH)§ MD5/Payload encryption (BGP over IPSEC)
Preparation
32
63Sigcomm 2005
BCP Control Plane
Receive ACLs
Preparation
64Sigcomm 2005
BCP Control Plane
Infrastructure ACLs
§ Filter traffic destined to core routers using§ Edge ACLs§ Unrouted (e.g. RFC 1918) address space§ MPLS
§ Infrastructure ACLs also provide anti-spoof filtering§ Deny RFC1918 space§ Deny multicast sources addresses (224/4)§ RFC3330 defines special use IPv4 addressing
§ Permit small list of required protocols that are sourced outsidenetwork and require access core routers§ eBGP peering, GRE, IPSec, etc (most don’t access peer routers)
Preparation
33
65Sigcomm 2005
BCP Control Plane
Infrastructure ACLs Issues
§ Breaks transparency: access from outside through pings, traceroute INTO the core does not work, PMTU, etc..§ As a consequence, makes troubleshooting difficult: from the outs ide,
and from the core§ Hard to deploy if core address space if not contiguous
§ Hardware does not support line speed ACLs on many platforms§ Not all vendor ACLs support fragments§ Fragments pose a security risk: by default they are not filtered by
ACLs
§ Difficult to maintain (e.g., infrastructure address space changes)
Preparation
66Sigcomm 2005
PR1 PR2
R1
CR1R4
R2R3
R5
SRC: 127.0.0.1DST: any
SRC: validDST: Rx (any R)
SRC: eBGP peerDST: CR1 eBGP
SRC: validDST: external to AS (e.g. customer)
CR2ACL “in” ACL “in”
ACL “in”ACL “in”
BCP Control Plane
Infrastructure ACLs in Action
Preparation
34
67Sigcomm 2005
BCP Control Plane
Infrastructure ACL Example (Cisco)
§! Deny our internal space as a source of external packets§access-list 101 deny ip our_CIDR_block any
§! Deny src addresses of 0.0.0.0 and 127/8§access-list 101 deny ip host 0.0.0.0 any§access-list 101 deny ip 127.0.0.0 0.255.255.255 any
§! Deny RFC1918 space from entering AS§access-list 101 deny ip 10.0.0.0 0.255.255.255 any§access-list 101 deny ip 172.16.0.0 0.0.15.255 any§access-list 101 deny ip 192.168.0.0 0.0.255.255 any
Preparation
68Sigcomm 2005
BCP Control Plane
Migration to MPLS
Increasingly core employs MPLS and is not visible to external IP world
Preparation
35
69Sigcomm 2005
BCP Control Plane
BTSH/GSTH
§ Protect TCP sessions using properties of TTL§ Formerly known as BGP TTL Security Hack/ Applied to other
functions, renamed Generalized TTL Security Hack (e.g., LDP)
§ Accept packets for this flow only with TTL of n or greater§ Can’t use lower value, else TTL itself could be spoofed§ Gaining some deployment traction, current implementation
status varies§ GTSH coupled with iACLs raises bar significantly§ Not available all vendors (not Juniper)
Preparation
70Sigcomm 2005
BCP Data Plane
§ Prevent spoofing and limit customers to their address space§ Ingress ACLs§ uRPF § But limited deployment due to conservative ISP engineering practices
and lack of operational confidence and inability to effectively manage configuration changes
§ Rate limit ICMP and other non-production traffic
§ Filter bogons and unused/RFC-1918 address space
§ Restrict multicast address space § Traffic to multicast address creates state, places load on router as traffic
forwarded to RP, etc.
§ Offramp capabilities
Preparation
36
71Sigcomm 2005
BCP Data Plane
Ingress Packet Filtering
§ Customers should not send any IP packets to the Internet with a source address space outside the address spaces allocated to them§ Filter as close to the edge as possible§ Filter as precisely as possible§ Filter both source and destination where possible
§ BCP 38/ RFC 2827 “Network Ingress Filtering: Defeating Denial of Service Attacks which Employ IP Source Address Spoofing”
Preparation
72Sigcomm 2005
BCP Data Plane
Ingress Packet Filtering
Internet
ISP’s Customer Allocation Block: 96.0.0.0/19BCP 38 Filter = Allow only source addresses from the customer’s 96.0.X.X/24
96.0.20.0/24
96.0.21.0/24
96.0.19.0/24
96.0.18.0/24
BCP 38 Filter Applied on Downstream
Aggregation and NAS Routers
ISP
Preparation
37
73Sigcomm 2005
BCP Data Plane
Source Address Validation
MIT Spoofer project
Preparation
74Sigcomm 2005
BCP Data Plane
Unicast RPF
§ Native (Cisco/Juniper) router capability to drop a packet if it contains a source address that’s not reachable via the interface through which it was received.§ Automates some configuration of BCP 38 functions § Deployed ubiquitously in some networks now
§ Limitations§ Strict-mode may break with asymmetry§ Loose-mode may allow spoofed packets to be forwarded§ Mostly applicable at edge of network
Preparation
38
75Sigcomm 2005
i/f 1
i/f 2
i/f 3
BCP Data Plane
Strict uRPF Check
i/f 1
i/f 2
i/f 3
FIB:. . . S -> i/f 1. . .
S D dataS D data
FIB:. . . S -> i/f 2. . .
S D dataS D data
Same i/f:Forward
Other i/f:Drop
router(config-if)# ip verify unicast reverse -pathor: ip verify unicast source reachable-via rx
Preparation
76Sigcomm 2005
i/f 1i/f 2
i/f 3i/f 1
i/f 2
i/f 3
FIB:. . . S -> i/f x. . .
S D dataS D data
FIB:. . . . . .. . .
S D dataS D data
Any i/f:Forward
Not in FIBor route à null0:
Drop
?
BCP Data Plane
Loose uRPF Check
router(config-if)# ip verify unicast source reachable-via any
Preparation
39
77Sigcomm 2005
BCP Data Plane
IP Address Spaces
Address delegation hierarchy:1. IANA assigns address blocks to Internet Routing
Registries (IRRS)2. IRRs (RIPE, ARIN, APNIC, LACNIC, AfriNIC)
assign address space to LIRs and Service Providers
3. SPs and LIRs assign address blocks to end users
Preparation
78Sigcomm 2005
BCP Data Plane
IP Address Types
§ Bogon (RFC 1918 and unallocated)§ Includes reserved blocks such as RFC 1918, as well as blocks not yet
assigned by IANA or being allocated to SPs/end users§ Only two types of traffic are destined to/from this address spac e:
§ Malicious (scans, backscatter, worms, etc.)§ Misconfigured (DNS, typo, etc.)
§ Assigned & Active
§ Assigned and “Dark”§ Assigned to a network operator but has not yet been sub-allocated to
customers or end systems§ More difficult for attackers to intentionally avoid this address space when
performing malicious activities
Preparation
40
79Sigcomm 2005
BCP Data Plane
Implications of Different Address Spaces
§ Traditionally, many DDOS attacks employed source address spoofing, often statistically evenly distributed across IPv4 unicast address spaces.
§ As a result, monitoring activity with packets from bogon sourcesprovided quite a reliable mechanism for inferring attack activity on the Internet
§ Techniques such as Backscatter Traceback (next section) are largely reliant upon attack source address spoofing
§ However, with wide-scale deployment of uRPF and BCP 38, source address spoofing is clearly falling out of vogue with attackers
§ Monitoring traffic destined to bogon address spaces often indicates random scanning, worm propagation attempts and misconfiguration
Preparation
Backbone Attack Detection and Mitigation Methodologies
SECTION IIb6 Phases of Infrastructure SecurityAttack Identification and Classification
Danny McPherson <[email protected]>Craig Labovitz <[email protected]>
Farnam Jahanian <[email protected]>
41
81Sigcomm 2005 81
Six Phases of Infrastructure Security
Preparation
§Prep the network§Create tools§Test tools§Prep procedures§Train team§Practice
Identification
§How do you know about the attack?§What tools can you use?§What’s your process for communication?
Classification
§What kind of attack is it?Traceback
§Where is the attack coming from?§Where and how is it affecting the network?
Reaction
§What options do you have to remedy?§Which option is the best under the circumstances?
Post Mortem
§What was done?§Can anything be done to prevent it?§How can it be less painful in the future?
Identification
82Sigcomm 2005 82
Attack Identification
§ Most attacks are NOT difficult to detect at end-system(s)§ 90% of DDoS are brute-force flooding
§ Challenges§ Detect DDoS upstream (e.g. within provider and aggregate OC48
traffic)§ Detect more complex, non-flooding application level attacks§ Actionable event correlation § Thousands of IDS alerts, MRTG graphs§ 100k flows per second
§ Traceback§ Mitigation
Identification
42
83Sigcomm 2005
Attack Identification
§ UMich Arbor survey of 40 tier1/2 ISPs§ Percentage respondents who indicated TCP Syn/UDP flooding comprised majority
of customer attacks
Detected Attack Types
0
20
40
60
80
100
TCP Syn or UDP Flooding Other/UnknownPer
cent
age
of S
urve
yed
Res
pond
ants
Identification
84Sigcomm 2005
Basic Detection Strategy
§ Build baselines to determine normal behaviour § Employ tools that enable network-wide correlation of
control and data plane characteristics§ CPU utilization
§ NetFlow§ SNMP data collection§ Route path attributes, stability and effects on traffic shifts
Identification
43
85Sigcomm 2005
Identification/Detection Techniques
§ How are attacked detected?1) Customer calls - “The Internet’s down!”2) End-system logs/traps3) Static thresholds4) Sinkhole5) BackScatter6) Commercial Tools
§ Traceback/Classification
Identification
86Sigcomm 2005
How are Attacks Detected?
85%
10% 5%Commercial Tools
Internal Tools
Customer PhoneCalls
§ UMich/Arbor survey of 40 tier1/tier2 ISPs§ ISPs in wholesale/transit mostly rely on NOC trouble tickets
Identification
44
87Sigcomm 2005 87
Slammer Data Plane Impact - A European SPs View
§ Some DDOS/worms easier to detect than others…
Identification
88Sigcomm 2005 88
Slammer Control Plane Impact – THE BGP PICTURE
Identification
45
89Sigcomm 2005
Detection Challenges
§ Hidden failures § E.g. load balancing, anycasting and automatic fail-over can
hide critical failures or regionalized attacks (recent DNS outages)
§ Event correlation§ Growing security event management (SEM) market§ Next slide: Was this a DDoS?§ Turns out that configuration/policy change lead to high CPU, § which lead to BGP dropping, § which lead to path change and new traffic on router
Identification
90Sigcomm 2005
Identification/Correlation
BGP Flaps
Packet Size
CPU
Identification
46
91Sigcomm 2005
Commercial Event CorrelationIdentification
92Sigcomm 2005
Detection: Thresholds
§ SNMP Data abortion can signal a network problem or a security incident.
CPU spike when nothing else is happening on the network and with
no one working on the router
Identification
47
93Sigcomm 2005 93
Detection: Sinkholes
§ Topological security feature -- somewhat analogous to a honeypot or honeynet§ Router(s) or workstation(s) built to accept traffic and assist in
analyzing attacks, collecting data, offloading router processing, etc..
§ Used to redirect attacks away from the customer – working the attack on a router built to withstand the attack
§ Used to monitor attack noise, scans, data from mis-configuration and other activity (via the advertisement of default or unused IP space)
§ Traffic is typically diverted via BGP route advertisements and/or routing policies
Identification
94Sigcomm 2005
Sinkhole Routers/Networks
Target of Attack
192.168.20.1 host is target
192.168.20.0/24 – target’s network
Sinkhole Network
Customers
Customers Customers
Router advertises 192.168.20.1/32
Identification
48
95Sigcomm 2005
Additional Sinkhole Data Collection
§ Employ network devices to export dark IP, bogon and RFC 1918 packet data..§ Packet flow data can be used to glean extremely useful
information from the network without diverting data streams§ Things like number of flows, infected hosts, general or
specific upticks in traffic types, etc..
§ No sinkhole required, most routers/switches support some form of flow information export (e.g., NetFlow, ipfix, sFlow, JFlow, etc..)
§ Provides mechanism to collect information network-wide - even for allocated/subscribed address blocks
Identification
96Sigcomm 2005
Sinkhole Safety
§ Do not allow advertisements to leak:§ BGP “no-export” community§ Explicit Egress Prefix Policies (community, prefix, etc.)
§ Do not allow traffic to escape the sink hole:§ Backscatter from a Sink Hole defeats the function of a Sink Hole
(egress ACL on the Sink Hole router)§ Advanced sink hole designs§ True honeypot potential --> protect resources in the sink hole§ Don’t become part of the attack§ Filter/rate limit outgoing connections
Identification
49
97Sigcomm 2005 97
Detection: Backscatter
FIB------------------------------------------
192.168.1.0 = Null0---------------------------------------------------------------------------------------------------------
ICMP Process------------------------------------------------------------------------------------
Null0
Packets Arrive
SRC = 172.16.10.70
DST = 192.168.1.1
Packets whose destination is unreachable (even Null0) will have a ICMP Unreachable sent back. This unreachable noise is referred to as backscatter. Data sent back (to bogon or pre-allocated addresses) is directed to sinkhole for analysis, ingress routers are identified
ICMP Unreachable to SRC 172.16.10.70
Identification
98Sigcomm 2005
§ CAIDA - Network Telescope
§ Internet Motion Sensor (IMS) - Blackhole w/Active Responders§ Team Cymru - DarkNets§ IUCC/IDC Internet Telescope§ iSink§ Related BGP off-ramping techniques (CenterTrack, SinkHoles)
Monitoring of unused address space very successful:
Detection: Backscatter
⇒ Investigating DDoS ⇒ Tracking worms ⇒ Characterizing emerging Internet threats
Identification
50
99Sigcomm 2005
§ A Blackhole sensor monitors an unused globally advertised address block that contains no active hosts
§ Traffic is the result of DDoS backscatter , worm propagation, misconfiguration, or other scanning
Distributed Blackhole (IMS)Identification
100Sigcomm 2005
Lightweight Active Responder (IMS)
§ Goal: obtain enough fidelity to differentiate threats with the minimum resource cost
§ TCP needs to establish a connection before payload data is sent
§ Use lightweight SYN-ACK Active ResponderReceive a SYN => Respond with a SYN-ACK
Identification
51
101Sigcomm 2005
Blaster Worm - Live Host
BufferOverflow
Backdoor:Fetch &Executeworm
Identification
102Sigcomm 2005
Blaster Worm - Passive Monitor
MissesImportantFeatures
Identification
52
103Sigcomm 2005
Blaster Worm - IMS
CapturesImportantFeatures
Identification
104Sigcomm 2005
Active Responder Limitations
§ Application-level response may be required to differentiate certain threats. (e.g. Agobot)- Threats like Agobot can be differentiated using
scanning behavior
§ Sensors can be fingerprinted and avoided- IMS focused on globally scoped threats (threat model
does not include targeted manual attacks)- Many sensors of different sizes in many networks
near live hosts makes avoidance very hard- Rotate active responders within address block
Identification
53
105Sigcomm 2005 105
Commercial DetectionIdentification
106Sigcomm 2005 106
Commercial DetectionA Recent Large Scale DOS attack
Identification
54
107Sigcomm 2005 107
Traceback
Traceback to ingress network perimeter1) Manual§ Packet filters (ACLs)
§ IP accounting § Disable interfaces
2) Backscatter3) Packet/CEF Accounting4) NetFlow/Jflow/sFlow
Identification
108Sigcomm 2005
Traceback: Manual
§ Steps§ Began with ACLs and counters at network egress to customer§ Filtered attack traffic as it was destined for customer premise§ Manually traced back through the network, hop-by-hop, interface by
interface - very time-consuming and tedious (automated with ACL scripting tools; I.e., dostracker.pl) - but error prone and may impact data path
§ ACLs applied at network ingress to drop traffic destined for victim IPs
§ Limitations§ Error-prone§ May impact service availability§ Tedious§ Very time consuming§ Fully characterizing and accounting for full impact of attack is still
unlikely.
Identification
55
109Sigcomm 2005
A
B C
D
E
FG
TargetTarget
Peer B
Peer AIXP W
IXP E
Upstream A
Upstream A
Upstream B
Upstream B
Upstream B
Upstream B
POP
CustomersCustomers
Traceback: Manual
Upstream A
Upstream A
Identification
110Sigcomm 2005
Traceback: Manual
§ Classification ACL (cACLs) applied to customer interface:
§ Once attack type is classified, Traceback ACL (tACLs) applied to egress then subsequent upstream interfaces back towards network ingress
access-list 101 permit icmp any any echoaccess-list 101 permit icmp any any echo-replyaccess-list 101 permit udp any any eq echoaccess-list 101 permit udp any eq echo anyaccess-list 101 permit tcp any any establishedaccess-list 101 permit tcp any any range 0 65535access-list 101 permit ip any any
interface serial 10/1/1ip access-group 101 out
access-list 102 permit icmp any any echoaccess-list 102 permit icmp any any echo-reply log -inputaccess-list 102 permit udp any any eq echoaccess-list 102 permit udp any eq echo anyaccess-list 102 permit tcp any any establishedaccess-list 102 permit tcp any anyaccess-list 102 permit ip any any
interface serial 10/1/1ip access-group 102 out
router# sh ip access -list 101Extended IP access list 101
permit icmp any any echo (2 matches)permit icmp any any echo-reply (2171374 matches)permit udp any any eq echopermit udp any eq echo anypermit tcp any any established (150 matches)permit tcp any any (15 matches)permit ip any any (45 matches)
router# sh log%SEC-6-IPACCESSLOGDP: list 170 permit icmp 1.1.1.1 (Serial0/1/1 *HDLC*) -> 192.168.1.1 (0/0), 1 packet%SEC-6-IPACCESSLOGDP: list 170 permit icmp 2.2.2.2 (Serial0/1/1 *HDLC*) -> 192.168.1.1 (0/0), 1 packet%SEC-6-IPACCESSLOGDP: list 170 permit icmp 3.3.3.3 (Serial0/1/1 *HDLC*) -> 192.168.1.1 (0/0), 1 packet%SEC-6-IPACCESSLOGDP: list 170 permit icmp 4.4.4.4 (Serial0/1/1 *HDLC*) -> 192.168.1.1 (0/0), 1 packet%SEC-6-IPACCESSLOGDP: list 170 permit icmp 5.5.5.5 (Serial0/1/1 *HDLC*) -> 198.168.1.1 (0/0), 1 packet
Identification
56
111Sigcomm 2005 111
Traceback: Backscatter
§ Combines the Sink Hole router, Backscatter Effects of Spoofed (D)DOS attacks, and remote triggered Black Hole Filtering§ Created by Chris Morrow and Brian Gemberling @ UUNET as a
means of finding the entry point of a spoofed DOS/DDOS
§ Basic Idea§ Configure all edge devices (routers, NAS, IXP Routers, etc)
with static route to Null0 (e.g. “TestNet” 192.0.2.0/24) § Announce BGP route with TestNet nexthop to remotely drop
traffic to victim at multiple routers§ Use sinkhole to “catch” ICMP backscatter for spoofed
dropped traffic
Identification
112Sigcomm 2005
Peer B
Peer A
Traceback: BackScatter
IXP-W
IXP-E
Upstream A
Upstream A
Upstream A
Upstream A
Upstream B
Upstream B Upstream
BUpstream
B
POP
TargetTarget
NOCG
sinkholeNetwork
sinkhole Router receive the
backscatter to 96/3 with entry points of
the attack
171.68.19.0/24
171.68.19.1
ICMP Unreachable backscatter will
start sending packets to 96/3
ICMP Unreachable backscatter will start sending packets to
96/3sinkhole router
advertises the /32 under attack with next-hop equal to the Test -
Net
Identification
57
113Sigcomm 2005
TraceBack: BackScatter
SLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.47.251.104 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.70.92.28 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.222.127.7 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.96.223.54 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.14.21.8 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.105.33.126 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18 -> 96.77.198.85 (3/1), 1 packetSLOT 5:3w1d: %SEC-6-IPACCESSLOGDP: list 150 permitted icmp 171.68.66.18-> 96.50.106.45 (3/1), 1 packet
Identification
114Sigcomm 2005
Traceback: BackScatter Limitations
§ Assumes attack is randomly spoofed (no longer always valid assumption)§ Requires ICMP Unreachables working. § ICMP Unreachable Overloads are a concern
(and they should be), rate-limit them (i.e. ip icmp rate-limit unreachable command)
Identification
58
115Sigcomm 2005 115
Traceback: Flow-based
§ Trace attack by matching fingerprint/signature at each interface via passive monitoring:§ Flow data (e.g., NetFlow, cflowd, IPFIX)§ Span Data§ PSAMP
§ Number of open source and commercial products evolving in market§ Non-intrusive, widely supported
Identification
116Sigcomm 2005 116
Commercial Detection and Traceback
5. Filter: Peakflow DoS Controller recommends filters (X), which the network engineer can implement to stop the attack before it brings down key routers, firewalls and IDS solutions, or the entire network.
2. Monitor: Peakflow DoS Collectors analyze traffic for anomalies without disrupting traffic flow to routers.
1. Profile: Peakflow DoS dynamically profiles traffic patterns in the network.
4. Trace: Peakflow DoS Controllers then quickly trace the attack to its source. 3. Detect: Peakflow DoS Collectors create and forward unique anomaly fingerprints to Peakflow DoS Controllers.
Service Provider AService Provider B
Service Provider C
Customer Web Server
IDS
Firewall
Service Provider AService Provider B
Service Provider C
Customer Web Server
IDS
Firewall
Service Provider AService Provider B
Service Provider C
Customer Web Server
IDS
Firewall
Identification
59
117Sigcomm 2005 117
Traceback: CommercialIdentification
118Sigcomm 2005 118
Commercial Traceback: Attack in More Detail
Identification
60
119Sigcomm 2005 119
Commercial Traceback: Attack in More Detail
Identification
120Sigcomm 2005 120
Commercial Traceback: Attack in More Detail
Identification
61
Backbone Attack Detection and Mitigation Methodologies
SECTION IIc6 Phases of Infrastructure SecurityAttack Mitigation
Danny McPherson <[email protected]>Craig Labovitz <[email protected]>
Farnam Jahanian <[email protected]>
122Sigcomm 2005 122
Six Phases of Infrastructure Security
Preparation
§Prep the network§Create tools§Test tools§Prep procedures§Train team§Practice
Identification
§How do you know about the attack?§What tools can you use?§What’s your process for communication?
Classification
§What kind of attack is it?Traceback
§Where is the attack coming from?§Where and how is it affecting the network?
Reaction
§What options do you have to remedy?§Which option is the best under the circumstances?
Post Mortem
§What was done?§Can anything be done to prevent it?§How can it be less painful in the future?
Mitigation
62
123Sigcomm 2005 123
Potential Mitigation Options
§ Do Nothing
§ Actively respond:1) Packet filters (e.g., ACLs) or rate-limit (e.g., CAR)2) BGP remote-triggered drop
• Blackhole (dst == Null 0/discard interface)• uRPF loose check (src == Null 0/discard interface)• Customer-performed
3) Intelligent filtering (e.g. divert to CloudShield, Cisco Guard)4) Peer/upstream filtering5) CPE filtering firewall, IDS or similar6) New directions (flowspec)
Mitigation
124Sigcomm 2005
Most Common Mitigation Approaches
§ UMich/Arbor Survey of 40 tier1/tier2 ISPs§ Most common approach is to BGP null-route destination (and complete DDoS)§ BGP destination more scalable than ACLs and most common mitigation approach
0 10 20 30 40 50
ACLs
BGP Destination
BGP Source uRPF
Off-ramp intelligent filters
Other
Percentage Surveyed Answers
Mitigation
63
125Sigcomm 2005 125
Data Plane Filtering Issues
§ Capabilities of linecard, router or switch impact where and what can be filterered§ Number of ACLs severely constrained (e.g. at most 1K and usually in the 100s)§ ACLs may impact forwarding performance (element specific as possible)§ Flexibility of filter language
§ Usually IP 5 tuple
§ Juniper support packet length
§ Related issues:§ Sequence of filters may impact performance (higher hit counts earlier in path)§ Configuration management (humans prone to error (e.g., employ tool or rancid)§ Impact of installing ACLs (e.g., application forwarding hit, rec ompilation to take
effect, etc..)§ Many ALCs do not filter fragments§ Avoid collateral damage
Mitigation
126Sigcomm 2005 126
Mitigation Issues: Avoid Collateral Damage
NOC
A
B C
D
E
F G
Target
Peer B
Peer AIXP-W
IXP-E
Upstream A
Upstream B
Upstream B
POP
Upstream A
Customers
Mitigation
64
127Sigcomm 2005
Filtering Example
2 Gbps Syn attack with randomized source but same packet length against /32 web site.
§ IDS/IPS filters are on wrong side of bottleneck. Need provider-based mitigation.
§ May be large enough to congest provider aggregation/PoP layer. May need to mitigate on peering edge.
§ Filter/offramp all Syn to /32, but this completes denial.§ On-net intelligent filtering/syn-proxy.
Mitigation
128Sigcomm 2005
ACLs on Core Routers (GSR)
§ ACL implementation depends on card§ Engines (0 is software, 1 w/ rev. 4+, 2 problem with GigE)§ Engine2 include limited microcode (choose between ACLs to
NetFlow)§ No ACLs supported with MPLS
§ ACL Limitations/Issues§ More advanced ACLs not implemented in ASIC, but require CPU
(e.g. port)§ Maximum ACLs size : 128 or 448 entries§ Drop packets on input interface (outbound ACLs really implements
on all input)
§ Committed Access Rate (CAR)
Mitigation
65
129Sigcomm 2005 129
Commercial Filter GenerationMitigation
130Sigcomm 2005
ACLs May Impact Performance
§ Marketing comparison of Enterysis 1805 and Cisco 1751 http://www.enterasys.com/performance/2003/tolly/Tolly -1805-screen.pdf
Mitigation
66
131Sigcomm 2005 131
Blackhole Routing
§ Blackhole Routing or Blackhole Filtering results in packets being forwarded to a router’s bit bucket , also known as:§ Null interface§ Discard Interface
§ Initially worked only based on IP destination address, per it’s exploit of a router’s forwarding logic (can work based on sourceas well w/uRPF)
§ Typically results in desired packets being dropped with minimal or no performance impact
§ At any given time, tier-1 providers average 500 active BGP null routes
Mitigation
132Sigcomm 2005 132
Exploits Forwarding Logic
FIB------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Ingress Packet Filter
------------------------------------------------------------------------------------
Null0/Discard
Packets
Arrive
• Forward packet to the Bit Bucket
• Saves on CPU and ACL processing
Egress
Interface
Mitigation
67
133Sigcomm 2005 133
Blackhole
NOC
A
B C
D
E
FG
Advertises List of Black
Holed Routes
Target
Peer B
Peer AIXP-W
IXP-E
Upstream A
Upstream B
Upstream B
POP
Upstream A
Mitigation
134Sigcomm 2005
Blackhole Trigger
§ Select a small unused block (e.g. TestNet 192.0.2.0/24)§ Configure static route with TestNet to Null 0
on every router§ Prepare BGP speaking router to act as trigger
device (next slide)
Mitigation
68
135Sigcomm 2005
Blackhole Trigger Configuration
router bgp 65501 .redistribute static route-map static-to-bgp.!route-map static-to-bgp permit 10 match tag 66set ip next-hop 192.0.2.1set local-preference 50set community no-export set origin igp!Route-map static-to-bgp permit 20
Redistribute Static with a route-map
Match Route Tag
Set BGP NEXT_HOP to
the Trigger
Set LOCAL_PREF
Mitigation
136Sigcomm 2005
Blackhole: Community Based Trigger
§ BGP Community-based triggering allows for more granular control over where you drop the packets.
§ Examples of flexibility§ Community #1 can be for all routers in the network.§ Community #2 can be for all peering routers. No customer routers – Preserves
customer-customer connectivity if the victim is within your AS.§ Community #3 can be for all customers (e.g., to push a inter-AS traceback to
the edge of your network).§ Trigger Communities per ISP Peer can be used to only black hole on one ISP
Peer’s connection. Allows for the DOSed customer to have partial service.
§ Three parts to the trigger:§ Static routes to Null 0 on all the routers.§ Trigger router sets the community and advertises the BGP update.§ Reaction Routers (on the edge) matches community and sets the next-hop to
the static route which maps to Null0.
Mitigation
69
137Sigcomm 2005
Customer Initiated Mitigation
§ Several providers accept more-specifics of customer routes with destination-based BGP blackholing community attached.§ No source-based blackholing§ Only accept more-specifics of customer
prefixes
Mitigation
138Sigcomm 2005
New Filter Options:Enhanced Policy Language
§ Use BGP to specify explicit prefix filters
§ Basic idea:§ A flow specification is an n-tuple consisting of several
matching criteria that can be applied to IP packet data.§ May or May not include reachability information (e.g.,
NEXT_HOP).§ Well-known or AS-specific COMMUNITIES can be used to
encode/trigger a pre-defined set of actions (e.g., blackhole, PBR, rate-limit, divert, etc..)
§ Application is identified by a specific (AFI, SAFI) pair and corresponds to a distinct set of RIBs.
§ BGP itself treats the NLRI as an opaque key to an entry in its database.
Mitigation
70
139Sigcomm 2005
Inter-Provider Cooperation
§ Attacks in 10 Gbps range require inter-provider cooperation§ NSPSec mailing list§ Fingerprint sharing § Share attack vector invariants§ Authenticated§ Integrated with detection, classification, workflow§ Preserve business/commercial privacy of data§ Controlled/selective distribution of fingerprint
Mitigation
140Sigcomm 2005
Inter-Provider Mitigation
F
TargetTarget
POP
ISP - A
ISP - B
ISP - C
ISP - DISP - H
ISP - G
ISP - EISP - F ISP - I
Mitigation
71
141Sigcomm 2005
Fingerprint Sharing
Operator selects exactly who to share fingerprint with.
Mitigation
142Sigcomm 2005
Intelligent Filtering
§ New hardware (e.g. CiscoGuard, CloudShield) supports > 1Gbps intelligent filtering
§ Provider (or customer) BGP offramps attack to mitigation device.MPLS/GRE tunnel back to aggregation router/cpe device.
§ Intelligence§ TCP, Web, DNS Proxy§ Fine-grain DDoS layer 1-7 recognition§ Zombie detection (100K+ src IP filters)§ Enforced baselines (via netflow or peace time traffic spanning/redirection)
§ Deployment Challenges§ Avoid loops!§ Provisioning adequate bandwidth through “scrubbers”§ Integrate with customer-initiated mitigation
Mitigation
72
143Sigcomm 2005
Intelligent Filtering (CiscoGuard)
§ Though all vendors claim dozens of advanced heuristics, linespeed TCP Syn and DNS UDP proxy provide 90% of the mitigation value…
144Sigcomm 2005
Intelligent Filtering
PeeringPoint
PeeringPoint
Core Router
Core Router
POP
POP
Enterprise A
Enterprise C
Enterprise BTargeted
Cisco Guard Cluster
Controller
Collectors
“Activate”
Mitigation
73
145Sigcomm 2005 145
Post Mortem
§ Analyze data & discuss attack§ Perform trending§ Maintain full history of attack data§ Determine what, if anything, could have been done to
be better prepared -- make appropriate adjustments as necessary
§ Remove any deployed mitigation mechanisms§ Clarify billing or other issues§ Involve your customers (encourage CPE filtering and
more importantly, patched systems!)§ Contact authorities as appropriate
Mitigation
146Sigcomm 2005
0
10
20
30
40
50
60
70
80
Ten or more Less than 10 None
Per
cent
ISP
Sur
veye
d
Legal and Law Enforcement§ Attacks referrred to law enfocrement in last 12 months
§ Though dozens of attacks per months, ISPs report average of 2-3 are reported to law enforcement per year§ Jurisdictional issue§ Online gambling techniquely illegal is US§ IRC users unloved
Mitigation
74
147Sigcomm 2005
First Successful US Investigation of a DDoShttp://darkwing.uoregon.edu/~joe/ddos-exec/ddos-exec.ppt
Mitigation
148Sigcomm 2005
… Dismissed (for now)Mitigation
75
Backbone Attack Detection and Mitigation Methodologies
SECTION IIcNew Directions
Danny McPherson <[email protected]>Craig Labovitz <[email protected]>
Farnam Jahanian <[email protected]>
150Sigcomm 2005
New Directions
§ Improved Mitigation§ Pushback§ Secure Overlay Services§ D-Ward
§ New Vulnerabilities§ Reflectors§ Botnets
§ Improved Detection§ Distributed telescopes§ Signal analysis, PCA,
inference§ Botnets
§ Improved Configuration§ DNS Errors, Router Configuration
§ High Speed Data Collection
§ New Testbed and Datasets§ Deter§ PREDICT
76
151Sigcomm 2005
Improved Mitigation
Push-back (Bellovin)http://www.cs.utsa.edu/~korkmaz/seminar/ddos_overview.ppt&e=7385
R2
R0
R1 R3
R7
R6
R5R4
Heavy traffic flow
Push-back messages
? Reactive mechanism? Accuracy of telling 'poor' packets from bad packets
152Sigcomm 2005
Improved Mitigation
SOS – Security Overlay Service
§ To protect a dedicated server from DDoS attacks§ Use high-performance filters to drop all the
packets not from secret servlets§ Path redundancy in overlay network is used to
hide the identities of secret servlets§ Legitimate users enter the overlay network at
the point of SOAP (secure overlay access point)
77
153Sigcomm 2005
Improved Mitigation
SOS (cont.)
Big time delayOverlay network
SOAP(s) Secret servlet(s)
ServerFilter
154Sigcomm 2005
New Vulnerabilities
DDOS with Reflectors
78
155Sigcomm 2005
New Vulnerabilities
Tracing when reflectors are used
• Easy for victim to ID reflectors since SRC IP is real.
• However, hard to trace back to slaves since reflectors are given victim’s IP as the source. Also involves cooperation from the reflector’s operator for analysis.
• Hard to use ITRACE since flow is low-volume for any given reflector (flow is diffused through many reflectors). Can use SPIE.
156Sigcomm 2005
New Detection
PCA/Subspace
§ Anukool Lakhina et al, “Diagnosing Network-Wide Traffic Anomalies”. SIGCOMM 2004.
§ Given link traffic measurements, diagnose the volume anomalies
§ Normal Subspace, : space spanned by the first kprincipal components
§ Anomalous Subspace, : space spanned by the remaining principal components
§ Then, decompose traffic on all links by projecting onto and to obtain:
79
157Sigcomm 2005
• Detectthe time of the anomaly
• Identifythe source & destination
• Quantifythe size of the anomaly
New Detection
PCA/Subspace
158Sigcomm 2005
New Detection
Detecting and Stopping Botnets
§ Little hard data on botnets§ Three areas of active research:
1. Prevent systems from getting infected2. Directly detect bot communications between bots and between bots and
bot controllers3. Detect the secondary features of a bot infection like propagation or
attacks
Attack
Communication
Propagation
DoS, SPAM Relay, Phishing Site…
IRC (can be encrypted)
Vulnerabilities, File Shares, P2P…{WORM
80
159Sigcomm 2005
§ Well developedmethods:
– Anti-virus– Firewalls– Patching
§ But:– Might not directly control of systems (ISPs)– Can’t upgrade certain systems (Win98 DAQ)– Complex infection vectors: App-level (javascript, AIM)– Custom threat (Israeli trojan)
§ Naïve to assume 100% protected
Prevent Infection
StillInfected
160Sigcomm 2005
Detect Bot Communication
§ Many bots use IRC for Command and ControlDetect IRCBot Commands
• OfframpTCP port 6667
• InspectPayloads(advscan…)[honeynet05]
• IRCBehavior[Racine04]
81
161Sigcomm 2005
Botnet Measurements – HoneypotEvan Cooke et al., SRUITI 2005
§ Windows 2000/XPHoneypot
§ Placed behind proxy:1. Rate limit traffic 12KB/s2. Disallow local network3. Log all traffic
§ 12 experimental runsover a month:
– 12-72 hour traces > 100MBs– Recruited into least 15 unique botnets– Bots used DCOM/RPC, LSASS=> Bots are extremely prevalent
Just 2 worm infections during the experiment!
162Sigcomm 2005
New Detection
IMS Global Deployment
60 monitored blocks at 20+ networks§ Service Providers§ Broadband Providers§ Large Enterprises§ Academic networks
Monitored blocks:§ One /8§ Ten /16§ Eighteen /17-/20§ Eight /21-/23§ Eighteen /24-/26§ …
3 Continents and growing
82
163Sigcomm 2005
New Detection
IMS Deployment Statistics
1. Internet Worms2. Reconnaissance/Scanning3. Distributed Denial of Service
§ 17M IPs monitored§ 1.25% of routed IPv4
space
§ 27 /8 blocks with an IMS sensor
§ 21% of all routable /8 blocks have at least one sensor
IMS has provided insight into:
164Sigcomm 2005
New Detection
IMS Distributed Detection
Each IMS block sees a very different traffic rate
§ Clearly more addresses are better, but…
§ Normalized by /24
§ Includes all protocols
§ Month long observation period
83
165Sigcomm 2005
§ V. Pappas et al. “Impact of Configuration Errors on DNS Robustness” SIGCOMM 2004§ Thousands or even millions of users affected§ All due to a single DNS configuration error
“Microsoft's websites were offline for up to 23 hours -- the most dramatic snafu to date on the Internet --because of an
equipment misconfiguration” -- Wired News, Jan 2001
DNS Configuration Errors
166Sigcomm 2005
§ Error Frequency:§ 15% of the zones§ 8% for the 500 most popular zones§ independent of the zone’s size,
varies a lot per TLD
§ Impact:§ 70% of the zones with errors lose half or more of the
authoritative servers§ 8% of the queries experience increased response times (up
to an order of magnitude) due to lame delegation
DNS Configuration Errors
84
167Sigcomm 2005
New TestBeds (DETER)Anthony D. Joseph, “Deter Testbed Update”, http://deter.cs.berkeley.edu/
§ NSF and DHS sponsored cyber-defense research project
§ DETER Goals:1. Design and construction of a testbed for network security
experiments,2. Research on experimental methodology for network security, and3. Research on network security.
§ DETER: focus on 1), but it needs to do some of 2) and 3)§ Goal: Duplicate observed attack effects in the testbed§ E.g., self-congestion for worms
168Sigcomm 2005
UC Berkeley
USC-ISI
ISI-EastInternetInternet
Cyber Defense Experiments run on Virtual Internet Network Traces
Deter Wide-Area Testbed Architecture
85
169Sigcomm 2005
Concluding Remarks
§ Disconnect between security research and commercial realities.
§ Security point solutions (e.g. CPE) not sufficient§ Security distributed problem and requires distributed solution§ Solution mix of technologies, devices and backbone layers
§ Increasingly complex security challenges require cooperation andintervention from service providers§ Infrastructure attacks require inter-provider fingerprint sharing and mitigation
§ Some suggestions for further research§ Detect and interrupt command and control infrastructure (irc)§ Distributed event correlation/aggregation (SEM)§ Configuration/ACL management
§ Feldman et al., “Network-Wide Inter-Domain Routing Policies: Design and Realization”§ OOB management§ …