DNS Architecture Idea: Modularization & Compartmentalization · 2019-06-18 · Attacks Interna...

1

DNS Architecture Idea: Modularization & Compartmentalization

1 1 1

2

Agenda

•  Consultation about the key “DNS” problems. •  Review of the key operational issue seen with

DNS robustness.

•  Modularization & Compartmentalization

3

Most DNS Today

Zone Slaves

Caching Resolvers Zone Master

Internally DNS

Infrastructure Only Only Slave Servers

External Resolution

The Soft Underbelly of the Internet

4

Protecting DNS like HTTP does not work

Zone Slaves

Caching Resolvers Zone Master

Internally DNS


External Resolution

Protective Anti-DDOS Box New Failure Point

5

DNS Resiliency Requires “Engineering”

•  DNS Resiliency requires engineers to execute “engineering.” –  The technology must be understood. –  DNS’s Interdependency and Coupled Dependency with

all parts of the other services must been mapped out. –  Architectural Plans must be drawn and tested.

•  Some of the world’s biggest company’s have had complete DNS failures …. where the root cause was based on throwing DNS into a network, putting a router/load balancer/anti-DOS device in front of it, and thinking it is going to “just work.”

•  Architectural Principles are the key to DNS Resiliency

6

Options

•  There are key options a provider has to “re-architect” their DNS. Two key requirements are: –  Investing in your own people to turn them into DNS

Gurus. –  Join DNS-OARC (https://www.dns-oarc.net/) –  Active Participation in your network operations

communities (RIPE and MENOG)

•  The “kick start” options to change fast include: –  Contracting with Internet Systems Consortium (

http://www.isc.org/) –  Outsourceing to a DNS provider (i.e. ISC) –  Work with one of the two big DNS product Vendors (ISC,

Nominum, or Infoblox).

7

Robust DNS Topology for Big Networks

Resolvers

Caching Forwarders (CFs)

Aggregate Caching Forwarders (ACFs)

(Optional)

Internal Resolvers (iRs)

External Resolvers (eRs)

Zone Slaves Zone Master

Internally Access Only

Internally DNS

Infrastructure Only Only Slave Servers Internet Accessible

8

Out Bound Recursion/Resolution

Resolvers


Aggregate Caching Forwarders (ACFs) Internal

Resolvers (iRs)



9

Compartmentalization Simplifies Security

•  Modularization and Role allow for distinct relationship to be turned into policy.

•  That policy can be enforced and monitored.

10

Roles and Security Realms

Resolvers



Resolvers (iRs)

External Resolvers (eRs) Zone Slaves Zone Master

Anycast Realm

Slaves Realm Master Realm External Access Realm

Agency Realm

11

Attack Vectors

Resolvers



Resolvers (iRs)


Anycast Realm


Agency Realm

External

Attacks

Internal Attacks

12

Configure Policy

Resolvers



Resolvers (iRs)


Anycast Realm


Agency Realm

Policy & Config Enforcing Policy

13

DNS Backscatter – Knowing when you are being Poisoned

13 13 13

14

Backscatter – ICMP Port Unreachable

Controller Proxy

Victim of Crime

DNS Recursive Server

Poison Engine

Miscreant Driving

the BOTNET

Wert543.example.com

Oihwoeif.example.com

Fdvakjnfvkjndaf.example.com

Send DNS Query to Controlled Domain

Poison Attempt w/RR “Hint”

My DNS Server

ns.example.com DNS Authority

www.example.com

ICMP Port Unreachable

Spoof ns.example.com

15

ICMP Unreachable & DNS

  ICMP Unreachable – specific port unreachable – are not normal packets which arrive at:   DNS Masters   DNS Slaves   DNS Split-Horizon Authoritative Servers

 Live Observation   Launching the attack results packets arriving on

closed ports of the recursive DNS Server.   This send ICMP Port Unreachable to the source

packet – which is the DNS Authority being spoofed.

16


 This will tell you that someone somewhere is poising somewhere so that they can be a man in the middle between you and your customer!

 How to monitor:   Classification ACLs (match ingress on ICMP port

unreachable)   Netflow   IDP/IPS   Firewalls   DPI Boxes

17

ACLs – How?

Controller Proxy

Victim of Crime


Poison Engine

Miscreant Driving

the BOTNET

Wert543.example.com





My DNS Server


www.example.com ACL on Router with SNMP trap


18

Netflow

Controller Proxy

Victim of Crime


Poison Engine

Miscreant Driving

the BOTNET

Wert543.example.com





My DNS Server


www.example.com Netflow Export


19

IDP/IPS

Controller Proxy

Victim of Crime


Poison Engine

Miscreant Driving

the BOTNET

Wert543.example.com





My DNS Server


www.example.com IDP/IPS


DNS Security (DRAFT)

Barry Raveendran Greene [email protected]

Version 0.7

21

Attack Vector #1

•  “Big Money Company’s” DNS Server get poisoned.

•  www.example.com is victimized •  Everyone going to the bad guy’s server is

victimized.

Home Users

Company Users

www.example.com

Bad Guy’s Server

DNS Poison

Big Money Company

SP’s DNS

22

Attack Vector #2

•  DNS Server get poisoned. •  Big Money Company is victimized •  Everyone going to the bad guy’s server is

victimized.

Home Users

Company Users

www.example.com

Bad Guy’s Server

SP’s DNS

DNS Poison

Big Money Company

23

Focus of the Industry

Chain of Victimization

Users Operator Domain Owner

www.example.com

Bad Guy’s Server

Target Target Means to a Target

Recursive DNS Resolver

24

Threat to any domain on the Internet!

Users Operator Domain Owner

www.example.com

Bad Guy’s Server

Target Target Means to a Target

Recursive DNS Resolver

25

These two attack vectors are just the start

•  Now that DNS Poison is easier, more attack vectors will be discovered.

•  This is a threat to the trust model(s) of the Internet.

26

Objective

•  This presentation is a consultation tool to help Juniper Networks have meaningful conversations about the tools and technique available to help mitigate issues around DNS Security.

•  The recent announcement by CERT/CC of a highly exploitable industry wide DNS vulnerability increases the urgency around DNS Security.

27

Agenda

•  What did CERT/CC Announce on July 8, 2008? •  Quick DNS Refresher •  What is the “DNS Problem?” •  DNS Threat Vectors •  DNS Architecture Idea: Modularization &

Compartmentalization •  CERT/CC #800113

Multiple DNS Implementations Vulnerable to Cache Poisoning Detailed Analysis

•  NATs Breaking the Source Port Randomization “Patch” •  How the Cyber-Criminal Might Use this Vulnerability (DNS

Poison – The BOT Version) •  How the Cyber-Criminal Might Use this Vulnerability (DNS

Poison Drive By) •  Spotting when someone is trying to Poison Your DNS

Identity

28

What did CERT/CC Announce on July 8, 2008?

29

The CERT/CC DNS Vulnerability Announcement

•  The Domain Name System (DNS) is responsible for translating host names to IP addresses (and vice versa) and is critical for the normal operation of internet-connected systems.

•  DNS cache poisoning (sometimes referred to as cache pollution) is an attack technique that allows an attacker to introduce forged DNS information into the cache of a caching nameserver.

•  DNS cache poisoning is not a new concept; in fact, there are published articles that describe a number of inherent deficiencies in the DNS protocol and defects in common DNS implementations that facilitate DNS cache poisoning.

30

Specific details are not announced, but …

•  This was a industry wide, multi-vendor, coordinated announcement!

•  This has never been done before. •  It indicates the urgency that should be placed on

the recommendation: 1.  Upgrade all DNS Servers which function as a recursive

caching forwarder ASAP! 2.  Slowly upgrade all DNS stub resolvers. 3.  Deploy BCPs for DNS Security.

31

Consequence of not acting now

•  The urgency and concern by the industry is based on how the criminals would use this new technique.

•  Not acting now would put your business, network, or operations at risk.

32

Quick DNS Refresher

33

What is DNS?

•  The Domain Name System (DNS) associates various information with domain names; most importantly, it serves as the "phone book" for the Internet by translating human-readable computer hostnames, e.g. www.example.com, into IP addresses, e.g. 208.77.188.166, which networking equipment needs to deliver information.

•  A DNS also stores other information such as the list of mail servers that accept email for a given domain. By providing a worldwide keyword-based redirection service, the Domain Name System is an essential component of contemporary Internet use. –  Source Wikipedia

34

Hierarchical Name Space

root

edu net org uk com ca

wisc ucb stanford cmu mit

cs ee

www

www.cs.stanford.edu

= 192.168.20.1

35

Zone = Juniper.net

DNS Server Functions/Roles

•  Zone (Domain): A DNS zone is a portion of the global Domain Name System (DNS) namespace for which administrative responsibility has been delegated.

Zone Administrator

Zone Files Master DNS Server

Slave DNS Server (s)

Dynamic Updates (DHCP & AAA)

Caching Forwarders

Resolvers

36


•  Zone Master (Primary): The authoritative server for a zone (domain). The Zone Master contains one or more zone files for which the DNS is authoritative. Other DNS Servers can automatically transfer zone files.

Zone Administrator




Caching Forwarders

Resolvers

37


•  Zone Slave (Secondary): A Zone Slave (also called a stub name server or secondary DNS), gets zone data from the Zone Master. When Zone Slave server starts up, it contacts its Zone Master, requesting a zone transfer. The goal of the Zone Slave is scaling (load) and zone resiliency (in case the Zone Master is down). You can have multiple Zone Slaves geographically distributed to increase resiliency.

Zone Administrator




Caching Forwarders

Resolvers

38


•  Caching Forwarders: Caching Forwarders (a.k.a. Proxy, Client, Remote) server forwards all requests to another DNS and caches the results. It is a scaling tool, speeding up responses, removing unnecessary traffic and simplifying administration The are also used as part of a Split Server configuration for perimeter defense and Anycast DNS architectures.

Zone Administrator




Caching Forwarders

Resolvers

39


•  Resolvers (customers): A resolver looks up the resource record information associated with nodes. A resolver knows how to communicate with name servers by sending DNS queries and heeding DNS responses.

Zone Administrator




Caching Forwarders

Resolvers

40


•  Stub Resolvers (customers): Stub Resolvers move the resolution function out of the local machine and into a name server which supports recursive queries. Little to no local caching happens.

Zone Administrator




Caching Forwarders

Stub Resolvers

41

DNS Server Functions/Roles (Options)

•  External Resolvers: External Resolvers are designed to proxy all queries from inside a large organization. It becomes one of the publicly visible addresses of the large network – allowing the internal DNS servers to be hidden (core hiding) and protected from outside attack.

•  Internal Resolvers: Internal resolvers are slaves configured in split horizon mode to allow for external zone transfers and authoritative responses. It becomes one of the publicly visible addresses of the large network – allowing the internal DNS servers to be hidden (core hiding) and protected from outside attack.

42

Internally DNS Infrastructure Only

DNS Server Functions/Roles (Options)

Resolvers






Only Slave Servers Internet Accessible

Zone Files


43

DNS Information Flow

1.  Zone Administrator (i.e. Juniper.net) updates information in the Zone files. These files are moved to the DNS Master.

Zone Administrator 1!

2!

4!

5!




Caching Forwarders

Resolvers

3!

44


2.  Dynamic Updates are sent by the DHCP or AAA server. The DNS Master updates its records.


2!

4!

5!




Caching Forwarders

Resolvers

3!

45


3.  Zone transfer is use to push copies of the Master’s Records to Slave DNS Servers. This allows for scaling and resilancy.


2!

4!

5!




Caching Forwarders

Resolvers

3!

46


4.  Caching Forwarders, Proxies, and Resolvers all query the Master/Slave DNS server to get authoritative information about the DNS Zone.


2!

4!

5!




Caching Forwarders

Resolvers

3!

47


5.  Resolvers query Recursive Caching Forwarders to have them get DNS records on their behalf. These are your local DNS servers set in most end devices.


2!

4!

5!




Caching Forwarders

Resolvers

3!

48

DNS Query Recursive Resolution

Question: www.juniper.net A

www.juniper.net A ?

www.juniper.net A ?

“go ask net server @ X.gtld-servers.net” (+ glue)

www.juniper.net A ?

“go ask ripe server @ ns.juniper.net” (+ glue)

www.juniper.net A ?

“192.168.5.10”

192.168.5.10

1! 2!

3!

4!

5!

6!

7!

Add to cache 9!

8!

10! TTL

Resolver Caching Forwarders

ROOT Server

GTLD Server

Juniper Server

49

DNS Query Non-Recursive Resolution

Question: www.juniper.net A

www.juniper.net A ?

Error – Go to Root “go ask net server @ X.gtld-servers.net” (+ glue)

www.juniper.net A ?

“go ask ripe server @ ns.juniper.net” (+ glue)

www.juniper.net A ?

“192.168.5.10”

1!

4!

5!

6!

7!

7!

Add to cache 8!

9! TTL


ROOT Server

GTLD Server

Juniper Server

2!

www.juniper.net A ?

13!

50

Non-Recursive: Partial Answers

Recursive: Full Answer to a Query

DNS Query Recursive vs Non-Recursive


ROOT Server

GTLD Server

Juniper Server

51

What is the “DNS Problem?”

52

Industry Wide Vulnerability

•  DNS has a highly exploitable architectural flaw. •  This is an industry wide vulnerability which

impact ever DNS Server on the planet. •  The Risk to the Industry is a general breach of

confidence and a feasible ability to break chains of commercial trust.

•  Demonstrated ability for the exploit to be commercially capitalized by the cyber-criminal economy (miscreant economy) –  See http://www.getit.org/Mediawiki/index.php?

title=Miscreant_economy –  Suspected – but not confirmed - active exploit today in

China.

53

Industry Risk DNS: Where is the Problem?

•  DNS Poison Entries in 4. •  Threat – “Botable” and Criminally Executable Threat

to the confident of the Internet.


2!

4!

5!




Caching Forwarders

Resolvers

3!

Computer with Hijacking Malware

54

Routers are Stub Resolvers

•  JUNOS is not “vulnerable” (i.e. the code is not broken)

•  JUNOS can be a victim of the Caching Forwarders are violated with a Poison Attack.


2!

4!

5!




Caching Forwarders

Resolvers

3!


55

DNS Threat Vectors

56

DNS is a “Couple Dependency”

•  Services depend on DNS to be there. •  Applications depend on DNS to be there. •  People depend on DNS to be there.

•  The Internet could be passing plenty of packets at line rate speeds, but if DNS is not working, the customer see the Internet as “not working.”

57

DNS Security – Protect the resolution path!

•  DNS Security is all about protecting the information that flows from one functional node to another.

Zone Administrator




Caching Forwarders

Resolvers

58

DNS Attack Vectors

Zone Administrator




Caching Forwarders

Resolvers

Corrupt Zone Data

DOS Servers

Poison Recursive Caching

Impersonating Master

Unauthorized Updates

Cache Impersonation

Redirection

59

Server Protection Data Protection

Divide the Problem in Half!

•  Policy, Tools, Protocols and Technique can be easily derived by dividing the problem in half:

Zone Administrator




Caching Forwarders

Resolvers

60

Zone Files

•  Are the Zone files protected? •  Are they edited on the Master or off on another

machine. •  Is the path between the Zone Administrator and

Master DNS Server protected?

Zone Administrator




Caching Forwarders

Resolvers

61

Master & Slave DNS Servers

•  Basic 101 of Server Security. The Master is a critical resource.

•  What happens if its gets DOSed? •  Who do you allow zone transfers to and from?

Zone Administrator




Caching Forwarders

Resolvers

62

Zone Transfer to Slave Servers

•  Data path between the Master and Slave needs protection.

•  File corruption of the zone transfer, hijacking the zone transfer, and DOS (low level) all could happen.

Zone Administrator




Caching Forwarders

Resolvers

63

Dynamic Updates

•  DHCP and other dynamic update tools need protection.

•  It could be a back door into the DNS System.

Zone Administrator




Caching Forwarders

Resolvers

64

DNS Cache Poising

•  DNS Cache poising is one of the most common attack vectors.

•  Anti-Spoofing and the new Source Port Randomization helps.

Zone Administrator




Caching Forwarders

Resolvers

65

“172.13.1.66”

DNS Poison Basic

•  DNS Poisoning is a by product of DNS using UDP. •  When a query goes out, the resolver will take the

first UDP packet back which seems to be “authoritative.”

•  It is a race to see who gets the UDP packet back first.

•  Once the “Caching Forwarder” is poisoned, all queries from all other resolvers will get the “poisoned” data.

www.juniper.net A ? www.juniper.net A ?


Juniper DNS Server UDP UDP

“192.168.5.10”

ME

ME

ME ME

ME ME ME ME

66

+---------------------------+---------------------------+ | ID | flags | +---------------------------+---------------------------+ | numbers of questions | numbers of answer | +---------------------------+---------------------------+ | number of RR authority |number of supplementary RR | +---------------------------+---------------------------+ | | \ QUESTION \ | | +-------------------------------------------------------+ | | \ ANSWER \ | | +-------------------------------------------------------+ | | \ Stuff etc.. No matter \ | | +-------------------------------------------------------+

“172.13.1.66”

DNS Poison – The Catch

•  You must match the transaction ID (query ID) of the DNS query – which means you need to sniff the wire



Juniper DNS Server UDP UDP

“192.168.5.10”

ME

ME

ME ME

ME ME ME ME

67

DNS Poison – Miscreant Workaround

•  If I cannot “sniff” the packets, but I can query the caching resolver, then I can brute force my way into a DNS Poison.

•  Instead of waiting for someone else to query, you send your own queries into the caching forwarder.

•  I can then brute force the query ID.



Juniper DNS Server UDP

“192.168.5.10”

ME

ME

ME ME

ME ME ME ME

68

DNS Poison – Better Yet – DOS the Server

•  DOSing the authoritative DNS Server(s) is one way to give the Miscreant Breathing room.

•  The DOS attack does not need to be big, just enough to clog up the DNS’s servers.

•  It might now be a flood. It could be a computational overload attack.



Juniper DNS Server

ME

ME

ME ME

ME ME ME ME

Low Level DOS

69

DNS Poison – Computational Overload

•  A computational overload attack makes the core functions of the application work really hard.

•  Send queries to the DNS server where each sub-domain = a name in a password cracking database.

•  Consequence: DNS Server is waiting for each domain to resolve – really nasty if you are forcing this to do recursive lookups.



Juniper DNS Server

ME

ME

ME ME

ME ME ME ME

a.juniper.net A aapple.juniper.net A aadvark.juniper.net A alvin.juniper.net A ake.juniper.net A A$#@.juniper.net A affrroo.juniper.net A (password crakcing file).juniper.net A

70

DNS Architecture Idea: Modularization & Compartmentalization

71

Credit

•  The ideas here are not new. Mathias Körber [[email protected]] crafted and deployed the basic in 1996.

•  Principles of Modularization and Compartmentalization are also not new.

72

Most DNS Today

Zone Slaves Caching Resolvers Zone Master


Only Slave Servers

External Resolution

The Soft Underbelly to IP NGN

73

Robust IPNGN DNS Topology

Resolvers


Aggregate Caching Forwarders (ACFs) (Optional)






Only Slave Servers Internet Accessible

74


Resolvers



Resolvers (iRs)



75

Compartmentalization Simplifies Security

•  Modularization and Role allow for distinct relationship to be turned into policy.

•  That policy can be enforced and monitored.

76


Resolvers



Resolvers (iRs)


Anycast Realm


Agency Realm

77

Attack Vectors

Resolvers



Resolvers (iRs)


Anycast Realm


Agency Realm

External Attacks

Internal Attacks

78

Configure Policy

Resolvers



Resolvers (iRs)


Anycast Realm


Agency Realm

Policy & Config Enforcing Policy

79

CERT/CC #800113 Multiple DNS Implementations Vulnerable to Cache Poisoning Detailed Analysis

80

CERT/CC Overview

•  The Domain Name System (DNS) is responsible for translating host name to IP addresses (and vice versa) and is critical for the normal operation of Internet-connected systems.

•  DNS cache poisoning (sometimes referred to as cache pollution) is an attack technique that allows an attacker to introduce forged DNS information into the cache of a caching nameserver.

•  The general concept has been known for some time, and a number of inherent deficiencies in the DNS protocol and defects in common DNS implementations that facilitate DNS cache poisoning have previously been identified and described in public literature.

81

Issue #1 - Insufficient transaction ID space

•  The DNS protocol specification includes a transaction ID field of 16 bits. If correctly implemented and randomly selected with a strong random number generator, an attacker will require, on average, 32768 attempts to successfully predict the ID.

•  Some flawed implementations may be utilizing a smaller number of bits for this transaction ID, meaning that fewer attempts will suffice.

•  Furthermore, implementation errors in the randomness of transaction IDs generated by a number of implementations have been identified.

•  Amit Klein researched several such affected implementations in 2007.

•  These vulnerabilities were published as: VU#484649 - Microsoft Windows DNS Server vulnerable to cache poisoning VU#252735 - ISC BIND generates cryptographically weak DNS query IDs VU#927905 - BIND version 8 generates cryptographically weak DNS query identifiers

82

Issue #2 'Birthday Attack

•  Multiple outstanding requests Some implementations of DNS services contain a vulnerability whereby multiple identical queries for the same resource record (RR) will generate multiple outstanding queries for that RR.

•  This condition leads to the feasibility of a 'Birthday Attack', significantly raising the chance of success for an attacker.

•  This problem was previously described in VU#457875. A number of vendors and implementations have already added mitigations to address this issue.

83

Issue #3 Fixed Source Port for Generating Queries

•  Some current implementations allocate an arbitrary port at startup (sometimes selected at random) and reuse this source port for all outgoing queries.

•  In some implementations, the source port for outgoing queries is fixed at the traditional assigned DNS server port number, 53/udp.

84

Add them together …

•  Recent additional research into these issues and methods of combining them to conduct improved cache poisoning attacks have yielded extremely effective exploitation techniques.

•  Caching DNS resolvers are primarily at risk, both those that are open (a DNS resolver is open if it provides recursive name resolution for clients outside of its administrative domain) and those that are not.

•  These caching resolvers are the most common target for attackers, however stub resolvers are also at risk.

85

Per-query source port randomization

•  Because attacks against these vulnerabilities all revolve around the ability for the attacker to predictably spoof traffic, the implementation of per-query source port randomization in the server presents a practical mitigation against these attacks within the boundaries of the current protocol specification.

86

Added Resiliency – Not the Final Solution

•  The use of randomized source ports can be used to gain an additional approximately 16 bits of randomness in the data that an attacker must guess. In practice, implementers will be restricted to less than 65535 in the actual number of source ports they can allocate (port numbers <1024 may be reserved, other ports may already be allocated, etc.) however a significant amount of additional attack resiliency can be achieved. It is important to note that in the absence of changes to the DNS protocol, these mitigations are insufficient to completely prevent cache poisoning. However, if properly implemented, they reduce the chances of success for an attacker by several orders of magnitude and make attacks impractical.

87

Restrict Access to Recursion

•  Administrators, particularly those who are unable to apply a patch, can limit exposure to this vulnerability by restricting sources that can ask for recursion. Note that restricting access will still allow attackers with access to authorized hosts to exploit this vulnerability.

•  Juniper Security Toolbox

88

Filter Traffic at Network Perimeters

•  Because the ability to spoof IP addresses is necessary to conduct these attacks, administrators should filter spoofed addresses at the network perimeter. IETF Request for Comments (RFC) documents RFC 2827, RFC 3704, and RFC 3013 describe best current practices (BCPs) for implementing this defense. It is important to understand your network's configuration and service requirements before deciding what changes are appropriate.


89

Run a Local DNS Cache

•  In lieu of strong port randomization characteristics in a stub resolver, administrators can protect their systems by using local caching full-service resolvers, both on the client systems and on servers that are topologically close on the network to the client systems, in conjunction with the network segmentation and filtering strategies mentioned above. •  Juniper Security Toolbox

90

Disable Recursion

•  Disable recursion on any nameserver responding to DNS requests made by untrusted systems. Securing an Internet Name Server contains instructions for disabling recursion in ISC BIND.


91

NATs Breaking the Source Port Randomization “Patch”

92

Source Port Randomization

•  The UDP Source port is randomized – making it harder to guess and spoof DNS transactions.

Zone Administrator




Resolvers

Sources port chances randomly with every DNS query


Really hard to DNS Poison

93

How do Firewalls respond?

•  If the DNS server is source port randomizing, what is the firewall doing?

•  What if the DNS server was RFC 1918 addressing with the FW NATing?

Zone Administrator




Resolvers

Sources port is predictable or does not change with every DNS query


Really easy to DNS Poison

94

Checkpoint Observation (Full-Disclosure)

•  I've had a report from someone with clue (and tcpdump) that a properly functioning DNS resolver that correctly uses randomised source ports magically becomes vulnerable once the traffic's passed through a Checkpoint firewall, where Dan Kaminsky's tool shows:

•  x.y.z.155:56978 TXID=712 •  x.y.z.155:56979 TXID=45713 •  x.y.z.155:56980 TXID=63532 •  x.y.z.155:56981 TXID=7243 •  x.y.z.155:56982 TXID=17620

•  (note the incrementing port numbers.)

•  Can anyone else confirm this behaviour?

95

Typical problem for our customers.

•  Our customers are going to patch their DNS server. •  Many of our enterprise customers will have their

DNS traffic pass through our firewalls (i.e. start with a ScreenOS box).

•  What will they see when they test their DNS Server with Dan Kamisky’s DNS Checker? –  http://www.doxpara.com/

Corporate NetScreen Firewall

Patched DNS Server Dan Kamisky’s DNS Check Tool

Sources port chances randomly with every DNS query

What does Dan’s Tool see from our ScreenOS box?

96

Test Setup

•  Goal: Simulate a bulk list of enterprise customers. –  Find out what is seen in default modes (NAT and non-

NAT) –  Look for any BCPs for the ScreenOS config that would

make it more effective.

Corporate NetScreen Firewall

Patched DNS Server Dan Kamisky’s DNS Check Tool

Bind or Microsoft DNS Server set up for recursive lookups. Typical Enterprise

Config as baseline

Nothing between our FW and Dan’s tool but routers.

PC using Recursive Server

97

Two “DNS Checkers” available

•  Dan Kamiski’s Tool: –  http://www.doxpara.com/

•  OARC’s Tool (https://www.dns-oarc.net/)

–  Use a DNS query tool such as dig to ask for the TXT record of porttest.dns-oarc.net:

•  $ dig +short porttest.dns-oarc.net TXT

–  You should get back an answer that looks like this: –  z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.pt.dns-oarc.net.

"169.254.0.1 is FAIR: 26 queries in 0.1 seconds from 25 ports with std dev 3843.00"

–  Your resolver's randomness will be rated either GOOD, FAIR, or POOR, based on the standard deviation of observed source ports. In order to receive a GOOD rating, the standard deviation must be at least 10,000. For FAIR it must be at least 3,000. Anything less is POOR. The best standard deviation you can expect to see from 26 queries is in the 18,000-20,000 range.

–  DNS records used in this test are given 60 second TTLs. To repeat the test you should wait at least 60 seconds.

–  Note that you can tell dig to test a specific resolver with an @-argument: –  $ dig @4.2.2.3 +short porttest.dns-oarc.net TXT

98

NetScreen “NAT” Modes

•  Dynamic IP Pool (DIP) – The translated address can come from a Dynamic IP (DIP) pool or from the egress interface of the security device. Default is to use Port Address Translation (PAT). PAT can be turned off.

•  Mapped Internet Protocol (MIP) a direct one-to-one mapping of one IP address to another. The security device forwards incoming traffic destined for a MIP to the host with the address to which the MIP points. Source Port matches the host.

99

DIP Mode

•  For custom applications that require a specific source port number to operate properly, performing PAT causes such applications to fail. To provide for such cases, you can disable PAT.

100

DIP Mode with PAT Passes Test

With realistic traffic and long up time, the DIP port mapping would not be predicable, which comply with the implementation PAT’s management algorithm.

Desktop PC executing Doxpara checker scripts

10.208.64.25

10.208.0.10 DNS cache server without patch

Egress interface, policy nat src with out fix port SSG550M, build: 5.4r8

trust

Untrust : 219.142.67.130

Reclusive query Public network, Doxpara domain server

101

Is PAT Random?

•  Based off lab test, the Doxpapra checker reported “safe” even the DNS server has not been patched, because the Firewall did the source port mapping in a non-predictable way.

•  With a clean firewall config, no existing sessions, and

just at boot up time, the source port allocation will be in a sequential range and it’s very easy to observer the source port number changed with incrementing 1 each time.

•  Very quickly, with traffic passing through and PAT’s allocation algorithm, the source port will change in a non-predictable pattern. PAT is not “random,” but it enough for the Doxpara checker deem it as ‘random’ – building resistance to an attack that needs a predictable source.

102

Why does DIP with PAT look “Random”

•  RBTree is used for efficient state management of the NetScreen: –  http://en.wikipedia.org/wiki/

Red-black_tree •  Once traffic is passed through,

the RBTree jumps around the port range.

•  While RBTree is not “Random,” traffic use has it “no likely” to be predictable

103

How do Firewalls respond? •  If the DNS server is source port randomizing,

screenOS will respond like below

Zone Administrator

Zone Files

Resolvers Win2K Server with patch

IXIA Load Simulator

192.168.54.99

Clients queried domain 192.168.54.145 port1.com 192.168.54.146 port2.com …… …… 192.168.54.245 port100.com

DNS Request

172.27.128.11

Eth0/2 192.168.54.140

Eth0/4 10.208.73.8

Use case 1: Using PAT disable at reboot time set interface ethernet0/4 dip 100 10.208.73.9 10.208.73.9 fix-port

set policy id 8 from “Trust" to “DMZ" "Any-IPv4" "Any-IPv4" "ANY" nat src dip-id 100 permit log

Trust DMZ

104

How do Firewalls respond? •  If the DNS server is source port randomizing,

screenOS will respond like below

Zone Administrator

Zone Files

Resolvers Win2K Server with patch

IXIA Load Simulator

192.168.54.99

Clients queried domain 192.168.54.145 port1.com 192.168.54.146 port2.com …… …… 192.168.54.245 port100.com

DNS Request

172.27.128.11

Eth0/2 192.168.54.140

Eth0/4 10.208.73.8

Use case 2: Using PAT enable at reboot time set interface ethernet0/4 dip 100 10.208.73.9 10.208.73.9 set policy id 8 from “Trust" to “DMZ" "Any-IPv4" "Any-IPv4" "ANY" nat src dip-id 100 permit log

Trust DMZ

105

DNS Random Source Port Path w/ NetScreen

•  DIP Mode in default Port Address Translation (PAT) mode requires traffic for the NAT to move from sequential to a non-predictable pattern.

•  DIP Mode with PAT turned off will use the random source ports of the DNS Server.

•  MIP mode will use the random source ports of the DNS Server.

106

Turning off PAT

•  When you define a DIP pool, the security device enables PAT by default.

•  To disable PAT, you must add the key word fix-port to the end of the CLI command, or clear the Port Translation option on the DIP configuration page in the WebUI.

•  For example, set interface ethernet3 dip 5 1.1.1.30 1.1.1.30 fix-port,

•  or •  Network >Interfaces > Edit (for ethernet3) > DIP: ID:

5; Start: 1.1.1.30; End: 1.1.1.30; Port Translation: (clear).

•  fix-port Keeps the original source port number in the packet header. Does not apply Port Address Translation (PAT).

107

Turning off PAT

•  What would happen in the worse case? –  Two resolvers both at for the same DNS query at the

same time? –  The two resolvers go to two different recursive caching

servers. –  The two resolvers which both do random source port

allocation, just happens to issue the same port number. –  The DNS queries both get to the NetScreen at the same

time.

–  What happens inside the NetScreen?

108

Normal Mode with DIP and no PAT

“Patched” DNS recursive caches

NetScreen using DIP with PAT Turned Off

Multiple servers doing multiple request all with random source ports

With PAT turned off, the source port is kept for the NAT session.

10.208.0.10 src 14001

172.14.0.6 src 14001

Authority for juniper.net

Get A Record

A Record

109

Concern - Normal Mode with DIP and no PAT

“Patched” DNS recursive caches

NetScreen using DIP with PAT Turned Off

What happens when you do get a port collision? Two Computers – both asking the same question – to the same authoritative DNS Zone – for the same piece of information – arrives at the same time.

10.208.0.10 src 14001

172.14.0.6 src 14001

Authority for juniper.net

Get A Record

172.14.0.1 src 14001 Get A Record

DNS’s Recursive Cache will retransmit on the same port after 1 – 3 seconds

110

How the Cyber-Criminal Might Use this Vulnerability DNS Poison – The BOT Version

111

My Tool Kit

Drive-By Secondary Malware

SPAM BOTNET Controller Proxy

Packer

Malware

Victim of Crime


Poison Engine

BOT Herder

112

Prepare Drive-by



Packer

Malware

Send Malware

Load Malware

Victim of Crime


Poison Engine

BOT Herder

113

Send SPAM to get People To Click



Packer

Malware

Victim of Crime


Poison Engine

Send SPAM

Click on me now

BOT Herder

114

Drive By Violation



Packer

Malware

Victim of Crime


Poison Engine

Click on me now

BOT Herder

115

Poison Checker



Packer

Malware

Victim of Crime


Poison Engine

Redirect to new domain

Use “Published” DNS “Check” Tools to Test a Poison Candidate BOT

Herder

116

Prepare Violated Computer



Packer

Malware

Victim of Crime


Poison Engine

Call to Secondary Malware Site Load Secondary Package

Tell Malware Downloader to Push the Poison Tool BOT

Herder

117

Poison Test #2



Packer

Malware

Victim of Crime


Poison Engine



BOT Herder

118

Poison Test #2 - Validation



Packer

Malware

Victim of Crime


Poison Engine

Poison Tester NS

Malware Test to see if the Poison with new NS is working.

BOT Herder

119

Poison Victory!

•  The BOT Herder now has an asset which can be cultivated and sold.

•  The BOT Herder can sell BOT for some good money.

•  Why?

120

Using the Poison - WWW

Controller Proxy

Victim of Crime


Poison Engine

Miscreant Driving the BOTNET

Wert543.example.com Oihwoeif.example.com




My DNS Server


www.example.com

121

Using the Poison - WWW

Controller Proxy


Poison Engine


Where is www.example.com?

My DNS Server

www.example.com

www.example.com

Victims of Crime

Yea! I’ve control their view!


122

Using the Poison – WWW Proxy

Controller Proxy


Poison Engine


Where is www.example.com?

My DNS Server

www.example.com

www.example.com

Victims of Crime

Yea! Copy what I want – like CREDIT CARDs and PASSWORD!


123

Using the Poison – E-mail

Controller Proxy


Poison Engine


I need to E-mail smtp.example.com?

My DNS Server

smtp.example.com

smtp.example.com

Victim of Crime Yea! I’ve got copies!


124

Using the Poison – Routers

Controller Proxy


Poison Engine


I need to telnet to my router ams-23-pos23.example.com

My DNS Server

NOC Team

Yea! I’ve got router Passwords!


125

Using the Poison – Routers

Controller Proxy


Poison Engine


I need to send a SNMP Trap to my Network Management Tool to my smtp-noc—server1.example.com

My DNS Server

Router Services

Yea! I’ve got SNMP Details!


126

How the Cyber-Criminal Might Use this Vulnerability DNS Poison Drive By

127

DNS Poison – The Drive-By Version

•  You do not need malware/BOTs to activate this attack vector.

•  All you need to do is to “drive” the resolver to a new domain and force a DNS query that you know.

•  You then trigger a poison. •  Can you say … “HTTP Redirect?”

128

My Tool Kit

Drive-By SPAM BOTNET Proxy

Victim of Crime


Poison Engine

Miscreant Driving the Poison Attack

129

Send SPAM to get People To Click


Victim of Crime


Poison Engine

Send SPAM

Click on me now


130

Drive By Violation


Victim of Crime


Poison Engine

Click on me now


131

Poison Checker


Victim of Crime


Poison Engine

Redirect to domain you control

Use “Published” DNS “Check” Tools to Test a Poison Candidate


A potentially poisonable recursive server. Trigger the Poison Attack

132

Poison via Redirect

Drive-By Proxy

Victim of Crime


Poison Engine Poison

Attempt w/RR “Hint”


www.example.com

Redirect to erowij.example.com Test Redirect to 49u0vfv.example.com Test Redirect to 943ofvoiv.example.com Test


133

Poison via Redirect Testing

Drive-By Proxy

Victim of Crime


Poison Engine

Poison Tester NS

Testing after each redirect tells you when you have succeeded


www.example.com

Once Poisoned server goes to test NS, you can stop


134

Spotting when someone is trying to Poison Your DNS Identity

135

Backscatter – ICMP Port Unreachable

Controller Proxy

Victim of Crime


Poison Engine






My DNS Server


www.example.com

ICMP Port Unreachable Spoof ns.example.com

136

ICMP Unreachable & DNS

•  ICMP Unreachable – specific port unreachable – are not normal packets which arrive at: –  DNS Masters –  DNS Slaves –  DNS Split-Horizon Authoritative Servers

•  Live Observation –  Lauching the attack results packets arriving on closed

ports of the recursive DNS Server. –  This send ICMP Port Unreachable to the source packet –

which is the DNS Authority being spoofed.

137


•  This will tell you that someone somewhere is poising somewhere so that they can be a man in the middle between you and your customer!

•  How to monitor: –  Classification ACLs (match ingress on ICMP port

unreachable) –  Netflow –  IDP –  NetScreen (any matches on ICMP Unreachable

138

ACLs – How?

Controller Proxy

Victim of Crime


Poison Engine






My DNS Server


www.example.com ACL on Router with SNMP trap


139

JUNOS Example

•  JUNOS can syslog and a syslog watcher could then alert the operator. The example below also adds a counter and discards (rather than rejects) the packets. ps@phillip> show configuration firewall family inet { filter discard-icmp-unreachables { term discard-traffic { from { protocol icmp; icmp-code port-unreachable; } then { count icmp-port-unreachables; syslog; discard; } } term explicit-accept { then accept; } } }

140

Netflow

Controller Proxy

Victim of Crime


Poison Engine






My DNS Server


www.example.com Netflow Export


141

NetScreen IDP

Controller Proxy

Victim of Crime


Poison Engine






My DNS Server


www.example.com NetScreen IDP


142

Credits

•  Isaac Ghansah and John Mitchell, Stanford U •  John (Jenya) Neystadt, Security Test Lead, Microsoft Israel R&D •  APNIC Training Team

Phase 1 – Prepare the Tools and Techniques

Anycast as a Security Tool

144

Agenda

•  DNS Server Roles •  DNS Server Communications •  DNS Architecture Layout •  Types of Attacks •  Protecting the DNS •  Monitoring and Forensics •  Summary

145

Types of DNS Servers

146

Six Phases to ISP Security Incident Response

–  Preparation –  Identification –  Classification –  Traceback –  Reaction –  Post Mortem

147

Design Principles

•  Functional/Role Based Design purpose: an action or use for which something is suited or designed Its function is to collect water. role: an activity or role assigned to somebody or something

•  Modular Design involving modules: made up of separate modules that can be rearranged, replaced, combined, or interchanged easily - modular construction techniques - a modular course structure

148

DNS Server Configurations

•  Master (Primary) •  Slave (Secondary) •  Cache •  Forwarder •  Resolver •  Stealth

149


•  Zone Master (Primary) •  Zone Slave (Secondary) •  External Resolvers •  Internal Resolvers •  Aggregate Forwarders •  Caching Forwarders •  Resolvers (customers)

150

Topology

Resolvers



Resolvers (iRs)




Internally DNS


Internet Accessible

151


Resolvers



Resolvers (iRs)



152

What do you gain?

•  Each Security Zone can deploy policies to protect that zone. –  Permit only traffic that needs access. –  Deny all other traffic.

•  ACLs, Host Access List, and Firewalls can all be used.

•  Differentiate between BGP Anycast (for external access) and IGP Anycast (for internal access).

153

IP Network Planes of Operation

User/Data Plane: The data plane receives, processes, and transmits network data between network elements, and represents the bulk of network traffic that passes to and through the router

Control Plane: The glue of the network. The control plane is where all routing control information is exchanged, making the control plane and its components a target. Because control plane resiliency depends on CPU processing power and scalability, "out-of-resources" attacks against the CPU are not uncommon.

Management Plane: The management plane is the logical path of all traffic related to the system management of the routing platform. In a distributed and modular environment, the management plane offers new levels of complexity, and hence, increased requirements to maintain secure access

Services Plane: Overlay “Layer 7” application flow built on the foundation of the other layers. Service insertion, application routing, application service flows and other flows separate, but dependent on the other layers.

Policy Plane: The business glue of the network. Rules execution, decision making, Identity Collection, Stores Session Identity/Credentials, Processes Command/Query requests, AAA, Service Manager, Manages/Caches Service Profiles, and all the other components to make a productize service.

154

Normal Expectations

Normal Policies Policy Plane

Service Plane

Management Plane

SSHv2 to all devices SNMPv3 to monitors

Control Plane IGP and BGP used for Anycast

User/Data Plane

DNS resolutions from customers. Only resolve customer DNS resolutions. Deny all others.

Policy Plane:

155

Roles

Threat Vectors Mapped to Plane

Resolvers Cashing

Forwarders

Aggregate Caching

Forwarders

Internal Resolvers

External Resolvers

Zone Slave

Zone Master

Distributed Denial of Service / Infrastructure

Break-ins/ Device takeover

Theft of Service/ Fraud mitigation

Reconnaissance

Threat Vectors

Data Plane Control Plane Management Plane Services Plane

Legend for Planes of Operation

Policy Plane

156


Resolvers



Resolvers (iRs)



157


Resolvers



Resolvers (iRs)



Anycast Realm


Agency Realm

158

Anycast Addressing to Build Symmetry

192.168.21.1 10.20.20.1

192.168.21.9 10.20.20.9

172.15.15.16 172.15.15.18

Request to Anycast Address

Response to Unicast Address

172.20.10.160

Resolver Resolver Zone Authority

Request to Anycast Address


Request to Unicast or Anycast Address


159

Anycast Addressing to Build Symmetry

Resolvers




GW

Internet

GW GW

Anycast Realm

192.168.21.1 10.20.20.1

192.168.21.1 10.20.20.1

192.168.21.1 10.20.20.1

192.168.21.9 10.20.20.9

172.15.15.19

172.15.15.15 172.15.15.16 172.15.15.17

172.15.15.18

192.168.21.20 10.20.20.90

192.168.21.20 10.20.20.90

192.168.21.20 10.20.20.90

171.68.10.70

171.70.10.70

171.80.10.70

160

Impact of Geography

Resolvers




GW Asia

Internet

GW NA

GW Europe

NA Anycast Realm

192.168.21.1 10.20.20.1

192.168.21.1 10.20.20.1 192.168.21.1

10.20.20.1

192.168.21.9 10.20.20.9

172.15.15.19

172.15.15.15 172.15.15.16 172.15.15.17

172.15.15.18

192.168.21.20 10.20.20.90

192.168.21.20 10.20.20.90

192.168.21.20 10.20.20.90

171.68.10.70

171.70.10.70

171.80.10.70

Europe Anycast Realm Asia Anycast Realm

161

Suggestion – Resolver Overlay

Resolvers




GW

Internet

GW GW

Anycast Realm

162

Suggestion – Resolver Overlay

Resolvers




GW

Internet

GW GW

Anycast Realm

Anycast and Security

165

Today’s Discussion

•  What we’ll be discussing - IPv4 Anycast

•  What we won’t be discussing - IPv6 Anycast

166

What is IPv4 Anycast?

IPv4 Anycast is simply an addressing technique which specifies the advertisement of non-unique IP addresses from multiple points of origin for the purpose of providing high availability, survivability, and/or a rough form of traffic/services load-balancing based upon route selection. It’s been in use for more than a decade!

167

Anycast DNS Caches

Peer B

Peer A IXP-W

IXP-E

Upstream A

Upstream A

Upstream B Upstream

B

POP

Customer

Primary DNS Servers

Sink Hole Network

171.68.19.0/24

171.68.19.1 DNS Caching Server Cluster

SAFE - Architecture

DNS Caching Server Cluster



DNS Secondary Server Cluster



168


•  IPv4’s Anycast technique can be used as a security tool. –  Provides topological separation. Making it harder to

attack a service (DNS, AAA, etc). –  Topological separation provides a means to put sink

holes through out the network. –  Two devices looking like one offers a way to have

customer iBGP origination points to be two routers vs one without the added IGP memory consumption.

169

What isn’t Anycast?

•  Not a protocol, not a different version of IP, nobody’s proprietary technology.

•  Doesn’t require any special capabilities in the servers, clients, or network.

•  Doesn’t break or confuse existing infrastructure.

Courtesy of Bill Woodcock Packet Clearing House (www.pch..net)

170

What is Anycast?

•  Just a configuration methodology.

•  Mentioned, although not described in detail, in numerous RFCs since time immemorial.

•  It’s been the basis for large-scale content-distribution networks since at least 1995.

•  It’s gradually taking over the core of the DNS infrastructure, as well as much of the periphery of the world wide web.


171

Really? That’s it?!

Yes - most of the mystique surrounding IPv4 Anycast is merely a function of the name. Unicast, Multicast, Anycast . . . In reality, IPv4 Anycast is just a form of ‘shared unicast’! It’s simple, requires no special software, hardware, etc. - just config-tweaking.

172

Benefits of distributing services via IPv4 Anycast.

Survivability - worms, DDoS, backhoes Load-balancing - spread services load across servers, links, etc. Availability - maintenance, upgrades, patching, hardware failures Performance/latency - bring services closer to clients

173

How Does Anycast Work?

•  The basic idea is extremely simple:

•  Multiple instances of a service share the same IP address.

•  The routing infrastructure directs any packet to the topologically nearest instance of the service.

•  What little complexity exists is in the optional details.


174

Example

Client

Server Instance A

Server Instance B

Router 1

Router 3

Router 2

Router 4


175

Example

Client

Server Instance A

Server Instance B

Router 1

Router 3

Router 2

Router 4

10.0.0.1"

10.0.0.1"

192.168.0.1"

192.168.0.2"


176

Client Router 1

Example

Server Instance A

Server Instance B Router 3

Router 2

Router 4

10.0.0.1"

10.0.0.1"

192.168.0.1"

192.168.0.2"

DNS lookup for http://www.server.com/"produces a single answer:""www.server.com. IN A 10.0.0.1"


177

Router 1

Example

Client

Server Instance A


Router 2

Router 4

10.0.0.1"

10.0.0.1"

192.168.0.1"

192.168.0.2"

Routing Table from Router 1:""Destination "Mask "Next-Hop "Distance"192.168.0.0"/29 "127.0.0.1 "0"10.0.0.1 "/32 "192.168.0.1 "1"10.0.0.1 "/32 "192.168.0.2 "2"


178

Router 1

Example

Client

Server Instance A


Router 2

Router 4

10.0.0.1"

10.0.0.1"

192.168.0.1"

192.168.0.2"



179

Router 1

Example

Client

Server Instance A


Router 2

Router 4

10.0.0.1"

10.0.0.1"

192.168.0.1"

192.168.0.2"



180

Router 1

Example

Client Server

Router 3

Router 2

Router 4

10.0.0.1"

192.168.0.1"

192.168.0.2"


What the routers think the topology looks like:"


181

An example of IPv4 Anycast

rdobbins@anabasis:~$ host f.root-servers.net

f.root-servers.net has address 192.5.5.241

182

An example of IPv4 Anycast

route-views.oregon-ix.net>sh ip bgp 192.5.5.241 BGP routing table entry for 192.5.5.0/24, version 3783472 Paths: (51 available, best #27, table Default-IP-Routing-Table) Not advertised to any peer 2914 3557 3557 3557 129.250.0.85 from 129.250.0.85 (129.250.0.85) Origin IGP, metric 61, localpref 100, valid, external Community: 2914:410 2914:2000 2914:3000 11537 6509 2884 25689 30123 3557, (aggregated by 30123 192.228.81.16) 198.32.8.196 from 198.32.8.196 (198.32.8.196) Origin IGP, metric 260, localpref 100, valid, external, atomic-aggregate Community: 11537:2501 10764 6509 2884 25689 30123 3557, (aggregated by 30123 192.228.81.16) 206.220.240.95 from 206.220.240.95 (206.220.240.95) Origin IGP, localpref 100, valid, external, atomic-aggregate 267 2914 3557 3557 3557 204.42.253.253 from 204.42.253.253 (204.42.253.253) Origin IGP, metric 0, localpref 100, valid, external Community: 267:2914 2914:410 2914:2000 2914:3000

And so on . . . .

183

What’s required to implement IPv4 Anycast?

•  A suitable service you wish to distribute. •  A couple of routers. •  A couple of peers. •  A couple of servers. •  Provider-independent address space. •  A bit of planning and configuration. •  And that’s it!

184

Checklist for IPv4 Anycasted DNS

  Servers running properly-configured BIND or other DNS of your choice

  Zebra, Quagga, other server-side routing daemon (not strictly a requirement, but recommended; static /32 routes plus downed server = queries dropped on the floor)

  Configuration of additional loopback(s) on servers for IPv4 Anycast addresses (same reasons we use loopbacks on routers)

  Injection of /32 routes from server-side routing daemon into IGP   Adjustments to egress filtering/uRPF to allow ‘spoofed’ responses

from servers (responses sourced from IPv4 Anycast address)   Consistent origin AS for IPv4 Anycast address block(s)   A way to ensure that the DNS service itself is up, not just the host   Distributed monitoring for distributed services

You’re done!

185

Core

Edge

Dist

Anycast IP - lo1 192.0.2.10 Admin IP - eth0 172.19.61.254

Simple IPv4 Anycast DNS topology

Peer A

192.0.2.0/24



Peer B

192.0.2.0/24 Peer C

192.0.2.0/24 Peer D

192.0.2.0/24

186

Building an Anycast Server Cluster

•  Anycast can be used in building either local server clusters, or global networks, or global networks of clusters, combining both scales.

•  F-root is a local anycast server cluster, for instance.


187

Building an Anycast Server Cluster

•  Typically, a cluster of servers share a common virtual interface attached to their loopback devices, and speak an IGP routing protocol to an adjacent BGP-speaking border router.

•  The servers may or may not share identical content.


188

Example

Router"

Eth0"192.168.1.2/30" Lo0"

10.0.0.1/32"

Eth0"192.168.2.2/30"

Eth0"192.168.3.2/30"

Lo0"10.0.0.1/32"

Lo0"10.0.0.1/32"

Server Instance A"

Server Instance B"

Server Instance C"

BGP" IGP"Redistribution"


189

Router"

Example

Eth0"192.168.1.2/30" Lo0"

10.0.0.1/32"

Eth0"192.168.2.2/30"

Eth0"192.168.3.2/30"

Lo0"10.0.0.1/32"

Lo0"10.0.0.1/32"

Server Instance A"

Server Instance B"

Server Instance C"


Destination "Mask "Next-Hop "Dist"0.0.0.0 "/0 "127.0.0.1 "0"192.168.1.0"/30 "192.168.1.1 "0"192.168.2.0"/30 "192.168.2.1 "0"192.168.3.0"/30 "192.168.3.1 "0"10.0.0.1 "/32 "192.168.1.2 "1"10.0.0.1 "/32 "192.168.2.2 "1"10.0.0.1 "/32 "192.168.3.2 "1"


190

Router"

Example

Eth0"192.168.1.2/30" Lo0"

10.0.0.1/32"

Eth0"192.168.2.2/30"

Eth0"192.168.3.2/30"

Lo0"10.0.0.1/32"

Lo0"10.0.0.1/32"

Server Instance A"

Server Instance B"

Server Instance C"


Destination "Mask "Next-Hop "Dist"0.0.0.0 "/0 "127.0.0.1 "0"192.168.1.0"/30 "192.168.1.1 "0"192.168.2.0"/30 "192.168.2.1 "0"192.168.3.0"/30 "192.168.3.1 "0"10.0.0.1 "/32 "192.168.1.2 "1"10.0.0.1 "/32 "192.168.2.2 "1"10.0.0.1 "/32 "192.168.3.2 "1"

Round-robin load balancing"


191

Building a Global Network of Clusters

•  Once a cluster architecture has been established, additional clusters can be added to gain performance.

•  Load distribution, fail-over between clusters, and content synchronization become the principal engineering concerns.


192

Example

Router 2"

Serv

er In

stan

ce D"

Serv

er In

stan

ce E"

Serv

er In

stan

ce F"


193

Example

Router 2"

Serv

er In

stan

ce D"

Serv

er In

stan

ce E"

Serv

er In

stan

ce F"

Region 1"

Region 2"

Region 3"


194

Example

Router 2"

Serv

er In

stan

ce D"

Serv

er In

stan

ce E"

Serv

er In

stan

ce F"

BGP Announcements"

10.0.0.1 /32"192.168.0.0 /22"192.168.0.0 /16"

10.0.0.1 /32"192.168.8.0 /22"192.168.0.0 /16"

10.0.0.1 /32"192.168.4.0 /22"192.168.0.0 /16"


195

Example

Router 2"

Serv

er In

stan

ce D"

Serv

er In

stan

ce E"

Serv

er In

stan

ce F"

IGP 1 Announcements"

10.0.0.1 /32"10.0.0.1 /32"10.0.0.1 /32"

192.168.1.0 /30"192.168.2.0 /30"192.168.3.0 /30"

10.0.0.1 /32"10.0.0.1 /32"10.0.0.1 /32"

192.168.9.0 /30"192.168.10.0 /30"192.168.11.0 /30"

10.0.0.1 /32"10.0.0.1 /32"10.0.0.1 /32"

192.168.5.0 /30"192.168.6.0 /30"192.168.7.0 /30"


196

Example

Router 2"

Serv

er In

stan

ce D"

Serv

er In

stan

ce E"

Serv

er In

stan

ce F"

IGP 2 Announcements"

10.0.0.1 /32"192.168.1.0 /30"192.168.2.0 /30"192.168.3.0 /30"

10.0.0.1 /32"192.168.9.0 /30"

192.168.10.0 /30"192.168.11.0 /30"

10.0.0.1 /32"192.168.5.0 /30"192.168.6.0 /30"192.168.7.0 /30"


197

Performance-Tuning Anycast Networks

•  Server deployment in anycast networks is always a tradeoff between absolute cost and efficiency.

•  The network will perform best if servers are widely distributed, with higher density in and surrounding high demand areas.

•  Lower initial cost sometimes leads implementers to compromise by deploying more servers in existing locations, which is less efficient.


198

Example

Geographic plot of user population density"


199

Example


Server deployment"


200

Example


Server deployment"Traffic Flow"


201

Example




202

Example




203

Example




204

Example

Drawing traffic growth away from a hot-spot"


205

Example



206

Example



207

Example



208

Example



209

Example


Topological watershed"


210

Example



211

Caveats and Failure Modes

•  DNS resolution fail-over

•  Long-lived connection-oriented flows

•  Identifying which server is giving an end-user trouble


212

DNS Resolution Fail-Over

•  In the event of poor performance from a server, DNS servers will fail over to the next server in a list.

•  If both servers are in fact hosted in the same anycast cloud, the resolver will wind up talking to the same instance again.

•  Best practices for anycast DNS server operations indicate a need for two separate overlapping clouds of anycast servers.


213

Long-Lived Connection-Oriented Flows

•  Long-lived flows, typically TCP file-transfers or interactive logins, may occasionally be more stable than the underlying Internet topology.

•  If the underlying topology changes sufficiently during the life of an individual flow, packets could be redirected to a different server instance, which would not have proper TCP state, and would reset the connection.

•  This is not a problem with web servers unless they’re maintaining stateful per-session information about end-users, rather than embedding it in URLs or cookies.

•  Web servers HTTP redirect to their unique address whenever they need to enter a stateful mode.

•  Limited operational data shows underlying instability to be on the order of one flow per ten thousand per hour of duration.


214

Identifying Problematic Server Instances

•  Some protocols may not include an easy in-band method of identifying the server which persists beyond the duration of the connection.

•  Traceroute always identifies the current server instance, but end-users may not even have traceroute.


215

A Security Ramification

•  Anycast server clouds have the useful property of sinking DOS attacks at the instance nearest to the source of the attack, leaving all other instances unaffected.

•  This is still of some utility even when DOS sources are widely distributed.


216

Bill Woodcock [email protected]

www.pch.net/documents/tutorials/anycast



Example Applications

218

Anycast and Security: Applications

•  DNS Services •  Distributed Sink Holes •  Dark IP Space Management (BGP Lock-up static

routes to Null0)

219

DNS & Anycast

•  Problem #1 – How to manage the load on those two DNS entries in customer’s TCP/IP Stack?

•  Problem #2 – How to manage saturation attacks targeted at your DNS infrastructure?

•  Answer – Anycast the DNS Caching Servers.

220

Anycast DNS Caches

Peer B

Peer A IXP-W

IXP-E

Upstream A

Upstream A

Upstream B Upstream

B

POP

Customer

Primary DNS Servers

Sink Hole Network

171.68.19.0/24


SAFE - Architecture







221

Anycast DNS Caches

Peer B

Peer A IXP-W

IXP-E

Upstream A

Upstream A

Upstream B Upstream

B

POP

Customer

Primary DNS Servers

Sink Hole Network

171.68.19.0/24


SAFE - Architecture







DNS Forwarded to the closed Caching

Cluster

222

DNS Anycast – What is needed?

•  Two IP Addresses to be used for the DNS Caching clusters.

•  Router to perform the load balancing and advertise the two IP addresses.

223

Anycast and Sink Holes

•  Sink Holes are designed to pull in attacks. •  Placement in the network requires mindful

integration. •  One Sink Hole might require major re-

architecting of the network. •  Anycast Sink Holes provide a means to distribute

the load through out the network.

224

Anycast Sink Holes Example

Core Backbone

Regional Node

Regional Node

Regional Node

Regional Node

Regional Node

Regional Node

Template Backbone with Regional Centers

ISPs ISPs ISPs

POPs

POPs

POPs

POPs

POPs

POPs

225

Anycast Sink Hole Placement

Core Backbone

Regional Node

Regional Node

Regional Node

Regional Node

Regional Node

Regional Node

Place Sink Holes in each of the Regional Nodes

ISPs ISPs ISPs

POPs

POPs

POPs

POPs

POPs

POPs

226

Anycast Sink Holes

•  Anycast Sink Holes are in their early stages. •  Placement and control of the trigger routers are

the two interesting challenges. •  These challenges will dissolve as more

operational experience is gained.

227

Relevant RFCs

•  RFC1546: Host Anycasting Service •  RFC2101: IPv4 Address Behavior Today •  RFC2181: Clarifications to DNS •  RFC2780: IANA Allocation Guidelines for IP •  RFC2893: Transition Mechanisms for IPv6 Hosts and Routers •  RFC2902: Overview of the 1998 IAB Routing Workshop •  RFC3068: An Anycast Prefix for 6to4 Relay Routers •  RFC3258: Distributing Authoritative Name Servers via Shared Unicast Addresses •  RFC3446: Anycast RP mechanism using PIM and MSDP

228

More Information

•  Kuro5hin.org -http://www.kuro5hin.org/story/2003/12/31/173152/86

•  Kevin Miller, CMU -http://www.net.cmu.edu/pres/anycast/

•  ISC - http://www.isc.org/pubs/tn/isc-tn-2003-1.html

DNS Architecture Idea: Modularization & Compartmentalization · 2019-06-18 · Attacks Interna...

Documents

Transcript of DNS Architecture Idea: Modularization & Compartmentalization · 2019-06-18 · Attacks Interna...