120 slides Mapping the Internet and Intranets Steve Branigan, Hal Burch, Bill Cheswick...

120 slides

Mapping the Internet and

Intranets

Steve Branigan, Hal Burch, Bill Cheswick

[email protected]

Mapping the Internet and intranets slide 2 of 120

Motivations

• Intranets are out of control– Always have been

• Highlands “day after” scenario

• Panix DOS attacks– a way to trace

anonymous packets back!

• Internet tomography

• Curiosity about size and growth of the Internet

• Same tools are useful for understanding any large network, including intranets


The Original Project

• Long term reliable collection of Internet and Lucent connectivity information– without annoying

too many people

• Attempt some simple visualizations of the data

– movie of Internet growth!

• Develop tools to probe intranets

• Extended database for researchers


Uses for the Internet data

• topography studies

• long-term routing studies

• publicly available database– (“open source”) for spooks

• interesting database for graph theorists

• combine with other mappers to make an actual map of the Internet


History of the Project

• Started in August 1998 at Bell Labs

• April-June 1999: Yugoslavia mapping

• July 2000: first customer intranet scanned

• Sept. 2000: spun off Lumeta from Lucent/Bell Labs

• June 2002: “B” round funding completed


Related Work

• See Martin Dodge’s cyber geography page

• MIDS - John Quarterman

• CAIDA - kc claffy

• Mercator

• Enter “internet map” in your search engine


Methods - data collection

• Single reliable host connected at the company perimeter

• Daily full scan of Lucent

• Daily partial scan of Internet, monthly full scan

• One line of text per network scanned– Unix tools


Methods - network scanning

• Obtain master network list– network lists from Merit, RIPE, APNIC, etc.– BGP data or routing data from customers– hand-assembled list of Yugoslavia/Bosnia

• Run a traceroute-style scan towards each network

• Stop on error, completion, no data– Keep the natives happy


Daily database

• 100-200MB of text

• compresses to 5-10MB

• daily Internet results available from mapping web page– have not checked to see who gets it!

• Saved to different partition, and offloaded to other secure computer


Traceroute

• Probes toward each target network with increasing TTL

• Probes are ICMP, UDP, TCP to port 80, 25, 139, etc.

• Some people block UDP, others ICMP


Traceroute

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

Application level

TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3Hop 4


Send a packet with a TTL of 1…

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

Application level

TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3Hop 4


…and we get the death notice from the first hop

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

Application level

TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3Hop 4


Send a packet with a TTL of 2…

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

Application level

TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3Hop 4


… and so on …

Application level

TCP/UDP

IP

Hardware

Client

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

IP

Hardware

Router

Application level

TCP/UDP

IP

Hardware

Server

Hop 1 Hop 2 Hop 3

Hop 3Hop 4


Advantages

• We don’t need access (I.e. SNMP) to the routers

• It’s very fast

• Standard Internet tool: it doesn’t break things

• Insignificant load on the routers

• Not likely to show up on IDS reports

• We can probe with many packet types


Limitations

• Outgoing paths only

• View is from scanning host only

• Takes a while to collect alternating paths

• Gentle mapping means missed endpoints

• Imputes non-existent links


The data can go either way

A

E F

D

B C


But our test packets only go part of the way

A

E F

D

B C


We record the hop…

A

E F

D

B C


The next probe happens to go the other way

A

E F

D

B C


…and we record the other hop…

A

E F

D

B C


We’ve imputed a link that doesn’t exist

A

E F

D

B C


Remediations

• Alternate routes not a factor on intranets

• Scan from several sources

• “stitching” needed– Traceroute in different directions gives

different interface IP addresses– Techniques needed to link multiple IP

addresses to a single host machine


Network scanning

• Custom program

• Concurrently scans towards 500 nets at once

• Throttled to 400 packets/sec– 100 p/s over dialup modems!

• Slow daily scan for host on destination network


Data collection complaints

• Australian parliament was the first to complain

• List of whiners (25 nets)

• Military noticed immediately– Steve Northcutt– arrangements/warnings to DISA and CERT


Visualization goals

• make a map– show interesting features– debug our database and collection

methods– hard to fold up

• geography doesn’t matter

• use colors to show further meaning


Colored byAS number


Map Coloring

• distance from test host

• IP address– shows communities

• Geographical (by TLD)

• ISPs

• future– timing, firewalls, LSRR blocks


Colored by IP address!


Colored by geography


Colored by ISP


Colored by distancefrom scanning host


US militaryreached by ICMP ping


US military networksreached by UDP

120 slides

Yugoslavia

An unclassified peek at a new battlefield

120 slides

Un film par Steve “Hollywood” Branigan...

120 slides

fin

120 slides

NYC after 9/11


CIDR and IP Counts

145K

150K

155K

160K

165K

170K

175K

180K

9/11 9/12 9/13 9/14 9/15 9/16 9/17 9/18 9/19 9/20 9/21 9/22

Date

Cou

nt

# Edges# CIDRs# IPs


Routers in New York City

1000

1100

1200

1300

1400

9/11 9/12 9/13 9/14 9/15 9/16 9/17 9/18 9/19 9/20 9/21 9/22

Date

# R

ou

ters


Internet before 9/11/2001


Internet after 9/11/2001

120 slides

Let’s look at some intranets

120 slides

Anything large enough to be called

an “intranet” isout of control


This is not the fault of network administrators!

• Robust internet design frustrates central control– Ad hoc growth– Mergers and acquisitions frustrate

long-term network planning and policies

• CIOs and auditors already know this


We call these “routing leaks”

• Easily-found holes in the intranet perimeter

• Show up nicely on the maps

• Leaking hosts or routers announce routes to other networks or the Internet

• Sometimes left over from an old corporate split

• Non-functional VPNs can show up


This wasSupposedTo be aVPN


The maps are useful, but not the main data

• We collect tens of megabytes of network data

• There were unexpected subtleties to this process

• How do you display all this information, given that different clients want different data?


The second technology: host leak detection

• Developed to find hosts that have access to both intranet and Internet

• Or across any privilege boundary

• Leaking hosts do not route between the networks

• May be a dual-homed host

• Not always a bad thing

• Technology didn’t exist to find these


Possible host leaks

• Miss-configured telecommuters connecting remotely

• VPNs that are broken

• DMZ hosts with too much access

• Business partner networks

• Internet connections by rogue managers

• Modem links to ISPs


Leak results

• Found home web businesses

• At least two clients have tapped leaks– One made front page news


Leak Detection Prerequisites

• List of potential leakers: obtained by census

• Access to intranet

• Simultaneous availability of a “mitt”


Leak Detection Layout

Internet intranet

Mapping hostA

Test hostB

mittD

C

• Mapping host with address A is connected to the intranet

• Mitt with address D has Internet access

• Mapping host and mitt are currently the same host, with two interfaces


Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• Test host has known address B on the intranet

• It was found via census

• We are testing for unauthorized access to the Internet, possibly through a different address, C


Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• A sends packet to B, with spoofed return address of D

• If B can, it will reply to D with a response, possibly through a different interface


Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• Packet must be crafted so the response won’t be permitted through the firewall

• A variety of packet types and responses are used

• Either inside or outside address may be discovered

• Packet is labeled so we know where it came from


Leak Detection

Internet intranet

Mapping hostA

Test hostB

mittD

C

• This describes outbound leaks

• Inbound leaks are usually much more serious


Possible problems

• NAT

• egress filtering

• transit of sensitive data over the public networks


Our new tools give new views of intranets

• The pictures are mostly for management

• Maps can show progress– red is bad, blue is good– we can color the maps in many ways

• The real value in the reports is the list of anomalies– network leaks, routing loops, open

routers, etc.


How we scan

• Via dialup, using RAS servers

• Secure tunnel, if you prefer– IP/SEC– PPTP– others?


What we do

• Probe the network for things not in the official list

• Run a host enumeration

• Run leak tests on each host found


Technology used

• Traceroute

• SNMP queries– Router type– Routing tables

• Pings

• Special leak detection probes– ICMP– UDP– Other possible if requested


Report

• HTML-based

• Delivered on CDROM or DVD

• Maps

• Executive summary shows highpoints

• Interactive map viewer tool for Windows


Competitors?

• Not yet, not quite– Many use the same terms, but offer

different services

• Some components are pretty easy and free– Host enumeration– But we do it better (!)

• A bit like HP OpenView– OpenView doesn’t scale– Much slower


Value

• Discovers unknown parts of the network

• Data feeds into existing tools, enhancing their value

• You can’t secure what you don’t know about

• Due diligence for intranets– Insurance?

• M&A activity

• Personnel turnover leaves legacy connections

• Business partners


Getting a report

• Web-based

• We can send you a CD-ROM

• You can access a web server– FreeBSD-based– One-time password authentication– Very paranoid server

120 slides

Sample report

120 slides

Internet report

120 slides

Intranet “Best current practices”

We are acquiring the data to produce a paper: statistics over a

variety of large intranets


Some intranet statisticsfrom Lumeta clients

Intranet sizes (devices) 7,900 365,000Corporate address space 81,000 745,000,000Address space usage efficiency% devices in unknown address space 0.01% 20.86%

% routers responding to "public" 0.14% 75.50%% routers responding to other 0.00% 52.00%

Outbound host leaks on network 0 176,000% devices with outbound ICMP leaks 0% 79%% devices with outbound UDP leaks 0% 82%

Inbound UDP host leaks 0 5,800% devices with inbound ICMP leaks 0% 11%% devices with inbound UDP leaks 0% 12%

% hosts running Windows 36% 84%

120 slides

Mapping the Internet and

Intranets

Steve Branigan, Hal Burch, Bill Cheswick

[email protected]

120 slides Mapping the Internet and Intranets Steve Branigan, Hal Burch, Bill Cheswick...

Documents

Transcript of 120 slides Mapping the Internet and Intranets Steve Branigan, Hal Burch, Bill Cheswick...