“There and back again”How The Internet Works

Photo: http://www.flickr.com/photos/86530412@N02/8210762750/ by StockMonkeys.com

A talk by Jon “The Nice Guy” SpriggsFirst given at PHPNW December 2012

It's simple, right?

It's all about perspective

● The previous slide was right “from a certain point of view”

● But it left out a lot of complicated bits● Things like:

– What does your router do?

– How does your ISP reach your server?

– What about the network where the server is?

● So, let's drill down a bit

Much closer, but still not quite right

Getting across it all...

● Many of you will already refer to all your connections as being TCP/IP connections– They're not all TCP/IP, some are UDP/IP, or ICMP/IP, or just, IP, or

ARP, or GRE/IP, or IPSEC/IP or … well, lots of things.

– And that doesn't get us anywhere near the actual application protocol

● It's actually explained through a few different models– Some refer to it as the OSI Model (ISO/IEC 7498-1)

– Some as the Internet Protocol Suite (RFC1112)

● But, the way you get from host to host, or host to network, is by piling a few different things on top of each other

● So, let's look at how that is supposed to work

Let's start with getting on your network. It's harder than it looks!

● We'll assume it's a CAT5e wired network with DHCP!● Plug in both ends of the ethernet cable and provide power to the NIC● Ethernet link (power & comms) detected, speed and duplex (optional), plus media type

(optional++) is negotiated● When using a switch, it learns the MAC address of the devices behind each port, and

only sends packets for that MAC address to that port. Hubs used to be much more common, because they wouldn't learn MAC addresses, and would broadcast the traffic across all it's ports. An attacker could just attach to a hub and see all traffic, but with switches they must convince the network they have the MAC of the router.

● Computer requests DHCP address using it's MAC address and the DHCP server replies with an address, netmask, (optional) default route and a lease time.

● When the computer tries to connect to an IP address, it uses it's routing table. If the address is “directly connected” on the same subnet, it requests the MAC address of the IP address, otherwise it requests the MAC address of the gateway.

● The computer communicates at “Layer 2” with the MAC address it learned, and the rest of the link is assumed to have worked*

● If the computer is resolving a DNS name, it's got to communicate with the DNS server to ask for the IP address of the server, so it can start the IP dance again.

WOW, wasn't that hard!

● And this stuff happens EVERY time you connect! To anything.

● It's a wonder anything on the internet EVER works!

● And in that explanation you've not even got past your LOCAL network.

● 99% of the time, you don't need to know about this stuff, but sometimes it helps when you've got some weird error log, or network issue to understand how this all works.

TCP and UDP

● TCP is considered reliable, as it ensures a conversation can occur by using a handshake to prove two-way connections.

– Used in HTTP, HTTPS, SMTP, LDAP, XMPP, FTP, SSH● UDP is a “fire and forget” protocol – the connection is not guaranteed,

which makes it a faster and efficient protocol, while not always reliable.

– Used in DNS, TFTP, Syslog, NTP, VPNs*● Some systems will use a combination of both TCP and UDP to perform

different roles, for example voice or video conferencing products will set up the link using TCP, then share media over UDP.

● DNS uses UDP for client queries, and TCP to exchange updates between authoritative nodes and replication targets.

● There are other L3 protocols, such as GRE or ESP which use neither TCP or UDP to communicate.

So how does this routing thing work then?

● Routes are defined in one of three ways– Statically assigned

● Have I been told which way to go?● A default gateway classes as this, as does anything added using route add

– Dynamically (e.g. BGP, EIGRP, RIP, etc.)● Has something else, which I trust, told me what networks live beyond it?

– Local (IP address and NetMask defined)● Does this address live in my subnet?

● A route has a “Metric” which defines the “cost” of using it, but there are rules, for example:– The metric is only relevant with equally specific networks, so, a route to the

network 10.8.0.0/24 is more specific than a route to the 10.8.0.0/16 network

– If we have two routes for 10.8.0.0/24, one with a Metric of 1, and the other with a Metric of 2, the Metric 1 “wins”, even if the Metric 1 gateway is down

– If we have two routes for 10.8.0.0/24, with the same Metric, but where the gateway for one is up, and the other isn't, the gateway which is up “wins”

– A local route can be overridden with a static route, but this gets messy FAST

WAN Accelerators

● A pair (or cluster) of WAN accelerators will tend to be deployed between your last router before the WAN router and the WAN router at either end of a high-volume or low capacity WAN link

● Each end builds a data dictionary which is exchanged with the other members and then sends just the dictionary entries

● Consider, much of IP traffic relates to the various packet headers, using a WAN accelerator can reduce the amount of traffic being sent over a known link, especially with high traffic targets (such as MS-AD servers or DNS)

Proxy

● Most of you will have seen/heard of these, particularly if you work for a company of any size above 2 or 3 employees.

● Usually deployed to broker a connection between you and a web server, a proxy will intercept the connection request to a remote server, do “stuff” with the content (cache it, filter it, strip stuff out, etc.) and then give you the content.

● It used to be very common before NAT was prevalent in network connections, letting several machines connect to the internet, appearing as one IP or service.

Reverse Proxy

● A reverse proxy exposes several services as one device sharing common ports

● Sometimes used to encrypt public traffic (HTTPS → HTTP), while permitting the private traffic to be intercepted and actions performed upon it (e.g. AV scans)

● Mostly seen with web servers, but sometimes mail, FTP or even several services on one port using a principal called multiplexing

● See also services such as pagekite which permits remote devices to share their web services with a public URL

Host Based Firewall

● Usually best examples of these are IPTables for Linux, Microsoft Firewall for Windows, PFSense for BSD.

● They prevent inbound connections where unexpected, and outbound connections where specified.

● Good for when you're on public wifi, 3g or raw internet. Useful if you've got a virus infection elsewhere in your local network.

Firewall (L2)

● A layer 2 firewall looks at the IP headers only (source IP and port, destination IP and port)

● It's called a layer 2 firewall, or sometimes a “bump in the wire”, because it's invisible to the devices either side of the firewall

● It will typically only have a management address, and will be connected between a switch and a router, or between a switch and a server.

Firewall (L3)

● Most common firewall deployment.● Inspect inbound and outbound connections

from a network, matching a white/black list.● A common place to perform NAT● This role, on a home network, is performed by

your cable modem or ADSL router, usually...

Firewall (L7)

● Otherwise known as an Application Firewall● This inspects traffic in known protocols (e.g.

HTTP, HTTPS, FTP, SMTP, etc.) and applies Accept/Deny/Drop rules to those protocols.

● It is usually considered to be slower than L3 firewalls (sometimes even 1/10th as fast)

● Frequently used to hand off AV scanning etc.● Usually deployed after an L3 Firewall

Intrusion Protection System

● Much like the L2 Firewalls, IPS devices tend to be “bump-in-the-wire” devices.

● They look for network anomalies● Typically, IPS systems are updated more

frequently than firewalls● Some IPS devices can update L2, L3 and L7

firewalls to protect immediately against network threats.

Load Balancer

● A load balancer MAY– Work with a device to see how much load it is under

– Look at how much traffic has been sent to a device

– Round-Robin traffic to each “up” member

– Poll each member to see whether it's still accepting requests

● And then will pass traffic from each request to one of the members of a cluster.

● Frequently seen in front of DNS servers, Web Servers, SMTP servers, etc.

VPN Terminators

● Usually seen in IPSec or SSL varieties● Will terminate lots of incoming connections from

external workers● Mostly implemented as a hardware appliance,

although many L3 firewalls will also perform the same role on existing hardware

● IPSec VPNs largely being replaced with SSL VPNs for “Road Warriors” and home workers, although Site-To-Site VPNs are still the domain of IPSec and Hardware gateways

Closer to your corporate networks

OK, that's enough devices, how about troubleshooting some of this?● Tools include:

– ifconfig/ipconfig● Check your interfaces are up, and passing traffic OK

– netstat● Check your routing table and active/listening connections

– ping/traceroute● Check your connectivity from host to host

– nslookup/dig/whois● Check your DNS responses

– nmap● Check a device's listening services, your Network Topology, OS and App fingerprints

– Packet Captures (e.g. wireshark, tcpdump, snoop)● Check the traffic looks right

If we've got time, we can talk about some protocols, if you're interested?● Hands up if you want to talk protocols

– Such as HTTP and HTTPS– Or why SFTP is not the same as FTPS or FTP

● And what is the difference between Active and Passive FTP

– Why SSH is better than Telnet, and SCP is better than FTP– How DNS works (I might need some help on this :D)– How DHCP works– Different VPN technologies (IPsec, OpenVPN, PPTP, SSH,

and more)– Or something else...... you choose!

“There and back again”How The Internet Works

Photo: http://www.flickr.com/photos/86530412@N02/8210762750/ by StockMonkeys.com

Questions?