What time is it, anyway? Securing NTP - Secure360 · •Developed code in SPITBOL (DEC’s SNOBOL)...

What time is it, anyway? Securing NTP

Presented by:

Shaun Kelly

@DefaultPermit

[email protected]

Disclaimer

While the information in this presentation is intended be accurate and up to date and all clocks should be synchronized to UTC…

…the presenter is speaking for herself, and all views, opinions and mistakes are her own and not the views, opinions, property or mistakes of any past, present, or future employer, organization or time travel agency.

Tweet along: #Sec360 www.Secure360.org

The Brief Bio

• Learned programming on the MECC/MTS Timeshare system, which was a CDC Cyber series mainframe.

• Actually used to “dial” to dial-in

• Developed code in SPITBOL (DEC’s SNOBOL)

• System admin, networking, lots of database background

• Built a former employer’s first (maybe 2nd) Web-based application

• Running a Public NTP Pool server and watching attacks in real time, now it’s personal

• Certs too – CISSP-ISSAP, CSSLP, CCSK


What time is it?


What time is it, really?


Segal’s Law

• “A man with a watch knows what time it is. A man with two watches is never sure.“ – Segal’s Law

• “But I would add further: A man with three clocks is more sure than a man with two clocks.” – LeapSecond.com

The Network Time Protocol addresses these issues.


Preview

• Why is accurate time important?

• A brief history of timekeeping

• NTP basics

• The NTP server pool

• NTP attacks

• Mitigations

• Build a Raspberry Pi Stratum 1 NTP server


Why is securing accurate time important?

• Replay attacks – expired credentials -> unexpired

• DOS attacks – unexpired credentials -> expired

• Log correlation becomes difficult or impossible

• Hiding other attacks by altering timestamps

• BGP attacks – Border Gateway Protocol, helps manage network routing.

• DDOS – Distributed Denial of Service attacks

• High speed trading? Security Cameras? Outside the box attacks? App dependent needs?

• Navigation or attacks on navigation

• Kind of a case study, parallels with other older software and protocols

• PCI-DSS


PCI Section 10.4(I am not a QSA but….)

“10.4 Using time-synchronization technology, synchronize all critical system clocks and times and ensure that the following is implemented for acquiring, distributing, and storing time.

Note: One example of time synchronization technology is Network Time Protocol (NTP).

10.4.1 Critical systems have the correct and consistent time.

10.4.2 Time data is protected.

10.4.3 Time settings are received from industry-accepted time sources.”

What this might mean specifically to you may depend on your QSA


NTP - Network Time Protocol

• Part of the “Plumbing of the Internet”

• Very good at getting correct time out of multiple input time servers

• Mostly just works

But:

• Designed back in the days when trust was abundant

• Uses UDP port 123 – messages can be spoofed

• As with many of the older protocols has a long list of CVEs

• Active target of researchers; new attacks being found (11 CVEs Apr 2016 -http://support.ntp.org/bin/view/Main/SecurityNotice#Recent_Vulnerabilities)


http://support.ntp.org/bin/view/Main/SecurityNotice#Recent_Vulnerabilities

What is UDP and why can it be spoofed?

• Not TCP

• User Datagram Protocol

• OSI Layer 4

• Low latency - good for setting time

• Still, why can it be spoofed easily where TCP can’t?

Let’s try a brief example


How do we agree on accurate time?

• Precision has become more important as we have found applications that can use that precision

• As we went from using human observable cycles (Sunset, Noon, Sunset, Midnight) to atomically measured time, standards organizations now define and agree on a central pool of extremely precise clocks which provide UTC – Universal Coordinated Time


So accurate time is importantHow do we measure it? A brief history of Time(Keeping)


Navigation away from the coasts was dangerous without accurate time• Solar fix gave latitude

• Longitude not so easy


A brief history of Precision Timekeeping


A brief history of Atomic Clocks


My Rubidium Clock


Admiral Grace Hopper’s Nanosecond

• https://en.wikipedia.org/wiki/Grace_Hopper

• Light travels 11.8 inches in a nanosecond

• If you have the precision to keep time in

nanoseconds, and can measure how long

it takes light (or radio) to travel….


https://en.wikipedia.org/wiki/Grace_Hopper

Atomic clocks in space make GPS possible


You thought changing clocksfor Daily Savings Time was a problem…• Before time zones, cities used local solar time – Sundial time

• You either set your watch at the jewelers, or you wait until noon and watch a time ball drop (think New Year’s Eve)

• Carleton College’s Goodsell Observatory provided local time by telegraph for most places West of the Mississippi (https://apps.carleton.edu/campus/observatory/histmain/story/four/

• Railroads would use the time of the headquarters city

• This made transferring, say, from a Chicago time train line

to an Omaha time based train line in Minneapolis…tricky


The Day of Two NoonsNovember 18, 1883


How do you distribute accurate time?

• Computer Networks

• GPS

• Radio (WWV, CHU)

• Dial-up Modem


Time over networks is often NTP

• Part of the “Plumbing” of the internet

• Uses UDP on port 123

• Does a good job at measuring and eliminating latency

• Does a good job at eliminating bad clocks – “falsetickers”

• Usually just works – which means usually goes without attention, which might be problematic

• Not kept up to date at in all distributions – 4.2.8p7 is current, but older is more common (4.2.6 for example)

• Configurations might not be up to date with current CVEs and mitigations


NTPD configuration

• Usually want to have at least 3 time sources configured – 4 or 5 might be better, allows a couple to go offline and still work well. Never use just two – which one is correct?

• Consider the built in cryptographic security (but cautions apply)

• There are “rules of engagement” for public time servers, use an internal mechanism to distribute if you can

• May wish to consider updating your leap seconds file if you have your own stratum 1s


Distribution

• If you have a lot of devices that need time, set up internal Time servers – may be provided by a data center provider

• Distribute in tiers:

GPS Source at DC1 Pool server 1 Pool server 2

Master TS1 Master TS2 Master TS3

Client1 Client2 Client3 Client4


The NTP pool project

• http://pool.ntp.org

• Volunteer run, part of default config of many NTP packages in various distros

• Just under 4k servers globally, divided into zones

• Geolocated DNS queries: 0.us.pool.ntp.org, 1.us.pool.ntp.org etc vs 0.south-america.pool.ntp.org

• Embedded devices/ IOT builder using NTP pool? Request your own custom pool DNS – 0.debian.pool.ntp.org for example


http://pool.ntp.org/

My pool server


Before the CVE list…What about other options or fixes?• NTP provides a symmetric key option, rarely used, has been the source of many CVEs.

(Use with caution).

• SNTP – Simple Network Time Protocol – sometimes used on endpoints, may work for some situations, just runs a query, sets the clock, exits.

• TLSDATE – a service using TCP, TLS to get date. Accurate to second or two, NTP over WAN is milliseconds, LAN can be microseconds – not updated, not widely adopted

• OpenNTPD – OpenBSD rewrite

• PTP – Precision Time Protocol (IEEE 1588). Standard, more accurate with hardware support, v2 (2008) has security extensions

• The White Rabbit Project - http://www.ohwr.org/projects/white-rabbit

• IETF draft - https://tools.ietf.org/html/draft-ietf-ntp-network-time-security-14

• STP – Server Time Protocol – z/OS (mainframe)

• And more….


http://www.ohwr.org/projects/white-rabbit

https://tools.ietf.org/html/draft-ietf-ntp-network-time-security-14

Issues

Hitting the same themes here:

• Older versions, 4.2.6 is very common

• I’ve seen 4.2.2 in use

• Latest build is….4.2.8p7

• Building from source isn’t terribly hard but may not scale well

• Default config may allow querying

• May not have enough servers – especially internal distribution, don’t use just one or two servers.


NTP attacks round 1 - simple attacks

• Since NTP uses UDP, DOS and DDOS amplification attacks possible (CVE-2009-3563, CVE-2013-5211). Made news in 2014.

• Forged input packets running queries – set ‘noquery’ or ‘disable monitor’ to block amplification, disable query altogether if just a client, not serving time requests outbound using ‘noserve’

• Example NTP server safe config:

IPv4: restrict default limited kod nomodify notrap nopeer noquery

• Remember IPv6:

IPv6: restrict -6 default limited kod nomodify notrap nopeer noquery

• Vulnerability scanners will pick up config which allows administrative queries (QID 121695 and others) but you need may need authenticated scans to keep up with older versions

• Use BCP 38!

• Buffer Overflows (CVE-2001-0414, CVE-2014-9296, CVE-2014-9295, etc)


Attacks round two –More interesting, more subtle• Boston University researchers trying to attack BGP found several

NTP flaws (http://www.cs.bu.edu/~goldbe/NTPattack.html)

• NTP sends a Kiss of Death (KOD) to those who query too often

• Since UDP can be forged, attackers can generate KOD

• If attackers can control the last remaining server that will still talks to you….

• Or send a stream of forged NTP replies to you from the sites after all are rate limiting you….

….they control your time.


http://www.cs.bu.edu/~goldbe/NTPattack.html

That 1970’s hack attack?Recent IOS issues

• Brick your phone by setting the clock to 1970?

• Needs to use a middleperson attack, redirect all your NTP calls from Apple to a malicious server(s) – usually hard to do

• Fixed in latest IOS builds

• But what else may be vulnerable to this class of attack?


Attacks round three –Some hope, though• More interesting: Refclock spoofing on 127.127.0.0/8 (CVE-2016-1551)

• More serious: Another symmetric key auth bug - Sybil vulnerability: ephemeral association attack (CVE-2016-1549) no fix, mitigations only

• Several others in the April batch, probably more coming

• Shodan was running IPv6 time servers in the pool to collect IPv6 addresses for later scans – some information is revealed to upstream time servers. If this is concern get your own stratum 1 servers

• A ray of hope? NTPSEC had only 8 of 11 of the April NTP CVEs

• What is NTPSEC?


The Future – NTPSEC?

• Being developed using secure development lifecycle SDLC

• Reduces the attack surface

• https://www.ntpsec.org/

• Available now for beta use

• Should be generally compatible with older NTPD

• Immune to 8 out of 11 of recently announced NTP CVEs (Eric Raymond’s Blog http://esr.ibiblio.org/?p=7167)


http://esr.ibiblio.org/?p=7167

Build your own Stratum 1

• Easiest to use GPS to get time these days, shortwave feasible, need serial port

• For most accuracy need a way to take Pulse Per Second signal from special GPS receiver and feed into something like GPIO pins on Raspberry Pi

• $100 give or take


What’s a Stratum?

• An authoritative time source is considered Stratum 0

• NTP servers synced with GPS, atomic clocks, modem to NIST, etc, are Stratum 1

• Servers connected to stratum 1 and serving clients are now Stratum 2, add one to strata for each NTP server in the tree back to time source

• Strata not the same as accuracy – stratum 3 connected on a low latency LAN might be within microseconds, stratum 1 pool server through many internet hops might be milliseconds off


Raspberry Pi Stratum 1 build

• Used to need a custom kernel to get the PPS by GPIO, part of latest kernels

• Lots of links out there:

http://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html

• Raspberry Pi 3 build with latest Raspbian had some issues• Serial port now controlled differently, some changes

• For a network connected server not a problem, PPS to GPIO works fine

• Let’s see if the demo works (Demo if we can get a GPS signal which seems unlikely)



Conclusion….

• Accurate time is important

• NTP security can be improved with attention to the basics (patching, more secure layout and configuration)

• Secure SDLC can help?

• For some use cases there may be better choices

• Questions?


References

https://timeandnavigation.si.edu/ - History of time and navigation

https://en.wikipedia.org/wiki/History_of_timekeeping_devices

http://leapsecond.com/ - For the Time Nuts

http://www.ntp.org/ - Reference and info

https://www.ntpsec.org/ - The NTPSEC project

https://apps.carleton.edu/campus/observatory/histmain/story/five/ - Carleton College Time history

http://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html - Pi build stuff

http://www.pool.ntp.org/en/ - The NTP server pool

Shaun Kelly - @DefaultPermit - [email protected]


https://timeandnavigation.si.edu/

https://en.wikipedia.org/wiki/History_of_timekeeping_devices

http://leapsecond.com/

http://www.ntp.org/

https://www.ntpsec.org/

https://apps.carleton.edu/campus/observatory/histmain/story/five/


http://www.pool.ntp.org/en/

What time is it, anyway? Securing NTP - Secure360 · •Developed code in SPITBOL (DEC’s SNOBOL)...

Documents

Transcript of What time is it, anyway? Securing NTP - Secure360 · •Developed code in SPITBOL (DEC’s SNOBOL)...