Risk Mitigation for Open NTP OpenNTP... · NTP clients synchronize their time with a local time...
Transcript of Risk Mitigation for Open NTP OpenNTP... · NTP clients synchronize their time with a local time...
Risk Mitigation for Open NTP
Copyright©2016,CyberGreen Sept2016
Agenda
1. Introduction2. AboutNTP3. MitigationrecommendationsforopenNTP4. Makingthecaseforimplementingmitigations
2 Copyright©2016,CyberGreen Sept2016
Introduction
WhencyberinfrastructureisinsecurethereisarisktotheglobalInternetcommunityNetworkTimeProtocol(NTP)isthestandardprotocolfortimesynchronizationfornetworkeddevicesNTPcanbefoundinnearlyeverynetworkenvironmentSynchronizedtimeiscriticaltologging,authentication,cryptographyandgeneralsystemadministrationNTPinfrastructureneedstobesecureandtrustworthy
3 Copyright©2016,CyberGreen Sept2016
About CyberGreen
• Globalnon-profitandcollaborativeorganizationfocusedonhelpingimprovethehealthofglobalCyberEcosystem
• WorkingtoprovidereliablemetricsandmitigationbestpracticeinformationtoCyberSecurityIncidentResponseTeams(CSIRTs),networkoperators,andpolicymakers
• Mission:helpCSIRTsandothersfocusremediationeffortsonthemostimportantriskso Helpunderstandwhereimprovementscanbemadeo Howwecanachieveamoresustainable,secure,and
resilientcyberecosystem
4 Copyright©2016,CyberGreen Sept2016
Copyright (c) 2016, CyberGreen
Thesematerialsaredistributedunderthefollowinglicense:Permissiontouse,copy,modify,and/ordistributethesematerialsforanypurposewithorwithoutfeeisherebygranted,providedthattheabovecopyrightnoticeandthispermissionnoticeappearinallcopies.THEMATERIALISPROVIDED"ASIS"ANDTHEAUTHORDISCLAIMSALLWARRANTIESWITHREGARDTOTHISMATERIALINCLUDINGALLIMPLIEDWARRANTIESOFMERCHANTABILITYANDFITNESS.INNOEVENTSHALLTHEAUTHORBELIABLEFORANYSPECIAL,DIRECT,INDIRECT,ORCONSEQUENTIALDAMAGESORANYDAMAGESWHATSOEVERRESULTINGFROMLOSSOFUSE,DATAORPROFITS,WHETHERINANACTIONOFCONTRACT,NEGLIGENCEOROTHERTORTIOUSACTION,ARISINGOUTOFORINCONNECTIONWITHTHEUSEORPERFORMANCEOFTHISMATERIAL.
5 Copyright©2016,CyberGreen Sept2016
About NTP
6 Copyright©2016,CyberGreen Sept2016
Network Time Protocol (NTP)
NetworkTimeProtocol(NTP)isstandardprotocolfortimesynchronizationfordevicesonanetwork,usedbyservers,mobiledevices,endpointsandnetworkingdevicesfromallvendorsThelatestdefinitionofNTPisversion4,asdescribedinRFC59051
1http://www.ietf.org/rfc/rfc5905.txt
7 Copyright©2016,CyberGreen Sept2016
Network Time Protocol (NTP)
NTPclientssynchronizetheirtimewithalocaltimeserver(liketheDomainControllerinWindowsenvironments),whichwillinturnsynchronize itsclockwithreliableNTPserversavailableontheInternetJusttogetthetime,veryfewtypesofmessagesareneeded• Additionalmessagesandmodesonlyneededfor
NTPserversthatneedtotalktoeachother
8 Copyright©2016,CyberGreen Sept2016
What is open NTP?
“OpenNTP”isaserverwhere• NPTisrunningonadeviceavailabletothepublic
Internet,and• NTPanswersMode6orMode7queries
o Thesequerieshavevulnerabilitiesthatcanbeexploitedbyattackers2
9 Copyright©2016,CyberGreen Sept2016
2https://community.rapid7.com/community/metasploit/blog/2014/08/25/r7-2014-12-more-amplification-vulnerabilities-in-
ntp-allow-even-more-drdos-attacks
How NTP works
10 Copyright©2016,CyberGreen Sept2016
Risks posed by open NTP
DevicesrunningopenNTPcanbeusedinreflectionattacks,atypeoftrafficamplificationattack• Denialofservice(DoS)– attackertriesmakeavictim’s
machineornetworkunavailabletoitsintendedusers• Amplification– whentheattackersendsasmallpacket
toaserverthatwillgeneratealargereplyInamplificationdistributeddenialofservice(DDoS)attacks,attackerssimultaneousabusemultipleamplifierssuchasNTPservers• Createshighly-distributedDoS attackconductedfroma
singlecommandandcontrolhost
11 Copyright©2016,CyberGreen Sept2016
Open NTP in reflection attacks
Attackertriestoexhaustthevictim'sbandwidthbyabusingthefactthatserversusingprotocolssuchasNTPallowspoofingofsenderIPaddressesReflectionattacksoftenexploitUserDatagramProtocol(UDP)traffic• UDPrespondstorequestswithoutanyvalidation
ofsenderidentity,i.e.IPaddress• UDPtrafficcanbespoofed(i.e.haveamisleading
apparentsourceIPaddress)andattackerisabletohidetrueidentity
12 Copyright©2016,CyberGreen Sept2016
NTP reflection amplification attack
ADDoSthatreliesonpublicallyaccessibleopenNTPserverstooverwhelmavictimsystemwithNTPresponsetraffic• Anattackerwithasingle1Gigabit/second(Gb/s)
connectioncantheoreticallygeneratemorethan200Gb/sofDDoStraffic3
Onlyscalableandeffectivemitigationistoreducenumberofserversthatcanbeusedbyattackers• Asof07/27/16,Shadowserverreported4,062,384
uniqueIPswithopenNTP;seehttps://ntpscan.shadowserver.org/stats/
13 Copyright©2016,CyberGreen Sept2016
3https://blog.cloudflare.com/understanding-and-mitigating-ntp-based-ddos-attacks
14 Copyright©2016,CyberGreen Sept2016
NTP amplification attack
AttackersgeneratealargenumberofUDPpacketsusingspoofedsourceIPaddressUDPpacketsaresenttoNTPserversonport123AttackersparticularlylikeNTPserversthatsupporttheMONLIST command4
• MONLIST commandreturnsalistwithlast600IPaddressesthatconnectedtotheNTPserver
• Actsasreconnaissancetoolforhackers:helpsbuildprofileoflocalnetwork
15 Copyright©2016,CyberGreen Sept2016
4AdiscussionofMONLIST canbefoundathttps://blog.qualys.com/securitylabs/2014/01/21/how-qualysguard-detects-
vulnerability-to-ntp-amplification-attacks
16 Copyright©2016,CyberGreen Sept2016
Real life attack using open NTP
Early2014reportofattackusingopenNTP5
• Generatedaround400Gbp/softrafficusing4,529NTPservers
• Eachserverreportedlysent87Mbp/softraffictothevictim
NTPamplificationattackscanresultinabandwidthamplificationfactorof556.96
5https://blog.cloudflare.com/technical-details-behind-a-400gbps-ntp-amplification-ddos-attack/
6http://www.christian-rossow.de/articles/Amplification_DDoS.php
17 Copyright©2016,CyberGreen Sept2016
Potential impacts from NTP attacks
Productivity• Serviceinterruptionorfailureofbusinessoperations
relyingonnetworkconnectivity,particularlyforseasonaloperations- e.g.onlineretailerswhereamajorityofsaleshappenbetweenThanksgivingandNewYears
• Timesensitiveoperations,e.g.collegeswithlimitedonlineregistrationperiodsoronlinewageringonupcomingsportingevents,etc.
Other potential NTP attack impacts
Brand• Lossofreputationwithcustomersandpartners• Becomingknownasa“DoSmagnet”inglobalcommunityTechnical• Networkserviceinterrupted• Isolationofvictimnetworkbynetworkprovidersfromthe
restofInternettomitigatecollateraldamagetoothercustomers
Financial• Lossofbusinessresultingfromserviceinterruption• CostofspecializedDDoSmitigationservices18 Copyright©2016,CyberGreen Sept2016
19 Copyright©2016,CyberGreen Sept2016
Indirect impacts from Open NTP attacks
YoumaybeimpactedifavictimorganizationsharesyourupstreamconnectivityOpenNTPdevicesonyournetworkmaybeusedtocontributetoanattackonanotherorganizationPotentialindirectimpactsinclude:Technical• Networkservicedegraded• Inboundoroutboundbandwidthmaybereduced• Networkprovidersmayisolateyournetwork(orat
leastyourinsecurerecursiveresolver)fromtherestofInternet
20 Copyright©2016,CyberGreen Sept2016
Other indirect impacts
Brand• Lossofreputationwithcustomersandpartnersduetoslow
orunreliablenetworkandsystemsFinancial• Unexpectednetworkusagecosts• Lossofbusinessresultingfromservicedegradation
Mitigate risks from open NTP
21 Copyright©2016,CyberGreen Sept2016
22 Copyright©2016,CyberGreen Sept2016
Mitigation options vary by environment
NotallmitigationbestpracticesareappropriateforallenvironmentsCyberGreenprovidesinformationrelevanttofourbasicenvironmentalprofilesLookfortheseiconstofindmitigationsforyourenvironment
1.
2.
3.
4.
23 Copyright©2016,CyberGreen Sept2016
Mitigate risks from open NTP
ThebestwaytomitigaterisksfromopenNTPmovingforwardistopurchaseanddeploydeviceswithminimalNTPconfigured,particularlyonoutsideinterfacesWorkwithyourinternalacquisitionandprocurementteams,orvendorsaboutotheroptions
24 Copyright©2016,CyberGreen Sept2016
Identify your open NTP risk
Evenifyoudon’tthinkyourdevicescurrentlyrunNTPacrosstheInternet,youshouldcheckyournetwork• ManydevicesmayberunningNTPwithoutyour
knowledge• NTPisoftenbuiltintoCustomerPremise
Equipment(CPE)gatewaysonnetworkequipmentsuchascablemodems,DSLrouters,“broadbandWiFi routers”,etc.
25 Copyright©2016,CyberGreen Sept2016
Find hosts running NTP
Thesimplestwayistouseaweb-basedprobe,suchastheoneathttp://openntpproject.orgTomanuallyidentifyNTPserverswithamplifiedresponsesenabled,runoneofthefollowingcommands:
ntpdc –n –c monlist 192.0.2.1
ntpdc –c sysinfo 192.0.2.1
ntpq –c readvar 192.0.2.1
Thecommandsonlyverifyifspecifiedfunctionsareenabled
26 Copyright©2016,CyberGreen Sept2016
Manually finding NTP hosts
Ifcommandwassuccessful,youwillseeastringofinformationlikethisfromtheIPyouqueried:
associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync, version="ntpd [email protected] Sun Oct 17 13:35:13 UTC 2010 (1)", processor="x86_64", system="Linux/3.2.0-0.bpo.4-
amd64", leap=00
27 Copyright©2016,CyberGreen Sept2016
Mitigation: Upgrade NTP
TheeasiestwaytomitigatetheriskistoupgradetoNTP-4.2.7p230(releasedin2011)orlater,whichremovestheMONLIST commandentirelyanddisablesMode7responsesbydefault• Protectsyournetworkfrominadvertentlybeing
usedinaDDoSattack• Protectsyournetworkfromunwanted
reconnaissance
28 Copyright©2016,CyberGreen Sept2016
Mitigation: Upgrade NTP
Ifyourenvironmentissofragilethatupgradingisnotanoption,modifytheNTPconffiletoaddthestatementdisable monitorandthenrestartyourNTPprocessesYoushouldalsoimplementanadditionalriskmitigation
29 Copyright©2016,CyberGreen Sept2016
Mitigation: Disable status queries or restrict access
NTPqueriesmayrevealinformationaboutthesystemrunningNTPthatyoudonotwantotherstoknow,suchastheoperatingsystemversionandntpdversionDisablingthesequeryfeaturesmayhelptoreducethelikelihoodofthisdataleakagetakingplace• Disablingthesequerieshasacost,asthesequery
capabilitiesalsoprovideusefulQ/Aanddebugginginformation
30 Copyright©2016,CyberGreen Sept2016
Mitigation: Restrict informational queries to authorized recipients
TodisableMONLIST functionalityonapublic-facingNTPserverthatcannotbeupdatedto4.2.7,addthefollowinglinestoyourntp.conf file:ForIPv4:restrict default kod nomodify notrap nopeernoquery
ForIPv6:restrict -6 default kod nomodify notrapnopeer noquery
Note:requiresarestartofthentpd servicetotakeeffect
31 Copyright©2016,CyberGreen Sept2016
Mitigation: Restrict access per network segment
Modifyyourntp.conf torestrictaccess:pernetworksegment(modifyline3tomatchyourLANsettings)*and*perhost(modifyline4):restrict default noquery
restrict localhost
restrict 192.168.0.0 netmask 255.255.0.0 nomodify notrap nopeer
restrict 192.168.1.27
Note:requiresarestartofthentpd servicetotakeeffect
32 Copyright©2016,CyberGreen Sept2016
Other NTP mitigations
ConsiderblockinglargeNTPpacketsatnetworkedge• Blockpackets234bytes– 482bytes(thesizeof
MONLIST replies)AdditionalguidelinesforsecuringtheNTPserviceondifferentplatformsandconfigurationsareavailablefromTeamCymru:http://www.team-cymru.org/secure-ntp-template.html
33 Copyright©2016,CyberGreen Sept2016
Mitigations for ASNs or ISPs
UsetrafficshapingonUDPservicerequests• Ensures repeated access toInternet resources isnotabusive
MonitorNTPinyournetworkforsignsofamplificationattacks(seehttps://www.us-cert.gov/ncas/alerts/TA14-017A)andgenerateabuseticketsforthesecustomers• Options:takeacustomer’smodemoffline,ornotifyviaphonecall
Notifyyourcustomersofissues,evenifyoucan’ttellthemhowtofixthem• TheymaynotbeintentionallyrunninganNTPserver - trafficmaybe
resultofmalfunctioninghomerouters thatCustomerCarehasnoideahowtoreconfigure
34 Copyright©2016,CyberGreen Sept2016
Spoofed Traffic Mitigation: Implement ingress filtering on networks
InternetEngineeringTaskForce(IETF)BestCurrentPractice(BCP)documentsConfigurationchangestosubstantiallyreducepotentialforsourceIPspoofedattacks,themostpopularDDoSattacktype• Howtofilternetworktrafficonnetworktoverify
thesourceaddressofapacket• Rejectpacketswithsourceaddressesthat
arenotreachableviatheactualpacket’spath
35 Copyright©2016,CyberGreen Sept2016
IETF BCPs recommended
AllnetworkoperatorsshouldperformnetworkingressfilteringasdescribedintheseBCPs:BCP-38NetworkIngressFiltering• DefeatingDenialofServiceAttackswhichemploy
IPSourceAddressSpoofing:https://tools.ietf.org/html/bcp38
BCP-84IngressFilteringforMultihomed Networks• https://tools.ietf.org/html/bcp84
36 Copyright©2016,CyberGreen Sept2016
More info on IETF BCPs
TestwhetheryournetworkcurrentlyfollowsBCP-38usingtoolsfromtheSpoofer Project:https://www.caida.org/projects/spoofer/
AdditionaldetailsabouthowtoimplementBCP-38:http://www.bcp38.info/index.php/Main_Page
37 Copyright©2016,CyberGreen Sept2016
Additional mitigations for ISPs
ISPsshouldensurethattheyhaveaDDoSdefensethatismulti-layered,anddesignedtodealwith:• Attacksthatcansaturatetheirconnectivity• “Lowandslow”sophisticatedapplicationlayer
attacks
38 Copyright©2016,CyberGreen Sept2016
Verify your fix
VerifyandmonitoryourinfrastructuretoensureitremainssecurebysubscribingtofreereportsfromShadowserver,availableathttps://www.shadowserver.org/wiki/pmwiki.php/Involve/GetReportsOnYourNetwork
39 Copyright©2016,CyberGreen Sept2016
Additional NTP resources
https://ntpscan.shadowserver.org/http://openntpproject.orghttp://www.us-cert.gov/ncas/alerts/TA14-017Ahttps://community.rapid7.com/community/metasploit/blog/2014/08/25/r7-2014-12-more-amplification-vulnerabilities-in-ntp-allow-even-more-drdos-attackshttp://www.acunetix.com/blog/articles/ntp-reflection-ddos-attacks
Making the case for implementing mitigations such as BCP 38
40 Copyright©2016,CyberGreen Sept2016
41 Copyright©2016,CyberGreen Sept2016
Making the case for mitigations
IHelpeveryoneunderstandthelevelofeffortneededtoimprovecyberhealthintheircommunityWhyshouldyouimplementthemitigationsinyourenvironment?1. ItistherightthingtodoasagoodInternet
neighbor2. YourorganizationmaybenexttobeattackedLet’sjointogetherandstopbadguysfromwinning!
42 Copyright©2016,CyberGreen Sept2016
Changing risk landscape
Increasedneedtodemonstrate“duecare”o Obtainingcyberinsuranceo Complyingwithriskframeworkstowinbusinesswith
local/nationalgovernmentsandlargecorporations
Ifwe(you!)don’tdoabetterjobofsecuringourowninfrastructureandreducingcyberrisk,governmentregulationmayforceadditionalmandatesand/orpenalties
43 Copyright©2016,CyberGreen Sept2016
Anticipated organizational benefits
Increasedproductivity• Fewerserviceinterruptionsandfailures
Improvednetworkperformance• Existingnetworkmore
reliableandresilient,withgreatercapacity
Improvedbrandreputation• Technicalreliabilityand
securityasellingpointtocustomers
44 Copyright©2016,CyberGreen Sept2016
More anticipated benefits
• Decreasedbudgetuncertaintyo FewerunanticipatedusagecostsforITo Budgetcanbeusedasplanned,e.g.- upgrading
technicalcapability/capacity,additionalpersonnel,etc.
• Systemadminsmayspendlesstimespenttryingtodealwithunexpectedproblems,whichinturnmayimprovetheirproductivityandreduceunexpectedovertime
45 Copyright©2016,CyberGreen Sept2016
What do you need to implement these mitigations?
Commandsandconfigurationdetailsformostimportantmitigationsarepublicallyavailable• Noadditionalsoftwaremustbepurchased• Implementingmitigationsdoesnotrequireanyspecial
knowledge,skills,orabilities
Note:AllmitigationsshouldbecarefullyreviewedinlightofyourspecificbusinessrequirementsandinfrastructureenvironmentbeforeproceedingAllorganizationalchangemanagementprocesses,includingtesting,shouldbefollowed
46 Copyright©2016,CyberGreen Sept2016
How long will mitigations take?
Systemadministratorsinsmallerorganizationsneedafewhourspernetworktoinvestigate,implementandverifyupgradeofNTP• Comparableeffortneededforothermitigations,suchas
disablingstatusqueriesandMONLIST functionality,andblockinglargeNTPpacketsatthenetworkedge
ISPsandlargeentitiescantakeadvantageofconfigurationmanagementsystemswithtaskexecution,suchasSaltandAnsible,toautomateadministrationofchanges
47 Copyright©2016,CyberGreen Sept2016
How long to implement BCP-38 network ingress filtering?
Smallbusinesses:fromafewminutestolessthananhour
Largerandmorecomplexorganizations:daystoweeks
Bonus:withnorealmaintenance,therecurringcostiseffectivelyzero!
Acknowledgement
48 Copyright©2016,CyberGreen Sept2016
CyberGreenwouldliketothanktheexpertswhomadethecreationofthisdocumentpossible:
Writtenby:- LaurinBuchanan,Applied Visions, Inc.– SecureDecisions Division
Contributed andReviewedby:- MattCarothers,CoxCommunications- Baiba Kaskina,CERT.LV- MotoKawasaki,JPCERT/CC- ArtManion,CERT/CC- Yoshinobu Matsuzaki, IIJ- JoeStSauver,Farsight Security- DavidWatson,ShadowServer Foundation
Disclaimer:CyberGreenbelievesthisguidanceandtheadvicefromourexpertsshouldbeofbenefittoanyonemitigatingariskconditions,butitisnotadvicespecifictoanyreaderornetwork.Ultimately,eachreaderisresponsibleforimplementinghisorherownnetwork remediationstrategyandweassumenoresponsibilityorliabilitytherefore.
Formoreinformationaboutriskmitigationbestpractices
pleasecontact:[email protected]
49 Copyright©2016,CyberGreen Sept2016