Bug Bounties andthe Path to Secure Software

ScottCrawford– ResearchDirector,InformationSecurity

What’s a Bug Bounty? (And why should you care?)

• Non-softwareproductsmustoftenfacerigoroustestingagainstreal-worldconditionstodemonstratetheirsafetyandreliability

• Butwhataboutsoftware?

4

“Hacker-powered security”• Testingisonlyasgoodastheexperts

applyingtheirknowledge• …and“users”areinfinitelycreative

• Bugsaren’tjustaboutsecurity• …butsecurityisatopconcern• …andsuccessinfinding&fixingisarace

againsttheclock

• Whynotengagethesameresearchersthatfindbugs,tohelpfixthem?

5

Anearly(andliteral)“bugbounty”:OS

company(andaptlynamed)

Hunter&Ready,1983

Photo: https://twitter.com/senorarroz/status/783093421204393985

Bug Bounty Programs: From concept to maturity

• From(asometimescontentious)opportunitytoformalizedfield– andforgoodreason

• Thedifferencebetweendiscoveringwhatothersknoworcouldfindout,andremaininginthedark

• “Everyonegetsafreepenetrationtest–whetherornottheygetacopyofthereportisuptothem.”

6

AtBlackHatUS2017,FacebookCSOAlexStamoshighlightedaconference– andanindustry– thathasgrownfromhackingtoanemphasisonmatureandintegrateddefense.BBPsalignboth.

Seeing results• Facebook,Feb2016:38%YOYincreaseinhigh-

impactsubmissions1

• Google,June2016:Upto50%increaseinamountspaidforhigh-qualityvulnerabilityreports2

• Positiveimpactonsafetyandlife-criticalissues,particularlywithgrowthofIoTand“smart”systems

7

1 https://www.facebook.com/notes/facebook-bug-bounty/2015-highlights-less-low-hanging-fruit/12251687441640162 https://security.googleblog.com/2016/06/one-year-of-android-security-rewards.html

Is a BBP for you?• Chiefconcern:Frombugtobadoutcome• Notjustsecurity• Safety,properoperation,(re)liability,

customerconfidence… evencheating!

• 3keyconsiderations:• Visibility• Criticality• Notoriety

• Nolongerjustfortechcompanies• HackerOne:41%ofbugbountieslaunched

in2016fromnon-techindustries3

8

3 https://www.hackerone.com/resources/hacker-powered-security-report

Where to begin?• Ifyourdigitalassetshaveany exposuretoinquisitive

minds…• Youmayfindthatsomeonehasdiscoveredabugor

vulnerability• Howwillyouhandleit?

• 94% oftheForbesGlobal2000donothaveknownvulnerabilitydisclosurepolicies4

• Every organizationwithapubicdigitalfootprintalready hasastakeinhacker-poweredsecurity

• Whynotdoitrightfromtheoutset?

94 https://www.hackerone.com/resources/hacker-powered-security-report

7 steps toward“hacker-powered” security

1: Create a VDP (and make it easy to find!)• Avulnerabilitydisclosurepolicyneedstobe

tablestakes foranyorganizationwithanypublicfootprint• Ensuresaclearprocessforcommunicating

issues• Enablesthemanywhoarewellmotivatedto

help!• Neednotbelimitedtobugs• Configerrorsorotherdetectableexposures

• Canbeassimpleasspecifyinganemailaddress• Butmoredetailwouldbeideal

Key elements of a VDP1. Contactinformation2. Cleardescriptionofreportableissuetypes3. Rulesforfindingandreportingbugs4. Listofsystemsavailableonwhichtoreportbugs5. Communicationexpectations:Whentoexpecttohearback

afterfirstcontact6. Rulesofengagement:HowmuchisOK,andhowmuchis

goingtoofar(i.e.potentiallybreakingthelaw)7. Guidanceonhowtotestmayalsobeprovided,suchasprovidingadetailed

summaryoftheissue,includingthe8. Target,steps,toolsandartifactsusedindiscovery(helpsthesubjectorgreproduce

theissue)

An international standard• ISO/IEC29147:Guidelinesforthe

vulnerabilitydisclosureprocess

• Freely availableathttp://standards.iso.org/ittf/PubliclyAvailableStandards/c045170_ISO_IEC_29147_2014.zip

• Related:ISO/IEC30111:Guidelinesforvulnerabilityhandlingprocesses(moreonthatshortly)

13

An NTIA template for VDP• Brandpromise("Thesafetyandsecurityof

ourcustomersisimportanttous…")• Initialprogramandscope:Whichsystemsand

capabilitiesare‘fairgame’vs.‘offlimits’• "Wewillnottakelegalactionif…":Clear,

statementstoguidegood-faithefforts• Communicationmechanismsandprocess• Non-bindingsubmissionpreferencesand

prioritizations• Versioningofthepolicy

14

https://www.ntia.doc.gov/other-publication/2016/multistakeholder-process-

cybersecurity-vulnerabilities

2: Corporate comms must know how to handle• Transparence andresponsivenesscangoa

longwaytowardmakingthebestofanincidentorreport

• Ensurethatcorporatecommunicationsstaffunderstandhowtorecognizeandhandleadisclosure

• Whatnot todo• Automatedemailswithnofollowup

• CasesofWin:• Bufferbreach• CloudBleed• GitLabDBincident

15

3: Document and practice vulnerability handling

16

ISO/IEC 29147 – Vulnerability disclosure process

ISO/IEC 30111 – Vulnerability handling process

A vulnerability handling process overview

17

Critical:• A clear,

common set of rules and expectations

• Easy to locate

Ready to take that next step?

18

4: Select a Bug Bounty Platform Provider ABBPPcanhelpshouldertheburden– orcompletelyoffload– manyprocessescriticaltoBBPsuccess:• HelpwithdesignofBBPs• Provideasoftwaresolutiontomanagesubmissions• ExpertguidanceandimplementationofprocessesvitaltoBBPsuccess• Responsetoreports• Triage• Disclosureassistance• Communitysupport• Accesstothetalentpool

19

• Managementplatformfeatures• Workflowintegration• Automationandorchestration• Flexibleprograms• Metricsforsuccess

BBPPs: Automation and orchestration• Soyou’regoingtoacceptincomingbugreports.

Maybealot ofthem• Thinkfixingissueswillbeyourbiggestproblem?• Howaboutsortingthroughthenoisetotriage

duplicates,falsepositives,orreportsoutofscope?

• Yelp:First100daysofapublicBBP:• 564reports• 322duplicates(57%)• 525notactionable- That’s93% ofreportsthat

peoplewouldhavehadtosortthroughwithoutthesupportoftriageandworkflowautomation

20

Measuring success: BBP metrics• Whattomeasure?Bugseverityor

quantity?Numberfixed?• Howaboutreducingthenumberfoundina

bountyinthefirstplace?

• Someexamplesthatmighthelpmeasureimprovementsinsoftwarequality:• Numberofissuesper1000linesofcode

(LOC)• Numberofcriticalflawsperdevelopment

cycle• Timetoresolve

21

5: Start conservative, with a private BBP, then6: Go public when comfortable

• Advantagesofaprivateprogram• Abilitytocontrolallconstraints• Choosetesters,limittheirnumber,improve

processesinprivate• Findingandfixingflawsbeforeproduction

release• Qualityandrelevanceofsubmissions

• Advantagesofapublicprogram• Actionableresultspotentiallymorequickly• Positivepublicimage

22

7: Refine and expand your program

23

Thank you!