Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, &...
Transcript of Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, &...
![Page 1: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/1.jpg)
Crowdsourced Security — The Good, the Bad, & the Ugly
Mike Shema [email protected]
(ISC)2 Security Congress September 25, 2017
![Page 2: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/2.jpg)
– Clint Eastwood, The Good, the Bad, and the Ugly.
“You see, in this world there’s two kinds of people, my friend: Those
with loaded guns and those who dig. You dig.”
![Page 3: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/3.jpg)
– Eli Wallach, The Good, the Bad, and the Ugly.
“There are two kinds of spurs, my friend. Those that come in by the door; those that come in by the
window.”
![Page 4: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/4.jpg)
Uneasy Alliances“What’s the price for this vuln?”
— Bounties “What’s the cost to fix this vuln?”
— DevOps “What’s the value of (& budget for) finding vulns?”
— CSOs
![Page 5: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/5.jpg)
Disclosure Happens
![Page 6: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/6.jpg)
Bounties are an imperfect proxy for risk, where price implies impact.
$0 — $15K~$800 avg.
$50Reflected XSS, self,
no auth
$10,000XSS any auth’d user,access sensitive info
![Page 7: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/7.jpg)
Bounties are an imperfect proxy for work, where earnings often diverge from effort.
80%
50%
100%
![Page 8: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/8.jpg)
~33%
~87%
![Page 9: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/9.jpg)
Noise increases cost of discovery and reduces efficiency.
![Page 10: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/10.jpg)
Baseline — Initial cost + Maintenance
Volume — Reports/day, Percent valid
Triage — Reports/hour, Hourly rate
![Page 11: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/11.jpg)
Clear, concise documentation
Scope*
Rules of engagement*
Practical SLAs for responses
Expectations of reasonable threat models
Filters
![Page 12: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/12.jpg)
![Page 13: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/13.jpg)
![Page 14: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/14.jpg)
Cost-effective, Efficient
Cost-ineffective, Efficient
Cost-ineffective, Inefficient
Cost-effective, Inefficient
![Page 15: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/15.jpg)
Where are the scanners?Overlaps, gaps, and ceilings in capabilities.
Fixed-cost, typically efficient, but still require triage and maintenance.
![Page 16: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/16.jpg)
Days since any report: 2, 5, 11
Days Since Valid (Any) Report
2016 7 (4) 16 (8) 33 (14)2015 4 (1) 10 (5) 23 (11)
50% 80% 95%
![Page 17: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/17.jpg)
Public, Private Bounties
![Page 18: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/18.jpg)
Pen Tests
![Page 19: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/19.jpg)
– Mike’s Axiom of AppSec
“We’ll always have bugs. Eyes are shallow.”
![Page 20: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/20.jpg)
BugOps vs. DevOpsChasing bugs isn’t a strategy.
![Page 21: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/21.jpg)
Where is threat modeling?DevOps exercise guided by security.
Influences design.
Informs implementation.
Increases security awareness.
![Page 22: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/22.jpg)
Risk reduction.
![Page 23: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/23.jpg)
“You’re not using HTTPS.”
“Use HTTPS.”
“Seriously. Please use HTTPS.”
Let’s Encrypt.
![Page 24: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/24.jpg)
Risk StrategiesDecrease rate of reports for ___ vulns.
Increase speed of deploying fixes for ___ vulns.
Deploy ___ to counter category of ___ vulns.
![Page 25: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/25.jpg)
![Page 26: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/26.jpg)
![Page 27: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/27.jpg)
Bounty ranges as a proxy for SDL,where price implies maturity.
$ 1 Experimenting$ 1,000 Enumerating$ 10,000 Exterminating$ 100,000 Extinct-ifying
![Page 28: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/28.jpg)
Based on realistic threat models.
Incentivized quality and effort.
Machine-readable reports.
Bounties
![Page 29: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/29.jpg)
CrowdsPublic bounty
Private bounty
Pen testing
Threat intel sharing
Fuzzing farms
![Page 30: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/30.jpg)
Create threat models.
Measure vuln discovery effort.
Strive for automation.
![Page 32: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/32.jpg)
Questions?
(ISC)2 Community — http://bit.ly/4416GBU
![Page 33: Crowdsourced Security - The Good, the Bad, and …...Crowdsourced Security — The Good, the Bad, & the Ugly Mike Shema mike@cobalt.io (ISC)2 Security Congress September 25, 2017 –](https://reader033.fdocuments.us/reader033/viewer/2022052722/5f0c8ff27e708231d43605db/html5/thumbnails/33.jpg)
R — www.r-project.org
RStudio — www.rstudio.com data.table ggplot