Money$ec Evolved
description
Transcript of Money$ec Evolved
![Page 1: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/1.jpg)
Money$ec EvolvedWherein not everything has a tidy baseball
analogyJared PfostChief Executive OfficerThird Defense
Brian KeeferSecurity ArchitectLeading SaaS Security Company
![Page 2: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/2.jpg)
Recap•Last year we applied baseball
“SABRmetrics” to InfoSec•We spent some time in the real
world•Oh yeah, some guy named Brad
was in a movie
![Page 3: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/3.jpg)
In case you missed it
How Analytics Changed Baseball
![Page 4: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/4.jpg)
Oakland A’s•Teams bid for players in Free Agent
market•Start of 2002 A’s had payroll
~$40M*•NY Yankees payroll ~$126M*•So poor teams have no shot at
winning, right?*From “Moneyball”
![Page 5: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/5.jpg)
1999-2001Team Wins Losses Est
Payroll*
NYY 280 203 $257M
OAK 280 205 $70M
*Estimate from baseball-reference.com
![Page 6: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/6.jpg)
Billy Beane•GM Billy Beane
defied convention
• i.e. he didn’t follow “best practices”
•made data-drive decisions
•Hired Paul DePodesta
![Page 7: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/7.jpg)
Traditional baseball•Talent is evaluated by scouts•Scouts are usually washed-up
players•i.e. “Industry veterans” or
“experts”•Value statements are largely
subjective
![Page 8: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/8.jpg)
Next-gen Baseball
•Started in 1977•Bill James wanted to see what
influenced game outcome•Realized stats created in 1859
didn’t properly attribute events
![Page 9: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/9.jpg)
Key lessons•Don’t make emotional decisions•At least recognize your bias
•Collect the “right” data•Look for correlations
•Set reasonable criteria for success•Don’t overspend
![Page 10: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/10.jpg)
This Applies to InfoSec
![Page 11: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/11.jpg)
Problem statement
•Every organization is competing with attackers
•Most don’t have Fortune 50 budget•How can you be effective?
![Page 12: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/12.jpg)
Conventional “wisdom”
•“Everyone knows” that you need•Firewall•Anti-virus•Change passwords frequently•Prohibit social networking•Etc.
![Page 13: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/13.jpg)
Do they work?
•Port 80 goes through the firewall•Anti-virus misses custom malware•Stolen passwords used quickly•Social networking key to marketing
and employee satisfaction
![Page 14: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/14.jpg)
Clearly this is not working
•Do we actually want a new strategy?
•What does winning look like?•How do we get started?
![Page 15: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/15.jpg)
Cheap & Easy
Spend to Comply
Fix Gaps Now!
Ok, how much do we really need...?
Are You Ready To Win?
Motivating Event
![Page 16: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/16.jpg)
•Winning is not losing...•No unacceptable risks realized •Cheap as possible
What Does Winning Look Like?
![Page 17: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/17.jpg)
So, about that...• Started collecting info• Realized it was far from
complete• Historical incident
rates were meaningless
• Minimal ability to measure what helps
• 12 metricsMoney$ec 1.0
![Page 18: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/18.jpg)
EvolutionMoney$ec 2.0
• Measure what’s easy
• Set Targets• Justify More• Optimize
Cost vs. Target
![Page 19: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/19.jpg)
Start With “Easy”• Incidents - # of High, Moderate, Annoying• Application- # of Post-production application bugs
• Passwords- % passwords easily guessed• Scanned Vulnerabilities- # Patch & config vulns not mitigated per Severity Service
Level
![Page 20: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/20.jpg)
Real Metrics Have Outcomes
• Stats are trendy, Metrics have Winners|Losers–Measure actual performance against target–Benefits
• Drives “acceptable risk” conversation with Management• Simplifies reporting e.g. are we above|below?
![Page 21: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/21.jpg)
Back To “Easy”• Scanned Vulnerabilities
- # Patch & config vulns not mitigated per Severity Service Level- Sev 1 Server Vulns Mitigated within 30 days- Sev 2 within 60 days
![Page 22: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/22.jpg)
You really can do this
![Page 23: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/23.jpg)
Ooooh, shiny!
![Page 24: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/24.jpg)
24
Expand Measurement• Access Management
- % Employee termination within policy- % Role/Access verification• Network- % critical systems monitored- Moving to % of full packet capture
• Vendors- % assessed per policy- # overdue findings• Employee- # of duplicate incidents• Change Management- # emergency or unplanned changes- % of changes with a regression
Every Metric Must Have A
Target
![Page 25: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/25.jpg)
Optimize Cost - Target•Is target too high?
67
75
84
92
100
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb
Proposed Target
![Page 26: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/26.jpg)
Cost - Benefit - AccountabilityRate Hrs Per
Test/Deploy# Personnel Cost Per
Server Update
$100/HR 40 10 $40,000
Evidence: Incidents, response performance, attack attempts
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10
DoS PostMalware Post
Worm Post
Or
http://code.google.com/p/openpert/
Current Target
Proposed Target
![Page 27: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/27.jpg)
Improve IR•Move IR out of IT?•Infections are incidents•Data is needed to evaluate
controls•Knowing root-cause guides future
controls and Targets
![Page 28: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/28.jpg)
Integrate Metrics Into Root Cause Analysis
Find Leading Indicators
![Page 29: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/29.jpg)
Parting Thought
•People implicitly decide not to measure.
•Money$ec says explicitly decide when you don’t.
![Page 30: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/30.jpg)
Security Reformation?
http://www.liquidmatrix.org/blog/2012/02/21/we-are-losing/
http://lifecypha.wordpress.com/
![Page 31: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/31.jpg)
Time to Share
•Data you find useful to collect?•Spotted any correlations?•Proved any controls too expensive?•What communities do you
participate in?
![Page 32: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/32.jpg)
Thanks!
Brian Keeferb: http://rants.effu.se
e: [email protected]: @chort0
Jared Pfostb: http://thirddefense.wordpress.com
e: [email protected]: @JaredPfost
![Page 33: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/33.jpg)
appendix
![Page 34: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/34.jpg)
Task InfoSec Control Owner Business Owner
Define Metric A,R R C
Define Target R R A,R
Report Metric A,R R I
Review Target A,R R R
R – ResponsibleA – AccountableC – Contribute
I - Informed(There can be only one “A”)
RACI in action
![Page 35: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/35.jpg)
2011 VZ DBIR vs. Money$ec
![Page 36: Money$ec Evolved](https://reader036.fdocuments.us/reader036/viewer/2022062501/56815f2f550346895dcdfbfd/html5/thumbnails/36.jpg)
Device Patch & Config Monitoring