The Most Important Thing: How Mozilla Does Security and What You Can Steal
-
Upload
mozillapresentations -
Category
Technology
-
view
1.121 -
download
0
description
Transcript of The Most Important Thing: How Mozilla Does Security and What You Can Steal
![Page 1: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/1.jpg)
The Most Important Thing
How Mozilla Does Security and What You Can Steal
Johnathan NightingaleHuman Shield
Mozilla [email protected]
![Page 2: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/2.jpg)
So you want to steal a security architecture...
Do you actually want to get better?
Do you care about responsiveness?
Can you let go of secrecy?
![Page 3: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/3.jpg)
Why steal from us?
We have been at it for a while...
in a phenomenally hostile environment...
with 180 million users...
and we seem to be doing alot of things right...
and you can see how we do it
![Page 4: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/4.jpg)
This Diagram is Stupid
Response
Design
Implementation
Testing
Metrics
![Page 5: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/5.jpg)
Good Security is a Feedback Loop
• The idea that security can be wholly top-down, with discrete one-way steps in an orderly flow from start to end is the worst kind of process management fiction
• Your security process should instead ask at every step, “How can we make sure problems like this never happen again?”
![Page 6: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/6.jpg)
The single most important thing you can do is find ways to
capture expensive knowledge so that you never pay for the same
lesson twice
![Page 7: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/7.jpg)
ResponseA security compromise is the most
expensive knowledge of all
![Page 8: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/8.jpg)
ResponsePrepare
Triage
Deploy
Fix
Schedule
Mitigate
Post-Mortem
Who should help?
With tests! (More later)
This is not the same as shipping! (More later)
Where is it written down?
![Page 9: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/9.jpg)
Learning from Response
• It’s okay for post-mortems to be short
• It’s not okay to skip them
• If you make them into blame-finding, they stop being useful (even for blame-finding!)
![Page 10: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/10.jpg)
Ask Questions
• Who did we have to bring in late?
• Why didn’t we notice that we broke the internet?
• How could we have dealt better with the original reporter?
• What were our bottlenecks?
Write down the answers for next time(there’s always a next time)
![Page 11: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/11.jpg)
TestingTesting is your best defense against forgetting, because you will forget
![Page 12: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/12.jpg)
Data Point
We run:
• 55,000 automated tests
• in 6 test frameworks
• on 4 platforms
• at least 20 times a day
![Page 13: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/13.jpg)
You Already Know Why
• Tests protect your features from security-based changes
• Tests protect your security from feature-based changes
• Tests capture and transfer expensive knowledge
• Tests reduce Bus Factor
![Page 14: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/14.jpg)
Now Make It Happen
• It must be easy to add new tests
• Yes, this is tricky at first
• Money can be exchanged for goods and services!
• Nothing lands without tests
• Nothing. Lands. Without. Tests.
![Page 15: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/15.jpg)
It’s Hard To Test <X>
• This is terrifying
• Steal another framework
• Don’t underestimate manual testing
![Page 16: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/16.jpg)
Power Tools
• Fuzzing
• Penetration Testing
• YMMV
![Page 17: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/17.jpg)
One More ThingTests that don’t run are a waste of everyone’s time
Option: Automatic GunfireBuy a box that sits in a corner and runs tests off
trunk every hour. Put a gun on it that shootspeople who break tests.
Option: Manual SlogMake check-in approval contingent on running
tests, every single time.
![Page 18: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/18.jpg)
Implementation“We have tests” is not an excuse
to keep breaking things
![Page 19: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/19.jpg)
Where Mistakes Are Made
• Strategic-level mistakes can be made in design, but most security bugs come from mistakes not caught during implementation
• Your ability to profit from expensive knowledge is highest here, but here is where you’re probably doing the worst job
![Page 20: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/20.jpg)
No-Brainers
• Static analysis tools
• assert()
• “Public” Betas
• Alphas?
• Source?
![Page 21: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/21.jpg)
Tougher
• Non-security bugs point to security bugs
• Do you have crash reporting?
• No bug happens once
• Where else are you assuming that a null pointer isn’t exploitable?
• Bad patterns - knowledge that you get to benefit from more than once.
![Page 22: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/22.jpg)
The Game Changer
• Socializes security knowledge by sharing it
• Gatekeeper against “This is little, it’ll be fine”
• P(Mistake1) * P(Mistake2) << P(Mistake1)
The most important change you can make atimplementation is mandatory review
![Page 23: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/23.jpg)
DesignEvery time you eliminate a threat class
an angel gets its wings
![Page 24: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/24.jpg)
Making Things Right
• Design for re-use
• Find areas that keep needing “temporary” field patches and fix them for good
• Design for testability
• Threat modelling
Make it easier to profit from expensive knowledge
![Page 25: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/25.jpg)
MetricsMeasure what matters, not what’s
easy to measure
Now with
12%
more bits!
![Page 26: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/26.jpg)
Don’t Know What Matters?
• Ask sales
• Ask your users
• Don’t ask your competitors, they are looking for the easy way out
![Page 27: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/27.jpg)
The #1 Grade-A Stupidest Metric of All
• A focus on bug counting creates perverse incentives for security
• Developers hide bugs from management
• You hide bugs from customers
Bug Count
Counting bugs teaches you to bury all theexpensive knowledge you should be sharing
![Page 28: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/28.jpg)
Think Harder
• Days of exposure
• Average time to deploy fix
• Better would be avg. time until > 90% of users are using the fix
• Customer downtime
![Page 29: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/29.jpg)
Get Creative
• Number of regressions per update cycle
• Number of all nighters
• Start using similar metrics when judging your own suppliers & platforms
• Tension between metrics can be a good thing, if it pulls people towards awesome
![Page 30: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/30.jpg)
Stupid Criticisms
• This model is totally reactive, not proactive
• This model is steady-state, not innovative
![Page 31: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/31.jpg)
Our tools, let me show you them
Tinderbox http://www.mozilla.org/tinderbox.html
Mochitest http://developer.mozilla.org/en/docs/Mochitest
Litmus http://wiki.mozilla.org/Litmus
MXR http://mxr.mozilla.org/
Dehydra http://developer.mozilla.org/en/docs/Dehydra
Bug Policy http://www.mozilla.org/projects/security/security-bugs-policy.html
Bugzilla https://bugzilla.mozilla.org/
Fuzzers http://www.squarefree.com/2007/08/02/introducing-jsfunfuzz/
![Page 32: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/32.jpg)
Remember This Slide
• Capture expensive knowledge everywhere, so that you don’t have to re-learn it
• Apply that knowledge everywhere
• Nothing lands without tests
• Nothing lands without code review
• Counting bugs is stupid, try harder
![Page 33: The Most Important Thing: How Mozilla Does Security and What You Can Steal](https://reader033.fdocuments.us/reader033/viewer/2022052905/5585827cd8b42aa7148b4a56/html5/thumbnails/33.jpg)
Credits
• Developer Kit, Sean Martell, http://developer.mozilla.org/en/docs/Promote_MDC• Waterfall, dave.hau, http://flickr.com/photos/davehauenstein/271469348/• Alarm, Shannon K, http://flickr.com/photos/shannonmary/96320881/• Oops, estherase, http://flickr.com/photos/estherase/24513484/• Card House, Bah Humbug, http://flickr.com/photos/gibbons/2294375187/• Bulldozer, Atli Harðarson, http://flickr.com/photos/atlih/2223726160/