One wrong click… Techniques to analyze malicious code Tyler Hudak.
Your Code is Wrong
-
Upload
nathanmarz -
Category
Technology
-
view
6.370 -
download
1
description
Transcript of Your Code is Wrong
![Page 1: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/1.jpg)
Your Code is Wrong
Nathan Marz@nathanmarz 1
![Page 2: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/2.jpg)
Let’s start with an example
![Page 3: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/3.jpg)
Storm’s “reportError” method
![Page 4: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/4.jpg)
(Storm is a realtime computation system, like Hadoop but for realtime)
![Page 5: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/5.jpg)
Storm architecture
![Page 6: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/6.jpg)
Storm architecture
Master node (similar to Hadoop JobTracker)
![Page 7: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/7.jpg)
Storm architecture
Used for cluster coordination
![Page 8: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/8.jpg)
Storm architecture
Run worker processes
![Page 9: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/9.jpg)
Storm’s “reportError” method
![Page 10: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/10.jpg)
Used to show errors in the Storm UI
![Page 11: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/11.jpg)
Error info is stored in Zookeeper
![Page 12: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/12.jpg)
What happens when a user deploys code like this?
![Page 13: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/13.jpg)
Denial-of-service on Zookeeper and cluster goes down
![Page 14: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/14.jpg)
Robust!
Designed input space Actual input space
![Page 15: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/15.jpg)
Your code is wrong
![Page 16: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/16.jpg)
Your code is literally wrong
![Page 17: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/17.jpg)
Your code is wrong
![Page 18: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/18.jpg)
![Page 19: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/19.jpg)
Why do you believe your code is correct?
![Page 20: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/20.jpg)
Your code
Dependency 1
Dependency 2
Dependency 3
![Page 21: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/21.jpg)
Dependency 1
Dependency 4
Dependency 5
![Page 22: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/22.jpg)
Dependency 4
Dependency 6
Dependency 9
Dependency 7
Dependency 8
![Page 23: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/23.jpg)
Dependency 3,000,000
Hardware
![Page 24: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/24.jpg)
Electronics
![Page 25: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/25.jpg)
Chemistry
![Page 26: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/26.jpg)
Atomic physics
![Page 27: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/27.jpg)
Quantum mechanics
![Page 28: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/28.jpg)
I think I can safely say that nobody understands
quantum mechanics.
Richard Feynman
![Page 29: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/29.jpg)
Your code is wrong
![Page 30: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/30.jpg)
Your code
...
![Page 31: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/31.jpg)
All the software you’ve used has had bugs in it
![Page 32: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/32.jpg)
Including the software you’ve written
![Page 33: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/33.jpg)
Your code issometimes correct
![Page 34: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/34.jpg)
That’s good enough!
![Page 35: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/35.jpg)
![Page 36: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/36.jpg)
Treat code as nondeterministic
![Page 37: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/37.jpg)
Embrace “your code is wrong”to design better software
![Page 38: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/38.jpg)
Robust!
Designed input space Actual input space
![Page 39: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/39.jpg)
Robust!
Designed input space Actual input space
![Page 40: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/40.jpg)
An example
![Page 41: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/41.jpg)
Learning from Hadoop
Jobtracker
Job
Job
Job
![Page 42: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/42.jpg)
Learning from Hadoop
Jobtracker
Job
Job
Job
![Page 43: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/43.jpg)
Learning from Hadoop
Jobtracker
Job
Job
Job
![Page 44: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/44.jpg)
Your code is wrong
![Page 45: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/45.jpg)
So your processes will crash
![Page 46: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/46.jpg)
Storm’s daemons are process fault-tolerant
![Page 47: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/47.jpg)
Storm
Nimbus
Topology
Topology
Topology
![Page 48: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/48.jpg)
Storm
Nimbus
Topology
Topology
Topology
![Page 49: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/49.jpg)
Storm
Nimbus
Topology
Topology
Topology
![Page 50: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/50.jpg)
Storm
Nimbus
Topology
Topology
Topology
![Page 51: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/51.jpg)
Storm
Nimbus
Topology
Topology
Topology
![Page 52: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/52.jpg)
Robust!
Designed input space Actual input space
![Page 53: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/53.jpg)
Robust!
Designed input space Actual input space
![Page 54: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/54.jpg)
The impact of code being wrong
![Page 55: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/55.jpg)
Robust!
Designed input space Actual input space
Failures!Bad performance!Security holes!
Irrelevant!
![Page 56: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/56.jpg)
Design principle #1
Measuring and monitoring are the foundation of solid engineering
![Page 57: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/57.jpg)
Measuring: Under what range of inputs does my software function well?
![Page 58: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/58.jpg)
Monitoring: What’s the actual input space of my software?
![Page 59: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/59.jpg)
Measure & MonitorLatencyThroughputStack tracesBuffer sizesMemory usageCPU usage#threads spawned...
![Page 60: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/60.jpg)
How you monitor your software is as important as its functionality
![Page 61: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/61.jpg)
Design principle #2
Embrace immutability
![Page 62: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/62.jpg)
Read/write databaseApplication
![Page 63: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/63.jpg)
MySQLApplication
![Page 64: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/64.jpg)
MongoDBApplication
![Page 65: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/65.jpg)
RiakApplication
![Page 66: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/66.jpg)
CassandraApplication
![Page 67: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/67.jpg)
HBaseApplication
![Page 68: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/68.jpg)
Your code is wrong
![Page 69: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/69.jpg)
So data will be corrupted
![Page 70: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/70.jpg)
And you may not know why
![Page 71: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/71.jpg)
ViewsImmutable,
ever-growing data
Application
Architecture based on immutability
![Page 72: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/72.jpg)
ViewsImmutable,
ever-growing data
Application
Lambda architecture
![Page 73: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/73.jpg)
Design principle #3
Minimize dependencies
![Page 74: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/74.jpg)
The less that can go wrong, the less that will go wrong
![Page 75: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/75.jpg)
Example:Storm’s usage of Zookeeper
![Page 76: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/76.jpg)
Worker locations stored in Zookeeper
![Page 77: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/77.jpg)
All workers must know locations of other workers to send messages
![Page 78: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/78.jpg)
Two ways to get location updates
![Page 79: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/79.jpg)
1. Poll Zookeeper
Worker Zookeeper
![Page 80: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/80.jpg)
2. Use Zookeeper “watch” feature to get push notifications
Worker Zookeeper
![Page 81: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/81.jpg)
Method 2 is faster but relies on another feature
![Page 82: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/82.jpg)
Storm uses both methods
Worker Zookeeper
![Page 83: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/83.jpg)
If watch feature fails, locations still propagate via polling
![Page 84: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/84.jpg)
Eliminating dependence justified by small amount of code required
![Page 85: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/85.jpg)
Design principle #4
Explicitly respect functional input ranges
![Page 86: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/86.jpg)
Storm’s “reportError” method
![Page 87: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/87.jpg)
Implement self-throttling to avoid overloading other systems
![Page 88: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/88.jpg)
Design principle #5
Embrace recomputation
![Page 89: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/89.jpg)
“Your code is wrong” meanings1. Design input space differs from actual input space2. The logic of your code is wrong3. Requirements are constantly changing
![Page 90: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/90.jpg)
You must be able to change your code to match shifting requirements
![Page 91: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/91.jpg)
Example: blogging software
![Page 92: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/92.jpg)
New requirement: search
![Page 93: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/93.jpg)
Have to build a search index
![Page 94: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/94.jpg)
![Page 95: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/95.jpg)
Recomputation gives you so much more
![Page 96: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/96.jpg)
ViewsImmutable,
ever-growing data
Application
![Page 97: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/97.jpg)
Building software no different than any other engineering
![Page 98: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/98.jpg)
The underlying challenges are the same
![Page 99: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/99.jpg)
![Page 100: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/100.jpg)
![Page 101: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/101.jpg)
What will break it?
![Page 102: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/102.jpg)
What are limits of my dependencies?
![Page 103: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/103.jpg)
How can I add redundancy to increase robustness?
![Page 104: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/104.jpg)
Can I isolate failures?
![Page 105: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/105.jpg)
Our raw materials are ideas instead of matter
![Page 106: Your Code is Wrong](https://reader034.fdocuments.us/reader034/viewer/2022051322/540dd83e8d7f72927e8b4a23/html5/thumbnails/106.jpg)
Thank you