Being Elastic -- Evolving Programming for the Cloud
-
Upload
randy-shoup -
Category
Documents
-
view
1.433 -
download
3
description
Transcript of Being Elastic -- Evolving Programming for the Cloud
![Page 1: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/1.jpg)
Being ElasticEvolving Programming for the Cloud
Randy ShoupeBay Chief Engineer
QCon San FranciscoNovember 4, 2010
![Page 2: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/2.jpg)
Cloud is a New Ecosystem
• Resource-Rich• Inexpensive• Available• Elastic• Managed
• Remote• Virtual• Variable• Ephemeral• Metered
![Page 3: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/3.jpg)
The Old and the New
flickr.com/photos/slapbcn/2035854990/
![Page 4: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/4.jpg)
Developers Must Adapt
• Benefits are huge• Constraints are real
• Adapt to the constraints to get the benefits• Most adaptations force otherwise good
development practices (!)
![Page 5: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/5.jpg)
Scaling for the Cloud
• To leverage scalable infrastructure, you need– Scalable application– Scalable development practices– Scalable culture
• Principles and practices discussed widely (particularly at QCon!)– Convergence of Architecture, Agile, DevOps, etc.
![Page 6: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/6.jpg)
Universal Scalability Law
• Throughput limited by two factors– Contention (α): queueing on shared resource– Coherency (β): communication and coordination
among multiple nodes• Formulated by Neil Gunther– Generalization of Amdahl’s Law– Grounded in real-world physical processes
http://www.perfdynamics.com/Manifesto/USLscalability.html
![Page 7: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/7.jpg)
Programming in the Cloud
• Parallelism• Layering• Services• State management• Data model• Failure handling• Testing
![Page 8: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/8.jpg)
Parallelism• Think parallel!– Simple parallel algorithm out-scales
“smarter” non-parallel algorithm– Exploiting elasticity fundamentally
requires parallelism
• Request processing– Parallelism through
• Routing requests• Aggregating services• Queueing work
– Async I/O and Futures are your friends
flickr.com/photos/sharif/2423144088/
![Page 9: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/9.jpg)
Parallelism• Offline computation– Parallelism through
• Workload partitioning• Partitioning data and processing
– E.g., pipelines, MapReduce
“There are 3 rules to follow when parallelizing large codes. Unfortunately, no one knows what these rules are.” – W. Somerset Maugham and Gary Montry
flickr.com/photos/sharif/2423144088/
![Page 10: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/10.jpg)
Layering• Strictly layered system– Each software layer handles a
single set of responsibilities– Cloud makes this particularly
important
• Common code in frameworks– Configuration– Instrumentation– Failure management
“All problems in computer science can be solved by another level of indirection … Except for the problem of too many layers of indirection.” – David Wheeler
![Page 11: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/11.jpg)
Services• Decompose system
functionality into services– Simple– Single-purpose– Modular– Stateless– Multi-instance
• Compose complex application behavior from simple elements
“Make everything as simple as possible, but not simpler.” – Albert Einstein
![Page 12: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/12.jpg)
Services
The service should be the fundamental unit of …• Composition– Combine simple services into complex systems
• Dependency– Depend on a service interface, not implementation
• Addressing– Talk to logical endpoint (URI), not IP:port
• Persistence– Abstract and isolate persistence within a service
• Deployment
![Page 13: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/13.jpg)
State Management• Stateless instances– Instances are ephemeral– Memory / storage is fast and
local, but transient and inconsistent
– Equivalent to a cache
• Durable state in persistent storage– (Many implementations)
“Here today, gone tomorrow.” – American proverb
![Page 14: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/14.jpg)
Key-Value Data Model• Distributed key-value stores– Simple– Horizontally scalable– (Many implementations)
• Constrained by design– Cannot express complex relationships– Limited schema– Predictable, bounded performance
• Greater burden on application– Joins, aggregations, sorts, etc.– Integrity and schema validation
![Page 15: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/15.jpg)
Key-Value Data Model• Plan to Shard– Partition by key– Parallelize writes for write throughput– Parallelize reads for read throughput
• Plan to Denormalize– Optimize for reads– Precalculate joins, aggregations, sorts– Use async queue to update– Learn to tolerate transient
inconsistency
![Page 16: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/16.jpg)
Failure Handling• Expect failures and handle them– e.printStackTrace() does
not count!
• Failure handling means– Graceful degradation– Timeouts and retries– Throttling and back-off
• Abstract through frameworks and policies
“Hope is not a strategy.” – Various
flickr.com/photos/davidwatts1978/3199405401/
![Page 17: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/17.jpg)
Testing• Test early and often– Test-driven and Test-first approaches work
particularly well in the cloud– Automated testing works particularly well– Incremental development and deployment
• Test end-to-end– Much easier to test at load– More challenging to simulate all combinations and
failure modes“In the data center, robust code is best practice. In the cloud, it is essential.” – Adrian Cockcroft
![Page 18: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/18.jpg)
Operating in the Cloud
• DevOps Mindset• Configuration Injection• Instrumentation• Monitoring• Metering
![Page 19: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/19.jpg)
DevOps Mindset
• In the Cloud, Developers Ops– Everyone will use tools for deployment, monitoring,
etc.• Manageability as an investment– Management interfaces need as much engineering
discipline as functional interfaces• Automate, automate, automate– Repeatable builds and deployments– Provisioning, scaling, failure management, etc.– If you do it more than twice, script it
![Page 20: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/20.jpg)
Configuration Injection• Boot instance from bare minimum package– Externalize all variability– Separate code and configuration
• Configuration should drive– Instance role– Service and resource dependencies– Behavior / capabilities
• Inject– At boot time– At runtime
“I do my work at the same time each day – the last minute.” – Unknown
flickr.com/photos/andresrueda/2983149263/
![Page 21: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/21.jpg)
Instrumentation• Fully instrument all components– Cannot attach debugger / profiler– Remotely diagnose, profile, debug– Avoid “monitoring fatigue”
• Logging is insufficient– Needs to be automatically
interpretable and actionable
• Use a framework– Make part of your infrastructure
flickr.com/photos/wwarby/3016549999/
![Page 22: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/22.jpg)
Monitoring• Monitor requests end-to-end– Trace all requests as they flow
from component to component
• Monitor component activity and performance– Metrics, metrics, metrics
• Understand your typical system behavior– Spiky, flat, sinusoidal?– Know the baseline so you know
what is abnormal
flickr.com/photos/pdstahl/3884421899/
![Page 23: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/23.jpg)
Metering• Fixed cost -> variable cost– Processing– Storage– Network
• Efficiency matters– Any savings goes directly to the bottom line– Any inefficiency is multiplied by many instances (!)
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.” – Donald Knuth
![Page 24: Being Elastic -- Evolving Programming for the Cloud](https://reader035.fdocuments.us/reader035/viewer/2022062312/554bb2b7b4c90594278b45c8/html5/thumbnails/24.jpg)
Evolving for the Cloud
“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.” – Charles Darwin