Architecture for the cloud deployment case study future

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


These are the last lectures for the course "Architecting for the Cloud". There are lectures on deployment, a case study, and future trends

Transcript of Architecture for the cloud deployment case study future

  • Architecting for the Cloud Len and Matt Bass Deployment
  • Deployment pipeline Developers commit code Code is compiled Binary is processed by a build and unit test tool which builds the service Integration tests are run followed by performance tests. Result is a machine image (assuming virtualization) The service (its image) is deployed to production. 2
  • Deployment Overview 3 Multiple instances of a service are executing Red is service being replaced with new version Blue are clients Green are dependent services VAVB VBVB UAT / staging / performance tests
  • Additional considerations Recall release plan. Each release requires coordination among stakeholders and development teams. Can be explicitly performed Can be implicit in the architecture (we saw an example of this) You as an architect need to know how frequently new releases will be deployed. Some organizations deploy dozens of times a day Other organizations deploy once a week Still others once a month or longer
  • Monthly deployment There is time for coordination among development teams to ensure that components are consistent There is time for release manager to ensure that a release is correct before moving it to production
  • Weekly deployment Limited time for coordination among development teams Time for release manager to ensure that a release is correct before moving it to production
  • Daily deployment No time for Coordination among development teams Release manager to validate release These items must be implicit in the architecture.
  • Deployment goal and constraints Goal of a deployment is to move from current state (N instances of version A of a service) to a new state (N instances of version B of a service) Constraints associated with Continuous Deplloyment: Any development team can deploy their service at any time. I.e. New version of a service can be deployed either before or after a new version of a client. (no synchronization among development teams) It takes time to replace one instance of version A with an instance of version B (order of minutes) Service to clients must be maintained while the new version is being deployed. 8
  • Deployment strategies Two basic all of nothing strategies Red/Black leave N instances with version A as they are, allocate and provision N instances with version B and then switch to version B and release instances with version A. Rolling Upgrade allocate one instance, provision it with version B, release one version A instance. Repeat N times. Other deployment topics Partial strategies (canary testing, A/B testing,). We will discuss them later. For now we are discussing all or nothing deployment. Rollback Packaging services into machine images 9
  • Trade offs Red/Black and Rolling Upgrade Red/Black Only one version available to the client at any particular time. Requires 2N instances (additional costs) Accomplished by moving environments Rolling Upgrade Multiple versions are available for service at the same time Requires N+1 instances. Both are heavily used. Choice depends on Cost Managing complications from using rolling upgrade 10 Update Auto Scaling Group Sort Instances Remove & Deregister Old Instance from ELB Confirm Upgrade Spec Terminate Old Instance Wait for ASG to Start New Instance Register New Instance with ELB Rolling Upgrade in EC2
  • Types of failures during rolling upgrade Rolling Upgrade Failure Provisioning Research topic Logical failure Inconsistencies to be discussed Instance failure Handled by Auto Scaling Group in EC2 11
  • What are the problems with Rolling Upgrade? Recall that any development team can deploy their service at any time. Three concerns Maintaining consistency between different versions of the same service when performing a rolling upgrade Maintaining consistency among different services Maintaining consistency between a service and persistent data 12
  • Maintaining consistency between different versions of the same service Key idea differentiate between installing a new version and activating a new version Involves feature toggles (described momentarily) Sequence Develop version B with new code under control of feature toggle Install each instance of version B with the new code toggled off. When all of the instances of version A have been replaced with instances of version B, activate new code through toggling the feature. 13
  • Issues Do you remember feature toggles? How do I manage features that extend across multiple services? How do I activate all relevant instances at once? 14
  • Feature toggle Place feature dependent new code inside of an if statement where the code is executed if an external variable is true. Removed code would be the else portion. Used to allow developers to check in uncompleted code. Uncompleted code is toggled off. During deployment, until new code is activated, it will not be executed. Removing feature toggles when a new feature has been committed is important. 15
  • Multi service features Most features will involve multiple services. Each service has some code under control of a feature toggle. Activate feature when all instances of all services involved in a feature have been installed. Maintain a catalog with feature vs service version number. A feature toggle manager determines when all old instances of each version have been replaced. This could be done using registry/load balancer. The feature manager activates the feature. 16
  • Activating feature The feature toggle manager changes the value of the feature toggle. Two possible techniques to get new value to instances. Push. Broadcasting the new value will instruct each instance to use new code. If a lag of several seconds between the first service to be toggled and the last can be tolerated, there is no problem. Otherwise synchronizing value across network must be done. Pull. Querying the manager by each instance to get latest value may cause performance problems. A coordination mechanism such as Zookeeper will overcome both problems. 17
  • Maintaining consistency across versions (summary) Install all instances before activating any new code Use feature toggles to activate new code Use feature toggle manager to determine when to activate new code Use Zookeeper to coordinate activation with low overhead 18
  • Maintaining consistency among different services Use case: Wish to deploy new version of service A without coordinating with development team for clients of service A. I.e. new version of service A should be backward compatible in terms of its interfaces. May also require forward compatibility in certain circumstances, e.g. rollback 19
  • Achieving Backwards Compatibility APIs can be extended but must always be backward compatible. Leads to a translation layer External APIs (unchanging but with ability to extend or add new ones) Translation to internal APIs Client Client Internal APIs (changes require changes to translation layer but do not propagate further)
  • What about dependent services? Dependent services that are within your control should maintain backward compatibility Dependent services not within your control (third party software) cannot be forced to maintain backward compatibility. Minimize impact of changes by localizing interactions with third party software within a single module. Keeping services independent and packaging as much as possible into a virtual machine means that only third party software accessed through message passing will cause problems. 21
  • Forward Compatibility Gracefully handle unknown calls and data base schema information Suppose your service receives a method call it does not recognize. It could be intended for a later version where this method is supported. Suppose your service retrieves a data base table with an unknown field. It could have been added to support a later version. Forward compatibility allows a version of a service to be upgraded or rolled back independently from its clients. It involves both The service handling unrecognized information The client handling returns that indicate unrecognized information. 22
  • Maintaining consistency between a service and persistent data Assume new version is correct we will discuss the situation where it is incorrect in a moment. Inconsistency in persistent data can come about because data schema or semantics change. Effect can be minimized by the following practices (if possible). Only extend schema do not change semantics of exist