Migrating Edmunds.com to AWS (re:Invent 2013 DMG205)
41
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. John Martin November 14, 2013 Edmunds.com on AWS
-
Upload
john-martin -
Category
Technology
-
view
418 -
download
1
description
Taking a stack composed of 30 web applications and their service dependencies to the cloud is no easy feat. Do you take the entirety of the stack or go the hybrid path? How transparent should the end result be to your technology teams? Does it look exactly the same in the cloud as it does in your data center? These are not rhetorical questions; they were very real for those tasked with the challenge of taking Edmunds.com to the AWS Cloud. This talk addresses these questions and many more, examining the challenges, successes, and lessons learned as the team took their first steps out of their own data centers. Accompanying video is available on YouTube: http://youtu.be/itbNET2dc3c
Transcript of Migrating Edmunds.com to AWS (re:Invent 2013 DMG205)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
John Martin
November 14, 2013
Edmunds.com on AWS
Moving isn’t easy
• A move isn’t easy • Taking something your familiar with elsewhere • How does that work again?
Moving Isn’t Easy
Today’s Agenda• Technology Overview • The Business Case • The Approach • Challenges • What’s Next?
John Martin @tekbuddha
WHO AM I?
John Martin @tekbuddha
WHO AM I?
• 15+ years in .com • 10+ years of Java • Old School Ops • New School Cultures
The Company
• Founded in 1966 • First online in 1994 as a gopher • First website in 1996
The Company
The Environment
• 30+ web applications across 300+ hosts. • Java on Redhat Linux • Tomcat, Solr, Coherence, Mongo, ActiveMQ
The Environment
The Environment
• Chef + Cloudstack/UCS • Perforce, Jenkins, Nexus, Selenium, JMeter • AppDynamics, Splunk, RTview, Zenoss
The Environment
The Deployment Pipeline
• OSS + Homegrown Tooling • All artifacts flow through the pipeline • Release Cycles: 1 Month > 3 Weeks > 1 Week
The Deployment Pipeline
The Business Case
• Simple: disaster recovery • Bad business not to evaluate cloud services
The Business Case
The Approach
• How to move 30+ apps and 300+ servers? • Make it run like it already does • Avoid biting off too much at once
The Approach
Objectives• Minimize change / leverage existing toolchain • Manage cost • Provide initial design patterns for future builds
Source: http://is.gd/YmewdR
• A move isn’t easy. • Making the things we were familiar with work
elsewhere wasn’t easy
Challenges
Source: http://is.gd/rayzL1
• Necessary from Day 1 • Familiar concepts • Infrastructure Is Code
CloudFormation
CloudStack
• Different structure, similar logic • Defining single hosts, not groups of services • No definitions of network resources
CloudStack JSON
CloudFormation
• Define network and services, not hosts • Live by cf-validate • Automate creation / avoid manual editing
CloudFormation JSON
Source: http://is.gd/rayzL1
• Still a core dependency • No great options in EC2 without refactor • Move static content to S3
NFS
Source: http://is.gd/rayzL1
• No physical load balancers • ELBs, HAproxy, and Chef were the key • Learn how Public/Private ELBs work with VPC
Load Balancing
Source: http://is.gd/rayzL1
• Worked but can leave behind a mess • Tooling must be prepared for ephermal nodes • Helped survive outages in US-EAST
[auto-]scaling
Testing… 1! 2! 3!
Source: http://is.gd/xKdI6E
• Additional live-traffic tests • Deployment of internal services • Prepare for full move out of data center
What Next?
Source: http://is.gd/D8bVaC
• Greater adoption of SOA principles • Full refactor of data pipelines (in and out) • Refactor for fragility of cloud environments
Refactor
THE BIG MOVE
• Work thus far completed by a small team • The Big Move will be all hands on deck • A two year goal to be fully cloud-based
The Big Move
Please give us your feedback on this presentation
As a thank you, we will select prize winners daily for completed surveys!
DMG205 Thank You
