Database Deployment Pipeline - SQL In The City Workshop 2014
-
Upload
red-gate-software -
Category
Software
-
view
137 -
download
1
description
Transcript of Database Deployment Pipeline - SQL In The City Workshop 2014
![Page 1: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/1.jpg)
Building an Automated Database Deployment Pipeline
Continuous delivery for databases
Grant FritcheyRed Gate Software
![Page 2: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/2.jpg)
Goals
Understand the technology and process requirements to work towards automation step-by-step in your release pipeline.
Learn about the organizational changes necessary to support process modifications.
Appreciate why these changes are necessary in support of modern development and deployment methodologies.
![Page 3: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/3.jpg)
scarydba.com
@gfritchey
Grant FritcheyProduct Evangelist, Red Gate Software
/in/scarydba
![Page 4: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/4.jpg)
ALM – and where the database fits in
Three core processes in Application Lifecycle Management:
Governance Development Operations
Requirements Design, build and testDeployment
and monitoring
![Page 5: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/5.jpg)
Natural friction across pipeline
Development → Operations
![Page 7: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/7.jpg)
Natural friction across pipeline
Development → Operations
Operations → Development
Why?
![Page 8: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/8.jpg)
Development focus is on speed
![Page 9: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/9.jpg)
Agile Scrum Lean Feature-driven Development Iterative
![Page 10: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/10.jpg)
Operations focus is on production protectionprotection
![Page 11: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/11.jpg)
Monitoring Deployment Integrity Data Management
![Page 12: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/12.jpg)
Databases as a bottleneck
Odd languages SQL Cubes X-Query
3 reasons why databases have traditionally slowed down deployments:
Data persistence Data outlives
applications Data can’t be
replaced
DBA paranoia Frankly…
1 2 3
![Page 13: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/13.jpg)
A quick primer on continuous delivery
Development Test Production
Continuous integration
Continuous deployment
Continuous delivery
Release pipeline
Approval gate
![Page 14: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/14.jpg)
The goals of continuous delivery for databases
• Faster feedback on changes made– Continuously integrate team changes– Automated testing– Releases rehearsed in testing environments before
deployed to Production
= repeatability, repeatability and a strong team process for changes
![Page 15: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/15.jpg)
Focus on the pipelineThink of the Toyota production system…
Start with Leano Focus on the customer, eliminate wasteo Continuously Improveo Empower the teamo Optimize the wholeo Plan for changeo Automate processeso Build quality in
![Page 16: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/16.jpg)
SOURCECONTROL
CONTINUOUS INTEGRATION
AUTOMATED TESTING
AUTOMATED DEPLOYMENT
Four key stages of the deployment pipeline
![Page 17: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/17.jpg)
Create development environment for automation
Empower Development Environment should let them work at their speed
Take part in Development Instead of stopping bad deployments, stop bad development
Automate your build process Supply clean production data or supply good sample data
Get started on writing tests Build your library
![Page 18: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/18.jpg)
Different types of testing for different stages of the pipeline:
DevelopmentIntegration
Testing QAPre-Production/
Staging Production
Automate testing
Unit tests Integration tests
Automated tests
Deployment validation
Behaviour validation
Other validations
![Page 19: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/19.jpg)
B C D
A
Always Be Continuously Delivering
Deliver
![Page 20: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/20.jpg)
Every delivery is practice
![Page 21: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/21.jpg)
DBAs must work with Devs…
Yes, you must protect the data for the business, but that must be tempered with helping deliver functionality.
Changes to philosophy
![Page 22: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/22.jpg)
Devs must work with DBAs…
Yes, you may know more about business needs, so educate rather than isolate.
Changes to philosophy
![Page 23: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/23.jpg)
Project Management must think of operations as part of development…
Yes: Deployment is part of development Release 1.1 and on are part of development
o And planning for 1.1 is not premature optimization Data retention is part of development
Changes to philosophy
![Page 24: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/24.jpg)
Changes to workplace
SOURCECONTROL
CONTINUOUS INTEGRATION
AUTOMATED TESTING
AUTOMATED DEPLOYMENT
Technology is only half the story…
![Page 25: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/25.jpg)
Changes to workplace for continuous database delivery
• Empower developers to do more
• Bring DBAs into development teams
• Management must buy-in
• Management must get out of the way
![Page 26: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/26.jpg)
How are other companies getting started?
![Page 27: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/27.jpg)
Case Study: Boxon• Global packaging and labelling company
• Based in Sweden; sub-division in China
• Three business areas: profitable packing solutions; intelligent marking; customized big-bags solutions for bulk handling
• One developer responsible for ASP.NET application with SQL Server backend, enabling customers to control consistent printing of labels wherever the print shop is located in the world
![Page 28: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/28.jpg)
Boxon – Initial Process• Deploying to production was scary…
• Database changes not version-controlled
• 2-hour window - at night - to complete deployments to production – difficult to find quiet period with a global customer base
PAINPOINTS
![Page 29: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/29.jpg)
Boxon – Improved Process• Deployments are less scary now…• Tickets are logged and prioritized in Unfuddle (unfuddle.com)• Ticket numbers are logged in SVN and TeamCity to track
items• Now use SQL Source Control and Subversion to version DB
changes– Source of authority on database build
• TeamCity (CI build server) is triggered on check-in of changes
– Fast response – changes checked in frequently
• Notification if build fails
• Build packaged into a Nuget file for deployment
BUILDING A PIPELINE
![Page 30: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/30.jpg)
Case Study: Move with Us• Risk management solution and
sales and marketing channel provider for residential property businesses in the UK
• Located near Cambridge, UK
• Customers put a heavy load on databases, so updates need to be efficient
![Page 31: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/31.jpg)
Move with Us – Initial Process• Database code often not checked in to
VCS due to risk of conflicts – changes were communicated by email instead
• Hot fixes sometimes done directly on integration server
• 1 day needed to release updates – a lot of work in building and testing
• Problems with merging code led to a lot of firefighting
PAINPOINTS
![Page 32: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/32.jpg)
Move with Us – Improved Process• Each developer has a local copy of database• SQL Source Control and Subversion used to
check in database changes to version control• TeamCity (CI build server) automatically
compiles changes on check-in
• Successful builds packaged into a Nuget file
automatically– Fast getting here – multiple database changes
checked in per day
• Nuget package deployed to Test on demand• If problems detected, return to Dev to fix
BUILDING A PIPELINE
![Page 33: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/33.jpg)
Case Study: Practice Fusion• Web-based electronic health record
(EHR) company
• Founded in 2005
• Based in San Francisco
• Hosts over 50 million patient records
• Used in all 50 states and by 150k+ physicians
• 3 DBA engineers; 1 data architect; 10+ developers
![Page 34: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/34.jpg)
Practice Fusion – Initial Process• Hand-crafted SQL scripts - inconsistent
• Database changes not always version controlled
• Changes released by opening dozens of files in SSMS
• Long-winded team comms if further changes needed
• Smoke-testing deployments showed some objects had been deployed straight to Prod and weren’t in deployment script
PAINPOINTS
![Page 35: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/35.jpg)
Practice Fusion – Improved Process
• SQL Source Control and Subversion to version DB changes
– Source of authority on database build
• Jenkins (CI build server) is triggered on check-in of changes
– More time to develop; less time managing scripts
• Build failed by Jenkins if problems detected– Fewer issues deploying to production
• Jenkins, SQL Compare and SQL Data Compare used to deploy changes to environments
• Custom scripts for replication
BUILDING A PIPELINE
![Page 36: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/36.jpg)
Summary• Eliminate or mitigate friction• Adopt lean methodologies and focus• A-B-C-D• Change your philosophy• Change your workplace
![Page 37: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/37.jpg)
Goals Understand the technology and process requirements to
work towards automation step-by-step in your release pipeline.
Learn about the organizational changes necessary to support process modifications.
Appreciate why these changes are necessary in support of modern development and deployment methodologies.
![Page 38: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/38.jpg)
Documentation and resources Continuous Delivery by Jez Humble and David Farley (Addison Wesley) The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win by Gene Kim,
Kevin Behr and George Spafford (IT Revolution Press) The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt and Jeff Cox (Gower
Publishing Ltd.) Agile Organization by the agile admin (theagileadmin.com)
Further resources: Database Delivery Learning program: www.red-gate.com/delivery
– Patterns and practices on Simple-Talk– Tutorials in Red Gate training academy
www.youtube.com/user/RedGateVideos - for recorded seminars
![Page 39: Database Deployment Pipeline - SQL In The City Workshop 2014](https://reader038.fdocuments.us/reader038/viewer/2022110120/558524dfd8b42ae4748b4f27/html5/thumbnails/39.jpg)
Want to try the tools you’ve just seen? Come to our Hands-on Labs