Challenges and Best Practices of Database Continuous Delivery
-
Upload
dbmaestro-database-devops -
Category
Technology
-
view
324 -
download
0
Transcript of Challenges and Best Practices of Database Continuous Delivery
![Page 1: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/1.jpg)
![Page 2: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/2.jpg)
▪You will be on mute for the duration of the event▪Please type a message in the Questions box in
the Control Panel if you can’t hear us (please check your speakers and GoToWebinar audio settings first)
▪If you have questions during the session, please submit them on the Q&A bar on your GoToWebinar dashboard and we will address them at the end
▪A recording of the full webinar will be put up online
Before we Begin
![Page 3: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/3.jpg)
Yaniv YehudaCo-Founder & CTO at DBmaestro
Spent the last years raising awareness about the challenges around database development and deployment, and how to support database Continuous Delivery.
Joined NessPRO as a product manager for the line of SAP complementary products and DevOps solutions for the DB.
Shachar FurmanProduct Manager, Central Systems, NessPRO
About the presenters
![Page 4: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/4.jpg)
Outline
▪ The Adoption of Continuous Delivery▪ The Reasons why the Database is Often Left Behind▪ Single Source of Truth for your Database▪ Best Practices for Including the Database in CD▪ The Bigger Picture – Automation and Stopping the
Line▪ Q&A
![Page 5: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/5.jpg)
NessPRO is Ness Technologies’ products group and the sole representative in
Israel of more than 30 international and Israeli organizations which develop
software products designated for the enterprise market.
About NessPRO
![Page 6: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/6.jpg)
About DBmaestroThe leading provider of DevOps for DatabaseDatabase development and deployment automation
![Page 7: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/7.jpg)
Media Coverage
![Page 8: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/8.jpg)
OperationsDevelopment
Smoother EffortLess Risk
Effort PeaksHigh Risk
Agile & DevOps
![Page 9: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/9.jpg)
The pain–Fortune 1000 by IDC
Application Downtime Cost
$2B/YDeployments/
month 2x Growth
Accelerate Delivery by 20%
Compliance & Audit
Enforcement
Infra Failure Hourly Cost
$100K
IDC DevOps Best Practices metrics: Fortune 1000 Survey, December 2014
Loss of Reputation
![Page 10: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/10.jpg)
▪Recently Conducted Survey
Continuous Delivery moving ahead!
![Page 11: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/11.jpg)
▪Every business is an IT business▪Customers demand that you deliver new features faster
− Agile Development− Process Automation− DevOps
▪Can’t wait 6 months for that next waterfall release…▪If you don’t, your competitor probably will
DevOps & CD: a must for every company
![Page 12: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/12.jpg)
Continuous Integration
Continuous Delivery
Continuous Deployment
Continuous Processes
![Page 13: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/13.jpg)
▪ More rapid changes▪ Fewer changes backed out▪ Better collaboration▪ Fewer defects
▪ Ultimately better service ▪ Happy customers ▪ Profitability
How Do I Measure Success?
![Page 14: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/14.jpg)
Why Continuous Delivery?
![Page 15: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/15.jpg)
15
But…what about the
database?
![Page 16: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/16.jpg)
Only 13% are actually performing basic CD practices for the DB!!!
Manual work: cant scale, cant match CD frequency
not repeatable, prone to error
Continuous Delivery is big and getting bigger, but...
![Page 17: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/17.jpg)
Manual steps lead to Chaos in Emergency Times
90%Rate this as a risk factor, but
53%Break the process and test urgent hot-fixes in pre-
production
![Page 18: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/18.jpg)
Old adage but true– The database is often neglected and
therefore can become the weakest link– Manual processes
Database/Code Silos exist…– Don’t always communicate effectively– Need to follow same procedures & best
practices Essential from a compliance and business point of view
Should be the strongest link
Is the database the Weakest link in a chain ???
![Page 19: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/19.jpg)
The Database is a constraint
![Page 20: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/20.jpg)
Only 13% automate… the rest do manual steps…
![Page 21: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/21.jpg)
What is the problem?
▪Root Causes for issues:− Challenging manual source control process− Static deployments code overrides and configuration drift− Dynamic deployments tools unaware of version control− No release automation red-flags – don’t know when to stop
the line…
![Page 22: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/22.jpg)
File Version Control Process
Today: Two isolated processes DB Development Process
Check-Out Script
Modify Script
Get updated Script from DB
Check-In Script
Compile Scriptin DB
Debug Scriptin DB
?
??
?
A
A’
Version control is out of sync from the database and cannot act as a Single Source of
Truth
![Page 23: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/23.jpg)
90%Rate this as a risk factor, yet
72%Admit database may not be in sync with the source repository
![Page 24: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/24.jpg)
Challenges:Code-overridesWorking on the wrong revisions Scripts do not always find their way to the version control solutionOut of process updates go unnoticedHard to locate outdated update scripts
Playing safe? What we really need: The upgrade scriptThe actual code of the objectA roll-back script
Scripts & version control
![Page 25: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/25.jpg)
X1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Database Deploy Script
DevDev
DevModel
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
Out of Process Change
XX
XXX
? 1.1.1
X
Challenge with static scripts…
Configuration drift…
![Page 26: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/26.jpg)
60%Of those manually building scripts have to fix or tweak them regularly as part of a
deployment process
![Page 27: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/27.jpg)
80%Rate this as a risk factor, yet
71%Follow manual processes to create their database
deploy scripts
![Page 28: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/28.jpg)
▪Scripts− Hard to test in their entirely (holistically)− Hard to test due to colliding dependencies− Need to run in a specific order…− Much harder to deal with project scope changes
▪Scripts, unless super sophisticated:− Unaware of changes made in the target environment− Time passed from their coding to the time they are run− Potentially overriding production hot-fixes or work done in parallel by
another team
Scripts are static…
![Page 29: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/29.jpg)
Test cases using compare & sync tools:
An index exists in source (QA) but not in target (Production)What should we do? Add the index or not?
Using tools
![Page 30: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/30.jpg)
Safe to automate?Sure… (?)
Challenge with ‘Compare & Sync’
![Page 31: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/31.jpg)
Safe to automate?No. Requires manual inspection…
Challenge with ‘Compare & Sync’
![Page 32: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/32.jpg)
70%of those using compare & sync tools have to review and fix the results as they can't always
trust them to automatically deploy correctly
![Page 33: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/33.jpg)
A compare & sync tool:Is unaware of any changes that occurred before the time it ranHas no knowledge of changes that took place at the target environment
Does not leverage version control for more informationUnable to deal with conflicts & merges between different teams
Requires manual inspection Requires detailed knowledge regarding each change as part of the process
So…no automation… We fear for automating problems into
production and a major risk!!!
Challenges
![Page 34: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/34.jpg)
The Solution to the challenges
![Page 35: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/35.jpg)
Modern VC integrated DB process
![Page 36: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/36.jpg)
Revision history
Actions
Standard IDE
Change management
Enforced and integrated to existing process
![Page 37: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/37.jpg)
LeverageVersion control knowledge
intoDeployment decisions
![Page 38: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/38.jpg)
1.11.21.31.41.51.61.7
*
Int QA Stage ProdDev
DevDev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.1 1.7
1.1.1 1.7
1.1 1.1 1.11.41.7
Out of Process Change
1.1.11.7 1.1.11.7
Validate
1.4 1.5
1.5 1.6
1.6 1.7
Configuration Drift prevention / conflict identification and Validated execution
1.4 1.5
1.5 1.6
1.6 1.7
OR
Baseline aware analysis
Validated execution / Build & deploy on demand
Validate
![Page 39: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/39.jpg)
Understand the nature of the changesRaise red flags on conflicts Support out-of-process changesUtilize baseline aware analysis
Safety Net Deployment Automation
![Page 40: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/40.jpg)
If we had the index in the baseline (previous version) and no longer in Dev (i.e. - removed) => we should take it down from production…
(Deploy Change)
Deploying changes if needed
Development BaselinePrevious Label /
Production Golden Copy
Production
![Page 41: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/41.jpg)
Development BaselinePrevious Label /
Production Golden Copy
Production
BUT… If no index in baseline => someone else added it to Production…we should protect the NEW index on production!!!
(Protect Target)
Or protecting target environment…
![Page 42: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/42.jpg)
Dealing with conflicts => merging changes
![Page 43: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/43.jpg)
Conflict Resolving – Meta Data/Content
![Page 44: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/44.jpg)
Continuous Delivery Pipeline Builder
• Define a process• Automate the process• Prevent/Alert out of process changes
![Page 45: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/45.jpg)
Raise red flags to stop the line…if requires human intervention
Impact Analysis! Not Damage Control…
![Page 46: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/46.jpg)
Automate “everything”– Package the deployment of database changes along with all your
other application components to give a unified picture Move the process upstream
– Easily promote the same package (including database changes!) from one environment to the next, handling environment-specific differences automatically
Create the deployment pipeline
For successful CD:
![Page 47: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/47.jpg)
Requited integrations into entire continuous delivery pipeline:
IDEs (Oracle, Microsoft, MySQL in dev)
Source control (MsTFS, Git, Subversion etc.)
Task based development and ticketing systems (Jira, MsTFS, IBM RTC etc.)
Build tools (Jenkins, bamboo etc.)
ARA tools (IBM uDeploy, CA release automation, Chef etc.)
Integrated CD world…
![Page 48: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/48.jpg)
▪ Focusing on changes rather than managing changes and dealing with re-work, boosted overall productivity of 250 developers. We estimate we were able to do 15% more with the same resources.
▪We went from several fix-centric deployments a day, to one feature-centric deployment a week.▪The amount of incidents in production has declined as well. We had 20% less incidents. ▪CIO @ Credit Card company
“Testimonials - Efficiency
“
![Page 49: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/49.jpg)
▪ Regulation requirement - SOX and derived 357▪Auditing - change management approach with change management auditing. You have to do it anyway, do it automatically and efficiently. ▪Change management in production itself is regulatory required (ITIL). But you cannot ensure it without managing the whole process starting at Dev.▪We have to comply with regulation - but the business benefits from it. ▪CIO @ Credit Card company
“Testimonials - Regulation
![Page 50: Challenges and Best Practices of Database Continuous Delivery](https://reader035.fdocuments.us/reader035/viewer/2022062503/58719b3b1a28ab044e8b5afb/html5/thumbnails/50.jpg)
Thank you!
Q & AShachar Furman Yaniv [email protected] [email protected] www.ness-tech.co.il www.dbmaestro.com