Challenges and Best Practices of Database Continuous Delivery

Post on 08-Jan-2017

324 views 0 download

Transcript of Challenges and Best Practices of Database Continuous Delivery

▪You will be on mute for the duration of the event▪Please type a message in the Questions box in

the Control Panel if you can’t hear us (please check your speakers and GoToWebinar audio settings first)

▪If you have questions during the session, please submit them on the Q&A bar on your GoToWebinar dashboard and we will address them at the end

▪A recording of the full webinar will be put up online

Before we Begin

Yaniv YehudaCo-Founder & CTO at DBmaestro

Spent the last years raising awareness about the challenges around database development and deployment, and how to support database Continuous Delivery.

Joined NessPRO as a product manager for the line of SAP complementary products and DevOps solutions for the DB.

Shachar FurmanProduct Manager, Central Systems, NessPRO

About the presenters

Outline

▪ The Adoption of Continuous Delivery▪ The Reasons why the Database is Often Left Behind▪ Single Source of Truth for your Database▪ Best Practices for Including the Database in CD▪ The Bigger Picture – Automation and Stopping the

Line▪ Q&A

NessPRO is Ness Technologies’ products group and the sole representative in

Israel of more than 30 international and Israeli organizations which develop

software products designated for the enterprise market.

About NessPRO

About DBmaestroThe leading provider of DevOps for DatabaseDatabase development and deployment automation

Media Coverage

OperationsDevelopment

Smoother EffortLess Risk

Effort PeaksHigh Risk

Agile & DevOps

The pain–Fortune 1000 by IDC

Application Downtime Cost

$2B/YDeployments/

month 2x Growth

Accelerate Delivery by 20%

Compliance & Audit

Enforcement

Infra Failure Hourly Cost

$100K

IDC DevOps Best Practices metrics: Fortune 1000 Survey, December 2014

Loss of Reputation

▪Recently Conducted Survey

Continuous Delivery moving ahead!

▪Every business is an IT business▪Customers demand that you deliver new features faster

− Agile Development− Process Automation− DevOps

▪Can’t wait 6 months for that next waterfall release…▪If you don’t, your competitor probably will

DevOps & CD: a must for every company

Continuous Integration

Continuous Delivery

Continuous Deployment

Continuous Processes

▪ More rapid changes▪ Fewer changes backed out▪ Better collaboration▪ Fewer defects

▪ Ultimately better service ▪ Happy customers ▪ Profitability

How Do I Measure Success?

Why Continuous Delivery?

15

But…what about the

database?

Only 13% are actually performing basic CD practices for the DB!!!

Manual work: cant scale, cant match CD frequency

not repeatable, prone to error

Continuous Delivery is big and getting bigger, but...

Manual steps lead to Chaos in Emergency Times

90%Rate this as a risk factor, but

53%Break the process and test urgent hot-fixes in pre-

production

Old adage but true– The database is often neglected and

therefore can become the weakest link– Manual processes

Database/Code Silos exist…– Don’t always communicate effectively– Need to follow same procedures & best

practices Essential from a compliance and business point of view

Should be the strongest link

Is the database the Weakest link in a chain ???

The Database is a constraint

Only 13% automate… the rest do manual steps…

What is the problem?

▪Root Causes for issues:− Challenging manual source control process− Static deployments code overrides and configuration drift− Dynamic deployments tools unaware of version control− No release automation red-flags – don’t know when to stop

the line…

File Version Control Process

Today: Two isolated processes DB Development Process

Check-Out Script

Modify Script

Get updated Script from DB

Check-In Script

Compile Scriptin DB

Debug Scriptin DB

?

??

?

A

A’

Version control is out of sync from the database and cannot act as a Single Source of

Truth

90%Rate this as a risk factor, yet

72%Admit database may not be in sync with the source repository

Challenges:Code-overridesWorking on the wrong revisions Scripts do not always find their way to the version control solutionOut of process updates go unnoticedHard to locate outdated update scripts

Playing safe? What we really need: The upgrade scriptThe actual code of the objectA roll-back script

Scripts & version control

X1.11.1.11.11.21.31.41.51.61.7

Int QA Stage Prod

Database Deploy Script

DevDev

DevModel

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

1.11.11.41.7

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

1.1 1.2

1.2 1.3

1.3 1.4

1.4 1.5

1.5 1.6

1.6 1.7

Out of Process Change

XX

XXX

? 1.1.1

X

Challenge with static scripts…

Configuration drift…

60%Of those manually building scripts have to fix or tweak them regularly as part of a

deployment process

80%Rate this as a risk factor, yet

71%Follow manual processes to create their database

deploy scripts

▪Scripts− Hard to test in their entirely (holistically)− Hard to test due to colliding dependencies− Need to run in a specific order…− Much harder to deal with project scope changes

▪Scripts, unless super sophisticated:− Unaware of changes made in the target environment− Time passed from their coding to the time they are run− Potentially overriding production hot-fixes or work done in parallel by

another team

Scripts are static…

Test cases using compare & sync tools:

An index exists in source (QA) but not in target (Production)What should we do? Add the index or not?

Using tools

Safe to automate?Sure… (?)

Challenge with ‘Compare & Sync’

Safe to automate?No. Requires manual inspection…

Challenge with ‘Compare & Sync’

70%of those using compare & sync tools have to review and fix the results as they can't always

trust them to automatically deploy correctly

A compare & sync tool:Is unaware of any changes that occurred before the time it ranHas no knowledge of changes that took place at the target environment

Does not leverage version control for more informationUnable to deal with conflicts & merges between different teams

Requires manual inspection Requires detailed knowledge regarding each change as part of the process

So…no automation… We fear for automating problems into

production and a major risk!!!

Challenges

The Solution to the challenges

Modern VC integrated DB process

Revision history

Actions

Standard IDE

Change management

Enforced and integrated to existing process

LeverageVersion control knowledge

intoDeployment decisions

1.11.21.31.41.51.61.7

*

Int QA Stage ProdDev

DevDev

Model

1.1 1.2

1.2 1.3

1.3 1.4

1.1 1.7

1.1.1 1.7

1.1 1.1 1.11.41.7

Out of Process Change

1.1.11.7 1.1.11.7

Validate

1.4 1.5

1.5 1.6

1.6 1.7

Configuration Drift prevention / conflict identification and Validated execution

1.4 1.5

1.5 1.6

1.6 1.7

OR

Baseline aware analysis

Validated execution / Build & deploy on demand

Validate

Understand the nature of the changesRaise red flags on conflicts Support out-of-process changesUtilize baseline aware analysis

Safety Net Deployment Automation

If we had the index in the baseline (previous version) and no longer in Dev (i.e. - removed) => we should take it down from production…

(Deploy Change)

Deploying changes if needed

Development BaselinePrevious Label /

Production Golden Copy

Production

Development BaselinePrevious Label /

Production Golden Copy

Production

BUT… If no index in baseline => someone else added it to Production…we should protect the NEW index on production!!!

(Protect Target)

Or protecting target environment…

Dealing with conflicts => merging changes

Conflict Resolving – Meta Data/Content

Continuous Delivery Pipeline Builder

• Define a process• Automate the process• Prevent/Alert out of process changes

Raise red flags to stop the line…if requires human intervention

Impact Analysis! Not Damage Control…

Automate “everything”– Package the deployment of database changes along with all your

other application components to give a unified picture Move the process upstream

– Easily promote the same package (including database changes!) from one environment to the next, handling environment-specific differences automatically

Create the deployment pipeline

For successful CD:

Requited integrations into entire continuous delivery pipeline:

IDEs (Oracle, Microsoft, MySQL in dev)

Source control (MsTFS, Git, Subversion etc.)

Task based development and ticketing systems (Jira, MsTFS, IBM RTC etc.)

Build tools (Jenkins, bamboo etc.)

ARA tools (IBM uDeploy, CA release automation, Chef etc.)

Integrated CD world…

▪ Focusing on changes rather than managing changes and dealing with re-work, boosted overall productivity of 250 developers. We estimate we were able to do 15% more with the same resources.

▪We went from several fix-centric deployments a day, to one feature-centric deployment a week.▪The amount of incidents in production has declined as well. We had 20% less incidents. ▪CIO @ Credit Card company

“Testimonials - Efficiency

▪ Regulation requirement - SOX and derived 357▪Auditing - change management approach with change management auditing. You have to do it anyway, do it automatically and efficiently. ▪Change management in production itself is regulatory required (ITIL). But you cannot ensure it without managing the whole process starting at Dev.▪We have to comply with regulation - but the business benefits from it. ▪CIO @ Credit Card company

“Testimonials - Regulation

Thank you!

Q & AShachar Furman Yaniv YehudaShachar.Furman@ness-tech.co.il yanivy@dbmaestro.com www.ness-tech.co.il www.dbmaestro.com