Webinar: Version Control Meets Database Control
DBmaestro Introductions
Presenters
Kyle Hailey @kylehhailey• Technical Evangelist at Delphix• Oracle ACE, member of the OakTable Network
Uri Margalit @UriMargalit • Director, Product Management• Presenter at world-wide conferences: ODTUG,
ilOUG, etc…
Before we start
• You will be on mute for the duration of the event
• We are now talking so please type a message in the Questions box in the Control Panel if you can’t hear us (please check your speakers and GoToWebinar audio settings first)
• There will be a Q+A session at the end but please feel free to type your questions in the Questions box in the Control Panel in advance
• A recording of the full webinar will be put up online
About Delphix
• Founded in 2008, launched in 2010• CEO Jedidiah Yueh (founder of Avamar: >$1B revenue))• Based in Silicon Valley, Global Operations• 10% of Fortune 500
About DBmaestro
• Founded in 2008• Headed by Yariv Tabac and Yaniv Yehuda• Headquartered in Israel
Version Control meets Data Control
Kyle Hailey & Uri Margalit
The Business Need
Copyright@2008, Juniper Networks, Inc.
80% of unplanned downtime is due to Change
50% More than
of this, half is due to human errors
40% of changes FAIL
The Technology Need
Dealing with Risk
Smaller and more focused changes are easier to manage (Agile…) Automation of repeating tasks lowers risk of (human) error Development and Operations should work in synergy (DevOps)
Source Control – Standard De Facto
Common version control tools:GitHubSVNPerforceTFSRTCVSS
The Database Challenge
• The Database is a crucial part of the Application— Schema Structure— PL/SQL Code— Lookup Content
• The Database is a central resource• Business Data Must be preserved
• The Database is not native to traditional version control
• Objects are not files on a file system• How can we manage Content?• How can we branch a Database?
Tradeoff: Speed, Quality, Cost
Good, Cheap, Fast : choose two
Fast
Good Cheap
What We’ve Seen
1. Inefficient QA: Higher costs of QA2. QA Delays : Greater re-work of code3. Sharing DB Environments : Bottlenecks4. Using DB Subsets: More bugs in Prod5. Slow Environment Builds: Delays
1. Inefficient QA: Long Build times
Build Time
QA Test
96% of QA time was building environment$.04/$1.00 actual testing vs. setup
Build
2. QA Delays: bugs found late require more code re-work
Build QA Env QA Build QA Env QA
Sprint 1 Sprint 2 Sprint 3
Bug CodeX
1 2 3 4 5 6 70
10203040506070
Delay in Fixing the bug
Cost To
Correct
Software Engineering Economics – Barry Boehm (1981)
3. Full Copy Shared : Bottlenecks
Frustration Waiting
Old Unrepresentative Data
4. Subsets : cause bugs
Production
4. Subsets : cause bugs
Classic problem is that queries that run fast on subsets hit the wall in production.
Developers are unable to test against all data
The Production ‘Wall’
5. Slow Environment Builds: 3-6 Months to Deliver Data
Management
DBA
System Admin
Storage Admin
Developers Submit Request
Disk Capacity?
Approve Request $$ (2 Weeks)
Approve Request $$
(1 Week)
RequestAdditional
Storage?ProvisionCapacity
File SystemConfigured?
Configure LUNS & Build File System
Coordinate Replication w/ Infrastructure
Re-Parameterize & Configure DB
Mount Recovery DB to
Specific PIT
Begin Work
Approve Request $$ (2 Weeks)
(3 Days)
(3 Days)
(2 Days)
(3 Days)
(3 Days)
.……1-2 Weeks of Approvals, Delays, and Provisioning……
20
5. Slow Environment Builds: 3-6 Months to Deliver Data
5. Slow Environment Builds: culture of no
What We’ve Seen
1. Inefficient QA: Higher costs2. QA Delays : Increased re-work3. Sharing DB : Bottlenecks4. Subset DB : Bugs5. Slow Environment Builds: Delays
Poll
Which of the following have you run into at your organization?1. Inefficient QA driving up costs2. QA Delays causing increased re-work of code3. Sharing DB causing development bottlenecks4. Subset DB database in development and QA
leading to bugs in production5. Slow Environment Builds causing project delays
60% Projects Over Schedule and Budget
CIO Magazine Survey:
Data is the problem
Solve the data problem.
TODAY.
UNLOCK YOUR DATA
Clone 1 Clone 3Clone 2
99% of blocks are identical
Clone 1 Clone 2 Clone 3
Thin Clone
Virtualization Layer
Virtualization
Three Physical Copies Three Virtual Copies
Install Delphix on x86 hardware
Intel hardware
Allocate Any Storage to Delphix
Allocate StorageAny type
One time backup of source database
Database
Production
Instance
File system
File system
DxFS (Delphix) Compress Data
Database
Production
Instance
Data is compressed typically 1/3
size
File system
Incremental forever change collection
Database
Production
Instance
File system
Changes
• Collected incrementally forever• Old data purged
Time Window
File system
Typical Architecture
Production
Instance
Development
Instance
QA
Instance
UAT
Instance
Database
File systemFile system
Database
File systemFile system
Database
File systemFile system
Database
File systemFile system
With Delphix
Database
File system
Production
Instance
Database
Development
Instance
Database
QA
Instance
Database
UAT
Instance
Three Core Parts
Production
Instance
Time Window
Instance
Virtual Database
1
2
3
1. Source Syncing
2. Storage (DxFS)
3. Self Service
Development
Fast, Fresh, Full
Database
Production
Instance
File system Time Window
Instance
Virtual Database
Free
Database
Production
Instance
File system Time Window
Instance
Virtual Database
Instance
Instance
Virtual Database
Virtual Database
Branching to QA
Database
Production
Instance
File system Time Window
Instance
Virtual Database
Instance
Virtual Database Dev
QA
Self Service
What We’ve Seen With Delphix
1. Efficient QA: Low cost, high utilization2. Quick QA : Fast Bug Fix3. Every Dev gets DB: Parallelized Dev4. Full DB : Less Bugs5. Fast Builds: Fast Dev, Culture of Yes
1. Efficient QA: Lower cost
Build
Ti
me
QA Test
1% of QA time was building environment$.99/$1.00 actual testing vs. setup
Build Time
QA TestBuild
2. QA Immediate: bugs found fast and fixed
Sprint 1 Sprint 2 Sprint 3
Bug CodeX
QA QA
Build QA Env QA Build QA Env QA
Sprint 1 Sprint 2 Sprint 3
Bug CodeX
3. Private Copies: Parallelize
4. Full Size DB : Eliminate bugs
Production
5. Self Service: Fast, Efficient. Culture of Yes!
Management
DBA
System Admin
Storage Admin
Developers Submit Request
Disk Capacity?
Approve Request $$ (2 Weeks)
Approve Request $$
(1 Week)
RequestAdditional
Storage?ProvisionCapacity
File SystemConfigured?
Configure LUNS & Build File System
Coordinate Replication w/ Infrastructure
Re-Parameterize & Configure DB
Mount Recovery DB to
Specific PIT
Begin Work
Approve Request $$ (2 Weeks)
(3 Days)
(3 Days)
(2 Days)
(3 Days)
(3 Days)
.……1-2 Weeks of Approvals, Delays, and Provisioning……
What We’ve Seen With Delphix
1. Efficient QA: Low cost, high utilization2. Quick QA : Fast Bug Fix3. Every Dev gets DB: Parallelized Dev4. Full DB : Less Bugs5. Fast Builds: Fast Dev, Culture of Yes
Challenges of Development & Release to Operation
Organizing the development of changes• Code• Database • Configuration • Metadata• Work Items
Development Test/ Staging/ UAT
Enabling safe migration into production• Moving the right
components• Enabling
Rollback & Recovery
Production
Development Operations
Moving just the right components needed for the release
• Release Approved Items
Agility dictates frequent changes & new tools are needed to streamline the process
ChangeManagement
ReleaseManagement
What is DBmaestro TeamWork
• Database Enforced Change Management+ Database version control+ Plugs into the ALM (change request, tickets &
work items)
+ Database change impact analysis+ Database deployment automation
• DevOps Solution for databases + Deployment, rollback & recovery+ Plugs into release management
The Challenges that DBmaestro Addresses
• Development delays• Silos in development, DBA and operations• Delays in deployment (internally and to operations)• Errors in production
Development Delays
• Different methodologies for the application & database
• Code overrides• Lack of history of changes (who did what, where, when and
why)
• Manual writing of delta scripts• Lack of automation
Silos in Development, DBA and Operations
• No sharing between the team• No visibility• Always looking for errors made by others
Delays in Deployment (Internally and to Operations)
• Deployment automation does not really include the database tier
• Database scripts generated out of the scope of automation
• Lack of confidence in automation
Errors in Production
• Missing changes• Deploying the wrong version of objects• What about the reference data?
Poll
Which challenges have you experienced?
1. Development delays
2. Silos in development, DBA and operations
3. Delays in deployment (internally and to operations)
4. Errors in production
How?
• Database version control – Enforced Check Out/In – Labels– Rollback/Undo – Audit trail reports
• Database impact analysis – Utilizes version control repository information – 3 way analysis
• Database deployment automation– API – Baselines – Conflict resolution – Customized business logic
Without DCM - Two isolated Processes
Check-Out Script
Modify Script
Get updated Script from DB
Check-In Script
Compile Scriptin DB
Debug Scriptin DB
Version Control Process Development Process
?
??
?
Development & Version Control Process
With DCM - One Enforced Process
Check-Out Object
Modify Object in DB
Run Applications’
Tests
Check-In Object
Safety Net For Automation of Deployment
Source vs. Target
Action
= No Action
≠ ?
Source vs. Baseline
Target vs. Baseline
Action
= = No Action
≠ = Override
= ≠ Ignore
≠ ≠ Merge
You do not have all of the information
With Baselines and 3 way analysis the unknown is now known
Simple Compare & Sync Baseline aware Deployment
Benefits - Development
• Database change repository• Follow SCM best practices (Check-Out/Check-
In) • All changes are documented• Manage who can do what, where, when & why
Benefits - Operations
• Integrated deployment engine• Business level audit• Roles & responsibilities enforcement
Benefits - Management
• Complete visibility into changes in progress• Management reports• No silos
Live Demo
• Clone 2 virtual copies of the Trunk1. Dev1
2. Dev2
• Make changes & merge them into the Trunk: Developer1 modifies Dev1 Developer1 merges changes into the Trunk Developer2 modifies Dev2 Developer2 merges changes into the Trunk
• Rely on enforced changes & automation
Time Window
Instance
Instance
Instance
Virtual DatabaseDev1
Dev2
Trunk
Virtual Database
Virtual Database
Developer 1 modify
Developer 2 modify
DBVC
Merge Dev1 & TrunkMerge to dev2
Dev2
Dev1Merge to dev1
Merge Dev2 & Trunk
Trunk
Merge Dev1 & Trunk
Merge Dev2 & Trunk
DBVC
Merge Dev1 to ForkMerge to dev2
Dev2
Dev1Merge to dev1
Merge Dev2 to Fork
Trunk
Merge Dev1 to Fork
Merge Dev2 to Fork
DBVC
Fork
Fork
Fork
Fork
Q&AKyle Hailey @kylehhaileyDelphix: delphix.com
Uri Margalit @UriMargalit DBmaestro: dbmaestro.com
Thanks
Top Related