Борис Трофимов. Continuous Database migration-это просто!
-
Upload
volha-banadyseva -
Category
Software
-
view
3.587 -
download
4
Transcript of Борис Трофимов. Continuous Database migration-это просто!
![Page 1: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/1.jpg)
Continuous DB migration
Boris Trofimov
@b0ris_1
www.btrofimoff.com
![Page 2: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/2.jpg)
Agenda
Dealing with changes
Perfect solution or Continuous DB migration
Carbon5 Introduction
Maven-driven mode
Embedded mode
Pros/Cos and best practices
![Page 3: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/3.jpg)
Dealing with Changes
Applications evolve. God, bless bugs, refactoring and
business changes.
We love applications and configure CI, VCS in order to
track source code/application version.
But what’s about DB?
![Page 4: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/4.jpg)
Dealing with Schema Change
FACT: Some modifications might require changes in DB
schema.
![Page 5: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/5.jpg)
Dealing with Schema Change
FACT: Some modifications might require changes in DB
schema.
Changes are simple and risky.
![Page 6: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/6.jpg)
Dealing with Schema Change
FACT: Some modifications might require changes in DB
schema.
Changes are simple and risky.
Simple changes:
– add one more table
– add index
![Page 7: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/7.jpg)
Dealing with Schema Change
FACT: Some application modifications might require
changes in DB schema.
Changes are simple and risky.
Simple changes:
– add one more table
– add index
Risky changes:
– rename/add column
– change foreign key
– migrate data from one table/column to another
– denormalization on a fly
![Page 8: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/8.jpg)
Dealing with Schema Change
What’s about old apps upgrade and multiple schema changes?
App v1
App v10
![Page 9: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/9.jpg)
The Impact
Risks to lose data
Painful downtime
Risks to break application (see #1 and #2)
Resources, efforts and budget
![Page 10: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/10.jpg)
Friday’s deployment in some teams
![Page 11: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/11.jpg)
How people solve it
Deny Friday’s deployments
Manual deployment
Hibernate-like schema update + SQLexec
Self-developed tools based on version number approach
![Page 12: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/12.jpg)
Perfect solution
Dedicated reusable framework
Every Feature request leads to separated SQL migration script.
Framework tracks which changes were not applied and applies
them.
Configurable time to launch apply procedure
Prevent Double changes
Have Change Log
Simple integration to existent project
![Page 13: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/13.jpg)
Continuous DB Migration approach
min + automatic migration = Continuous DB Migration
![Page 14: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/14.jpg)
Carbon V
![Page 15: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/15.jpg)
Why Carbon V
DB-migration framework
Open Source project
Lightweight framework
Simple usage
![Page 16: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/16.jpg)
How it works
Dedicated project folder for SQL migration scripts. This folder
should be available when framework takes control.
Each SQL migration script is pure SQL/DDL file with migration
SQL code for specific feature
Each Delta file should have name
YYYYMMDDHHMMSS_<FEATURE>.sql
When framework take control it checks and applies only new
changes.
C5 uses own JDBC connection.
Way to re-usue existent DataSources with connection params
![Page 17: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/17.jpg)
No magic, just…
C5 creates own table “schema_version” inside your database
C5 controls this table by itself (no needs to create or update).
sql_file_name date duration
201405170900_user_auth.sql 2014-05-17 13:00:00
8 sec
201404130400_award_4565.sql 2014-04-14 14:00:00
6 sec
![Page 18: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/18.jpg)
When to initiate migration procedure?
Maven-driven approach
Embedded mode
![Page 19: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/19.jpg)
Maven-driven mode
![Page 20: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/20.jpg)
Maven-driven mode
![Page 21: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/21.jpg)
Configure Maven project
Update POM definition
Configure Explicit DB connection
Can be configured depending on specific maven profiles
(staging, production)
![Page 22: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/22.jpg)
Add SQL migration files
Use Project/src/main/db/migrations directory
![Page 23: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/23.jpg)
Launch it
performs migration
$ mvn db-migration:migrate
resets migration
$ mvn db-migration:reset
check if DB is up to date
$ mvn db-migration:validate
create new feature
$ mvn db-migration:new -Dname=new_feature
![Page 24: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/24.jpg)
Use cases
Manual runs from developer machine
Part of Continuous Deivery process – Add just one more Maven action inside specific CI configuration before
deployment action
![Page 25: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/25.jpg)
Embedded mode
![Page 26: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/26.jpg)
Embedded mode
Schema Check is constituent part of application.
SQL changes are built-in into the application
Any checks and possible DB migration is performed every time
when application launches.
Dedicated bean to carry out this responsibility
![Page 27: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/27.jpg)
Create application bean import com.carbonfive.db.migration.DataSourceMigrationManager;
import com.carbonfive.db.migration.DriverManagerMigrationManager;
import com.carbonfive.db.migration.MigrationManager;
import com.carbonfive.db.migration.ResourceMigrationResolver;
public class DBMigratorImpl {
DataSource dataSource;
...
public void init(){
DataSourceMigrationManager migrationManager = new
DataSourceMigrationManager(dataSource);
migrationManager.setMigrationResolver(new
ResourceMigrationResolver("classpath:/db/migrations"));
migrationManager.migrate();
}
}
![Page 28: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/28.jpg)
Inject the bean into application context
Seamless integrated with Spring DI It is possible to initialize ResourceMigrationResolver
DataSourceMigrationManager directly though Spring DI without any line of code
Two places where bean might be integrated inside Spring context Before Persistent beans
After Persistent beans
![Page 29: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/29.jpg)
Embedded vs maven driven
E: Consistent and grace application upgrade
E: No delays between deployment and schema upgrade
M: Simple integration with CI
M: External commands like validate & reset
M: Quick manual migrations (hotfixes for instance)
![Page 30: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/30.jpg)
Carbon5 Benefits
Lightweight framework
Brilliant extensibility (thanks to reasonable design)
Support many DBMS in unofficial mode
Simple integration
Real continuous DB migration
![Page 31: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/31.jpg)
Need more features?
![Page 32: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/32.jpg)
![Page 33: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/33.jpg)
Flyway framework
Rich support for different build tools
The same integration approach on Carbon V
Pluggable architecture
Java-based migrations (two options: through JDBC Connection &
Spring JDBCTemplate)
Extended service table
![Page 34: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/34.jpg)
Best practices
Feature driven development (each SQL feature change in own
file)
Make sql files safe in order to prevent it from double execution
Keep database connection settings in single place (use shared
DataSource)
Do not make structure changes to the DB outside of your
framework
![Page 35: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/35.jpg)
What’s about schemaless
?
![Page 36: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/36.jpg)
![Page 37: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/37.jpg)
NoSQL migration approach
No dedicated explicit migration procedure (solved on business
level in case of needs)
No shutdown or downtime
Rolling update – Every domain entity has version number
– your v2 application should handle the v1 db format
– Write code to convert entity to v2 on a fly (repository level)
– Write modified entities to v2 on demand
Some DBs like RavenDB are able to perform auto-migration
![Page 38: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/38.jpg)
References
https://code.google.com/p/c5-db-migration/
http://flywaydb.org/
Martin Fowler Evolutionary Database Design
http://www.martinfowler.com/articles/evodb.html
Refactoring Databases
![Page 39: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/39.jpg)
Presentation in a nutshell
Keep in mind this problem
Do not invent wheels, use external frameworks/techniques
And do not be afraid of Friday’s deployment then.
![Page 40: Борис Трофимов. Continuous Database migration-это просто!](https://reader033.fdocuments.us/reader033/viewer/2022052522/554f439bb4c905cd048b5602/html5/thumbnails/40.jpg)
Thank you!
Q&A