Magnet: solution for database delta

deliveries

Martins Kemme

The Challenge

You can’t deliver all database objects every time, because

• it may take too much time to recreate all database objects

• there are objects (like tables) that can’t be deleted and reinstalled (valuable data are there!)

Delta Delivery

“Delta delivery” - deliver only the items, that were modified recently

TimePreviously delivered changes

Changes that are new and need to be delivered now

Delta Delivery Challenges

• Take only modified items plus related items

• Decide what to do if this item has been modified several times?

• Find out detailed info about each file• what it is? • how it must be installed?

• Know the installation sequence

• Make a note that now these changes have also been delivered

Magnetfor Subversion repositories

What is Magnet?

Java tool, that operates on SVN repositories and can

• retrieve from Subversion repository only these files that are referenced in selected revisions

• create installation file for files referenced in selected revisions

• collect Subversion log messages from selected revisions into one text file (i.e. create documentation for the delivery)

• add specific properties to revisions in Subversion repository marking that these revisions are collected

The Algorithm

1. Determine file type either by location (directory) or by file name

2. Determine file installation method from file type

3. Determine installation method parameters

4. Create installation script that contains calls of installation methods for each file

Magnet: the result

• The following slides will present some examples of installation scripts generated automatically by the Magnet

• These scripts can be run automatically by continuous integration engine or by person manually

Unix Script Example#!/bin/bash

### Load all installation functions. ./installation_functions.sh

### Check if required Login Ids are entered else prompt to get themCHKSCHEME "APPS_ID" "APPS_ID"

echo [INSTALLER] $SYSDATE Starting installation...

echo [INSTALLER] Installing revision 5930INSTALLBINARY XXCMP "../applnode-concurrent/appl_top/xxcmp/11.5.0/bin/xxcmp_ar_loader.ctl"

echo [INSTALLER] Installing revision 5935RUNSQLPLUS $APPS_ID "@../dbinstall/apps/data_scripts/CMP-5048_DATAFIX.sql"

echo [INSTALLER] $SYSDATE Installation finished.

SQL*Plus Script ExampleSET verify ONSET timing ONSPOOL &spool_fileSET define off prompt [INSTALLER] Starting installation... prompt [INSTALLER] Installing TABLEconn &&ZV_HORIZON@branches/horizon/DB_scripts/ZV_HORIZON/table/gl_invoices.sql prompt [INSTALLER] Installing PACKAGE_BODYconn &&ZRZ_OWNER@trunk/source_code/DB_scripts/ZRZ_OWNER/package/p_legal_persons-b.sql prompt [INSTALLER] Installing GRANTconn &&ZV_ADMIN@trunk/source_code/DB_scripts/ZV_ADMIN/grant/771_setup_new_form.sql

Windows BAT file@echo offecho [INSTALLER] Starting installation... echo [INSTALLER] Installing TABLEcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/table/dmf_parts.sqlcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/table/labels.sql

echo [INSTALLER] Installing PACKAGE_SPECIFICATIONcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_zc_prices-h.sqlcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_legal_persons-h.sql

echo [INSTALLER] Installing PACKAGE_BODYcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_zc_prices-b.sqlcall:RUNSQL %ZV_OWNER% @trunk/DB_source/ZV_OWNER/package/p_legal_persons-b.sql

echo [INSTALLER] Installation finished.GOTO :EOF :RUNSQL

...etc...

Magnet

Setup file(either .sh, or .bat or .sql)

Folder of collected files

List of deleted filesConsolidated log messages

Updated repository

Martins.Kemme (at) gmail.com

Thank you!