Migrating to RDSSeamless Migrations using Bucardo

–Everyone

Why the F@#$%@#$% would I want to use RDS?

• Difficult to scale • Difficult to recover • Difficult to failover • Expensive • Complicated

–Jack Burton

Sit tight, hold the fort and keep the home fires burning. And if we're not back by dawn... call the

president.

–Jack Burton

Are you crazy... Is that your problem?

Create your RDS InstanceUse RDSTune to get a default parameter set

https://bitbucket.org/davidkerr/rdstune

> gem install rdstune > rdstune -t web -m 2.75 -c 200

Create your RDS Instance

Create your RDS Instance

Other best practices1 Database per RDS Instance

Use Multi-AZ Deployments for production

Use SSDs/Provisioned IOPS

Create UsersOne user will be your replication user (usually the app user / database owner)

Bucardo setup says that this user needs access to session_replication_role parameter which is granted by the rds_superuser role.

Session Replication Role

The session_replication_role parameter allows bucardo to disable triggers.

Which is critical for Bucardo to manage replication.

Be Aware

export PGOPTIONS='-c session_replication_role=replica' psql -h mydb.mycompany.com -U dba mydb Password for user dba: psql: FATAL: permission denied to set parameter "session_replication_role"

RDS is finicky in setting the session_replica_role

mydb=> set session_replication_role=replica; SET mydb=> show session_replication_role; session_replication_role -------------------------- replica (1 row)

Prep your RDS Instance

> pg_dump -s -n <schema> <database> | psql -h <rds host> -U <rds user> <database>

• Set the database owner • Create any extensions • Create an empty schema

BucardoBucardo provides is Multi-Master replication for Postgres

This is logical replication (trigger based) and won’t interfere with Postgres streaming replication

Is Asynchronous (which is perfect for this)

You can find installation instructions here

https://bucardo.org/wiki/Bucardo#Obtaining_Bucardo

Configuration> createdb bucardo

> bucardo install

> bucardo add db local dbname=myDb addalltables

> bucardo add db remote db=myDb \ pghost=rds.xxyzrtl2.us-west-2.rds.amazonaws.com

> bucardo add dbgroup RDS local:source remote:source

> bucardo add sync RDSSync tables=all \ dbs=RDS autokick=0

Copy the Data

> pg_dump -Fc --data-only -N bucardo myDb | pg_restore -Fc --disable-triggers -U myUser \ -h rds.xxyzrtl2.us-west-2.rds.amazonaws.com \ -d myDB

Fix your sequencesCREATE or REPLACE FUNCTION bsequencer(sequence_name text,offset_by int,even boolean) RETURNS VOID AS $$ DECLARE myval bigint; BEGIN EXECUTE 'select last_value +'||offset_by|| ' from '||sequence_name INTO myval; IF ( even AND ( (myval % 2) != 0) ) OR ( NOT even AND ( (myval % 2) = 0) ) THEN myval := myval + 1; END IF; EXECUTE 'SELECT setval( ''' || sequence_name || ''',' || myval || ')'; EXECUTE 'ALTER sequence ' || sequence_name || ' increment by 2'; END; $$ LANGUAGE plpgsql;

local => select bsequencer('sequence_name',100,true);

RDS => select bsequencer('sequence_name',100,false);

/Kick your Bucardo

> bucardo update sync RDSSync autokick=1 > bucardo reload config

Repoint your Apps

• Repoint the DNS entry for your database host to the RDS host.

• Apps will slowly switch over to the RDS host as connections age and get re-established

• Or implement close to a release to force a reconnect

Tear down the old

• Once all of the connections have drained off the old DBs you can de-commission them.

• Remember to remove Bucardo!

Thanks!