Move to Managed: Migrating Open-source Databases from Self ...

© 2021, Amazon Web Services, Inc. or its Affiliates.

Chirag Dave

Sr. Database Specialist Solutions Architect

Move to Managed: Migrating Open-source

Databases from Self-managed to Amazon RDS or

Amazon Aurora

Peter Celentano

Database Specialist Solutions Architect


Agenda

• Why move to managed database services

• An overview of AWS managed database services

• Available options to migrate self-managed database to

RDS/Aurora

• Demo 1: MySQL migration

• Demo 2: PostgreSQL migration

• Migration best practices

• Q&A


Hardware & software installation, configuration, patching, backups

Performance and high availability issues

Capacity planning and scaling clusters for compute and storage

Security and compliance

Self-managing databases is time consuming, complex,

and expensive


Amazon Relational Database Service (Amazon RDS) M A N A G E D R E L A T I O N A L D A T A B A S E S E R V I C E W I T H A C H O I C E O F P O P U L A R D A T A B A S E E N G I N E S

Easy to administerEasily deploy and maintain hardware, OS, and DB software; built-in monitoring

Performant & scalableScale compute and storage with a few clicks; minimal downtime for your application

Available & durableAutomatic multi-AZ data replication; automated backup, snapshots, and failover

Secure & CompliantData encryption at rest and in transit; assists customers achieve their compliance goals through tooling

Microsoft SQL Server

Oracle


Amazon AuroraM Y S Q L - A N D P O S T G R E S Q L - C O M P A T I B L E R E L A T I O N A L D A T A B A S E W I T H P E R F O R M A N C E A N D A V A I L A B I L I T Y O F C O M M E R C I A L - G R A D E D A T A B A S E S A T 1 / 1 0 T H T H E C O S T

Performance

and scalability

Availability

and durabilityHighly secure Fully managed

Performance of commercial-

grade databases; scale-out up

to 15 read replicas

Fault-tolerant, self-healing

storage; six copies of data

across three Availability

Zones; continuous backup to

Amazon S3

Network isolation,

encryption at

rest/transit, compliance

and assurance programs

Managed by RDS:

No server provisioning,

software patching, setup,

configuration, or backups


Amazon RDS & Amazon Aurora - fully managedS P E N D T I M E I N N O V A T I N G & B U I L D I N G N E W A P P S , N O T M A N A G I N G I N F R A S T R U C T U R E

Schema design

Query construction

Query optimization

Automatic fail-over

Backup & recovery

Isolation & security

Industry compliance

Push-button scaling

Automated patching &

upgrades

Advanced monitoring

Routine maintenance

You AWS


Choosing your relational database deployment in AWS

Amazon RDS

Self-managed


Move to managed relational databasesM I G R A T E O N - P R E M I S E S O R C L O U D - H O S T E D R E L A T I O N A L D A T A B A S E S T O M A N A G E D S E R V I C E S

Reduce DB administrative burden

No need to rearchitect existing applications

Get better performance, availability, scalability, and security

Amazon AuroraMySQL, PostgreSQL

Amazon RDSMySQL, PostgreSQL, MariaDB,

Oracle, SQL Server


Migration options for MySQL


MySQL migration options at a glanceC O N S I D E R A L L O P T I O N S I N C O N T E X T O F Y O U R R E Q U I R E M E N T S A N D L I M I T A T I O N S

Physical Restore

from Backups

Logical Export

and Import

AWS Database

Migration Service

Physical migration

+ logical replication

Using native or

ecosystem tools

+ logical replication

Using AWS DMS and

SCT for conversion and

data migration

1 2 3

Homogenous OnlyHeterogeneous

Homogenous


Physical restore from backupsT Y P I C A L L Y T H E P R E F E R R E D O P T I O N C O M I N G F R O M M Y S Q L

• Simple option if you already use

Percona XtraBackup for backups

• Upload backup files to Amazon S3

• Physical restore is typically faster

than logical export and import

• Supports MySQL 5.6, 5.7 and 8.0 as

source

• Use native binlog replication for

capturing changes and minimizing

cutover downtime

Corporate

data center

AWS Cloud

Amazon S3

Amazon Aurora

MySQL

MySQL

Some limitations apply: encrypted backups not supported; stored procedures, functions and timezone setting not imported

automatically; only works with default file name for innodb_data_file_path parameter; some definer policies not usable

1.Percona Xtrabackup

3. Binlog Replication

2. Aurora MySQL

Restore


Migrate using logical export and importT Y P I C A L L Y A R E L I A B L E O P T I O N I F D A T A S E T S A R E S M A L L , O R I M P O R T S A R E P A R A L L E L I Z E D

• Typically only used for homogenous migrations

• Using native tools or third party tools

- Native tools: mysqldump tool suitable only for small data sets

- Third party tools: mydumper/myloader suitable for parallelized export and import

operation

• Initial load depends on data set, degree of parallelization,

DB instance size, storage characteristics

• Use native logical replication for capturing changes

and minimizing downtime


Migration options summary for MySQL

Options

Near Zero

Downtime

Required?

Supported

VersionsMigration Type

Physical restore from

backups using Percona

XtraBackup Can minimize

downtime with

native logical

replication5.6, 5.7, 8.0 (RDS

MySQL only)

Homogenous

Logical replication using

mysqldumpHomogenous

Logical replication using

mydumper/myloaderHomogenous

AWS DMS with CDC YesHomogeneous,

Heterogeneous


Demo 1:

Creating an RDS MySQL DB Instance from

S3-based backups


Migration options for PostgreSQL


PostgreSQL Migration options at a glanceC O N S I D E R A L L O P T I O N S I N C O N T E X T O F Y O U R R E Q U I R E M E N T S A N D L I M I T A T I O N S

Logical Dump Logical

Replication

AWS Database

Migration Service

pg_dump/pg_restore Native logical

replication/pglogical

Using AWS DMS and

SCT for conversion and

data migration

1 2 3

Homogenous Only

Heterogeneous

Homogenous


Migration to RDS PostgreSQL – Native tools

pg_dump/pg_restore

• Simple option

• Does not support CDC

• Supports all major versions

• May not be suitable for 24/7

workloadsSelf-

managed

pg_dump/

pg_restore

Amazon RDS


When to choose pg_dump?

We recommend that you use nat ive PostgreSQL database migrat ion too ls

under the fo l lowing cond i t ions :

• You have a homogenous migrat ion , where you are migrat ing f rom a

database w i th the same database eng ine as the target database .

• You are migrat ing an ent i re database .

• Smal l database where downt ime dur ing backup and restore i s accepted .


On-Premise/EC2 to Amazon RDS replicationI N T R O D U C I N G I N B U I L T L O G I C A L R E P L I C A T I O N A N D P G L O G I C A L E X T E N S I O N

• PostgreSQL 10 (onwards)

supports inbuilt logical

replication using

publication/subscription

• Pglogical Extension available in

Amazon RDS for PostgreSQL >=

9.6.10

9.6.10

Amazon RDS>=10

10.x +


Migration Options Summary for PostgreSQL

Options

Near Zero

Downtime

required

Supported Versions Migration Type

pgdump/pg_restore No

All PostgreSQL

versions Homogenous

AWS DMS with CDC Yes 9.4 and higherHomogeneous

Heterogeneous

Pglogical extension Yes 9.4 and higherHomogenous

PostgreSQL Native

Pub/Sub replicationYes 10.x and higher

Homogenous


AWS DMS: Migration Option for both MySQL

& PostgreSQL


Migrate using AWS DMS and SCT

AWS Database Migration Service (DMS)easily and securely migrates and/or replicate

your databases and data warehouses to AWS

AWS Schema Conversion Tool (SCT)converts your commercial database and data

warehouse schemas to open-source engines or

AWS-native services, such as Amazon Aurora and

Redshift


How does AWS DMS work? F I R S T , C O N V E R T S C H E M A S A N D P E R F O R M I N I T I A L D A T A L O A D S

Source DB

AWS SCT

Native Tool

Target DB

AWS DMS

Database

Source DBTarget DB

Step 1:

Step 2:


How does DMS work? - contN E X T , C A P T U R E C H A N G E S ( K E E P A P P S R U N N I N G D U R I N G M I G R A T I O N )

Customer

premises

Application users

AWS

Internet

VPN

Start a replication instance

Connect to source and target databases

Select tables, schemas, or databases

Let AWS DMS create tables, load data, and keep them in sync

Switch applications over to the target at your convenience

AWS

DMS


Demo 2:

Migrating from Self Managed PostgreSQL to

Amazon Aurora using Logical Replication


Migration Best Practices


Migration best practices

• Do not use Mul t i -AZ for in i t ia l load

• Ensure a l l the parameters and set t ings you need for the source and

target are set beforehand

• Test per formance by restor ing the snapshot of the migrated instance

• Use nat ive too ls where i t makes sense

• For AWS DMS: The s i ze and instance c lass for your DMS instance i s a

common i ssue for per formance


How to

get started

• Learn more about AWS database services

• aws.amazon.com/products/databases/

• Engage with your AWS account team

early. They are here to help!

• Participate in workshops, Immersion

Days, or Data Lab

• Get support from ProServe or

AWS Partner for database migration

https://aws.amazon.com/products/databases/


Chirag Dave: [email protected]

Peter Celentano: [email protected]

Move to Managed: Migrating Open-source Databases from Self ...

Documents

Transcript of Move to Managed: Migrating Open-source Databases from Self ...