(DAT304) Amazon RDS for MySQL: Best Practices

49
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Abdul Sathar Sait, Principal Product Manager, RDS October 2015 | Las Vegas, NV DAT 304 Amazon RDS for MySQL Best Practices Kevin Rice, Director of Engineering, Airbnb

Transcript of (DAT304) Amazon RDS for MySQL: Best Practices

Page 1: (DAT304) Amazon RDS for MySQL: Best Practices

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Abdul Sathar Sait, Principal Product Manager, RDS

October 2015 | Las Vegas, NV

DAT 304

Amazon RDS for MySQL

Best Practices

Kevin Rice, Director of Engineering, Airbnb

Page 2: (DAT304) Amazon RDS for MySQL: Best Practices

What’s new in Amazon RDS MySQL

AWS Key Management

Service (AWS KMS)

Page 3: (DAT304) Amazon RDS for MySQL: Best Practices

Selected RDS MySQL customers

Tens of thousands of customers. Hundreds of thousands of DB instances.

Page 4: (DAT304) Amazon RDS for MySQL: Best Practices

What we will cover in this session

Quick introduction to Amazon Relation Database Service (Amazon RDS)

Making the most out of RDS MySQL

Securing your data—encryption at rest

Database migration with minimal downtime

Improving performance by cache warming

Burst mode resources to reduce cost

Learn it from the experts—Airbnb

Page 5: (DAT304) Amazon RDS for MySQL: Best Practices

Why choose Amazon RDS?

Schema design

Query construction

Query optimization

High availability

Backup and recovery

Isolation and security

Industry compliance

Push-button scaling

Automated patching

Advanced monitoring

Routine maintenance

Amazon RDS takes care of your time-consuming database

management tasks, freeing you to focus on your applications and

business

You

RDS

Page 6: (DAT304) Amazon RDS for MySQL: Best Practices

We made it highly available, secure, easier, and cheaper

Push-button provisioning; automated scaling, patching, security, backups,

restores, and general care and feeding

Lower TCO because we manage the muck

► Get more leverage from your teams

► Focus on the things that differentiate you

Built-in high availability and cross-region replication across multiple data

centers

Now even a small startup can leverage multiple data centers to design highly

available apps with over 99.95% availability

Page 7: (DAT304) Amazon RDS for MySQL: Best Practices

High availability with Multi-AZ deploymentsEnterprise-grade fault tolerance solution for production databases

An Availability Zone is a physically distinct, independent infrastructure

Your database is synchronously replicated to another AZ in the same AWS region

Failover occurs automatically in response to the most important failure scenarios

Page 8: (DAT304) Amazon RDS for MySQL: Best Practices

Customers love Multi-AZ

26%

40%

25%

30%

35%

40%

45%

Multi-AZ instances as a share of all RDS instances

Page 9: (DAT304) Amazon RDS for MySQL: Best Practices

Choose cross-region read replicas for faster disaster

recovery and enhanced data locality

Promote a read replica to a

master for faster recovery in the

event of disaster

Bring data close to your

customer’s applications in

different regions

Promote to a master for easy

migration

Page 10: (DAT304) Amazon RDS for MySQL: Best Practices

Choose cross-region snapshot copy for even

greater durability, ease of migration

Copy a database snapshot to a different AWS

region

Warm standby for disaster recovery

Or use it as a base for migration to a different

region

Page 11: (DAT304) Amazon RDS for MySQL: Best Practices

Amazon RDS provides levels of security

difficult to achieve on-premisesAmazon RDS gives each database instance IP firewall protection

RDS offers transparent encryption at rest and SSL protection for data in transit

Amazon VPC lets you isolate and control network configuration and connect securely to your IT infrastructure

AWS Identity and Access Management (IAM) provides resource-level permission controls

AWS has achieved major compliances

Page 12: (DAT304) Amazon RDS for MySQL: Best Practices

Securing your data at rest

Page 13: (DAT304) Amazon RDS for MySQL: Best Practices

Do you encrypt your database?

Protect your data at rest

Premium feature for most commercial databases included at no

additional cost to RDS customers

Data stored at rest in the underlying storage is encrypted, as are

its automated backups, read replicas, and snapshots

May be needed for compliance (HIPAA and FedRamp)

Page 14: (DAT304) Amazon RDS for MySQL: Best Practices

AWS Key Management ServiceIntegrated with IAM console

Page 15: (DAT304) Amazon RDS for MySQL: Best Practices

Services integration with AWS KMS

Two-tiered key hierarchy using envelope encryption

• Unique data key encrypts customer data

• AWS KMS master keys encrypt data keys

Benefits:

• Limits risk of compromised data key

• Better performance for encrypting large data

• Easier to manage small number of master keys

than millions of data keys

• Centralized access and audit of key activity

Data Key 1

Amazon

S3 ObjectAmazon

EBS

Volume

Amazon

Redshift

Cluster

Data Key 2 Data Key 3 Data Key 4

Custom

Application

Customer Master

Key(s)

Page 16: (DAT304) Amazon RDS for MySQL: Best Practices

Your Application or

AWS Service

+

Data Key Encrypted Data Key

Encrypted

Data

Master Key(s) in

Customer’s Account

AWS KMS

1. Application requests encryption key to use to encrypt data, passes reference to master key in account

2. Client request authenticated based on master key permissions

3. New data encryption key created—copy encrypted under master key

4. Plaintext and encrypted data key returned to the client

5. Plaintext data key used to encrypt data and then deleted

6. Encrypted data key stored for later use and sent back to AWS KMS for when decryption occurs

How keys are used to protect your data

Page 17: (DAT304) Amazon RDS for MySQL: Best Practices

Encryption using AWS KMS demo

Page 18: (DAT304) Amazon RDS for MySQL: Best Practices

Database migration to AWS with

minimal downtime

Page 19: (DAT304) Amazon RDS for MySQL: Best Practices

Move data to the same or different database engine

Keep your apps running during the migration

Start your first migration in 10 minutes or less

Replicate within, to, or from Amazon EC2 or RDS

AWS Database

Migration Service

Page 20: (DAT304) Amazon RDS for MySQL: Best Practices
Page 21: (DAT304) Amazon RDS for MySQL: Best Practices
Page 22: (DAT304) Amazon RDS for MySQL: Best Practices
Page 23: (DAT304) Amazon RDS for MySQL: Best Practices
Page 24: (DAT304) Amazon RDS for MySQL: Best Practices
Page 25: (DAT304) Amazon RDS for MySQL: Best Practices
Page 26: (DAT304) Amazon RDS for MySQL: Best Practices

Migrate from Oracle and SQL Server

Move your tables, views, stored procedures,

and data manipulation language (DML) to

MySQL, MariaDB, and Amazon Aurora

Highlight where manual edits are neededAWS Schema

Conversion Tool

Page 27: (DAT304) Amazon RDS for MySQL: Best Practices
Page 28: (DAT304) Amazon RDS for MySQL: Best Practices
Page 29: (DAT304) Amazon RDS for MySQL: Best Practices

Architecting for lower cost

Page 30: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode—GP2 and T2

GP2—SSD based Amazon EBS storage

• 3 IOPS per GB base performance

• Earn credits when usage below base

• Burst to 3000+ IOPS

T2—Amazon EC2 instance with burst capability

• Base performance + burst

• Earn credits per hour when below base performance

• Can store up to 24 hours’ worth of credits

• Amazon CloudWatch metrics to see credits and usage

Page 31: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode—GP2 and T2

Page 32: (DAT304) Amazon RDS for MySQL: Best Practices

T2—CPU credits

Page 33: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode vs. standard vs. Provisioned IOPS

0

1000

2000

3000

4000

5000

6000

7000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Tra

ns

ac

tio

ns

pe

r S

ec

on

d (

TP

S)

Hours

100% read—20 GB data

db.m1.medium + 200GB standard

$0.575 per hour

Page 34: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode vs. standard vs. Provisioned IOPS

0

1000

2000

3000

4000

5000

6000

7000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Tra

ns

ac

tio

ns

pe

r S

ec

on

d (

TP

S)

Hours

100% read—20 GB data

db.m1.medium + 200GB standard

db.m3.medium + 200G + 2000 IOPS

$0.575 per hour

$0.408 per hour

Page 35: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode vs. standard vs. Provisioned IOPS

0

1000

2000

3000

4000

5000

6000

7000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Tra

ns

ac

tio

ns

pe

r S

ec

on

d (

TP

S)

Hours

100% read—20 GB data

db.m1.medium + 200GB standard

db.m3.medium + 200G + 2000 IOPS

db.m3.large + 200G + 2000 IOPS

$0.575 per hour

$0.408 per hour

$0.508 per hour

Page 36: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode vs. Standard vs. Provisioned IOPS

0

1000

2000

3000

4000

5000

6000

7000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Tra

ns

ac

tio

ns

pe

r S

ec

on

d (

TP

S)

Hours

100% read—20 GB data

db.m1.medium + 200GB standard

db.m3.medium + 200G + 2000 IOPS

db.m3.large + 200G + 2000 IOPS

db.t2.medium + 200GB gp2

$0.105 per hour

$0.575 per hour

$0.408 per hour

$0.508 per hour

Page 37: (DAT304) Amazon RDS for MySQL: Best Practices

Burst mode vs. standard vs. Provisioned IOPS

0

1000

2000

3000

4000

5000

6000

7000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Tra

ns

ac

tio

ns

pe

r S

ec

on

d (

TP

S)

Hours

100% read—20 GB data

db.m1.medium + 200GB standard

db.m3.medium + 200G + 2000 IOPS

db.m3.large + 200G + 2000 IOPS

db.t2.medium + 200GB gp2

db.t2.medium + 1TB gp2

$0.105 per hour

$0.575 per hour

$0.233 per hour

$0.408 per hour

$0.508 per hour

Page 38: (DAT304) Amazon RDS for MySQL: Best Practices

Selected Amazon RDS customers

Page 39: (DAT304) Amazon RDS for MySQL: Best Practices
Page 40: (DAT304) Amazon RDS for MySQL: Best Practices

AirbnbUsing technology to provide unique global

travel experiences.

Exponential growth in: traffic, users, bookings,

data, number of engineers.

Engineers deploy their own code at any time of

day. Rapid experimentation.

Search. Discovery. Global payments. Trust and

safety. Customer experience.

Page 41: (DAT304) Amazon RDS for MySQL: Best Practices

The basics

Master

Replica Replica

Application

Server

Application

Server

Multi-AZ

Page 42: (DAT304) Amazon RDS for MySQL: Best Practices

Binlog settings

Master

Replica Replica

Application

Server

Application

Server

Multi-AZ

Page 43: (DAT304) Amazon RDS for MySQL: Best Practices

Snapshots for data analytics

Master

Replica Replica

Multi-AZ

Batch

ReplicaSnapshot

Temp

Instance

Daily

Application

Server

HDFS

Page 44: (DAT304) Amazon RDS for MySQL: Best Practices

Binlog streaming: “SpinalTap”

Master

Replica Replica

Multi-AZ

Batch

Replica

SpinalTap

Replica

Application

Server

Page 45: (DAT304) Amazon RDS for MySQL: Best Practices

Disaster recovery

Master

Replica Replica

Multi-AZ

Batch

Replica

Application

Server

S3

Application

Server

To separate

account and

region

Page 46: (DAT304) Amazon RDS for MySQL: Best Practices

Summary and future work

Page 47: (DAT304) Amazon RDS for MySQL: Best Practices

http://bit.ly/awsevalsDAT304

Page 48: (DAT304) Amazon RDS for MySQL: Best Practices

Thank you!

Page 49: (DAT304) Amazon RDS for MySQL: Best Practices

Remember to complete

your evaluations!