Getting Started with Amazon Aurora
-
Upload
amazon-web-services -
Category
Software
-
view
200 -
download
1
Transcript of Getting Started with Amazon Aurora
![Page 1: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Steve Abraham, Solutions Architect, AWS
July 28, 2016
Getting Started with Amazon Aurora
![Page 2: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/2.jpg)
Meet Amazon Aurora ……Databases reimagined for the cloud
Speed and availability of high-end commercial databases
Simplicity and cost-effectiveness of open source databases
Drop-in compatibility with MySQL
Simple pay as you go pricing
Delivered as a managed service
![Page 3: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/3.jpg)
Not much has changed in last 30 years
Even when you scale it out, you’re still replicating the same stack
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Storage
Application
![Page 4: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/4.jpg)
Reimagining the relational database
What if you were inventing the database today?
You wouldn’t design it the way we did in 1970
You’d build something
that can scale out …
that is self-healing …
that leverages existing AWS services …
![Page 5: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/5.jpg)
A service-oriented architecture applied to the database
Moved the logging and storage layer into a
multi-tenant, scale-out database-optimized
storage service
Integrated with other AWS services like
Amazon EC2, Amazon VPC, Amazon
DynamoDB, Amazon SWF, and Amazon
Route 53 for control plane operations
Integrated with Amazon S3 for continuous
backup with 99.999999999% durability
Control planeData plane
Amazon
DynamoDB
Amazon SWF
Amazon Route 53
Logging + Storage
SQL
Transactions
Caching
Amazon S3
1
2
3
![Page 6: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/6.jpg)
Rapid adoption of Amazon Aurora
![Page 7: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/7.jpg)
Fastest growing service
in AWS history
Aurora customer adoption
![Page 8: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/8.jpg)
Expedia: On-line travel marketplace
Real-time business intelligence and analytics on a
growing corpus of on-line travel marketplace data.
Current Microsoft SQL Server–based architecture
is too expensive. Performance degrades as data
volume grows.
Cassandra with Solr index requires large memory
footprint and hundreds of nodes, adding cost.
Aurora benefits:
Aurora meets scale and performance
requirements with much lower cost.
25,000 inserts/sec with peak up to 70,000. 30 ms
average response time for write and 17 ms for
read, with 1 month of data.
World’s leading online travel
company, with a portfolio that
includes 150+ travel sites in 70
countries.
![Page 9: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/9.jpg)
Higher performance, lower cost
Safe.com lowered their bill by 40% by switching
from sharded MySQL to a single Aurora instance.
Double Down Interactive (gaming) lowered their
bill by 67% while also achieving better latencies
(most queries ran faster) and lower CPU
utilization.
Aurora benefits:
Due to high performance and large storage
support, sharded MySQL instances can be
consolidated in fewer Aurora instances.
High performance allows for smaller instances.
Automated storage provisioning removes the need
for storage “headroom.”
No additional storage for Read Replicas.
![Page 10: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/10.jpg)
“When we ran Alfresco’s workload on Aurora, we were blown away to find that
Aurora was 10x faster than our MySQL environment,” said John Newton,
founder and CTO of Alfresco. “Speed matters in our business, and Aurora has
been faster, cheaper, and considerably easier to use than MySQL.”
Amazon Aurora is fast
![Page 11: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/11.jpg)
SQL benchmark results
• Four client machines with 1,000 threads each
WRITE PERFORMANCE READ PERFORMANCE
• Single client with 1,000 threads
• MySQL SysBench
• R3.8XL with 32 cores and 244 GB RAM
![Page 12: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/12.jpg)
Reproducing these results
h t t p s : / / d 0 . a w s s t a t i c . c o m / p r o d u c t - m a rk e t i n g / Au r o r a / R D S_ Au r o r a _ Pe r f o r m a n c e _ As s e s s m e n t _ Be n c hm a r k i n g _ v 1 - 2 . p d f
AMAZON
AURORA
R3.8XLARGE
R3.8XLARGE
R3.8XLARGE
R3.8XLARGE
R3.8XLARGE
• Create an Amazon VPC (or use an existing one).
• Create 4 EC2 R3.8XL client instances to run the SysBench
client. All 4 should be in the same Availability Zone (AZ).
• Enable enhanced networking on your clients.
• Tune Linux settings (see whitepaper referenced below).
• Install SysBench version 0.5.
• Launch a r3.8xlarge Amazon Aurora DB instance in the
same VPC and AZ as your clients.
• Start your benchmark!
1
2
3
4
5
6
7
![Page 13: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/13.jpg)
Beyond benchmarks
If only real-world applications saw benchmark performance.
POSSIBLE DISTORTIONS
Real-world requests contend with each other.
Real-world metadata rarely fits in the data dictionary cache.
Real-world data rarely fits in the buffer cache.
Real-world production databases need to run at high availability.
![Page 14: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/14.jpg)
Writes scale with connection count
• OLTP Workload
• Variable connection count
• 250 tables
• Query cache (default on for Amazon Aurora, off for MySQL)
8xU P TO
FA S T E R
![Page 15: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/15.jpg)
Consistent performance as table count increases
• Write-only workload
• 1,000 connections
• Query cache (default on for Amazon Aurora, off for MySQL)
• i2.8XL instances for MySQL SSD and RAM
• r3.8XL instances for Aurora and Amazon RDS MySQL
11xU P TO
FA S T E R
![Page 16: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/16.jpg)
Consistent performance with growing dataset
• SysBench write-only workload
• Aurora r3.8XL
• Amazon RDS MySQL r3.8XL with 30K IOPS (single AZ)
67xU P TO
FA S T E R
![Page 17: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/17.jpg)
Running with read replicas
SysBench write-only workload
250 tables
Updates per
second Amazon Aurora
RDS MySQL
30 K IOPS (single AZ)
1,000 2.62 ms 0 s
2,000 3.42 ms 1 s
5,000 3.94 ms 60 s
10,000 5.38 ms 300 s
500xU P TO
L O W E R L A G
![Page 18: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/18.jpg)
DO LESS WORK
Do fewer I/Os
Minimize network packets
Cache prior results
Offload the database engine
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
BE MORE EFFICIENT
How do we achieve these results?
![Page 19: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/19.jpg)
Aurora cluster
Amazon S3
AZ 1 AZ 2 AZ 3
Aurora Primary
instance
Cluster volume spans 3 AZs
![Page 20: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/20.jpg)
Aurora cluster with replicas
Amazon S3
AZ 1 AZ 2 AZ 3
Aurora Primary
instance
Cluster volume spans 3 AZs
Aurora Replica Aurora Replica
![Page 21: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/21.jpg)
Aurora requires fewer I/Os
Binlog Data Double-write bufferLog records FRM files, metadata
T Y P E O F W R I T E S
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
MYSQL WITH STANDBY
SEQUENTIAL
WRITE
SEQUENTIAL
WRITE
EBSAmazon Elastic
Block Store (EBS)
Primary
Instance
Standby
Instance
AZ 1 AZ 3
Primary
Instance
Amazon S3
AZ 2
Replica
Instance
AMAZON AURORA
ASYNC
4/6 QUORUM
DISTRIBUTED
WRITES
![Page 22: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/22.jpg)
I/O Traffic in Aurora (read replica)
Logical: Ship SQL statements to replica.
Write workload similar on both instances.
Independent storage.
Can result in data drift between master and
replica.
PAGE CACHE
UPDATE
Aurora Master
30% Read
70% Write
Aurora Replica
100% New Reads
Shared Multi-AZ Storage
MySQL Master
30% Read
70% Write
MySQL Replica
30% New Reads
70% Write
SINGLE-THREADED
BINLOG APPLY
Data Volume Data Volume
Physical: Ship redo from master to replica.
Replica shares storage. No writes performed.
Cached pages have redo applied.
Advance read view when all commits seen.
MYSQL READ SCALING AMAZON AURORA READ SCALING
![Page 23: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/23.jpg)
Amazon Aurora is highly available
![Page 24: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/24.jpg)
Amazon Aurora is highly available
Highly available storage
• Six copies of data across three AZs
• Latency tolerant quorum system for read/write
• Up to 15 replicas with low replication lag
Survivable caches
• Cache remains warm in the event of a database restart
• Lets you resume fully loaded operations much faster
Instant crash recovery
• Underlying storage
replays redo records
on demand as part of a
disk read
• Parallel, distributed,
asynchronous
AZ 1 AZ 2 AZ 3
Amazon
S3
SQL
Transactions
Caching
T0
![Page 25: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/25.jpg)
Faster, more predictable failover
AppRunningFailure Detection DNS Propagation
Recovery Recovery
DBFailure
RDS MYSQL
App
Running
Failure Detection DNS Propagation
Recovery
DB
Failure
AURORA WITH MARIADB DRIVER
1 5 – 3 0 s e c
5 – 2 0 s e c
![Page 26: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/26.jpg)
Traditional databases
Have to replay logs since the last
checkpoint.
Typically 5 minutes between checkpoints.
Single-threaded in MySQL; requires a
large number of disk accesses.
Amazon Aurora
Underlying storage replays redo records
on demand as part of a disk read.
Parallel, distributed, asynchronous.
No replay for startup.
Checkpointed Data Redo Log
Crash at T0 requires
a re-application of the
SQL in the redo log since
last checkpoint.
T0 T0
Crash at T0 will result in redo logs being
applied to each segment on demand, in
parallel, asynchronously.
Instant crash recovery
![Page 27: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/27.jpg)
High availability with Read Replicas
Amazon S3
AZ 1 AZ 2 AZ 3
Aurora Primary
instance
Cluster volume spans 3 AZs
Aurora Replica Aurora Replica
db.r3.8xlarge db.r3.2xlarge
Priority: tier-1
db.r3.8xlarge
Priority: tier-0
![Page 28: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/28.jpg)
High availability with Read Replicas
Amazon S3
AZ 1 AZ 2 AZ 3
Aurora Primary
instance
Cluster volume spans 3 AZs
Aurora ReplicaAurora Primary
Instance
db.r3.8xlarge db.r3.2xlarge
Priority: tier-1
db.r3.8xlarge
![Page 29: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/29.jpg)
ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]
ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN
[DISK index | NODE index] FOR INTERVAL interval
ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type
[TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval
Simulate failures using SQL
To cause the failure of a component at the database node:
To simulate the failure of disks:
To simulate the failure of networking:
![Page 30: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/30.jpg)
Amazon Aurora is easy to use
“Amazon Aurora’s new user-friendly monitoring interface made it
easy to diagnose and address issues. Its performance, reliability, and
monitoring really shows Amazon Aurora is an enterprise-grade AWS
database.” —Mohamad Reza, information systems officer at United
Nations
![Page 31: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/31.jpg)
Simplify storage management
• Automatic storage scaling up to 64 TB—no performance impact
• Continuous, incremental backups to Amazon S3
• Instantly create user snapshots—no performance impact
• Automatic restriping, mirror repair, hot spot management, encryption
Up to 64 TB of storage—auto-incremented in 10 GB units
up to 64 TB
![Page 32: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/32.jpg)
Simplify monitoring with AWS Management Console
Amazon CloudWatch
metrics for RDS
CPU utilization
Storage
Memory
50+ system/OS metrics
1–60 second granularity
DB connections
Selects per second
Latency (read and write)
Cache hit ratio
Replica lag
CloudWatch alarms
Similar to on-premises custom
monitoring tools
![Page 33: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/33.jpg)
Simplify data security
Encryption to secure data at rest
• AES-256; hardware accelerated
• All blocks on disk and in Amazon S3 are encrypted
• Key management by using AWS KMS
SSL to secure data in transit
Network isolation by using Amazon VPC by
default
No direct access to nodes
Supports industry standard security and data
protection certifications
Storage
SQL
Transactions
Caching
Amazon S3
Application
![Page 34: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/34.jpg)
Delivered as a managed service
![Page 35: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/35.jpg)
If you host your databases on-premises
Power, HVAC, net
Rack and stack
Server maintenance
OS patches
DB software patches
Database backups
Scaling
High availability
DB software installs
OS installation
you
App optimization
![Page 36: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/36.jpg)
If you host your databases in Amazon EC2
Power, HVAC, net
Rack and stack
Server maintenance
OS patches
DB software patches
Database backups
Scaling
High availability
DB software installs
OS installation
you
App optimization
![Page 37: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/37.jpg)
If you choose Amazon RDS
Power, HVAC, net
Rack and stack
Server maintenance
OS patches
DB software patches
Database backups
App optimization
High availability
DB software installs
OS installation
you
Scaling
![Page 38: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/38.jpg)
Amazon Aurora saves you money
![Page 39: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/39.jpg)
Enterprise grade, open source pricing
Simple pricing
No licenses
No lock-in
Pay only for what you use
Discounts
44% with a 1-year Reserved Instance
63% with a 3-year Reserved Instance
vCPU Mem Hourly Price
db.r3.large 2 15.25 $0.29
db.r3.xlarge 4 30.5 $0.58
db.r3.2xlarge 8 61 $1.16
db.r3.4xlarge 16 122 $2.32
db.r3.8xlarge 32 244 $4.64
• Storage consumed, up to 64 TB, is $0.10/GB-month
• IOs consumed are billed at $0.20 per million I/O
• Prices are for US East (N. Virginia) region
![Page 40: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/40.jpg)
Cost of ownership: Aurora vs. MySQLMySQL configuration hourly cost
Primary
r3.8XL
Standby
r3.8XL
Replica
r3.8XLReplica
R3.8XL
Storage6TB/10K PIOP
Storage6TB/10K PIOP
Storage6TB/5K PIOP
Storage6TB/5K PIOP
$3.78/hr
$3.78/hr
$3.78/hr $3.78/hr
$2.42/hr
$2.42/hr $2.42/hr
Instance cost: $15.12/hr
Storage cost: $8.30/hr
Total cost: $23.42/hr
$2.42/hr
![Page 41: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/41.jpg)
Cost of ownership: Aurora vs. MySQLAurora configuration hourly cost
Instance cost: $13.92/hr
Storage cost: $4.43/hr
Total cost: $18.35/hr
Primary
r3.8XL
Replica
r3.8XL
Replica
R3.8XL
Storage/6 TB
$4.64/hr $4.64/hr $4.64/hr
$4.43/hr
*At a macro level, Aurora saves over 50% in
storage cost compared to RDS MySQL.
21.6%
Savings
No idle standby instance
Single shared storage volume
No PIOPS—pay for use IO
Reduction in overall IOP
![Page 42: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/42.jpg)
Cost of ownership: Aurora vs. MySQLFurther opportunity for saving
Instance cost: $6.96/hr
Storage cost: $4.43/hr
Total cost: $11.39/hrStorage IOPS assumptions:
1. Average IOPS is 50% of maximum IOPS
2. 50% savings from shipping logs vs. full pages
51.3%
Savings
Primary
r3.8XL
Replica
r3.8XLReplica
r3.8XL
Storage/6 TB
$2.32/hr $2.32/hr $2.32 hr
$4.43/hr
r3.4XL r3.4XL r3.4XL
Use smaller instance size
Pay-as-you-go storage
![Page 43: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/43.jpg)
Migration to Aurora is easy
![Page 44: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/44.jpg)
Start your first migration in 10 minutes or less
Keep your apps running during the migration
Replicate within, to, or from Amazon EC2 or RDS
Move data to the same or different database
engine
AWS
Database Migration
Service
![Page 45: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/45.jpg)
Customer
premises
Application users
AWS
Internet
VPN
Keep your apps running during the migration
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let AWS Database Migration Service
create tables, load data, and keep
them in sync
Switch applications over to the target
at your convenience
AWS
Database Migration
Service
![Page 46: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/46.jpg)
Migrate from Oracle and SQL Server
Move your tables, views, stored procedures, and data manipulation language (DML) to MySQL, MariaDB, and Amazon Aurora
Know exactly where manual edits are needed
Download at aws.amazon.com/dms
AWS
Schema Conversion
Tool
![Page 47: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/47.jpg)
Know exactly where manual edits are needed
![Page 48: Getting Started with Amazon Aurora](https://reader031.fdocuments.us/reader031/viewer/2022030311/58eec3001a28ab9f368b46f9/html5/thumbnails/48.jpg)
Thank you!