Dynamo: Amazon’s Highly Available Key-value Store

DAAS – Database as a service

CAP Theorem

Motivation

• Build a distributed storage system:– Scalable– Simple: key-value And Document types– Highly available– Guarantee Service Level Agreements

(SLA)

Structure

• Table, Items and AttributesDifferent from SQL database structure1. Each attribute is name –value pair

2. An attribute can be scaler, a json document or a set

3. One table can have different category of items. Some books, cars, classes etc. But all have unique primary key to fetch an item.

Items, Item and key

Primary key

• Partition key – Input primary key is converted into a hash function to get a position for it to be placed. That is called partition key. Partition key value is unique.

• Partition key and Sort key – (Composite Key) For items having same partition key. All items with the same partition key are stored together, in sorted order by sort key value. It is possible for two items to have the same partition key value, but those two items must have different sort key values.

Table Pet

DynamoDB Data Types

• Amazon DynamoDB supports the following data types:

• Scalar types – Number, String, Binary, Boolean, and Null.

• Document types – List and Map.

• Set types – String Set, Number Set, and Binary Set.

Why is it called DAAS?

• No hardware needed

• No configurations required

• No memory used

• Accessible from anywhere.

• You just pay for the services you use.

Always writable!!!

• Assume N = 3. When A is temporarily down or unreachable during a write, send replica to D.

• D is hinted that the replica is belong to A and it will deliver to A when A is recovered.

• Again: “always writeable”

Basic Vocab

• Low latency allows human-unnoticeable delays between an input being processed and the corresponding output providing real time characteristics.

• Throughput is the rate at which something gets processed.

DynamoDB has objective of providing low latency and high throughput.

Service Level Agreements (SLA)

• Application can deliver its functionality in abounded time: Every dependency in the platform needs to deliver its functionality with even tighter bounds.

• Example: service guaranteeing that it will provide a response within 300ms for 99.9% of its requests for a peak client load of 500 requests per second.

DynamoDB

• Fully managed NoSQL database

• Seamless scalability

• Can store and retrieve any amount of data

• Serve any level of request traffic at a time.

• Automatically spreads data and traffic over significant number of servers to handle requests

Difference

Pricing

• Free Tier – 25GB storage 25 write capacity units

25 read capacity units

Enough throughput for 200 million requests/month

• Paid Tier Varies (0.001 – 0.1 $/hour) depending on

Region

Read/Write requests per month

Failures

• Like Google, Amazon has a number of data centers, each with many commodity machines.– Individual machines fail regularly– Sometimes entire data centers fail due to

power outages, network partitions, tornados, etc.

• To handle failure of entire centers, replicas are spread across multiple data centers.

Pros

• Fast, Consistent• Highly Scalable• Flexible- Both document and key-value data

structures• Fine grained Access Control• Fully managed - by Amazon cloud• Distributed• Cost-Effective

Cons• 64kb limit on row size

• 1MB limit on querying

• No complex queries can be done

• Size is multiple of 4kb for read operations

• Joins are not possibleIts glitch effected big companies like Netflix.

An outage occurred when Dynamo db servers could not query the metadata service within the allocated time.

DEMO

PRACTICAL USAGE of DYNAMODB