NoSQL, No SQL!!, No, SQL?

Raj Nair, Penton

Variety is the spice of life

Key-Value stores

Document

storesColumnFamilyGraphHybrid

Spice can lead to

heartburn

General Advantages

Programmer friendly

Web friendly

No pre-defined schemas

Better value @scale

General ChallengesMore onus on the application

Lose the power of SQL

Different needs can require different stores

How much do you care about “immediate consistency” ?

Good knowledge of access patterns for schema design

Key-Value stores

Simple

Fast

Uses

• hash table/dict of keys and values

• Super fast for key based access

• Session Management• High frequency atomic

operations• Caching

Document stores

Rich

Powerful

Uses

• JSON format• Structure

• Indexing capabilities• REST interfaces

• Non-key based queries• Schema• All of that at Scale

Columnar Stores (Special Mention)

• Physical storage is “column” based

• Rows only materialized in memory

• Great for analysis/warehouse type workload• Billions of rows, you want only a handful of columns • Faster aggregation

Column Family

Flexible

High Scale

Uses

• No format imposed

• Read/write • Low latency

• Logs• Messaging• Temporal/TimeSeries

Graph databases

Niche

Uses

• Model Pairwise relations

• Networked systems• Triple stores or RDF• Recommendation

engine base (matches on dating sites?)

“Other”

Postgres – has built KV storage, HStore

RDF / triple stores – specialized graph stores

XML stores

New SQL – yeah really!!

My “Hypothesis” – Long Tail, DataAp

plic

ation

s u

sing

da

ta

Data available for active use

PB100 TB50 TB 500 TB500 GB

Single server

RDBMS

Clustered RDBMS

Upto low 100s of GB

100’s of GB

to few TBs

** Not to Scale** Illustrative only

NoSQL

Workload Economics

Mins to hours$ per GB

<200 ms

$$ per GB

$$$$ per GBIn-memory - Few GBs

Operational NoSQL - 100’s GBs to few TBs

Analytic NoSQL - TBs to 100’s TB

Hadoop - TBs to PBs

> 200 ms to 2 secs$$$ per GB

Few seconds to minutes

* Nieman Marcus approach presented at TDWI Solution Summit 2014

Popular Document StoresMongoDB CouchDB/CouchBase

Benefits Developer friendlyIndexing

Operations friendly,Easy scale outCaching support

Challenges Harder to scale , sharding, keys, write locks

Unique query design, complex index creation

CAP Leans towards “C” Leans towards “A”

So when do I use it?

- Your RDBMS is growing out of a single server environment or you are in clustered RDBMS mode - You’d rather respond correctly or not give an answer at all - You are anticipating a 2-4 node cluster

- When low latency is a high priority for you - Additionally, you want richness of document store with query flexibility- Scale out management needs to

be friendly- You prefer that app always gets a

response

ColumnFamilyHBase Cassandra

Benefits Really scales!! Columns and Rows! SQL “like” query language

Challenges Leaves a lot to the application codeNo query language

Unique query design, complex index creation

CAP Leans towards “C” Leans towards “A”

So when do I use it?

When eventual consistency is not an option , but you can live with “less availability”When you have a finite number of key-based access patternsWhen your programmers are comfortable buidling queries You are working on the Hadoop stack and prefer strong Hadoop integration

When you need When query patterns are more complicated and you need to use secondary indexesWhen your developers prefer an “SQL like” interface for queries

Everyone has a sayDictionaries,

variables, objects, arrays

How many systems? Who has the skills?

Stability Easy to meet changing

needs, flexibility

SLAs

App Developer

OpsBusiness

Customer