NoSQL: An Architects Perspective

03.04.13

NoSQL: An Architect‘s Perspective Eberhard Wolff Architecture and Technology Manager adesso AG

About me

►  Eberhard Wolff

►  Architecture & Technology Manager at adesso

►  adesso is a leading IT consultancy in Germany

►  Speaker

►  Author (e.g. first German Spring book)

►  Blog: http://ewolff.com

►  Twitter: @ewolff

►  [email protected]

Back in the Days….

03.04.13 NoSQL aus Sicht eines Architekten

NoSQL Is All About the Persistence Question


Key-Value Stores

►  Maps keys to values

►  Just a large globally available Map

►  i.e. not very powerful data model

►  No complex queries or indices

►  Just access by key

►  Redis: Think cache + Persistence

►  Riak: Think massive scale

Key Value 42 Some

data


Wide Column

►  Add any "column" you like to a row

►  Not key-value - "key-(column-value)”

►  Column families are like tables

►  E.g. in the "Users" column family >  "someuser" è ("username"è"someuser"),

("email" è"[email protected]")

►  Columns named: indexing possible

►  So queries possible

►  Apache Cassandra

►  Amazon SimpleDB

►  Apache HBase

►  All tuned for large data sets

XX

XX XX XX XX

XX XX XX

XX XX XX XX

XX XX XX XX

XX XX XX XX

XX XX XX XX

XX XX

XX XX XX

XX XX XX

XX XX XX

XX XX XX XX

XX XX XX XX

XX xX XX XX XX


Document Stores

►  Aggregates are typically stored as "documents“ (key-value collection)

►  JSON quite common

►  No fixed schema

►  Indexes possible

►  Queries possible >  E.g. "find all baskets that contain the product 123"

►  Still great horizontal scalability

►  Relations might be modeled as links

►  MongoDB, CouchDB


Graph

►  Nodes with Properties

►  Typed relationships with properties

►  Ideal e.g. to model relations in a social network

►  Easy to find number of followers, degree of relation etc.

►  Hard to scale out

►  Neo4j


NoSQL Benefits


Costs •  Scale out instead of Scale Up •  Cheap Hardware •  Usually Open Source

Flexibility •  Schema in code not in

database •  Easier to upgrade schema •  Easier to handle

heterogeneous data No Object/relational impedance mismatch •  NoSQL database are more OO like

Ops

Dev

Drivers

Exponential Data Growth

More Connected Data

Semi Structured Data

Scale Out

Key Value

Wide Column

Document

Graph

Cost

Flexibility


Document-oriented Databases are the best NoSQL database


For at least one definition of “best”

Document-oriented databases

►  Offer scale out >  Unless you need huge amounts of data

►  Offer a rich and flexible data model > …and queries

►  Other databases have other sweet spots

>  Huge data sets >  Graph structures >  Analyzing data

►  Niches or mainstream?


Cost

Flexibility

Polyglot Persistence in Ecommerce Application


Financial Data

RDBMS

Product Catalog

Document Store

Shopping Cart

Key / Value

Recommendation

Graph

Needs transactions & reports. Data fit well in tables.

High Performance & Scalability No complex queries

Complex document-like data structures and complex queries

Based on friends, their purchases and reviews

The NoSQL Game


Financial Data

RDBMS

Product Catalog

Document Store

Shopping Cart

Key / Value

Recommendation

Graph

Needs transactions & reports. Data fit well in tables.

High Performance & Scalability No complex queries

Complex document-like data structures and complex queries

Based on friends, their purchases and reviews

0 1000

900 800

2700 High Score!

Just Like the Patterns Game!


Points for each Pattern used Extra points if one class implements multiple Pattern

This is not how Software Architecture works.


Why not?


More is worse!

More hardware More Ops Trouble •  Installation • Backup • Disaster Recovery • Monitoring • Optimizations

More Developer Skillz Not necessarily bad

But: Polyglott Persistence Has a Point


Object-oriented Databases did it wrong •  Strategy: Replace RDBMS •  Enterprises will stick to RDBMS •  Pure technology migration basically

never happens •  …only vendors think differently

Example: Archive Database •  Store current data in RDBMS •  Store archive in NoSQL (MongoDB) •  Archive contains mainframe data

•  Benefit: Use flexibility to allow for many data formats •  Benefit: No need to convert mainframe data •  Benefit: Store lots of data cheaply

Complex Document Processing System


MongoDB Redis elastic search

Document-oriented

Meta Data for quick access

Search index

Documents

Key/value in memory Search

engine

Alternative: Only elasticsearch


elastic search •  Stores original documents as well

•  (like a key/value store) •  Support for complex queries •  Very powerful features also for data

mining / analytics

Alternative: Only MongoDB


MongoDB •  Now with (limited beta) fulltext search •  Quite fast – memory mapped files •  So why Redis? •  Map/Reduce support

What about Redis?


Redis •  Like a Swiss Knife •  Cache •  Messaging •  Central coordination in a distributed

environment

Your Choice – a trade off!


Typical architecture decision

Who Does What? RDBMS

Developer / Architect

► Schema design (at least partly) ► Access code

DBA

► Performance tuning ► Indices ► Query optimization

► Changes do not influence code


Data Access: RDBMS


RDBMS

•  Schema •  Stored Procedures

Data Model

Data Access •  Queries •  Other code

Architect/ Developer

Optimizations

•  Indices •  Tables

spaces No need to change code •  …

DBA

RDBMS separate data from data access


Joins and normalization allow flexible data access patterns

Indices

Data Access MongoDB


MongoDB

•  Influences access patterns

Data Model

Data Access •  WriteConcerns

how much do love your data?

•  Shard key •  Consistency


Optimizations

•  Only basic indices

Other optimizations must be done in code

DBA

Cluster: RDBMS

► Works somehow

► A special setup of hardware and RDBMS software


DBA

Cluster: MongoDB ►  CAP theorem

>  Consistency >  Availabilty >  Partition tolerance >  Choose any two

►  Deals with replication ►  MongoDB has master /

slave replication



MongoDB

►  Write Concerns: >  Unacknowledge >  Acknowledged >  Jounrnaled >  Some nodes in the

replica set

►  Queries might go to master only or also slaves

►  Influences consistency

More Power and more Responsibility


DB Admin

Architect

Architects

►  Architect has always been a multi-dimensional problem

►  Need to choose persistence technology

►  Need to think about operations

►  Needs to do DBA work 03.04.13 NoSQL aus Sicht eines Architekten

NoSQL Is All About the Persistence Question


NoSQL: An Architects Perspective

Technology

Transcript of NoSQL: An Architects Perspective