NoSQL: The Good, the Bad and the Ugly

Rob Harrop

About Me

• Co-founder of Spring Source

• Writer of books

• Now co-founder and CTO at Bamboo

• UK-based consumer finance startup

• I like data

• My business likes data

• NoSQL matters to me!

Naming

NoSQL

Not Only SQL

–Martin Fowler

“When we say "x is a NoSQL database" I think it's silly to interpret NoSQL as "Not Only"

because that would render the term meaningless. (You could then reasonably argue that SQL Server (say) is a NoSQL database.)”

• Not using the relational model (nor the SQL language)

• Open source

• Designed to run on large clusters

• Based on the needs of 21st century web properties

• No schema, allowing fields to be added to any record without controls

Polyglot Persistence

IMHO…

Landscape

Data Model• Relational

• Key-value

• Document

• Column

• Graph

• And many more…

Query Model• SQL

• Key/value

• Map/reduce

• JSON path

• Search

Storage

• Distributed

• Replicated

• In-memory

Durability

• Ephemeral stores

• System-of-record stores

• Archive store

Consistency

• Linearised

• Eventual

• Causal

Operations

• Backup

• Failover

• HA

• SaaS

Good, Bad and Ugly

Good: Choice

• Right tool for the job

• Choose the simplest tool

• Have more fun

Bad: Choice

• Making the choice is hard

• Danger of analysis paralysis

• Might head down the wrong road

Bad: Complexity

• Consistency is front and centre

• Data models aren’t always straightforward

• Might head down the wrong road

Good: Complexity

• Illusions of reliability and consistency are removed

• Deeper understanding cannot hurt

Bad: Marketing

• Many elaborate claims

• Plenty of dis-information

• Difficult for beginners to find the kernel of truth

Good: Science!

• Good quality experience reports

• Solid base of literature

• Deep analysis

• Kyle Kingsbury - Jepsen

• Testing isn’t all that hard

Ugly: Trolling

• Sheer number of choices leads to many opinions

• Exaggerated claims compound with opinions

• Claims can be difficult to verify

• Evangelism leads to emotional investment

Good: Many Models

• No longer one big data model

• Compose models to solve a problem

• Different models with different requirements

• Model per micro-service?

• Split by use-case

• Read model for analytics

• Search model

• Low latency write model

• Layer within use-case

• Caching

• System of record

• Separate write model (CQRS)

Good: SaaS

• AWS

• Redshift, DynamoDB, Elasticache, S3, RDS

• Google Cloud

• SQL, Storage, BigQuery, Datastore

• Azure

• SQL, DocumentDB, Redis, Storage

Good: SQL Innovation

Doing NoSQL

Learn

• Understand the concepts

• Don’t get overwhelmed!

• Read the literature

• The Morning Paper from Adrian Colyer

Specify• What are your performance/reliability/scalability

requirements?

• How will you handle differing consistency models?

• Who owns what data?

• How long does data live for?

• Does data live in only one place?

Get Started• Implement something small

• Caching, search, analysis

• Use ancillary technologies

• Blob store

• Archive storage

Get Serious• Get the best out of your SQL database

• Identify core functional areas for NoSQL

• Low-latency

• Document

• Migrate gradually

• Consider parallel runs

Get Sophisticated

• Segment system into multiple models

• Identify consistency/durability/reliability boundaries

• Fit the right store to each model

• Layer stores within a model as needed

Q&A@robertharrop