OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

NoSQL & MySQL: The Best of Both Worlds

Andrew Morgan (@andrewmorgan)www.clusterdb.com

Principal MySQL Product Manager

11th June 2014

Safe Harbour Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

MySQL Customers

World’s Most Popular Open Source Database

WebSaaS, Hosting

Enterprise OEM / ISV’s

Telecom

SQL NoSQL

Session Agenda

NoSQL – What are people looking for? RDBMS – What advantages do they still have? How MySQL Delivers the Best of Both Worlds

– MySQL Cluster NoSQL attributes: Scale-out, performance, ease-of-use, schema

flexibility, on-line operations NoSQL APIs

– Key-Value store access to InnoDB (Memcached)

Types of NoSQL stores

Key-Value

• Cassandra• Memcached• BigTable• Hadoop• Voldermort

Document

• MongoDB• CouchDB

• Neo4J• FlockDB

Key-Value Store

Document Store

Graph Database

What NoSQL must deliver

Massive scalability– No application-level sharding

Performance High Availability/Fault Tolerance Ease of use

– Simple operations/administration– Simple APIs– Quickly evolve application &

schema

ScalabilityPerformance

HAEase of use

Still a role for the SQL (RDBMS)?

Simple access patterns

Compromise on consistency for performance

Ad-hoc data format

Simple operation

Complex queries with joins

ACID transactions

Well defined schemas

Rich set of tools

No best single solution fits all

Mix and matchScalability

PerformanceHA

Ease of useSQL/Joins

ACID Transactions

The Temptation to Jettison Relational Model Relational Model:

– Data stored in multiple tables

– Many lookups to retrieve a user’s timeline

– Is it worth the effort in setting up this complex data model?

Simpler just to store as one document?

The allure of document stores

Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy

friend

comment

commenter

The Temptation to Jettison Relational Model

Document Model:– Entire timeline in a single

document (row)– Single lookup to retrieve the

user’s timeline– Brilliantly efficient model when

the document truly contains self-contained information

Like a real-world document!

{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}

These are all people who have their own data that users will want to view or click through:

– Name– url– Timeline

Easy to represent with FKs in a relational model

But when the data isn’t self contained…

friend

comment

commenter

Do I store all data for all friends and likers again at every point they appear in the document?

– Massive amount of repeated data

Wasted space Needs to be kept in sync (and

you don’t have transactions to provide consistency

{name: ‘Joe’, url: ‘…’ stream:[ {friend:{ name: ‘Jane’, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: ‘Joe’}, {…} ], likes: [‘Joe’, ‘Fred’, ‘Billy’] }, {…},{…},… ] }, {…},{…},…}

The Temptation to Jettison Relational ModelThe allure of document stores

{name: 83746251, url: ‘…’ stream:[ {friend:{ name: 9384726153, posts:[{content: ‘…’, comments:[ {comment: ‘…’, commenter: 83746251}, {…} ], likes: [83746251, 750730283, 2938493820] }, {…},{…},… ] }, {…},{…},…}

The reality is that the developer will store the user-ids instead

Developer is responsible for implementing ‘joins’ in the application

– e.g. scanning all timelines for where a user made a comment or liked a post

No transactions to ensure consistency between documents

MySQL Cluster: Overview• Auto-Sharding, Multi-Master• ACID Compliant, OLTP + Real-Time Analytics

HIGH SCALE, READS + WRITES

• Shared nothing, no Single Point of Failure• Self Healing + On-Line Operations

99.999% AVAILABILITY

• Open Source + Commercial Editions• Commodity hardware + Management, Monitoring ToolsLOW TCO

• Key/Value + Complex, Relational Queries• SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++SQL + NoSQL

• In-Memory Optimization + Disk-Data• Predictable Low-Latency, Bounded Access TimeREAL-TIME

Who’s Using MySQL Cluster?

MySQL Cluster Architecture

MySQL Cluster Data Nodes

Data Layer

Clients

Application Layer

Management

Data Layer

Clients

Application Layer

ManagementManagement

Data Layer

Application Layer

ManagementManagement

Clients

On-line Operations

Scale the cluster (add & remove nodes on-line) Repartition tables Upgrade / patch servers & OS Upgrade / patch MySQL Cluster Back-Up Evolve the schema on-line, in real-time

Scale-Out Reads & Writes on Commodity Hardware

8 x Commodity Intel Servers– 2 x 6-core processors 2.93GHz – x5670 processors (24 threads)– 48GB RAM

Infiniband networking flexAsynch benchmark (NDB API)

2 4 80

1,200SELECT Queries per Minute

Number of Data Nodes

204060

120UPDATE Queries per Minute

Number of Data Nodes

JOIN Performance

33K rows over 11 tables

Must Analyze tables for best results

mysql> ANALYZE TABLE <tab-name>;

MySQL Cluster 7.1 MySQL Cluster 7.20

102030405060708090

Query Execution Time Seconds

Scalability aPerformance a

HA aEase of useSQL/Joins a

ACID Transactions

70xMore

performance

MySQL Cluster 7.4

Performance gain over 7.3– 47% (Read-Only)– 38% (Read-Write)

Better performance and operational simplicitylabs.mysql.com

Faster node restarts– Recovering nodes rejoin the cluster

faster

Creating & running your first ClusterThe traditional way (pre-MCM & Auto-Installer)

Download & Extract•edelivery.oracle.com

•www.mysql.com•dev.mysql.com

Configure•Cluster-wide “config.ini”

•Per-mysqld “my.cnf”

Start processes•Management Nodes•Data Nodes•MySQL Servers

HA aEase of use aSQL/Joins a

ACID Transactions

MySQL Cluster 7.3: Auto-Installer

Fast configuration Auto-discovery Workload optimized Repeatable best practices

Specify Workload

Auto-Discover

Define TopologyDeploy

MySQL Cluster Manager

1. Download MCM/Cluster package from edelivery.oracle.com:2. Unzip3. Run agent, define, create & start Cluster!

$> bin\mcmd --bootstrapMySQL Cluster Manager 1.1.2 started

Connect to MySQL Cluster Manager by running "D:\Andrew\Documents\MySQL\mcm\bin\mcm" -a NOVA:1862

Configuring default cluster 'mycluster'...

Starting default cluster 'mycluster'...

Cluster 'mycluster' started successfully

ndb_mgmd NOVA:1186

ndbd NOVA

mysqld NOVA:3306

mysqld NOVA:3307

ndbapi *

Connect to the database by running "D:\Andrew\Documents\MySQL\mcm\cluster\bin\mysql" -h NOVA -P 3306 -u root

MCM: Upgrade Cluster

mysql> upgrade cluster--package=7.3 mycluster;

HA aEase of use aSQL/Joins a

ACID Transactions

NoSQL Access to MySQL Cluster data

ClusterJMySQL

JNIPython Ruby

ClusterJPA

Apps Apps Apps Apps Apps

Node.js

mod-ndb

Apache

ndb-eng

Memcached

Apps Apps

NDB API (C++)

PHP PERL

Schema-Free apps Rapid application evolution

– New types of data constantly added

– No time to get schema extended

– Missing skills to extend schema

– Initially roll out to just a few users

– Constantly adding to live system

Cluster & Memcached – Schema-Free

<town:maidenhead,SL6>

key value

<town:maidenhead,SL6>key value

Key Valuetown:maidenhead SL6

generic table

Application view

SQL view

Cluster & Memcached - Configured Schema

<town:maidenhead,SL6>

prefix key value

<town:maidenhead,SL6>key value

Prefix Table Key-col Val-col policytown: map.zip town code cluste

rConfig tables

town ... code ...

maidenhead ... SL6 ...

map.zip

Application view

SQL view

MySQL 5.6 Memcached with InnoDB

8 32 128 5120

MySQL 5.6: NoSQL Benchmarking

Mem-cached APISQL

Client Connections

Clients and Applications

MySQL ServerMemcached Plug-in

innodb_memcached

local cache(optional)

Handler API InnoDB API

InnoDB Storage Engine

mysqld process

SQL Memcached Protocol

Up to 9x Higher “SET / INSERT” Throughput

MySQL Cluster 7.3: Node.js NoSQL API Native JavaScript access to MySQL Cluster

– End-to-End JavaScript: browser to the app & DB

– Storing and retrieving JavaScript objects directly in MySQL Cluster

– Eliminate SQL transformation Implemented as a module for node.js

– Integrates Cluster API library within the web app Couple high performance, distributed apps,

with high performance distributed database Optionally routes through MySQL Server

MySQL Cluster NoSQL API for Node.js & FKs FKs enforced on all APIs:{ message: 'Error', sqlstate: '23000', ndb_error: null, cause: { message: 'Foreign key constraint violated: No parent row found [255]',

sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } }

Next StepsLearn More• www.mysql.com/cluster• Authentic MySQL Curriculum:

http://oracle.com/education/mysql

Try it Out• dev.mysql.com/downloads/cluster/

Let us know what you think• clusterdb.com• @clusterdb• forums.mysql.com/list.php?25

OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

Software

Transcript of OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

MySQL Cluster - 2012.nosql- · PDF fileMySQL Cluster Scaling to Billion Database Queries with MySQL Cluster Bernd Ocklin MySQL Cluster Engineering bernd.ocklin@oracle.com

Developing for Node.JS with MySQL and NoSQL

Crazy NoSQL Data Integration with Pentaho NoSQL Data Integration with Pentaho NoSQL Matters, ... Sybase IQ All different in ... • Insert MySQL data into MongoDB and CouchDB

SQL and NoSQL on MySQL - Percona€¦ · SQL and NoSQL on MySQL Wonderful things you can do with MySQL 5.7 Peter Zaitsev CEO, Percona July 23, 2016 Data Day Seattle

MySQL Integration with NoSQL, Big Data and the Cloudeffectivemysql.com/downloads/MySQLIntegrationwith... · MySQL Integration with NoSQL, Big Data and the Cloud Ronald Bradford New

Your Guide to MySQL (no "NoSQL" stuff)

Асинхронная репликация MySQL без цензурыЗнакомимся • Писал OLAP, OLTP и NoSQL базы данных (QD, MySQL, SciDB) • Помогаю

MySQL NoSQL APIs

PerconaLiveNYC2011 How MySQL and NoSQL Coexist

NoSQL and MySQL: News about JSON

NoSQL Plus MySQL From MySQL Practitioner\'s Point Of View

MySQL & noSQL Conference: MySQL for Sysadmins

NoSQL and MySQL webinar - best of both worlds

MySQL 5.7 - NoSQL Возможности NoSQL, JSON, PluginX · Percona Live Europe Call for Papers & Registration are Open! •Championing Open Source Databases MySQL, MongoDB,

MySQL: Combining SQL and NoSQL · MySQL: Combining SQL and NoSQL Tomas Ulin, VP MySQL Engineering ... functionality described for Oracle’s products remains at the sole discretion

NoSQL with MySQL

MySQL do ISAM ao NoSQL

Simple Way for MySQL to NoSQL

How MySQL and NoSQL Coexist - Percona · How MySQL and NoSQL Coexist Matt Yonkovit - Percona. DIAMOND SPONSORSHIPS THANK YOU TO OUR ... Oracle …

MySQL 8 : The Best gets even better · 2018-07-27 · MySQL 5.7 MySQL InnoDB Cluster - MySQL Group Replication - MySQL Router - MySQL Shell MySQL 8.0 - NoSQL Document Store - JSON