MariaDB versus NoSQL; Best of 2 worldsMariaDB and TokuDB MariaDB and Tokutek have agreed to make...

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.

MariaDB versus NoSQL; Best of 2 worlds ?

MariaDB or NoSQL(or both)

Oliopäivät 2012Michael “Monty” WideniusEntrepreneur, MariaDB [email protected]://mariadb.com/

Introducing Maria

mailto:[email protected]

http://mariadb.com/


What is MariaDB

● Open Source, binary compatible, superset of MySQL:● True drop in replacement of MySQL

● Same data on disk and on the wire.● Same file names, same sockets and ports.

● Created and maintained by the same people that created MySQL.● Open development: Developed together with the community.● True fork, not just a patch set upon MySQL.

● MariaDB is not depending on MySQL for future development.● More plugins, more features, faster, better code quality.

● 30 man years more development than MySQL ● GPL-only server license (no closed source extensions)


Why MariaDB was created

“Save the People, Save the Product”

● To keep the MySQL talent together● To ensure that a free version of MySQL always exists● To get one community developed and maintained branch● Work with other MySQL forks/branches to share knowhow

and code


MariaDB server releases

● MariaDB 5.1 was released as stable in February 2010● MariaDB 5.2 was released as stable in November 2010● MariaDB 5.3 was released as stable in April 2012● MariaDB 5.5 was released as stable in April 2012● MariaDB 10.0 was released as alpha in November 2012

● MariaDB-Galera (multi-master) was released as Alpha in September 2012 after a lot of testing.

●Gamma/stable to be released shortly (December)


MyISAM Segmented key cache

● Blue line is without segmented key cache.● Solves one of the major read bottlenecks for MyISAM● We see up to 250% performance gain depending on the

amount of concurrent users.● Fix applies to all MyISAM usage with many readers!


query_size=1 query_size=2 query_size=30

500

1000

1500

2000

2500

3000

quer

y tim

e, s

ec (

smal

ler

is b

ette

r)

New Batched Key Access Speedups

● Join benchmark with BKA

select max(l_extendedprice) from orders, lineitem whereo_orderdate between $DATE1 and $DATE2 andl_orderkey=o_orderkey

10x7x

3x

BKA, join_buffer_size=32M

Regular join


Group commit, verified

Source: Marc Callaghan's facebook blog for a server with 400 microsecond fsync latency


New thread pool for 5.5

8 16 32 64 128 256 512 1024 2048 40960

1000

2000

3000

4000

5000

6000

7000

8000

pool-of-threadsthread-per-connection

Concurrent Clients

OL

TP

(ro

) T

hro

ug

hp

ut,

tp

s


MariaDB and TokuDB

MariaDB and Tokutek have agreed to make TokuDB a native plugin in MariaDB 5.5 and MariaDB 10.0 by end of Q1 2013.

This means that the official MariaDB binary will be able to dynamically load the TokuDB storage engine directly (no patches needed for MariaDB).

TokuDB will be added to the MariaDB buildbot test suite to ensure that the combination is properly tested on all supported platforms.

TokuDB will be available for download from https://downloads.mariadb.org/ together with MariaDB.

About TokuDB

- TokuDB uses Fractal Tree® indexing to improve insert and query speed, compression, replication performance, and online schema flexibility.

- TokuDB is created by Tokutek Inc. See www.tokutek.com for details.


Connect storage engine

MariaDB 10.0 will include the Connect storage engine by Olivier Betrand.

With the connect storage engine you can read, write and update files in a lot of different storage formats:● Various fixed and dynamic text formats● .DBF (dBASE format)● .CSV● .INI● .XML● ODBC ; Table extracted from an application accessable

with ODBC


Why MariaDB instead of MySQL

● MariaDB has 30 man years of more development than MySQL (and the gap will continue growing).

● MariaDB is maintained by the people that originally created MySQL and has the best knowledge of the MySQL code.

● MariaDB is binary compatible with MySQL, so its trivial to replace MySQL with MariaDB (minutes).

● Reasons to switch to MariaDB● Faster queries thanks to XtraDB (InnoDB plugin fork from Percona), a better optimizer and replication and better code.

● Open source development: Anyone can be part of the development at all stages. Dev meetings are public.

● More features, including critical ones like microseconds, multi-source and dynamic column support.

● Less risk as MariaDB will not remove features like MySQL is doing (thread pool, storage engines, safemalloc (developer feature), etc)


Big data is here

Source: IDC Digital Universe study 2011


Big data at end of 2011

● 3146M emails (71% spam)● 555M web sites (300M new)● 220M registered domain names● 800M Facebook users (200M new)● 225M twitter accounts (100M active)● 91250M tweets (250M/day, stored in MySQL)● 2100M internet users (591M with broadband)● 1200M active mobile broadband users● 1 trillion video playbacks from youtube● 6 billion photos on Flickr

Data from royal.pingdom.com


Big data driven by Open Source

● MySQL / MariaDB● SQL database with flexible replication

● Apache Cassandra● Tunable consistency● Keys map to multiple values, which are grouped into column families● CQL language

● MongoDB● Stores structured data as JSON like documents

● CouchDB● MVCC, ACID, eventually consistent

● Hbase (+ Haddop + Hadoop distributed file system)● Written in Java. Compression, in-memory operation and Bloom filters (Is data part

of a set).● Redis

● In memory database (+ snapshots to disk)● Optional durability


Database usage 2011

MySQL SQL Server PostgreSQL Oracle MongoDB MariaDB DB2 Cassandra Reddis0

10

20

30

40

50

60

70

80

%

Based on 451 Research surveys: Sample 205 DB users


Predicted MySQL usage

2009 2011 2012 2014 2014 20170

10

20

30

40

50

60

70

80

90

MySQL forks

Survey 2012

Survey 2009

%

Based on 451 Research surveys

Sample 285/205 users


Big data driven by Open Source

Predicted database usage growth 2012-2017 of total market:● MariaDB (3.9 %)● Apache/Cassandra (3.4 %)● Apache CouchDB (3.0 %)● Apache Hbase (2.9 %)● Redis (2.5 %)● PostgreSQL (2.4 %)● DB (-1.5%)● Oracle (-5.9%)● SQL Server (-9.3%)● MySQL (-26.4%)

Based on 451 Research survey 2012


Reasons to use NoSQL

The main reasons for using NoSQL are:●Handling of unstructured data (not everything is table and fixed number of columns)

●Faster replication (usually with 'unconventional' shortcuts)

●Fast and easy key/value access●Data is often stored in memory

● The same way MySQL with it's storage engine interface can handle both transactional and datawarehousing, we are extending MariaDB to be a bridge between SQL and NoSQL.


SQL is catching up

Replication is getting better and faster● MariaDB has multi-master replication (galera) and Tungsten replication is getting traction

● PostgreSQL now includes replication● MariaDB added to 5.3 (one year ago) dynamic columns which

allows each row to have different set of columns.● Twitter, the company that started the NoSQL hype with their

blog about Cassandra, is still using MySQL to store tweets.● Teradata just announced a Hbase connection● The main benefit of most NoSQL solutions is that data is

mostly in memory.● With similar memory resources you can usually keep SQL data in memory too.


MariaDB as a bridge between SQL and NoSQL

● Up to 50 % faster HANDLER commands (HANDLER READ …)● Up to 530,000 queres/second measured(*)

● HandlerSocket compiled in (Direct access to InnoDB)● Dynamic columns (each row can have different set of columns)

● Building block for NoSQL storage engines● Storage engine for Cassandra

● You can read, write, update and join with Cassandra● We are working on a storage engine for LevelDB

(*) Stephane Varoqui's blog:

http://varokism.blogspot.com/2011/01/20-to-50-improvement-in-mariadb-53.html


Dynamic columns in MariaDB 5.3

● With dynamic columns all extra columns are stored in one or many packed blobs, maintained by the database.

● You can instantly add more columns, remove or query them for a row.

● You can access columns in the server or retrieve the full blob to the client and manipulate it there.

● You can use virtual columns to create indexes on some values.

●True indexes for dynamic columns is planned for later.● Implemented trough functions to enable use by ODBC etc.● First implementation uses an integer to access columns.

●In MariaDB 10.0 columns can be accessed by name.


Dynamic columns in MariaDB 5.3

● Simple set of functions (available in server and client):●COLUMN_CREATE(column_nr, value,[column_nr,....])●COLUMN_ADD(blob,column_nr, value, [column_nr,...])●COLUMN_DELETE(blob, column_nr, column_nr...);●COLUMN_EXISTS(blob, column_nr);●COLUMN_LIST(blob, column_nr);●COLUMN_GET(blob, column_nr, type);

As a proof of concept we have in MariaDB 10.0 created a storage engine for Cassandra where we use dynamic columns as a bridge.


Reason to not use NoSQL

● Can't (easily) join/combine data● No standardized language; Hard to move data and

applications between systems● Complete lock in to one system● Normally few connections to computer languages or existing

applications that needs a database.● Allows one to initially ignore solving the database layout

problem as one can store data 'as such'● Initially easier to use, MUCH harder later

● No sustainable scalable business models for most open source NoSQL solutions (as most of them relay on the BSD or Apache licenses)


NoSQL and the money

MySQL, NoSQL and NewSQL revenue

NoSQL$20m

NewSQL$12M

MySQL$171m

NoSQL$215m

CAGR: 82%

NewSQL$112M

CAGR: 75%

MySQL$664m

CAGR: 40%

451 Research

2011 2015 (estimated)


Problems to overcome with BIG data

Most companies don't need 'big data' solutions● Storage (but not normally the big problem)

● Memory (Very expensive)● SSD (Expensive but slower)● Hard disk (Inexpensive. Slow when doing random reads, fast on sequential reads)

● Access patterns● Get it up when things fails

●Recovery; Can take days or weeks●Get the caches warm (you need > 90 % hit rate with buffers!)

● Replication and hot standby●Must have, but makes big data more expensive


Why big data solutions is a hype

Most companies don't need 'big data' solutions● Most companies will not have as much data as Facebook, Twitter and Walmart.

● Solutions that they have to use are not applicable for others.● In the near future one can run what most companies think is

big data on a few machines:● Memory now costs about $10/G; Anyone can afford 24G-100G RAM.● 512G SSD is now $400 (Read: 500MB/sec 80K IOPS, Write: 260-400M/sec write, 36K IOPS)

● 1.2T fusionIO cards (can be linked to 2.4T) gives you persistent NAND memory at close-to-memory speed (2.5G/sec write, 3G/sec read)

● Violin gives you 22TB flash memory (2G/sec)● By 2016 we can expect consumer harddisk with 60TB (Seagate)

● Most companies don't have many terabytes of real data that needs to be accessed 'at once'.


MariaDB Foundation Overview

The Foundation is the new driver of the MariaDB project

Custodian of the code, Guardian of the community

Foundation can never to be controlled by a single entity or person

Foundation designed to be self-sustaining


MariaDB Foundation Details

A US 501c(3) not-for-profit8 founders and 2 sponsors so far (including all MySQL founders!)Open for more founders until FebruaryIn February the founders will the choose board etc.

In the interim we have:● Monty as CTO● Andrew Katz as COO and Legal Person (checking all the

paperwork)● Niall McCarthy as Business Development (signing up

founders)


MariaDB Foundation Goals

That MariaDB be actively developed in the community and to:

● Increase adoption of MariaDB● Ensure sustainable high-quality efforts to build, test and

distribute MariaDB● Ensure that community patches are reviewed and adopted● Guarantee a community voice● Keep MariaDB compatible with MySQL● Maintain mariadb.org


MariaDB Foundation

More founders and sponsors are welcome!

If you care about the future of the MySQL ecosystem, please contact us andand ask how you can get involved!

Niall McCarthy [email protected] Widenius [email protected] Katz [email protected]




Thanks

Q & A

MariaDB versus NoSQL; Best of 2 worldsMariaDB and TokuDB MariaDB and Tokutek have agreed to make...

Documents

Transcript of MariaDB versus NoSQL; Best of 2 worldsMariaDB and TokuDB MariaDB and Tokutek have agreed to make...