Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA
-
Upload
severalnines -
Category
Technology
-
view
1.281 -
download
0
Transcript of Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA
![Page 1: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/1.jpg)
Confidential
Polyglot Persistence
December 22nd, 2015
Art van Scheppingen
Severalnines
![Page 2: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/2.jpg)
Confidential
Copyright Severalnines AB
2
Logistics
Webinar is recorded Replay available soon Feel free to ask questions at any time Use your control panel to contact us Or email us as well: [email protected]
![Page 3: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/3.jpg)
Confidential 3
Agenda What is Polyglot Persistence? Example project Is Polyglot Persistence good or bad? Leverage ClusterControl for Polyglot Persistence
(demo) Q&A
Copyright Severalnines AB
![Page 4: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/4.jpg)
Confidential
Copyright Severalnines AB
4
What is Polyglot Persistence?
![Page 5: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/5.jpg)
Confidential
Copyright 2013 Severalnines AB
5
Polyglot Persistence
Using multiple specialized persistent stores rather than
one single general-purpose database
![Page 6: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/6.jpg)
Confidential
Copyright Severalnines AB
6
Where does the term come from?
The way we work is changing Enterprise applications are becoming more complex Separate (devops/agile) teams Ownership of applications (Micro)services
Everyone has their preference Various programming languages Various storage systems
![Page 7: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/7.jpg)
Confidential
Copyright Severalnines AB
7
Where does the term come from?
Monoglot Programming Only one programming language allowed Readability
All code is in the same language Support
One platform to support Knowledge
Everybody is an expert Is there a jack-of-all-trades language?
![Page 8: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/8.jpg)
Confidential
Copyright Severalnines AB
8
Monoglot Programming
Does a carpenter only use just one tool?
![Page 9: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/9.jpg)
Confidential
Copyright Severalnines AB
9
Monoglot Programming
Carpenters use a broad variety of tools
![Page 10: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/10.jpg)
Confidential
Copyright Severalnines AB
10
Polyglot Programming
Polyglot Programming Use programming languages for what they are good at Flexibility
Use Java for a secure API Use Scala for real time stream processing Use Python for text analysis Tie everything together using AngularJS
Knowledge Everybody is expert at one or more languages
![Page 11: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/11.jpg)
Confidential
Copyright Severalnines AB
11
Polyglot Programming
![Page 12: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/12.jpg)
Confidential
Copyright Severalnines AB
12
Monoglot Persistence
![Page 13: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/13.jpg)
Confidential
Copyright Severalnines AB
13
Data storage landscape changes
Relational data stores (RDBMS) Key-Value data stores Columnar data stores Document data stores Graph data stores
![Page 14: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/14.jpg)
Confidential
Copyright Severalnines AB
14
Data stores overview
SoftwareRDMBS MySQL,
PostgreSQL, Oracle
Key-Value Redis, RiakColumnar Cassandra,
Hbase, InfiniDB
Document MongoDB, Couchbase
Graph Neo4J, Titan
![Page 15: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/15.jpg)
Confidential
Copyright Severalnines AB
15
Data stores overview
Software AWS GoogleRDMBS MySQL,
PostgreSQL, Oracle
RDS, Aurora CloudSQL
Key-Value Redis, Riak DynamoDB DatastoreColumnar Cassandra,
Hbase, InfiniDB
Redshift Big Query
Document MongoDB, Couchbase
SimpleDB Bigtable
Graph Neo4J, Titan Titan + DynamoDB
-
![Page 16: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/16.jpg)
Confidential
Copyright Severalnines AB
16
Even Hadoop becomes a polyglot
![Page 17: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/17.jpg)
Confidential
Copyright Severalnines AB
17
Polyglot Persistence
Complex problems require different storage systems Use the right tool for the job, for example
Use PostgreSQL for financial data Use MySQL for website contents Use MongoDB for user profiles Use Cassandra for real time streams Use Neo4J for recommendation analysis
![Page 18: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/18.jpg)
Confidential
Copyright Severalnines AB
18
Use the right tool for the right job Document storage: MongoDB
![Page 19: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/19.jpg)
Confidential
Copyright Severalnines AB
19
Use the right tool for the right job Columnar storage: Cassandra
![Page 20: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/20.jpg)
Confidential
Copyright Severalnines AB
20
Use the right tool for the right job Graph storage: Neo4J
![Page 21: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/21.jpg)
Confidential
Copyright Severalnines AB
21
Polyglot Persistence
![Page 22: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/22.jpg)
Confidential
Copyright Severalnines AB
22
Example project
![Page 23: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/23.jpg)
Confidential
Copyright Severalnines AB
23
Project management lifecycle
![Page 24: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/24.jpg)
Confidential
Copyright Severalnines AB
24
Project management lifecycle
![Page 25: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/25.jpg)
Confidential
Copyright Severalnines AB
25
Project management lifecycle
![Page 26: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/26.jpg)
Confidential
Copyright Severalnines AB
26
Development
Requirements are gathered Create a recommendation system Storage must be (blazingly) fast Document must contain user details Data must be kept until the next visit
Choose the right tool for the right job Store data as a document Use MongoDB
Development
![Page 27: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/27.jpg)
Confidential
Copyright Severalnines AB
27
But what happens after development?
Delivery Ops sets up the application on the infrastructure Monitoring is added to see if the application works If everything is fine the product will be handed over
What’s next? Systems monitoring Backups Scaling Software updates BI
![Page 28: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/28.jpg)
Confidential
Copyright Severalnines AB
28
The ops perspective
![Page 29: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/29.jpg)
Confidential
Copyright Severalnines AB
29
Systems monitoring
Supported data stores so far: MySQL Memcached
How do we monitor MongoDB? Integration with Nagios / Zabbix / Shinken Integration with Graphite / OpenTSDB / InfluxDB
What should we monitor? Key performance metrics SQL vs Mongo queries
![Page 30: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/30.jpg)
Confidential
Copyright Severalnines AB
30
Backups
Supported backups so far Xtrabackup streams
How do we backup MongoDB? What should we backup from MongoDB? How do we restore from a backup? How do we provide a backup for development?
![Page 31: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/31.jpg)
Confidential
Copyright Severalnines AB
31
Scaling out
We are data hoarders and like to keep everything Scaling options in MySQL
Read-bound workloads: add more slaves Write-bound workloads: sharding or NDB Cluster
Scaling in MongoDB? When should we scale out? How do we add a new shard? How do we redistribute between shards? How do we add a new read slave?
![Page 32: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/32.jpg)
Confidential
Copyright Severalnines AB
32
Software updates
When do we need to update with MySQL? Once we hit a bug? Every major release? Every minor release?
When do we need to update with MongoDB? How do we know the impact?
![Page 33: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/33.jpg)
Confidential
Copyright Severalnines AB
33
Business Intelligence
BI with MySQL? Export full database from a slave Allow schemas/tables to be queried for specific tables Most BI tools can connect to MySQL
BI with MongoDB? Native MongoDB analytics tools Write your own exports BI connector (since version 3.2)
![Page 34: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/34.jpg)
Confidential
Copyright Severalnines AB
34
A new project emerges
![Page 35: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/35.jpg)
Confidential
Copyright Severalnines AB
35
With new requirements
Requirements are gathered Allow users to view their earnings Data must always be 100% reliable Storage must be ACID compliant
Choose the right tool for the right job Store data relational Use PostgreSQL
![Page 36: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/36.jpg)
Confidential
Copyright Severalnines AB
36
Is Polyglot Persistence good or bad?
![Page 37: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/37.jpg)
Confidential
Copyright Severalnines AB
37
The drawbacks
More specific (domain) knowledge is needed Additional tooling may be necessary Need more hands to maintain the systems
![Page 38: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/38.jpg)
Confidential
Copyright Severalnines AB
38
The benefits
More flexibility to your company Driven and dedicated devops who take ownership Use the right tool for the right job
![Page 39: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/39.jpg)
Confidential
Copyright Severalnines AB
39
The challenges
Deploy various data stores Handle HA and failovers Scaling and sharding Interpret graphs and monitoring Schedule backups and handle recovery Handle updates/upgrades
![Page 40: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/40.jpg)
Confidential
Copyright Severalnines AB
40
Polyglot Persistence is here to stay
Variety of data storage needs keeps growing Polyglots in the cloud Better be prepared for it
![Page 41: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/41.jpg)
Confidential
Copyright Severalnines AB
41
Leverage ClusterControl for Polyglot Persistence (demo)
![Page 42: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/42.jpg)
Confidential
Copyright Severalnines AB
42
Q&A
![Page 43: Slides: Polyglot Persistence for the MongoDB, MySQL & PostgreSQL DBA](https://reader033.fdocuments.us/reader033/viewer/2022050614/58f2fde71a28ab28618b45a3/html5/thumbnails/43.jpg)
Confidential 43
Thank You! ClusterControl
www.severalnines.com/product/clustercontrol
ClusterControl – Getting Started www.severalnines.com/getting-started
Polyglot Persistence meetups http://goo.gl/64Ga5z
Severalnines Blog www.severalnines.com/blog
Contact: [email protected]
New ClusterControl Tips & Tricks