Post on 31-May-2020
Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite
Peter Zaitsev, Denis Magda Santa Clara, California | April 25th, 2017
About the Presentation
Problems
Existing Solutions
Denis Magda will show the power of Apache Ignite
My Conviction
There is no silver bullet in technology!
Why ?
All design decisions comes with their own benefits and drawbacks
Technologies not Technology
Large Scale applications tend to use more than one technology on
data layer
Works especially well with Open Source!
Additional Components do not require hefty license fees
Easy to prototype and test out
Open Source Community is good at building bridges
Balance is Needed
•Use as many technologies as you need, but no more
MySQL
•MySQL is no Exception. It is not Great for Everything.
Some of the Problems
Hot Data
Highly Volatile Data
Large Data Volume
Analytical Processing
Full Text Search
Hot Data
For example “Cache”
Large volume of simple requests
High overhead due to SQL
No good Memory focused Engine
Not Designed for very high Concurrency
Solutions
•MySQL•MemcacheD interface•Thread Pool
•External•MemcacheD•Redis
Highly Volatile Data
Lots of updates, especially to a single row
Design around full Transactional ACID semantics
Disk Log based durability
Pessimistic Logging
Solutions
•MySQL•Data Design•Configuration Tuning•Parallel Replication
•External•MemcacheD•Redis
Large Data Volume
MySQL is designed as single node system
Limited in CPU, Memory
Manual “Sharding” solutions are painful
Especially with complex queries
Solutions
•MySQL•Manual Sharding•Vitess•ProxySQL
•External•Shading for MemcacheD and Redis•MongoDB•Cassandra
Analytics (OLAP)
MySQL does not support column based storage
MySQL optimizer is limited for complex queries
MySQL does not do parallel query execution
MySQL does not do distributed query execution
Solutions
•MySQL•Configuration and Schema Design (Limited)
•External•Hadoop & Spark•Vertica•ClickHouse
Full Text Search
Can handle basic Full Text Search
Does not scale well with data volume
No parallel processing
Limited search relevance options
Hard To do GIS searches; Facets
No language processing
Solutions
•MySQL•Small Scale search applications only•Supported with Innodb tables since MySQL 5.6
•External•Elastic •Solr•Sphinx
New Solutions constantly appear
•Always be on lookout for a better solutions!
DENIS MAGDAProduct Manager, Apache PMC Chair
Apache Ignite - In-Memory Data FabricBeyond the Data Grid
www.ignite.apache.org #apacheignite
• In-Memory Data Fabric• Advanced Clustering• In-Memory Compute Grid• In-Memory Data Grid• In-Memory SQL Grid• In-Memory Service Grid• In-Memory Streaming & CEP• Plug-n-Play Hadoop Accelerator
IN-MEMORY DATA FABRICSTRATEGIC APPROACH TO IMC
IN-MEMORY COMPUTING WHY NOW?
ADVANCED CLUSTERING & DEPLOYMENT
Ease of Getting StartedAutomatic Discovery
Any EnvironmentPublic CloudPrivate CloudHybrid CloudLocal Laptop
Zero-DeploymentAuto-Deploy Code
Full Cluster Management
Pluggable Design
IN-MEMORY COMPUTE GRID
Direct API for MapReduce
Zero Deployment
Cron-like Task Scheduling
State Checkpoints
Load Balancing
Automatic Failover
Full Cluster Management
Pluggable SPI Design
IN-MEMORY DATA GRID
Distributed In-Memory Key-Value StoreReplicated and Partitioned dataTBs of data, of any typeOn-Heap and Off-Heap StorageHighly Available In-Memory ReplicasAutomatic Failover Distributed ACID Transactions SQL queries and JDBC driverCollocation of Compute and Data
IN-MEMORY SQL GRID
ANSI-99 SQL CompliantAggregations, group by, sortingCross-cache joins, unions, etc.
DML and DDL
JDBC, ODBC, Native APIs
DistributedAlways consistentFault tolerant
IN-MEMORY SERVICE GRID
Distribute Any Data StructureAvailable Anywhere on the GridAutomatic Remote Access via Proxies
Controlled DeploymentSupport for Cluster SingletonSupport for Node SingletonSupport for Custom TopologyLoad Balanced
Guaranteed AvailabilityAuto Redeployment in Case of Failures
IN-MEMORY STREAMING AND CEP
Streaming Data Never Ends
Branching Pipelines
Pluggable Routing
Sliding Windows
Real Time Analysis
IN-MEMORY HADOOP ACCELERATOR
Plug and Play installation
2x to Nx Acceleration
In-Memory Native MapReduce
In-Process Data Colocation
IgniteFS In-Memory File System
Read-Through from HDFS
Write-Through to HDFS
Sync and Async Persistence
SPARK INTEGRATIONSHARED RDD & IN-MEMORY FILE SYSTEM
Turbocharge your SQL queries in-memory with Apache® Ignite™ 25 April - 4:50 PM - 5:15 PM @ Room 210
THANK YOU!
www.ignite.apache.org #apacheignite
Thank You to All of Our Sponsors!!