Kickfire June 18

8/14/2019 Kickfire June 18

1/33

2008 Kickfire, Inc. All rights reserved.

Turbocharging MySQL Reportingand Data Warehousing


2/33


Agenda

MySQL intro

Kickfire background Technology overview

Demo

Customer case studies Q&A


3/33


Agenda

MySQL intro


Demo



4/33


Growing use for data warehousing and BI MySQL DW deployed at 28% of MySQL customers

Strong DW ecosystem support

MySQL Data Warehousing Market


5/33


MySQL and Kickfire Break Records

100 GB TPC-H records1

#1 performance, non clustered #1 price/performance

300 GB TPC-H records2

#1 performance, non clustered

#1 price/performance

System tested is fully ACID compliant

1. The Kickfire Database Appliance Series 2300 achieved 49,228 QphH@100G. The price performance is $0.70/QphH@100GB USD.The total Kickfire system price over three years is $34,425 USD.

2. The Kickfire Database Appliance Series 2400 achieved 54,895 QphH@300G. The price performance is $0.89/QphH@300GB UDS.The total Kickfire system price over three years is $48,790 USD.

The appliances will be available October 14, 2008. TPC-H is a registered trademark of the TPC council.


6/33


Agenda

MySQL intro


Demo



7/33


8/33


DW and Reporting Problems We Hear About

Technical issues Reports and queries take too long to run

Lots of tuning and maintenance necessary

Difficulty supporting ad-hoc queries or complex reports

Key features needed Parallel query

I/O subsystem performance

Data warehouse-focused optimizer

Business impact Difficulty in growing the system with the business

Too much time, money, and effort spent on maintenance

Value for business and for customers limited


9/33


Who is Kickfire?

Worlds first high-performance appliance for MySQL

Makes MySQL rock for reporting and queries Affordable, low-power, load-and-go appliance Scalable from GBs to TBs


10/33


11/33


Quick Poll #2

Do you have a dedicated MySQL reporting and/or data

warehousing server today?

Yes

No

No, but plan to


12/33


The Technology (1/2)

SQL: MOPS

=

1:10

Query Parallelization

done on SINGLE SQCdrastically reducing

h/w footprint

SQC uses column-store, compression,

intell indexes and pre-fetches from storage

SQC operates directly

out of Memory, NOT

limited by Registers

Parallel Query

=

Massive h/w buildout

von Neumann bottleneck

=

CPUs idle

SQL: Assembly Code

=

1:1,000,000,000,000

I/O bottleneck

=

Massive storage


13/33


The Technology (2/2)

Dataflow Pipelined Parallel Processing (SQL Chip)

Storage

Array

Unified

Database

Memory

Dataflow

Engine

(SQL Chip)

Unified

Database

Memory

Host

Memory

x86

x86


14/33


1. SQL execution

2. Memory management

3. Loader acceleration

1. Connectivity

2. Security

3. Administration

1. Optimizer

2. Column store & cache

3. Transactional engine

KDBMySQL

Whats in the Box


15/33


Deploying the Box

Setup Pre-configured appliance to deliver high performance on broad set

of schemas and queries

Data movement and loading Utility to migrate from existing MySQL data sources

Fast Loader can load 100GB/hr; Incremental Loader for changes

Works with ETL tools certified for MySQL

Reporting and queries Supports full MySQL SQL syntax Works with business intelligence tools certified for MySQL


16/33


Product Line Overview

2000 Series

Size: 2 RU Capacity: up to 0.8TB

Power use: ~600 W

Price: starting at ~$20k

3000 Series Size: 3 RU

Capacity: up to 3TB

Power use: ~700 W

Price: starting at ~$65k


17/33


Quick Poll #3

If you use MySQL for reporting or data warehousing, how

much data are you reporting on today?

1GB - 49GB

50GB 99GB

100GB 999GB

1TB 5TB

> 5TB


18/33


Demo


19/33


Agenda

MySQL intro


Demo

Customer case studies

Q&A


20/33


Customer Case Study #1: Overview

Customer Public company with large online communities

Problems Canned reports are too slow and report development too slow

because need too many aggregate tables

Cant create a new revenue service based on ad hoc queries

Customer environment

1 TB MySQL DW with multiple customer DB of clickstream data

Two dual-CPU servers with 16GB RAM

Kickfire system

Kickfire Database Appliance 2200 with 32GB RAM


21/33


Customer Case Study #1: Results

Test platform

One 45GB customer DB 50 million rows in fact table

Sample query below

Results

Average 35X improvement


22/33


Customer Case Study #2: Overview

Customer Successful high-tech startup in network management domain

Problems Cant scale to larger data volumes of network data

Revenue tied to scaling of network monitoring service

Customer environment Up to 1 TB DW of customer network data

Kickfire system Kickfire Database Appliance 2300 with 64GB RAM


23/33


Customer Case Study #2: Results

Test platform

450M rows in each fact table Over 125 tables

450GB total

Sample query below

Results

Average 600X improvementover current system results

In lab were able to improve 60Xwith partitions etc. but requiredmajor app re-architecture


24/33


Q&A

Additional information

Product is currently in beta; program currently oversubscribed If interested in trials, please email [email protected]

For more info, download our white paper and data sheet

Stay in touch Read our blog www.kickfire.com/blog

Send us an email [email protected]

Call us at 408.450.5400


25/33


Appendix


26/33


TPC-H Queries

Multiple aggregations over large number of rows (Q1) Multiple tables in the query (Q2, Q5, Q7, Q8, Q9) Like predicate [over a large number of rows] and string functions

(Q2, Q9, Q13, Q16, Q22) Order by limit (top 10-100 rows) (Q2, Q3, Q10, Q18, Q21) Correlated subqueries (Q4, Q17, Q20, Q21, Q22) Case expression (will highlight how well you can handle conditional

expressions) (Q8, Q12, Q14) Large groupby key (Q10, Q18) Left Outer join ( Q13) Queries over views (Q15) Complex filters involving AND-OR (Q19)

Randomized data and query parameters


27/33


100GB Performance: Kickfire vs. MyISAM

Ran all 22 queries tocompletion

MyISAM timed outafter three hours on 12of the queries

Over 1000x faster onaverage

Query 22

Query 21

Query 20

Query 19

Query 18Query 17

Query 16

Query 15

Query 14

Query 13

Query 12Query 11

Query 10

Query 9

Query 8

Query 7

Query 6

Query 5

Query 4

Query 3

Query 2

Query 1

3.7

23.6

2.1

2.2

260.6

5.9

4.4

1.5

36.8

52.7

4

9.9

0.8

5.1

2.3

4.6

3.6

7.5

2.2

34.1

Kickfire (secs)

100.72

10800

1450.42

4682.19

108001895.55

693.56

10800

2345.68

10800

108002726.2

10800

10800

4122.62

4328.22

10800

10800

10800

10800

10800

3737.54

MyISAM (secs)

27

458

691

2128

4153159

118

2455

1564

293

21601010

2700

1091

5153

849

4696

2348

3000

1440

4909

110

Speedup


28/33


p0(a)p0(a)

p1(a,b)

p1(a,b)

T1.a

Chip Exploits fine-grained Parallelism

Pipelined parallelism

Data-partitioned parallelism

Independent-operator parallelism

Inter-query parallelism

f(.)f(.)

T1.a

T2.bT2.bp0(a)p0(a)

T1.a

p1(a,b)

p1(a,b)

T2.bT2.b

p2(c,d)

p2(c,d)

T2.dT2.c


29/33


KDB: Transactional Storage Engine

Full ACID compliant

KDB supports serializable isolation level

TPC-H benchmark requires full ACID

Automatic crash recovery using the write-ahead logs

Automatic check-pointing for fast recovery from crashes

Automatic deadlock detection and rollback

Support for streaming DML for operational BI

Future release to allow concurrent updates with long running queries

Similar to MVCC but without the expensive overhead


30/33


Loading Utilities

Fast loader for initial load

Efficiently loads data using bulk processing - 100GB in 1hr

Leverages the SQL Chip to execute the bulk operations

Incremental Loader for periodic loading

Bulk loading of Inserts, Deletes, and Updates SQL chip to offload

Row store like efficiency on column store data

Data Migration tools utilize Fast and Incremental Loaders

Schema migration + data type mapping + index migration

Foreign key migration or declaration


31/33


KDB supported Indexing

Basic indexing is automatically generated

All primary keys have a B+ tree index automatically

All Foreign keys have a FK-index automatically

The following indices can be created/deleted by user

Any Date/Time column can have a Date-range index

Any string column can have Word index B+ tree index for any data type

Any automatically created indices can be deleted


32/33


Leveraging MySQL SE Architecture

KDBStorageEngine

Parsedquery isinterceptedby KDB &

executed inChip

Answer isreturnedthrough

MySQL

MySQLOptimizer &execution is

by-passed

Source: MySQL 5.0 Pluggable Storage Engine Architecture document


33/33


Typical Features in DW Queries

Many columns; Complex expressions; Aggregations;

In-line-views (Subselect)

Complex selection conditions: Large IN lists; Case/If; OR conditions;

Functions: Extract, Substring, Like

Complex Join conditions: FK joins; EQ joins

Exists, NOT Exists, IN, NOT IN subselect (Correlated)

Multi-column, Expressions

Condition on Aggregations

Orderby on multi-columns and Aggregations, with limit

From

Select

Having

Where

Group By

Order By

Proj. Optimization

Primary key condition

OrderBy limted values

Functions/HW limits

Kickfire June 18

Documents

Transcript of Kickfire June 18