Download - Teradata QueryGrid to MongoDB Lightning Introduction

Transcript
Page 1: Teradata QueryGrid to MongoDB Lightning Introduction

Teradata QueryGrid to MongoDB Lightning Introduction

Rich Charucki - Teradata

Page 2: Teradata QueryGrid to MongoDB Lightning Introduction

2

What is a Teradata Data Warehouse?

• Analytic database– In-memory, in-database

• Scale-out MPP– 30+ petabyte sites– 35PB, 4096 cores

• Self service BI– Dashboards, reports, OLAP– Predictive analytics

• Complex SQL– 20-50 way joins– 350 pages of SQL

• Real time access/load

• Mixed workloads

Datascientists

Powerusers

Sales,partners

1024 nodes

IntelCPUs

512GB

IntelCPUs

512GB

IntelCPUs

512GB

IntelCPUs

512GB

Page 3: Teradata QueryGrid to MongoDB Lightning Introduction

3

JSONPath inside SQL

Color Size Prod_ID Create_Time----- ----- ------- -------------------Blue Small 96 2013-06-17 20:07:27

SELECT box.MFG_Line.Product.Color AS "Color", box.MFG_Line.Product.Size AS "Size", box.MFG_Line.Product.Prod_ID AS "Prod_ID", box.MFG_Line.Product.Create_Time AS "Create_Time"

FROM mfgTable WHERE CAST(box.MFG_Line.Product.Create_Time AS TIMESTAMP) >= TIMESTAMP'2013-06-16 00:00:00' AND box.MFG_Line.Product.Prod_ID = 96;

Page 4: Teradata QueryGrid to MongoDB Lightning Introduction

4

Math and Stats

DataMining

BusinessIntelligence

Applications

Languages

Marketing

ANALYTIC TOOLS &

APPSUSERS

UNIFIED DATA ARCHITECTURE

MarketingExecutives

OperationalSystems

FrontlineWorkers

CustomersPartners

Engineers

DataScientists

BusinessAnalysts

INTEGRATED DATA WAREHOUSE

DISCOVERY PLATFORM

DATA LAKE

REAL TIME PROCESSING

ERP

SCM

CRM

Images

Audio and Video

Machine Logs

Text

Web and Social

SOURCES

Page 5: Teradata QueryGrid to MongoDB Lightning Introduction

5

MONGODB

NoSQLDatabase

Teradata and MongoDB: QueryGridIDW

TERADATA DATABASE

Discovery

ASTER DATABASE

Business users Data scientists

TERADATA ASTER SQL,

SQL-MR,SQL-GR

Teradata Systems

TERADATA DATABASEHADOOP

Push-down

to Hadoop

SAS, Perl, R, Python,

Ruby

LANGUAGES

Page 6: Teradata QueryGrid to MongoDB Lightning Introduction

6

Integration Export / Import

Direct Connect

Page 7: Teradata QueryGrid to MongoDB Lightning Introduction

7

Teradata and MongoDB

• Operational + Analytical

– Rich MongoDB applications

– Rich Teradata analytics

– Complementary

• Teradata pulls directly from MongoDB sharded clusters

• Teradata pushes back to MongoDB deployments

MongoDB Teradata

Operational Data

Analytics

Page 8: Teradata QueryGrid to MongoDB Lightning Introduction

8

Scale-out NoSQL + Scale-out DW SQL Application

Primary

Shard 1

Primary

Shard 2

Primary

Shard N

Primary

Shard 3

Query router Query router Query router

NoSQL

SQL

AMPAMP

PE

AMPAMP

PE

AMPAMP

PE

AMPAMP

PE

Page 9: Teradata QueryGrid to MongoDB Lightning Introduction

9

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

Contract Phase

Teradatanode

PE

SQL

EAH

AMP

AMP

AMP

AMP

Page 10: Teradata QueryGrid to MongoDB Lightning Introduction

10

Contract Phase

Teradatanode

AMP

AMP

AMP

AMP

EAH

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

PE

Page 11: Teradata QueryGrid to MongoDB Lightning Introduction

11

Data Export to Shards

Teradatanode

EAH

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

AMP

AMP

AMP

AMP

PE

Page 12: Teradata QueryGrid to MongoDB Lightning Introduction

12

Import Data from Shards

Teradatanode

EAH

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

AMP

AMP

AMP

AMP

PE

Page 13: Teradata QueryGrid to MongoDB Lightning Introduction

13

Back-office context to the Front-office operations

Use cases

Page 14: Teradata QueryGrid to MongoDB Lightning Introduction

14

Data Warehouse

eCommerce in Action: A Virtuous Circle

Buyer preferencesSales catalogCampaigns

Recent purchasesProfitability

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Page 15: Teradata QueryGrid to MongoDB Lightning Introduction

15

Data Warehouse

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Shardreal time

Call Center Efficiency: A Virtuous Circle

Trouble ticketsCustomer profilesPayment history

ClaimsNext best offer

web logs

Page 16: Teradata QueryGrid to MongoDB Lightning Introduction

16

• Context from the DW– Enriching MongoDB applications

• Integration– Import/export – Teradata QueryGrid

• Two scale out architectures– OLTP scale-out – Analytics scale-out

• JSON in the data warehouse

Conclusions

Page 17: Teradata QueryGrid to MongoDB Lightning Introduction

1717