Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

42
Cloud architecture and deployment: The Kognitio checklist 20 September 2012

description

CloudOps Summit 2012, Frankfurt, 20.9.2012 Track 2 - Build and Run by Nigel Sanctuary, VP Propositions at Kognitio (www.kognitio.com) http://cloudops.de/sprecher/#nigelsanctuary Find the video of this talk at http://youtu.be/wQrHQNOMlKc

Transcript of Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Page 1: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Cloud architecture and deployment: The Kognitio

checklist

20 September 2012

Page 2: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Question

IS THE CLOUD

JUST THE LATEST BUSINESS COUTURE?

BI Process outsourcing

Page 3: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Answer: Well, Yes…..But…

ASP model of the 1990’s

Managed Services

SaaS

Business Process Outsourcing

Virtualisations

Flexibility

Established

New

ish/ig

Page 4: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

The DB traditional performance management model

Develop & Manage + Capacity = Cost

Cost of Data Environment

Complexity and demand of Business intelligence

Growing Data Volumes

Time

Cost + Capacity (Demand & Supply)

System Performance

High

Low

Page 5: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Cost and risk aligned with need and budget

Complexity and demand of Business intelligence

Growing Data Volumes

Time

Cost + Capacity (Demand & Supply)

Performance aligned to incremental growth

High

Low

Low Cost Data Environment Increments

Page 6: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

If these are the drivers, what must we check?

CIO Research Center

Page 7: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Let’s start with a definition

Cloud Computing: Cloud computing is a model for enabling openly available, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

U.S. Government's National Institute of Standards and Technology

Page 8: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Cloud Types

Page 9: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Cloud Service Types

Software as a Service (SaaS): Application only – Salesforce.com

Page 10: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

•  Platform as a Service (PaaS): Applications provided on a preconfigured hardware and configured by customer – Kognitio, Engineyard, SAAVIS

PaaS

Database Systems

Portal Products Testing

Backup & Recovery

Network Management

Integration tools

Business Process Management

State management

Page 11: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

•  Infrastructure as a Service (IaaS): The consumer uses "fundamental computing resources" such as processing power, storage, networking components. AWS, SAVVIS

IaaS Servers

Switching

Networks Virtualisation

Virtualisation

Page 12: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

My Scope

PaaS

Database Systems

Portal Products Testing

Backup & Recovery

Network Management

Integration tools

Business Process Management

State management

Lot’s of experience

Page 13: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Kognitio’s qualifications: Cloud Customers

Page 14: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Kognitio’s Cloud Service

Webs Site

Customer

Private Cloud Public Cloud

Volume Time Location BI App. store

Page 15: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Data moved into high speed memory

Queries run directly against data in memory

Many CPU cores Shared-nothing

Very high performance Handles any analytical workload

RAM merged together across

servers into a shared fabric

CPUs

CPUs

CPUs

CPUs

What the client’s buy : High performance analytical platforms

Server 1

Server 2

Server 3

Server 4

Page 16: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Made possible by the collapse of in Memory costs

Source: Feb, 2012, John C. McCallum

$367.002

$202.375

$135.266 $156.238

$76.546 $85.983

$40.894

$18.874 $22.020

$5.138 $-

$50.000

$100.000

$150.000

$200.000

$250.000

$300.000

$350.000

$400.000

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

RAM $/TByte (10 Years)

RAM $/TByte

Costs have/are falling

Page 17: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

The Check List: For Database services •  What do you NEED to achieve •  What are the benefits you want from Cloud v in house

–  In house V outsource –  Pricing Options –  Contract terms v price

•  Security Levels: –  “Collected data should be kept secure from abuse” EE Data Protection Directive

–  Managing Security – Buying services/responsibility –  How can you put the data in the cloud (Hipaa)

•  Availability –  Volume –  Flexibility - Scaling –  User Base and workload –  Service Levels/ Responsibility –  Application integration/System Integration

Page 18: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

If these are the drivers, what must we check?

CIO Research Center

Page 19: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

The question: Which component

Operational Data Store

Computer/ Telephony Integration

Automated Call Delivery

Interactive Voice Response

PSTN

Analytical Data Platform

Analysis Application

Operational Data staging

Campaign Management

Decision Engine

Risk Management/

Decisions

Product systems

Other op’s Systems

Agent Platform/

Co-browsing

Marketing Analysis/ Decisions

Correspondent

Telephone outbound campaigns

Operational Reporting

Branch

Data

Campaign Data

Campaign

Mail

Scores

Customer/event flags

Contact centre Business

Intelligence Reporting

Campaign

Transactional database

Analytical Database

Operational database

Delivery Environment

Web

Con

tent

Cookies

Tags

Transactional Database

Page 20: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

What do you need it for

Analytical batch Processing

•  Solved through PaaS, SaaS

•  Mid/Large batch data volumes •  Off line analytical processing

(MPP/SMP)

•  Need speed, complexity management, capacity

•  Smaller user community

Application Processing •  Common solutions in IaaS,

SaaS

•  High transactions volumes

•  OLTP

•  Need response, flexibility, management, capacity

•  Dynamically expanding and contracting user community Real Time Reporting

•  Common solutions in PaaS, SaaS (Microstrategy

•  High transactions volumes

•  SMP/MPP

•  Processing dependent, management, capacity

•  Large User Community

Page 21: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

BIG DATA – More in it

Transactions

Contact

Business Operations

Page 22: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

What is the anticipated workload Self learning algorithms

Process Reporting

Statistical Analysis

Behaviour modelling

On-Line Fraud detection

Dynamic Content

Technology/Automation

Ana

lytic

al C

ompl

exity

Clustering

Campaign Management

Analytical Platform as a Service

Page 23: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

What is the User Community

Impact on process capacity •  Demand Spikes

•  Complex Queries

•  Scheduled Capacity

•  Volume of users

Page 24: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Cost and Trade Off •  What are the benefits you want from Cloud v in house

–  In house V outsource –  Pricing Options –  Contract terms v price

Page 25: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Pricing Option Trade Off’s

In-House v Outsource-

1  =  No  Cost  10=High  CostIn  House Outsource

SLA  Control 5 5Capex  Budget 10 3

Budget  Control 8 6

Performance  Control 4 6

System  Management  Costs 8 6System  Maintenance  responsibility 10 4

Software  Maintenance 10 7

Implementation  Costs 10 3Total  Cost  Score 65 40

Comment

Premiums  for  higher  SLA's  in  Outsource,  Shared  Service  costOutsource  No  Capex  options  availableFlexibile  per  usage  models  allow  budget  control.  SAAVIS  and  AWS  both  offer  good  but  different  solutionsIn  House  tend  to  have  faster,  focused  performance  control,  Private  Cloud  is  best  if  this  is  a  priorityShared  resources  and  system  performance  management  is  a  key  benefit  of  cloudThe  key  advantage  of  Outsource  modelsPaaS  Outsource  include  charges  for  maintenance  and  upgrade  using  shared  resourcesIntegrated  architectures  are  pre  constructed  requirinf  only  confirguration  in  the    outsource  modelAbout  38%  improvement  if  you  optimised  the  benefits

Page 26: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Pricing Option Trade Off

Public V Private

Page 27: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Cost savings •  Considerations:

–  Long term v Short term costs in the cloud

–  Trade off of savings v functionality

–  Total Cost of Ownership – Management, Skills, Hardware, Applications, Networks

–  Create you measure of success

–  Target your improvement goal and buy components and services to support it

Page 28: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Security

•  Security Levels: –  “Collected data should be kept secure from abuse” EE

Data Protection Directive

–  Managing Security – Buying services/responsibility

–  How can you put the data in the cloud (Hipaa)

Page 29: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Security

It’s an emotional issue

Page 30: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

The fear

Unauthorised access

Data theft Rogue user abuse

Data in transit

Physical Security

Page 31: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Investment in Cloud Security

Tier 3 data centres VPN/SSH Access Control Monitoring Systems Encryption

http://aws.amazon.com/security/

http://www.rackspace.co.uk/managed-hosting/business-need/security-compliance/

Some good examples

Page 32: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Security service options

RAM merged together across

servers into a shared fabric

CPUs

CPUs

CPUs

CPUs

Data

Data

Data

Data

VPN/ PGP

File Encryption

SysAdmin SSH/FixedIP

Client monitoring

RootAdmin Monitoring

SysAdmin User Privileges

LinuxAdmin Manage S’ware

Dep

loym

entm

in

Ser

ver F

arm

X

Page 33: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Planning for use

•  Availability

–  Volume

–  Flexibility - Scaling

–  User Base and workload

–  Service Levels/ Responsibility

–  Application integration/System Integration

Page 34: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Volume management

Month

TB

TB

Hour

•  Manage to Demand

•  Understand Usage pattern

•  Purchase capacity for storage & Processing Separately.

Page 35: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Manage Analytical Platform System to Demand

Data

Data

Data

Data

Source

Persistent Data Store

SSD/Disk

RAM Servers

•  Store Increasing Data

•  Flexible, granular views of any data

•  Use RAM capacity according to demand

•  Drop and Pin Data in RAM

•  Valuable RAM is controllable

•  Cheap Disk allows you to lose nothing Data Views/

Queries

Page 36: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Near real time/Real time

Page 37: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Public Cloud

Can you create persistent data layers independent from processing? Can you switch data between operational schemas within one DB installation? Can you flex hardware scale to scheduled demand patterns?

Page 38: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Manage and re-configure for capacity

Single Capacity, Multi-Use Kognitio Licence

ETL 1

ETL 2

ETL 3

Kognitio Appliance: Industry Standard Hardware

System management & Data Staging for Import/ Export

Application Source

data

Geo Location Management App

Reporting Application

Analytics Application

Development/Test interface

Export to application

data

Applications

Contact History & Quota

Geo Location Info Service

Product & Service

Catalogue

Loyalty Management

Sandpit 1

Sandpit 2 Inventory

Management Repository

Permissions Database

Data Analytics Database

B2 Customer data Store

Offer and Assets

Repository

Device Catalogue

Capacity Management

Development

Test Test

Test & Development

Analysis Single DB instance

Single DB instance

Single DB instance

Single DB instance

Single DB instance

Dynamic platform management

Page 39: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Private/Hybrid Cloud

•  Partition the system to support differing workloads within one DB installation

•  Support Mass reporting and data mining from one platform

Page 40: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

A sample solution

Kognitio Cloud app store

Firewall

Application Server Cloud Servers (Public or private)

Client 3

Client 4

Source data

Tools

ASP DB Team

Client/ Prospect Fixed IP

Client 2

Client 1

Kognitio Service Support Service & Access requirements

Access /Service Config. Management Report/

Query Requests

Results

Internet

SFTP/other media Data Load

Data

Client Direct Interface

Direct Interface

Client Internet Interface

Page 41: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

The Check List: For Database services

•  What do you NEED to achieve •  What are the benefits you want from Cloud v in house

–  In house V outsource –  Pricing Options –  Contract terms v price

•  Security Levels: –  “Collected data should be kept secure from abuse” EE Data Protection Directive

–  Managing Security – Buying services/responsibility –  How can you put the data in the cloud (Hipaa)

•  Availability –  Volume –  Flexibility - Scaling –  User Base and workload –  Service Levels/ Responsibility –  Application integration/System Integration

Page 42: Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, Kognitio

Thank you

Nigel Sanctuary VP Propositions [email protected]