How Open Source Databases Can Help Your Business - IBM · PDF fileHow Open Source Databases...

How Open Source Databases Can Help Your Business

Mark A. Short

Lead Migration Consultant

IBM Migration Factory

Talking PointsTalking PointsTalking PointsTalking Points

• IntroductionIntroductionIntroductionIntroduction

• Open Source Database Options

• Some Open Source Considerations

• Open Source SQL options to replace Oracle and MySQL

• Open Source NoSQL options to enhance a companies database environment

• Typical Use Cases

• The Risks and Benefits of Implementing Open Source Databases

• Implementing Open Source Databases on IBM Platforms

• Implementation and Migration Services Available

• Who to Contact

The changing database landscape

“Cost Optimization with

Open-Source DBMSS”

Donald Feinberg

VP & Distinguished Analyst

Red Hat Summit September 2009

– Gartner

“In the current economic climate, the number of inquiries to Gartner about the use of open-source DBMSs in place of a commercial DBMS is escalating rapidly.”

The changing database landscape

“By 2018, more than 70% of new in-house

applications will be

developed on an

OSDBMS.” 1

– Gartner

The digital economy is driven by big data.

To deal with it, companies require more agile, flexible, and scalable tools.

Source: The State of Open-Source RDBMSs, 2015, Gartner, Donald Feinberg, Merv Adrian, April 2015

Open-Source DBMS Maturity

2009

Open-Source DBMS Maturity

2015

OSDBMS MaturityOSDBMS MaturityOSDBMS MaturityOSDBMS Maturity


• Introduction

• Open Source Database OptionsOpen Source Database OptionsOpen Source Database OptionsOpen Source Database Options








• Who to Contact


• Introduction


• Some Open Source ConsiderationsSome Open Source ConsiderationsSome Open Source ConsiderationsSome Open Source Considerations







• Who to Contact

RDBMs and NoSQL system typesRDBMs and NoSQL system typesRDBMs and NoSQL system typesRDBMs and NoSQL system types

• Relational database management systems (RDBMS) Relational database management systems (RDBMS) Relational database management systems (RDBMS) Relational database management systems (RDBMS) support the relational

(=table-oriented) data model. The schema of a table (=relation schema) is

defined by the table name and a fixed number of attributes with fixed data

types. A record (=entity) corresponds to a row in the table and consists of the

values of each attribute.

• Document Databases Document Databases Document Databases Document Databases (eg – MongoDB) store data in Documents, Documents

contain one or more Fields. Data can be queried based on any combination of

fields in a document. The appeal of these systems is that that are very general

purpose, have large application ecosystems and map very nicely to support

and enable many of today’s object oriented programing styles.

RDBMs and NoSQL system types (Cont.)RDBMs and NoSQL system types (Cont.)RDBMs and NoSQL system types (Cont.)RDBMs and NoSQL system types (Cont.)

• Key Value Store Databases Key Value Store Databases Key Value Store Databases Key Value Store Databases (eg – Redis) are the most basic type of non-

relational DBs. They store a Key and associated Values.

• Wide Column Stores Wide Column Stores Wide Column Stores Wide Column Stores (example – Cassandra) vary in the number of Columns

that are stored. The appeal of these systems is around their very high

performance and scalability. For example, some customers running Cassandra

and their tables have > 30,000 columns. Oracle RDBMS can have a maximum

of 1000 columns per table, 200-300 is more "normal".

• Graph Databases Graph Databases Graph Databases Graph Databases – (eg – Neo4j) focus on storing simple and complex

relationships and can be queried to discover simple and more complex

relationships between the data.

NoSQL DBs usually NoSQL DBs usually NoSQL DBs usually NoSQL DBs usually ScaleoutScaleoutScaleoutScaleout, RDBMS usually , RDBMS usually , RDBMS usually , RDBMS usually ScaleupScaleupScaleupScaleup Brewer’s CAP Theorem(1998)Brewer’s CAP Theorem(1998)Brewer’s CAP Theorem(1998)Brewer’s CAP Theorem(1998)

• Pick Any Two *Pick Any Two *Pick Any Two *Pick Any Two *• ConsistencyConsistencyConsistencyConsistency–All nodes see the same

data at the same time (not the same “consistency” in the ACID definition)

• AvailabilityAvailabilityAvailabilityAvailability–every request receives a response about whether it succeeded or failed

• Partition TolerancePartition TolerancePartition TolerancePartition Tolerance-The system will continue to function despite arbitrary partitioning (access to nodes) due to network failure.

* In reality it’s not this absolute

C

P

ARD

BM

S

CA:

PostgreSQL/EnterpriseDB

DB2/Oracle/SAP HANA

Neo4j

CP:

MongoDB

BigTable

HBase

Redis

AP:

Cassandra

CouchDB

Riak

AWS

DynamoDB


• Introduction



• Open Source SQL options to replace Oracle and MySQLOpen Source SQL options to replace Oracle and MySQLOpen Source SQL options to replace Oracle and MySQLOpen Source SQL options to replace Oracle and MySQL






• Who to Contact

Which Open Source Databases are good candidates to replace Oracle and MySQL and why?

Oracle

• Enterprise DB – Postgres Plus Advanced Server

• “Deep database compatibility with Oracle” plus “familiar Oracle tools like EDB*Plus, EDB*Loader, …”

• Splendid Data – PostgresPURE

• “Matches functionality of Oracle” and is 100% Open Source

• 2nd Quadrant – Providing PostgreSQL professional services and support

MySQL

• MariaDB – MariaDB Enterprise

• MySQL fork therefore a compatible replacement for MySQL

Target Database Candidates

Enterprise DB

• This is a fork of Postgres (therefore is based on a highly stable OLTP platform) that has been extended to provide a high level of support for Oracle procedural objects and provides built in tools for executing the DDL conversion

• Compared to all other target candidates, EBD has the most compatibility and includes “look-alike” utilities for Oracles’ SQL*Plus, SQL*loader, many dictionary views as well as excellent PL/SQL support

• The Oracle extensions are a licensed product


MariaDB

• A fork of MySQL, taken by the original developer (Monte Widenius) just after the acquisition of MySQL by Oracle

• It now ships as the default Open Source RDBMS with all major Linux distros – Red Hat, Ubuntu, SUSE

• Like Postgres, it is a highly stable OLTP platform but there are no Oracle specific extensions for it, in comparison to those provided in EDB

• MariaDB is a drop in replacement for MySQL - not Oracle


MariaDB (Continued)

• Being fully open source and supported by the MariaDB Foundation and Community, there are no license fees

• The current support for the Enterprise Edition on Power is being migrated to the MariaDB Community (mariadb.org)

• Commercial support will continue to be available from MariaDB(mariadb.com) as our relationship evolves


Like for Like

• In the “like for like” space, EDB again is the strongest candidate

• peripheral processes that support a DSS/OLAP environment such as ETL processing, real time EDI data interchange, desktop and middle tier query engines all need to be evaluated and migrated

Green Field

• In many cases, customers are trying to retrofit unstructured data into traditional SQL databases such as Oracle.

• Traditional database vendors have tried to extend their databases but these traditional databases are not really designed to optimally handle unstructured data.

• Therefore there may be some Oracle DSS/OLAP customers that warrant a partial or complete move away from a SQL target database to one of the non-sqlplayers

It’s not all about the databaseIt’s not all about the databaseIt’s not all about the databaseIt’s not all about the databaseSome other considerations for migrations

• Database Connectivity

• Desktops and application servers will connect to the Oracle database typically using SQL*NET so these will need to be changed to leverage JDBC

• SQL Optimizations

• Application SQL will have been optimized for Oracle performance either within the database via the optimizer or with embedded hints so this will need to be addressed

• OS level software

• ETL applications, batch schedulers, data scrubbers, performance monitors, backup software etc will need to be migrated to target versions or functional equivalents

• EDI / Event Messaging

• Many databases will exchange data with external entities (either directly with other databases or indirectly via EDI backbones such as TIBCO) so this will need to be addressed

Operating System (OS)

OS resident applications

OS resident support services

Database

Network

Application Servers

Desktop

The Onion – Glass to Metal


• Introduction




• Open Source NoSQL options to enhance a companies database Open Source NoSQL options to enhance a companies database Open Source NoSQL options to enhance a companies database Open Source NoSQL options to enhance a companies database environmentenvironmentenvironmentenvironment





• Who to Contact

All in the NoSQL FamilyAll in the NoSQL FamilyAll in the NoSQL FamilyAll in the NoSQL Family

• Document databases Document databases Document databases Document databases are generally useful for content management systems, blogging platforms, web analytics, real-time analytics, ecommerce-applications.

• Graph databases Graph databases Graph databases Graph databases are very well suited to problem spaces where we have connected data, such as social networks, spatial data, routing information for goods and money, recommendation engines

• KeyKeyKeyKey----value databases value databases value databases value databases are generally useful for storing session information, user profiles, preferences, shopping cart data

• Column family databases (Wide Column Stores) Column family databases (Wide Column Stores) Column family databases (Wide Column Stores) Column family databases (Wide Column Stores) are generally useful for heavy write volume such as log aggregation / Internet of Things data

MongoDBMongoDBMongoDBMongoDB

• MongoDB is a NoSQL Open Source Document Database

• JSON-like documents with dynamic schemas (MongoDB calls the format BSON)

• As of May 2016, MongoDB was the fourth most widely mentioned database engine on the web, and the most popular for document stores.

• ibm.co/29Rr51J - getting started with MongoDB Enterprise on Linux on Power

• MongoDB Enterprise LoP Tuning Guide: ibm.co/28Qv5iK

MongoDB (Cont.)MongoDB (Cont.)MongoDB (Cont.)MongoDB (Cont.)

• Forbes gains critical insight into the social sharing of their articles, to capitalize on stories going viral in real-time.

• Otto, Europe’s second-largest e-commerce company, continually updates its catalog of over 2M products to provide a one-to-one shopping experience for 30M shoppers and drive €2.3B in revenue.

• Bosch has built its Internet of Things suite on MongoDB, bringing the power of big data to a new range of Industrial Internet applications including manufacturing, automotive, retail, energy and many others.

• Expedia is a virtual concierge – Using MongoDB, it pushes special travel offers to users in real time by tracking their searches and comparisons across its site.

RedisRedisRedisRedis

• Best used: For rapidly changing data with a foreseeable database size (should fit mostly in memory), e.g. to store real-time stock prices, Real-time analytics, Leaderboards, etc.

• Exploits Power 8:

• Redis Labs on Power utilizes IBM POWER8 servers, the Flash System, the IBM CAPI-Flash card and the Redis Labs Enterprise Cluster (RLEC) for Flash software.

• Other features:

• Master-slave replication

• automatic failover

• Redis on Power systems https://redislabs.com/solutions/redis-labs-and-ibm-power-systems

RedisRedisRedisRedis (Cont.)(Cont.)(Cont.)(Cont.)

• InstagramInstagramInstagramInstagram – Instagram uses Redis heavily to run their main feed, activity feed and session store. You can check out some of the articles about their infrastructure here and here, see how they scaled in a great presentation by one of the founders, and read about their switch from Cassandra to Redis. Lastly, check out Redis-Faina, a query analyzer that the engineering team at Instagram built.

• GithubGithubGithubGithub – Github is using Redis for exception handling and queue management. They also use Redis for configuration management, and as a persistent key/value store for routing all kinds of data.


• Introduction





• Typical Use CasesTypical Use CasesTypical Use CasesTypical Use Cases




• Who to Contact

26

http://i0.wp.com/www.jamesserra.com/wp-content/uploads/2015/04/pp.png

Example Example Example Example ---- EEEE----commerce app leveraging commerce app leveraging commerce app leveraging commerce app leveraging multiple Databasesmultiple Databasesmultiple Databasesmultiple Databases

From Polyglot Persistence – Martin Fowler & Pramod Sadalage (Thoughtworks)

What might a client environment look like What might a client environment look like What might a client environment look like What might a client environment look like ––––Relational and NonRelational and NonRelational and NonRelational and Non----Relational data storesRelational data storesRelational data storesRelational data stores

Semiconductor, ASE Singapore (ASEAN)

Company Overview

ASE Singapore Pte. Ltd. is one of the total test solutionsproviders in Singapore with the primary objective of providinga more integrated supply chain with real time services to meetcustomers’ needs of semiconductor packaging and testing inthe ASEAN cluster.

Client Requirements

• The backend core systems needed to supportmanufacturing of semiconductor products have to berefreshed to cater for growing workload demands andactivities.

• This core system is for tracking their manufacturingprocess and yield/fault reporting.

• Client’s primary focus is on Postgres DB Server as it’svery core to their manufacturing’s speed of delivery.They have a team of skilled Postgres DBA (with focuscapability in DB tuning).

• Key consideration in platform choice is based onreliable RAS and price/performance.

• Existing platform is Lenovo System x server, and theyneed a new open platform that is able to give them theperformance they need to align with business demand.

Solution

� Hardware: 1 x S812L with 10 cores & 96 GBmemory

� Software: CentOS� DB: PostgreSQL distribution.

Business Results

• Yield and fault reporting now takes only few minutes to generate (Took about 1 hr previously when run on x86) – At least 80% faster in reporting on Power.

• Senior Management is now able to • make quicker decision (at least 50% faster

decision time)• minimize back-end rework and testing

(improve 3x productivity)• IT is able to provide a much better SLA (at least

2x better SLA) for the business, as the Power Linux systems is much more robust than x86.

• Linux on Power provides an open platform for running open source application and next generation workloads, which aligned to ASE’s IT strategy moving forwards.

• Parts Precision/IBM team delivered a price-performance advantage over x86 with enterprise grade RAS.

Company Overview

Conseil Départemental (formerly Général) des Hauts de Seine (CG92) is the assembly of the french department of Hauts-de-Seine. Haut-de-Seine covers the western inner suburbs of Paris. It is France's second wealthiest département (behind Paris) and one of Europe's richest areas. It's population reaches 1.6 million people.

Client Requirements• Running numerous Oracle databases on x86 servers.• The equipement is under-utilized, increasing the high

licencing costs of the databases environments. • Following the french government recommendations, the

IT department intends to minimise proprietary solutions and develop solutions based on Open Source software.

• Need Lower licensing costs for databases environment• Need to Increase storage performance to support high

performance servers• Highly available architecture

Solution• 2 x E870 servers• 8 IFL for 16 cores and 128 GB memory per server• 2 FlashSystems storage with 2 TB• Open Source Splendid Data PostgreSQL software

Business Results

• Licencing costs will be much lower with Postgres than Oracle (80 oracle licences)

• Infrastructure will be simplified, more manageable, denser

• By using much fewer enterprise-grade servers instead of entry x86 servers, SLA will be much higher

Public Sector, Hauts-de-Seine (Europe)

Financial Services, North America

Company Overview

A multinational financial services corporation in North Americaand accounts for a significant of the credit card transactions inthe US.

Client Requirements

• Customer was undertaking a Next GenerationInfrastructure (NGI) project to move to open sourcebased applications on Linux for their web presence

• Port many of the software packages were not formallysupported on RHEL v7.1

• IBM had to demonstrate a clear price-performanceadvantage over x86 Linux solutions.

Solution

• The solution consisted of (48) Power S824L servers running RHEL v7.1 LE.

• Linux on Power is the basis for the customers Next Generation Infrastructure

• MongoDB, RedisLabs Enterprise Cluster, Nginx, Docker, JBoss, and Node.js

Business Results

• Performs 4x to 6x faster than X86-RHEL platform

• MongoDB’s dynamic Schema adds more velocity for Agile

Application Development

• Cardmember’s online experience can be easily

personalized using MongoDB as backend

• Faster response time fuels faster personalized content

delivery

79883

47771

188907905

29538

6981

16 500

# O

ps

Pe

r S

eco

nd

Concurrent Users

1447

6

2042

3

16 200

# O

ps

Pe

r S

eco

nd

Concurrent Users

IBM P8 X86 (IB) X86 Xeon (SB)

1 core CPU 1 core CPU 4 core CPU

16 GB RAM 4GB RAM 24GB RAM

100% Writes 100% Reads

Company Overview

Ranks among the nation's largest fashion apparel, cosmetics and home furnishings retailers. This Company focuses on delivering style, service and value to its shoppers; complemented by exceptional customer care. Stores offer a broad selection of merchandise and feature products from both national and exclusive brand sources. Operating in nearly 300 locations plus an Internet store.

Client Requirements

This company needed to run an asset management tool (for management of assets/inventory) as well as foundations team (transaction based tool). Their existing platform wasn’t allowing enough of the system to be available and the company was looking for better performance for their critical apps.

Solution• IBM RHEL 7.2 LE + MongoDB + Linux +

Power8 + Sirius• 24 cores, 356G memory• Rapid Build Program allowed for versatility in

solution offered to company (working around supported features and licensing for appealing price/performance), quick set up and convincing POC.

Business Results

• Performance: MongoDB on POWER8

improved performance to be 12X faster

compared to existing Virtualized Intel system

with approximately 10% CPU usage.

• Tasks that previously took >24 hours to

complete are now done in ~1 hour.

• MongoDB on the POC Power8 environment

processed a higher volume of data faster

than the same on their existing

environment.

Major Retail, North America

Implementation was quick, easy and

done in ONLY ONE business day!

“We realized that one of our longest

running queries on MariaDB was almost

10 times faster than the same query

executed on MySQL. And once we built

the system on MariaDB with IBM POWER8

the execution time of the same query

improved from “hours” to “seconds”.

- Paolo Messina, CEO of Foedus Group.

Company Overview

Foedus Group is to companies as the most advanced Technological Partner and Consulting with leading-edge tools and highly innovative solutions, with synergistic collaboration of partners and qualified consultants with whom share values and goals.

Client Requirements• A new web based ERP & Enterprise Management

System• Multiplatform and multi-device• Scalability• Beat the competitiion

Solution

• E870 servers• Open Source Maria DB

Business Results

• Completely open source• Minimal migration costs• Excellent performance on IBM POWER8• Cloud ready• Quality of support

Commercial Services, Foedus Group (North America)


• Introduction






• The Risks and Benefits of Implementing Open Source DatabasesThe Risks and Benefits of Implementing Open Source DatabasesThe Risks and Benefits of Implementing Open Source DatabasesThe Risks and Benefits of Implementing Open Source Databases



• Who to Contact

35% 80% 70+%

Software Infrastructure

Budget on DBMS (1)

In House DBMS Apps

Candidates to Move Off

Expensive DBMS (2)

Cost Savings (3)

(1) Gartner, Forrester and EDB analysis

(2) The State of Open-Source RDBMS, 2015, Gartner, April 2015

(3) The State of Open-Source RDBMS, 2015, Gartner, April 2015; EDB analysis of

customer experiences and market prices

Finding Money in IT

Applications InventoryApplications InventoryApplications InventoryApplications Inventory

Which applications are good candidates for migration and Which applications are good candidates for migration and Which applications are good candidates for migration and Which applications are good candidates for migration and which ones are not and why?which ones are not and why?which ones are not and why?which ones are not and why?

Good Candidates

• Apps of Innovation (Web and Mobile)

• Apps of Differentiation (Custom Apps, ISV Apps)

• Custom Code (Java, C/C++, ProC, PL/SQL)

• High performance transaction based business applications

Unlikely Candidates

• Oracle packages (eBusiness Suite, PeopleSoft, Retek, Siebel)

• SAP

• Most ISV packages unless they support an Open Source database

• Custom Code applications that have made extensive use of special Oracle functions for which there are no analogs in the target Open Source database

• Applications that have currency issues (i.e. very old unsupported versions of compilers, databases, libraries etc.)


• Introduction







• Implementing Open Source Databases on IBM PlatformsImplementing Open Source Databases on IBM PlatformsImplementing Open Source Databases on IBM PlatformsImplementing Open Source Databases on IBM Platforms


• Who to Contact

Open Source Databases on Linux on Power Open Source Databases on Linux on Power Open Source Databases on Linux on Power Open Source Databases on Linux on Power


• Introduction








• Implementation and Migration Services AvailableImplementation and Migration Services AvailableImplementation and Migration Services AvailableImplementation and Migration Services Available

• Summary

Services to help clients migrate to Open Services to help clients migrate to Open Services to help clients migrate to Open Services to help clients migrate to Open Source alternatives like Enterprise DBSource alternatives like Enterprise DBSource alternatives like Enterprise DBSource alternatives like Enterprise DB

ISVs like EDB, MariaDB, Splendid Data and others as well as IBM Systems Lab Services Migration Factory offer a wide range of services to help their clients make the transition to an Open Source database alternative.

• PrePrePrePre----SalesSalesSalesSales

• Client calls, road shows, conferences

• Migration Assessments

• Proof of Concepts

• PostPostPostPost----Sales MigrationsSales MigrationsSales MigrationsSales Migrations

• On-shore migration teams

• Off-shore resources when approved by client

• Migration tools

• Other ServicesOther ServicesOther ServicesOther Services

• Performance Tuning

Migration Factory Offering and CapabilitiesMigration Factory Offering and CapabilitiesMigration Factory Offering and CapabilitiesMigration Factory Offering and Capabilities

We help clients migrate existing Oracle/MySQL databases to open-source alternatives such as EnterpriseDB (EDB) and MariaDB. We also help clients determine the best way to implement open-source NoSQL databases such as MongoDB in their existing database environments.

• Open Source Database Migration AssessmentOpen Source Database Migration AssessmentOpen Source Database Migration AssessmentOpen Source Database Migration AssessmentThe Open Source Database Migration Assessment is the first step in planning a migration to Open Source databases like EnterpriseDB (EDB) and MariaDB, or in determining the best way to implement Open Source NoSQL databases such as MongoDB. We perform a rapid assessment of existing applications and databases and help select suitable candidates for migration.

Migration Factory Offering and Capabilities Migration Factory Offering and Capabilities Migration Factory Offering and Capabilities Migration Factory Offering and Capabilities (Cont.)(Cont.)(Cont.)(Cont.)

• Migration Planning and Solution DesignMigration Planning and Solution DesignMigration Planning and Solution DesignMigration Planning and Solution DesignMigration Factory experts work with client technical personnel to explore database and application details, to better understand dependencies and to uncover potential issues. Migration Factory helps design and build a migration solution and maps out a formal transition plan.

• Migration AssistanceMigration AssistanceMigration AssistanceMigration AssistanceMigration Factory experts are available to work alongside client technical personnel to help deliver key aspects of a migration project. For example, a Proof of Concept demonstrates the process of migrating a database to an Open Source database like EnterpriseDB (EDB) or MariaDB.


• Introduction









• Who to ContactWho to ContactWho to ContactWho to Contact

| 44

Who to ContactWho to ContactWho to ContactWho to Contact

• Via your local IBM Sales Teamor IBM Representative or Business Partner

• For more information on the IBM Migration Factory, please visit us at:

• http://www-03.ibm.com/systems/services/labservices/migrationfactory/index.html

How Open Source Databases Can Help Your Business - IBM · PDF fileHow Open Source Databases...

Documents

Transcript of How Open Source Databases Can Help Your Business - IBM · PDF fileHow Open Source Databases...