MarkLogic - Managing a New Generation of Data
-
Upload
marklogic -
Category
Data & Analytics
-
view
96 -
download
1
Transcript of MarkLogic - Managing a New Generation of Data
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Managing A New Generation of Data By Diane Burley, Chief Content Strategist, MarkLogic Corporation Finally, an agile data environment for seamless application development
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 2
Are you on offense or defense?
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 4
Data is Growing at a Staggering Rate
44 ZB
8 ZB
2015 2020 Source: IDC
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 5
But Which Database?
NOSQL
NEWSQL
HADOOP
KEY VALUE STORES
DOCUMENT DATABASES
GRAPH DATABASES
IN-MEMORY DATABASES
MAPREDUCE
OBJECT ORIENTED
WIDE COLUMN STORES
SEARCH ENGINES
ANALYTICS
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 6
Enterprise IT Faces Unprecedented Challenge Leveraging Both Heterogeneous and Unstructured Data
OLTP Warehouse
Data Marts
?
Reference Data
Archives
12% Structured 88% Unstructured
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 7
Relational Databases Are Not Designed to Solve This Problem
Inability of Companies to Store, Manage, and Search Their Data
OLTP Warehouse
Data Marts
Unstructured Data
?
Reference Data
Archives
0
10
20
30
40
50
2015 2020Structured Unstructured
Source: IDC
44 ZB
8 ZBs
Explosion of Heterogeneous Data
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 8
The Endless Cycle of Data Normalization
1 2
3 4
Take snapshot of current data Build master data model based on initial view
Extract, transform, & load data into data model
Revise static model & restart process for new data
x
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 9
The Endless Cycle of Data Normalization
1 2
3 4
Take snapshot of current data Build master data model based on initial view
Extract, transform, and load data into data model
Revise static model and restart process for new data
x
2-5 years $5M++
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 11
Any Structure Era “For all your data!” Massive scale Built for heterogeneous
and unstructured data Faster time-to-results Commodity hardware Fraction of the cost
Generational Shift in Database Market
Relational Era “For all your structured data!” Bad for unstructured Difficult for heterogeneous Proprietary hardware Expensive
Hierarchical Era “For your application “data!" Proprietary hardware Expensive
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 12
Gartner Online Transaction Processing RDBMS Magic Quadrant by Betsy Burton and Kevin H. Strange, May 2, 2002
Operational Database Market Static for Over a Decade 2013 2002
Gartner Magic Quadrant for Operational Database Management Systems by Donald Feinberg, Merv Adrian, Nick Heudecker, October 21, 2013
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 13
*Gartner Magic Quadrant for Operational Database Management Systems by Donald Feinberg, Merv Adrian, Nick Heudecker, October 16, 2014
2014: MarkLogic − Only NoSQL Vendor in Leaders Quadrant 2014
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 14
Enterprise Capability: A Corporate IT Requirement
ACID: ATOMIC, CONSISTENT, ISOLATED, DURABLE
Uncompromised Data & Transaction Resiliency
"Don't lose your data!"
SECURITY
Enterprise-grade, Fine-grained Access
"Protect your data!"
HIGH AVAILABILITY DISASTER RECOVERY
Automatic Failover, Replication, Backup/Recovery
"Prepare for the worst!"
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 15
Core Differentiator: Purpose-built for the Enterprise
RELATIONAL OPEN SOURCE
ACID TRANSACTIONS ✔ ✔ ✗
SECURITY ✔ ✔ ✗
HIGH AVAILABILITY & DISASTER RECOVERY ✔ ✔ ✗
SCHEMA-AGNOSTIC ✗ ✔ ✗
SCALE-OUT ✗ ✔ ✔
ELASTIC ✗ ✔ ✗
TIERED STORAGE ✗ ✔ ✗
SEMANTICS ✗ ✔ ✗
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 20
Front Office Systems
Workflow Components
Reporting System
Derivatives Trading Platform MarkLogic Reduces Operational Complexity & Lowers Cost
CHALLENGES The legacy Sybase system was insanely complex with 20 copies to ensure required uptime. Each new bespoke front office system necessitated expensive ETL into this complex system, and all of the reporting applications also had to be woven together. The bank needed to understand the risk in the portfolio and needed a simpler solution. REQUIREMENTS • ACID transactions to ensure no data is lost • Enterprise-level uptime with proven HA/DR • Security that meets the bank's high standards • API's and Application services to support
new reporting requirements and future application needs
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 22
Healthcare.gov Marketplace & Data Services Hub MarkLogic Fast Time-to-Results Meets Deadlines in 12 Months
CHALLENGES Scoped to be built on Oracle, CMS quickly realized that they would not be able to meet the 2013 deadline because schema changes and data modeling would take too long. REQUIREMENTS • ACID Transactions to support
operational workload • Schema flexibility to support
changing data from payers and changing requirements from other Federal agencies
• Security to ensure private data stayed safe
• Massive scalability – 80,000 concurrent users & 170,000 request per minute
Health Insurance Payers Income and Eligibility Confirmation
State Exchanges Healthcare.gov
HIM Data Services Hub
DSH
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 24
Unified Government Counter-Terrorism System MarkLogic Unified Platform Supports Explosive Data Growth
CHALLENGES As intelligence data poured into various databases, they were unable to move data to various disparate systems because ETL was too costly and took too long. Unified search was necessary to save lives. REQUIREMENTS • ACID transactions to support updates to
intelligence • Schema flexibility to quickly load any and all
data • Fine grained security to ensure appropriate
access controls • Massive scalability to support an enormous
and growing system • Alerts and ability to automate workflows as
data is added
Applications for Data Analysts
XLS
Energy Life Sciences Healthcare
Transportation Retail Retail
Manufacturing Law
Enforcement Media &
Entertainment
Education
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 26
Making the Mental & Organizational Shift
Are you stuck in the cycle of data modeling and ETL while the applications and data changes faster than you can build?
Are you developing and iterating with agility to continuously add value to your organization by using all the data that's available?
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 27
Any Structure Era “For all your data!” Massive scale Built for heterogeneous
and unstructured data Faster time-to-results Commodity hardware Fraction of the cost
Generational Shift in Database Market
Relational Era “For all your structured data!” Bad for unstructured Difficult for heterogeneous Proprietary hardware Expensive
Hierarchical Era “For your application “data!" Proprietary hardware Expensive
Mainframe
Relational / SQL
NoSQL