Powering Self Service Business Intelligence with Hadoop and Data Virtualization
-
Upload
denodo -
Category
Data & Analytics
-
view
186 -
download
1
Transcript of Powering Self Service Business Intelligence with Hadoop and Data Virtualization
Powering Self Service Business Intelligence with Data VirtualizationSean Roberts, HortonworksMark Pritchard, DenodoJanuary 2017
2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Powering the Future of DataSean RobertsPartner Engineering
3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Data Doubles Every Two Years
44ZB By 2020
3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
The Old WaySystem Centric
Procedural
Hierarchical
Scheduled
Homogeneous
The New WayUser Centric
Agile
Dynamic
Real-Time
Contextual
4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Data Doubles Every Two Years
44ZB By 2020
5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Capture streaming data
Deliverperishable insights
Combinenew & old data
Store data forever
Access a multi-tenant data lake
Modelwith more data
DATA AT RESTDATA IN MOTION
ACTIONABLEINTELLIGENCE
Perishable Insights Historical Insights
6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Secure
Real-time
Streaming
Integrated
Hortonworks DataFlow for Data in MotionPowered by Apache NiFi, Kafka, and Storm
7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Hortonworks Data Platform for Data at RestPowered by Open Enterprise Hadoop
Open
Interoperable
Ready
Central
8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Claims Optimization
SocialSentiment
Cohort Selection Bill ShockPhysician
NotesDevice
Monitoring
R & DQuality Benchmarks
PatientExperience
Seasonal Staffing
Net Promoter
Score
Supply Chain
SentimentAnalysis
PatientOutreach
360°PatientView
Patient Throughput
Customer Churn
Analysis
STARS Ratings
Genomics
Remote Monitoring
Drug Diversion
CensusProactive Maintenance
PreventativeMedicine
Inventory
MedicationSafety
OPEXReduction
Lab Notes Archive
MainframeOffloads
Device Data
Ingest
Rapid Reporting
DigitalProtection
Dataas a
Service
FraudPrevention
Real-time Decision Support
I N N OVAT E
R E N OVAT E
E X P L O R E O P T I M I Z E T R A N S F O R M
A C T I V EA R C H I V E
E T LO N B O A R D
DATAE N R I C H M E N T
DATAD I S C OV E RY
S I N G L EV I E W
P R E D I C T I V EA N A LY T I C S
H EA LT H C A R E
Care-path Best Practices
OR Optimization
HCAHPSScores
Staffing Predictions
Proactive Outreach
Legacy System
DataImaging Archive
Historical PatientRecords
Improved Drug Yields
9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Actionable Intelligence Makes Healthcare Precise and Personal
Patient Records
Lab Data
Pharmacy Data
Patient Locations
Wearables
Intra-Network Data
Sensor Data
Claims Data
Social Media Physician
NotesPatient
Satisfaction Data
Clinical (EMR) Data
SINGLE VIEW OF PATIENT
REAL-TIME VITAL S IGN MONITORING
BILLING & REIMBURSEMENTS
EMR OPTIMIZATION
SUPPLY CHAIN OPTIMIZATION
10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Mercy’s Journey
Mercy Medical System Sought a Data Lake for a Single View of its Patients – “One Patient, One Record”
Existing platform impeded goal of enriching Epic data for 1 million patients over 35 Hospitals and 500 clinicsMoving Epic EMR data to Clarity EDW took 24 hours and was “never goingto enable real-time analytics”. Now that takes 3-5 minutes with HDP.Improved billing processes resulted in $1M additional annual revenuefrom newly documented secondary diagnoses and care
11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Better Health Through “One Patient, One Record”
ACTIVE ARCHIVELab Notes
DATADISCOVERY
OPEX Efficiency
SINGLE VIEWBilling
DATA DISCOVERYVital Signs
SINGLE VIEWSingle Patient
Record
ACTIVE ARCHIVEEpic EMR
Replication
ACTIVEARCHIVEPrivacy
Database
DATA ENRICHMENT
Epic Enrichment
PREDICTIVE ANALYTICS
Device Data Ingest
Move to Clarity wouldn’t enable real-time analytics
Existing platform impeded goals
Data enrichment needed for 1 million patients
Move off Epic took over 24 hours
S I T UAT I O N
3-5 Minutes
$1M Additional Annual Revenue
From “Never”to “Seconds”
900x Faster
move data off Epic to Clarity with HDP
from improvedbilling process
acceleratedresearcher insight
ingest of ICU vital signs
P R E D I C T I V EA N A LY T I C S
Preventive Care
ETL OFFLOADMedical Decision
Support
12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Interoperable with Leading Technologies
Partners
13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Information spread across different systems
IT responds with point-to-point data integration
Takes too long to get answers to business users
The Self-Service Challenge Data Is Siloed Across Disparate Systems
MarketingSales ExecutiveSupport
Database
AppsWarehouse Cloud
Big Data
Documents AppsNo SQL“Data bottlenecks create business bottlenecks.”
– Create a Road Map For A Real-time, Agile, Self-Service Data Platform, Forrester Research, Dec 16, 2015
14 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Denodo and Modern Data Architecture
Powering Self Service Business Intelligence with Data VirtualizationMark PritchardJanuary 2017
Agenda1.Data Virtualization and Benefits
2.Case in Point – Vizient
3.Modern Data Architecture
4.Self-Service BI on Distributed Datasets
5.Denodo Platform for Data Virtualization
6.Q&A
Data Virtualization and Benefits
18
-Source: “Gartner Market Guide for data virtualization – 2016”
Data virtualization technology can be used to create virtualized and integrated views of data in memory (rather than executing data movement and physically storing integrated views in a target data structure), and provides a layer of abstraction above the physical implementation of data.”
Data Virtualization – Definition
19
Data VirtualizationReal-time Data Integration
“Data virtualization integrates disparate data sources in real time or near-real time to meet demands for analytics and transactional data.”
– Create a Road Map For A Real-time, Agile, Self-Service Data Platform, Forrester Research, Dec 16, 2015
Publishes the data to applications
Combines related data into views
Connects to disparate data sources
2
3
1
20
Denodo and Modern Data Architecture
21
Benefits of Data Virtualization“Get it Real-time and Get it Fast!”
Better Data IntegrationLower integration costs by 80%.
Flexibility to change.
Real-time (on-demand) data services.
Complete InformationFocus on business information needs.
Include web / cloud, big data, unstructured, streaming.
Bigger volumes, richer/easier access to data.
Better Business OutcomeProjects in 4-6 weeks.
ROI in <6 months.
Adds new IT and business capabilities
“Benefits of Data Virtualization: get it real-time and get it fast!” – William McKnight, President, McKnight Consulting Group
Case in Point - Vizient
23
Who is Vizient?Network for non-profit hospitals and alliance of academic medical centers
Network of not-for-profit healthcare organizations to
improve performance and efficiency in
clinical, financial and operational
management
Combination of VHA, University
HealthSystem Consortium, Novation,
MedAssets Spend, Clinical Resources
Management and SG2
Experts with purchasing power,
insights and connections that
accelerate performance for
members
24
Purpose, Mission and Strategic Aspirations
Mission To connect members with the
knowledge, solutions and expertise that accelerate performance
Strategic Aspirations To become an indispensable partner to healthcare organizations
Purpose To ensure
members deliver exceptional, cost effective care
25
Vizient delivers brilliant, data-driven resources and insights — from benchmarking and predictive analytics to cost-savings — to where they’re needed most.
Empowering Brilliant Connections
Modern Data Architecture
27
Modern Data Architecture
HTTP
FTP
HL7
Flat File
Share
Subscribe
Attach
Initial
Event
Share
Persist
Event
Share
Ongoing
Event
Usage
Purchase Data
OpenData
RAW CLEAN AGGREGATED ENRICHED
RDBMS
RDBMS
ODS
DW
STAGESOURCE PROCESS PERSIST SERVE
Rules
Batch
Human Process
Hadoop
Hadoop
Lake
Machine Learning
Analytics
Consulting
28
Modern Data Architecture
HTTP
FTP
HL7
Flat File
Share
Subscribe
Attach
Initial
Event
Share
Persist
Event
Share
Purchase Data
OpenData
RAW CLEAN AGGREGATED ENRICHED
RDBMS
RDBMS
ODS
DW
STAGESOURCE PROCESS PERSIST
Ongoing
Event
Usage
SERVE
Rules
Batch
Human Process
Hadoop
Hadoop
Lake
Machine Learning
Analytics
ConsultingData
Virt
ualiz
atio
n
Powering Self-Service Discovery with Data Virtualization
30
Financial Data MartPrimary Use Caseo Unify disparate accounting and finance data marts across various legacy
organizations into a shared repository
Secondary Use Cases o Provide a unified source for key BI initiatives like the GPO Dashboardo Support reporting needs as legacy systems are migrated or replaced during
integration of Vizient and L-MDAS (dbVision, etc.)o Provide a final resting place for archived legacy sources like Solomon, Epicor,
etc. VHA
MedAssets
UHC
Vizient
31
Financial Data MartArchitectural Approach
Denodo was selected as the data platform in order to utilize the following features of the software:
o Data Virtualization allows sources in various mediums and locations to be integrated without physically moving the data
o Data Abstraction allows data to be represented consistently within the DataMart while data sources are moved or replaced behind the scenes
o Data Integration allows for a single seamless view to be created across a subject area (e.g. “Supplier Sales”) with varied data transformation rules for each data source within the subject area (PRS, dbVision).
32
GPO DashboardPrimary Use Case
o Provide a consolidated view of supplier sales data across all customers of legacy Vizient & Med Assets organizations.
Architectural Approach
o Financial DataMart (on Denodo) for data source o Denodo TDE Exporter Tool for daily data extracts to Tableau:
Report Data Report User Security
o Tableau for report development and distribution
33
GPO DashboardKey Challenges
o Balance between data timeliness and report performance Tableau reports performed best utilizing the TDE format (cached/extracted dataset) as
opposed to a live connection This meant that the report caches required daily refreshes, and data extraction had to be
appropriately tuned Denodo features such as dataset statistics and indexing greatly contributed to this
performance tuning
o Provisioning user security at cell level The requirement for some internal report users to be restricted to the
members/customers to which they are assigned meant that a new report security approach was needed
Reliance on TDEs for report data necessitated the integration of security in the reporting layer
Tableau’s “data blending” feature allows user security to be specified within a separate dataset
This also supports reuse of the security view in other reporting environments.
34
Contract Sales Actualizer DashboardPrimary Use Caseo Integrate Member Spend and Supplier Sales data from all Vizient organizations to identify
opportunities for increasing contract utilization
Other Use Caseso Maintain consistency (Single Source Of Truth) with GPO dashboard regarding:
Supplier Sales Data Dimension Data User Security
Architectural Approach o Data source utilizes Denodo to reuse overlapping datasets (sales, dimensions,
security) while allowing separate virtualized views to be created for new datasets (member spend) which can be also be reused by future projects
o Reporting components match approach used by GPO Dashboard
35
Contract Sales Actualizer DashboardKey Challenges
o Successful integration of Exadata RDM as a data source for Denodo. Approach utilizes the strength of Exadata RDBMS for aggregating large
quantities of data quickly Denodo to integrate the data with similar legacy SQL Server data sources to
create a comprehensive view of Vizient member spend
o Scalability/Configuration Management Advances were made to support parallel development of this project and
continued efforts on GPO dashboard Compartmentalization features within Denodo allow for code changes in each
project to be version controlled and assessed for dependencies Process guidelines are being authored to allow for multiple development efforts
on the same datasets
Denodo Platform for Data Virtualization
37
Accelerate Your Fast Data Strategy with Denodo Platform 6.0New Release of Denodo Platform Delivers Breakthrough Performance, Accelerates Adoption, and Expedites Business Use of Data
Breakthrough PerformanceDynamic Query Optimizer delivers breakthrough performance for big data, logical data warehouse, and operational scenarios.
Data Virtualization In the Cloud
Denodo Platform for AWS accelerates adoption of data virtualization.
Self-service Data Discovery and SearchSelf-service data discovery and search expedites use of data by business users.
“Very happy with Denodo version 6. Well done!” – Claudia Imhoff, President, Intelligent Solutions
38
Common Data Virtualization Use CasesData Virtualization
BIG DATA, CLOUD INTEGRATION
Advanced Analytics Data Warehouse Offloading Big Data for Enterprise Cloud / SaaS Integration
AGILE BUSINESS INTELLIGENCE
Logical Data Warehouse Virtual Data Marts Self-Service BI Operational BI / Analytics
SINGLE VIEW APPLICATIONS
Single Customer View - Call Centers, Portals Single Product View - Catalogs Single Inventory View - Inventory Reconciliation Vertical Specific - Single View of Wells
DATA SERVICES
Unified Data Services Layer Logical Data Abstraction Agile Application Development Linked Data Services
39
Why Denodo Data Virtualization?Data Virtualization
Exceptional pre-sales service and customer support
Technical expertise and response time rated best by costumers.
Enhancement requests typically made available in days / weeks.
Lower overall TCO and ROI Address more use cases with a single platform. Aggressive “all-in-one” pricing for entire platform;
flexible pricing models.
Broad functionality and winning innovation in a fully integrated platform
Flexible and broad use cases. Integrated purpose-built for data virtualization – very
easy to develop / deploy / change models.
Q&A
Thanks!
www.denodo.com [email protected]© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.