INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big...
16
Mike Combs, VP of Marketing 978-996-3580 • [email protected] INTEGRATING Z/OS DATA WITH LINUX AND HADOOP
Transcript of INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big...
Mike Combs, VP of Marketing 978-996-3580 • [email protected]
INTEGRATING Z/OS DATA WITH LINUX AND HADOOP
Big Data and Hadoop § Volume, Velocity, Variety (3 Vs) § Challenge traditional solutions § Enables discovery, decision-making,
and process optimization
§ Discovery § Exploratory, needs short iterations § Traditional reporting: Often an
lengthy IT project that restructures data for periodic reports
2
Hadoop SQL
Scale-Out via parallel compute, I/O
Scale-Up
Key/value pairs (text docs, XML)
Relational tables
Functional programming Declarative queries
Offline Batch (write once, read many)
Online Transactions
MapReduce: Distributes jobs Hive: Access via SQL subset, not transactional HDFS: Distributes files
Increasing Needs for Detailed Analytics § Baselining & Experimenting § Parkland Hospital analyzed records
to find and extend best practices
§ Segmentation § Dannon uses predictive analytics to
adapt to changing tastes in yogurt
§ Data Sharing § US Gov Fraud Prevention shared
data across departments
3
§ Decision-making § Lake George ecosystem project
uses sensor data to protect $1B in tourism
§ New Business Models § Social media, location-based
services, mobile apps
http://www.veristorm.com/content/big-advantages-big-data
The Veristorm Solution 4
§ Mainframes process 60% of commercial transactions and contain 70% of all enterprise data
§ Proprietary data may contain insights unavailable to your competition
§ Logs provide visibility into behavior before and beyond transactions
Apps
Mobile
Things
Web Customers Transactions Proprietary App logs
System logs
Near real-time access to mainframe data provides a 360° view for better insights and decisions.
Long Services Engagements
Technical Challenges
COBOL copy books Compressed data SQL usage Staging build-out Aggregation
Staging
Transform
Load
Hadoop
Security? Governance? Agility? TCO?
System z EAL 5
IMS
VSAM
DB2
5
vStorm Enterprise 6
§ Point-and-click software data copy § No programming required. COBOL
copy books, compressed data handled automatically
§ Target: File system or HDFS/Hive § No staging build-out: 33% storage
savings § No ETL: Avoid MIPS usage charges § Near real-time access § For security and compliance sensitive
data, data can be kept on mainframe
z/OS Linux
Logs
IMS
VSAM
System z Mainframe
DB2
Linux
vStorm Enterprise
zDoop Big Insights, Mongo DB,
Cloudera, Splunk, Pentaho,
Hortonworks, Red Hat
Hadoop
7
8
Authenticate for z/OS source
9
Browse tables and contents on z/OS
10
Authenticate for Linux HDFS destination
11
Select destination for copy
12
13
Copybooks are no problem
14
Direct to HIVE for SQL manipulation or
analytics
Unlock New Insight and Reduce Cost § Do More § Analyze large amount of information
in minutes § Offload batch processes to IFLs to
free up batch window
§ Reduce Cost § Take advantage of IFL pricing and
Linux ecosystem
§ Application extensibility achieved through newly available skillset § Linux, Java, Python, Hadoop § Bring dev teams together
15
System z Workloads
Batch Real-time
Stop by our booth: #106 Ask for a free trial: http://www.veristorm.com/content/free-trial-
vstorm-enterprise
Thank you! 16
