Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016
-
Upload
ibm-ims -
Category
Technology
-
view
139 -
download
0
Transcript of Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016
![Page 1: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/1.jpg)
1
Rocket Data Virtualization
Thursday, November 10, 2016
![Page 2: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/2.jpg)
2
What is Data Virtualization?
Enabling data
structures that were
designed
independently to be
leveraged together,
from a single source,
in real time, and
without data
movement
Mainframe
Web/ Mobile
RDBMS
Cloud Data
Big Data
Unstructured
Logical Data Source
© 2015 Rocket Software, Inc. All Rights Reserved.
![Page 3: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/3.jpg)
3© 2014 Rocket Software, Inc. All Rights Reserved.
Data Virtualization Drivers
![Page 4: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/4.jpg)
4
What is Data Virtualization
© 2016 Rocket Software, Inc. All Rights Reserved.
Data Virtualization: a virtualized data
services layer that integrates data from heterogeneous data sources and content in real-time, near-real time, or batch as needed to support a wide range of applications and processes.
Forrester Research – March 2015 - Noel Yuhanna
Data Virtualization
![Page 5: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/5.jpg)
5
Rocket Data Virtualization
© 2016 Rocket Software, Inc. All Rights Reserved..
![Page 6: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/6.jpg)
6
How We Lower Mainframe TCO
GPP zIIP
Eligible Workloads Can Run Outside of GPP within zIIP
� Mainframes have multiple processors• General purpose processor
� all processing counts against capacity
• Specialty Engines� Eligible workloads don’t count against
capacity
� Rocket DV has patented technology that allows it to run 99% of its own processing in the zIIP engine• Enables mainframe data to be
integrated in-place without processing “penalty
![Page 7: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/7.jpg)
7
Why Data Virtualization
Need to accommodate volume,
variety and velocity of data
Mobile driving need for more
real-time, accurate information
Increased adoption of advanced
analytics and self-service discovery
Need for agile data services
with high security
Rules of the Game Have Changed
![Page 8: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/8.jpg)
8
Challenge of Skills and Data Compatibility
� The skills necessary to work with mainframe
data are diminishing
� More programmers today are familiar with
SQL or Java
Mainframe non-relational data structure Transformed into relational format
![Page 9: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/9.jpg)
10
What is Wrong with Status Quo?
“There is not enough time in the
day to move all the data.” “My mobile users expect to see
current data, not yesterday’s data.”
![Page 10: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/10.jpg)
11
Reporting
Ad-hoc
OLAP
Data WarehouseStaging Server
Staging Server
Staging Server
Moving Data Via ETL Tools
Represents ETL
S
Q
L
Data Integration Limitations
Complex, high mainframe costsData inconsistency – High latency
DB2
VSAM
IMS
Adabas
Physical Sequential
CICS
IMS
Natural
IDMS
![Page 11: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/11.jpg)
12
Data Warehouse
Using Connectors for Data Access
Data Integration Limitations
ETL Server
Rigid, difficult to change, expensiveLots of connections – high complexity
DB2
VSAM
IMS
Adabas
Physical Sequential
CICS
IMS
Natural
IDMS
© 2016 Rocket Software, Inc. All Rights Reserved..
![Page 12: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/12.jpg)
13
A Closer Look at Rocket Data Virtualization
![Page 13: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/13.jpg)
14
Mapping DB2
LUW and DB2
for z/OS
Mapping mainframe
non-relational (Adabas,
VSAM) and DB2 for
z/OS data sources
Map Once Use Many
![Page 14: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/14.jpg)
15
Virtualize & Use
© 2016 Rocket Software, Inc. All Rights Reserved..
![Page 15: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/15.jpg)
21
21
![Page 16: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/16.jpg)
22
z13 Exploitation
© 2015 Rocket Software, Inc. All Rights Reserved.
• Takes advantage of SIMD• Single Instruction Multiple Data (SIMD) Accelerator exploitation in
Data Virtualization core engine.
• Takes advantage of SMT2 • Simultaneous Multi-Threading 2 exploitation due to 100% zIIP offload
of DVS
• Heavily exploits zEDC for network I/O • minimum 5x improvement – depending on network topology and
speed) in elapsed time for large result sets • bidirectional exploitation in DVS, we see > 5x times reduction in
elapsed times with 0 z13 GPP cycles consumed.• 100% zIIP offload (2 zIIP per GPP ratio)
• Exploits SMC-R• Load of optimized DV engine based on hardware• for the z13, we use ARCH(11) capability in Metal C 2.1.1 compiler• we built the Rocket DV core engine with Metal C and ship optimized
builds for current z Systems platforms (z196, EC12, z13)
![Page 17: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/17.jpg)
23© 2015 Rocket Software, Inc. All Rights Reserved.
• Exploits AT-TLS, because we use it, we inherit all of the new crypto/encryption advances
• VSAM – new SRB mode support (not ICI based)
• Log Streams – new READ parallelism
• Exploitation of 64 bit storage, Shared Memory Objects and more importantly z Flash Express to take advantage of Pageable Large Pages (reducing DAT code path)
• Flash Express exploitation for reduced Dynamic Address Translation (DAT) overhead. Exploited for all above the bar Private and Shared Memory Objects (buffer pools, Metal C heap, data areas)
• MapReduce (reading different sections of the same dataset in parallel then aggregating the Virtualization engine
• Parallel I/O, only keeping a file open for milliseconds, network and file I/O is done in parallel
• Full intra SQL and intra-partition parallelism
z13 Exploitation
![Page 18: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/18.jpg)
24
Rocket ® Data Virtualization for IBM® z13™ and z13s™ Systems
© 2016 Rocket Software, Inc. All Rights Reserved.
![Page 19: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/19.jpg)
25
Hybrid Cloud Data Services
© 2016 Rocket Software, Inc. All Rights Reserved.
![Page 20: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/20.jpg)
26
Bring Mainframe Data to Spark
© 2014 Rocket Software, Inc. All Rights Reserved.
![Page 21: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/21.jpg)
27
IBM DB2 Analytics Accelerator Loader
for z/OS Enterprise Edition
Accelerator Loader Server
IBM DB2 Analytics
Accelerator
DRDA Sources
(Oracle)
SQL Result Set
BatchDSNUTILB
SourceSQL
Statement
AcceleratorLoader
2 Back
![Page 22: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/22.jpg)
28
RDV Use Case – Transactional Data Access
IBM z/OS Connect allows native applications
to be discovered and invoked via a
RESTful API
![Page 23: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/23.jpg)
29
IBM z/OS Connect
Allows native applications to
be discovered and invoked via
a RESTful API
REST API consumers
Cloud / Bluemix
apps
Mobile apps
Web apps
z/OS ConnectEnterprise Edition
MobileFirstPlatform
CICS
DB2
IMS
MQ
WAS
![Page 24: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/24.jpg)
30
IBM z/OS Connect with Data Virtualization
REST API consumers
Cloud / Bluemix
apps
Mobile apps
Web apps
z/OS ConnectEnterprise Edition
MobileFirstPlatform
CICS
DB2
IMS
MQ
WAS
VSAM
SMF
SYSLOG
Adabas / Natural
IDMS
Cloudant
Hadoop
DB2 LUW
Oracle
SQL Server
…
Allows native applications and
data to be discovered and
invoked via a RESTful API
Data
Apps
![Page 25: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/25.jpg)
Questions
![Page 26: Rocket Data Virtualization - IMS Phoenix UG Nov 10th 2016](https://reader031.fdocuments.us/reader031/viewer/2022021920/5871ecf81a28ab5c348b4b8f/html5/thumbnails/26.jpg)