zVM zLinux InfoSphere Experiences 2011 - Infocura - · PDF filezVM – zLinux –...
-
Upload
dangkhuong -
Category
Documents
-
view
222 -
download
2
Transcript of zVM zLinux InfoSphere Experiences 2011 - Infocura - · PDF filezVM – zLinux –...
date 05/07/2011 - page 1
zVM – zLinux – InfoSphere Experiences 2011
Information Management Solutions with Linux on System z
28 June 2011
Tom Van den Bossche – Hein Vandenabeele
Toyota Motor Europe
date 05/07/2011 - page 2
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Our Conclusions
date 05/07/2011 - page 3
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Our Conclusions
date 05/07/2011 - page 4
Toyota – in the World
• Established in 1937
• 77 manufacturing companies in 27 countries
• Vehicles sold in more than 170 countries worldwide
• Over 8.4 million vehicles sold worldwide in 2010
• Market share: 47% in Japan (2009), 15% in US (2010),
4.4% in Europe (2010)
• Over 3 million cumulative hybrid sales
• €142 billion net revenue in FY 2008-09
• Approx. 318,000 employees worldwide
date 05/07/2011 - page 5
Regional Headquarters
Toyota Motor
Corporation
(TMC)
Toyota Motor Asia
Pacific Pte Ltd.
(TMAP-MS)
Toyota Motor Europe
NV/SA (TME)
Toyota Motor North
America, Inc. (TMA)
date 05/07/2011 - page 6
Toyota – in Europe
• Began selling cars in 1963
• 9 manufacturing plants in 7 countries
• Over €7 billion invested since 1990
• Approx. €5 billion spent with European-based suppliers per year
• 808,320 vehicles sold in 2010
• Over 300,000 hybrid vehicles sold in Europe to date
• 4.4 % market share in 2010
date 05/07/2011 - page 7
• 30 NMSCs
• 56 Countries
• 272 Lexus retailers
• 2,856 Toyota retailers
National Marketing & Sales Companies
Not shown: Toyota Caucasus LLP, Union Motors Ltd (Israel), Toyota Motor Kazakhstan LLP
date 05/07/2011 - page 8
14 Parts Logistics Centres
11 Vehicle Logistics Centres
Logistic Companies
Zeebrugge
Vehicle Centre Toyota Parts
Centre Europe
date 05/07/2011 - page 9
TMIP - Engine Plant
Toyota Caetano Portugal Commercial vehicles
TMUK - Engine Plant
TPCA - Aygo
TMMF - Yaris
TMMP - Engine & Transmission Plant
TMMR - Camry
TMMT - Auris & Verso
TMUK - Auris & Avensis
Manufacturing Facilities
date 05/07/2011 - page 10
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 Including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Our Conclusions
date 05/07/2011 - page 11
TME – Current IT Infrastructure
Mainframe: 2 x z196 UNIX: 320 images (AIX: Power4, Power5, Power7 - Solaris: 40 Sun Blades) Linux: Under VM/Ware on HP Blades and HP Proliant AS/400: 18 Lpars 3 physical machines Windows: 30 VM Ware ESX Servers Hosting 500 virtual machines Lotus Notes, File Sharing, Printing, Web, …
date 05/07/2011 - page 12
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Our Conclusions
date 05/07/2011 - page 13
New z196 Including z/Linux
TME had 2 x z9: - Acquired Jan 2007 - PSLC Pricing Scheme - Not upgradable after June 2011 (IBM decision ) TME has now 2 x z196: - Acquired Jan 2011 - Rationale: 1) Be sure to have enough capacity for the future 2) Switch PSLC to AWLC to save €€€
- Negotiated inclusion of 2x2 IFLs, zVM for future opportunities
date 05/07/2011 - page 14
z/VM - z/Linux – Opportunities
A number of applications rely on heavy TCPIP communication with DB2 on z/OS: - InfoSphere DataStage on Sun Solaris - WebSphere Application Server on Sun and AIX - Cognos BI on Windows Performance Issue: Network configurations and the huge number of communications cause delay in the applications
date 05/07/2011 - page 15
z/VM - z/Linux – Opportunities
Z196
zOS
DB2 V9
High Volume
Latency
Important
High Volume Low Volume
High network traffic is an obstacle for good application response times
Application response time not good
date 05/07/2011 - page 16
z/VM - z/Linux – Opportunities
Solution:
This can be mended by System z HiperSockets:
- Network in a box
- Latency very close to 0 msec which will prove to be
significant for performance
date 05/07/2011 - page 17
z/VM - z/Linux – Opportunities
Z196 TASC
DWH Lpar
zOS
DB2 V9
z/Linux / z/VM
Hip
erS
ockets
Hig
h V
olu
me
Low Volume
Low Volume
High network traffic has been moved to a zero latency network-in-the-box
Application response times a lot better (50% - 80%)
date 05/07/2011 - page 18
Server Consolidation: * Simulation was done by PSSC (IBM Montpellier) based on performance data - for Windows (Cognos) - for Solaris (DataStage, WebSphere App Server) * It shows that in our specific case we can achieve positive consolidation ratios * Ratio depends on a number of parameters
z/VM - z/Linux – Opportunities
date 05/07/2011 - page 19
Server Consolidation: * Advantages Software Licence Cost (PVU Scheme is used) Software Annual Support Costs Number Of Servers CR Space and Cooling System Management Efficiency Networking infrastructure Hardware maintenance DR Strategy …
z/VM - z/Linux – Opportunities
date 05/07/2011 - page 20
z/Linux - PSSC Sizing Study - Capacity
Peak
Total Equivalent
MIPS on z196
Different Physical Servers That Will be consolidated in the simulation
* the graph shows the consolidation of the workload of the intended (physical) machines * goal is to determine how many IFLs are needed to replace the open processors
1 IFL
2 IFLs
date 05/07/2011 - page 21
z/Linux – Typical Sizing Estimate for TME
IBM Software Distributed System z196
Datastage 16 cores (50 PVUs per core)
800 PVUs
2 cores (120 PVUs per IFL)
240 PVUs
WebSphere 40 cores (50 PVUs per core)
2000 PVUs
4 cores (120 PVUs per IFL)
480 PVUs
Cognos 12 cores (50 PVUs per core)
600 PVUs
4 cores (120 PVUs per IFL)
480 PVUs
Additional Software
management tools
68 cores (50 PVUs per core)
3400 PVUs
4 cores (120 PVUs per IFL)
480 PVUs
* z/Linux uses the PVU (Processor Value Unit) pricing scheme * You pay more PVUs for an IFL but if you can consolidate, you might have a €€€ bonus
SW Invoice Benefit because
LESS PVUs
date 05/07/2011 - page 22
z/Linux - PSSC Sizing Study - Results
• 3 x z196 IFL engines are needed to support the consolidation
of 12 distributed servers
Distributed core / IFLs consolidation ratio of around 22
(68/3)
TME decided to install 4 IFL to have a configuration in
parallel
date 05/07/2011 - page 23
z/Linux - PSSC Sizing Study - Results
Impact of the consolidation project:
- Savings on Software Licenses and Software annual support costs
- Savings on Power and Cooling
- Savings on Data Center room
- Savings on Network infrastructure
- Savings on System Management efficiency
- Savings on Hardware maintenance costs
- Improvements on Disaster Recovery plan
- Improvements of overall Quality of service
- Less network bandwidth needed
- …
To be confirmed case
by case for each
environment !
date 05/07/2011 - page 24
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Conclusions
date 05/07/2011 - page 25
z/VM – Installation - Experiences
* : z/VM and z/Linux on IBM System z – The Virtualisation CookBook for SLES11SP1 (SG24-7931-00) is of excellent quality * Order / Download z/VM + Maintenance via ShopzSeries * Write zip files on DVD * Installation using HMC using ‘Integrated 3270 Console’ * z/VM lpar needs to be defined in IOCDS with specific profile (IFLs !) * DASD used was in same range as zOS DASD: not recommended !
date 05/07/2011 - page 26
z/VM – Profile Definition
date 05/07/2011 - page 27
z/VM – Installation - Experiences
* Installation of Vanilla z/VM took 2h15min - 1h00 real effort - 1h15 waiting for copying operations to DASD to complete * IPL z/VM using HMC * No problems at all * Vanilla z/VM (‘Out Of The Box’): - No RACF - No DIRMAINT
date 05/07/2011 - page 28
z/VM – Up and … running !
date 05/07/2011 - page 29
z/VM – Experiences (2)
- z/VM commands need a bit of getting used to
link * cf1 cf1 mr
acc cf1 f copy profile xedit z = = a
- Getting some knowledge before is a plus - Excellent HELP is provided: ‘HELP command’’
date 05/07/2011 - page 30
z/VM – Future
Remaining questions - opportunities: * Getting a better knowledge of the product * Backup * Recovery * Installation of a test z/VM system * Standardisation (naming, …) * Performance and Tuning * Availability * Security: Implement RACF * …
date 05/07/2011 - page 31
z/Linux – RedHat or SUSE
* Only these two are supported on System z * IBM doesn’t care which one you choose. You can run whichever you want. * Differences ? * Packaging * Price * Personal Experience / Preference * YAST: Yet Another Setup Tool (SUSE) * SUSE is mostly used in Belgium TME has chosen SUSE
date 05/07/2011 - page 32
z/Linux – Installation
* : z/VM and z/Linux on IBM System z – The Virtualisation CookBook for SLES11SP1 (SG24-7931-00) is of excellent quality * Download ISO image from SUSE and mount ISO images on NFS server: - Tried (free) PC NFS servers. Didn’t work - Asked AIX colleagues for help with their NAS * Create parameter files for new zLinux machine * IPL from bootstrap files * YAST to configure
date 05/07/2011 - page 33
z/Linux – Installation
Cloner Image
Golden Image
User Image User
Image User Image User
Image
This image will CLONE the
Golden Image
This image is the CLONE
source These images are
the usable images. Creation in 5 mins
(snapshot)
date 05/07/2011 - page 34
Hipersockets
- High speed network within one System z (Internal LAN) - Created by in memory µcode communication - 100% secure: no wires, no sniffing, no routers, no adapters, no functional differences, no encryption needed - Latency = 0 or not measurable (using PING) - Easy implementation
date 05/07/2011 - page 35
Hipersockets - IODF
-> IO Configuration: chpid type IQD
date 05/07/2011 - page 36
Hipersockets - Devices
date 05/07/2011 - page 37
Hipersockets – TCPIP config
date 05/07/2011 - page 38
Hipersockets – VM definition
date 05/07/2011 - page 39
Hipersockets - Linux
date 05/07/2011 - page 40
Hipersockets - example
Database access Round trip (in=32K, out=32K)
1,000
1,500
2,000
2,500
3,000
3,500
4,000
0 20 40 60 80 100
Observation number
Mic
roseco
nd
s
HiperSocket
Wired dev
Wired prod
DWH Lpar
z/OS
DB2 V9
192.0.1.9
z/Linux / z/VM
Hip
erS
ock
ets
Db
2p
ing
(3
2K
) 192.0.1.11
date 05/07/2011 - page 41
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Our Conclusions
date 05/07/2011 - page 42
Datastage at TME
- Graphical E(xtract)T(ransform)L(oad) tool
SOURCE
(DW2P tables +
delta files)
TARGET
(DW2P tables)
EXTRACT
TR
AN
SF
OR
M
LOAD
- Might replace COBOL batch for future ETL projects - Current version 8.0.1 running on Sun Solaris - Processes are delayed due to high number of lookups from Solaris to DB2 on z/OS
date 05/07/2011 - page 43
Datastage on z/Linux – Phase 1
POC testing at PSSC – January 2011 * PSSC provided an infrastructure with - Datastage 8.5 on z/Linux using HiperSockets - DB2 for z/OS V9 as source/target environment
* Conclusions: - Migration (export-import-compile) from Datastage 8.0.1 to 8.5 proves to be easy (very minor application modifications needed) - 1 IFL is not enough (1 process can block entire CPU) – need at least 2 (virtual) – parallelism ! - Enormous performance improvements (> 50% reduction in elapsed times)
date 05/07/2011 - page 44
Datastage on z/Linux – Phase 2
TME implementation – March 2011
* Install Datastage 8.5 on z/Linux using Hipersockets - gcc compiler problems needed to use
SLES10 SP3 (solved in meantime) - testing ongoing – still some memory problems (related to version of datastage)
date 05/07/2011 - page 45
Datastage on z/Linux – TPC-H
TPC-H is a decision support benchmark Sort and load sequential file of 1 GB into DB2 table
TPCH lineitem scale 1GB load (elapsed in hh:mm:ss)
00:00:00
00:07:12
00:14:24
00:21:36
00:28:48
00:36:00
00:43:12
00:50:24
2 4 8 16 32
Nodes
Ela
psed
tim
e
Solaris
Windows
zLinux 1 IFL
zLinux 2 IFL
-80%
Degradation due to overhead
Increased nodes (parallelism)
date 05/07/2011 - page 46
Datastage on z/Linux – TPC-H
the number of nodes (parallellism inside Datastage) on
zLinux is much more scalable then on Solaris, resulting in major throughput advantages
TPCH lineitem scale 1GB load (throughput)
0
10,000
20,000
30,000
40,000
50,000
60,000
2 4 8 16 32
Nodes
# r
ow
s / s
ec Solaris
Windows
zLinux 1 IFL
zLinux 2 IFL
X 10
date 05/07/2011 - page 47
Datastage on z/Linux - TME applications
TME processes (Elapsed)
00:00:00
01:12:00
02:24:00
03:36:00
04:48:00
06:00:00
07:12:00
08:24:00
09:36:00
SDR WAT-1 WAT-2
Testcase
Ela
psed
tim
e
Solaris
zLinux 1 IFL
zLinux 2 IFL
-50%
-50% -80%
Elapsed time (hh:mm:ss) reduction > 50% !
date 05/07/2011 - page 48
Agenda
• Overview of Toyota Motor Europe (TME)
• TME - Current IT Infrastructure
• New z196 including zLinux
• Installation zVM and zLinux
• InfoSphere DataStage
• Our Conclusions
date 05/07/2011 - page 49
Our Conclusions
* HiperSockets dramatically enhances the application response time. This is the main benefit in our case. If your data is elsewhere, then HiperSockets might not be of big use to you
* Possibility to consolidate an important number of ‘open’ processors onto a limited number of IFLs with all associated benefits. We should consider benefits in licensing, management, environmentals, …
* Both z/VM and z/Linux have a (significant) learning curve * Still lots of things to be investigated (sizing, load balancing, DR, security, …)
date 05/07/2011 - page 50
Questions ?
date 05/07/2011 - page 51
Thank You !