Lessons Learned With Oracle9i on Linux for S/390 Denny Dutcavich [email protected].
-
Upload
baldric-newton -
Category
Documents
-
view
216 -
download
0
Transcript of Lessons Learned With Oracle9i on Linux for S/390 Denny Dutcavich [email protected].
• Introduction– Objectives
– Overview
• Lessons learned– CPU
– Memory
– Paging and Swap Space
– VM and Linux Guest Setup
– Horizontal and Vertical Scaling
– Async I/O
– Storage (disk) considerations
– Backup Options
– Measurement Tools
– Oracle
• Summary of Key Factors
• New Workshop
Agenda
Introduction
• The objective of this presentation is to highlight our experiences with monitoring and performance tuning gained from:– The development team at Oracle in preparing for GA – Early adopter customers– Testing done at Oracle and IBM Boeblingen
• These topics are covered in the following Redbooks– Experiences with Oracle9i for Linux for S/390 SG24-6552 – Linux on IBM eServer zSeries Performance Measurement
and Tuning SG24-6926
CPU
• Plan for sufficient CPU resource– How many Linux guests can run on an IFL?
• Options– Linux guest under z/VM
– Linux native LPAR, IFL or general CPUs • CPU Allocation
– Virtual CPUs per constrained guests should equal Real CPUs
– Never have Virtual CPUs per guest greater than Real CPUs
– Use QUICKDISP for constrained quests
• Speed Matters – Similar clock speeds will exhibit similar performance
• Insure proper expectations when consolidating servers– Spectrum of performance runs from CPU intensive to I/O
intensive
Operational Choices
z/VM 4.3
Linux guests
MVS01MVS01 MVS03 MVS04
z/VM 4.3
MVS02 Linux guests
LPARs
IFLs
Memory
• Plan size of the Linux Guest– Linux uses all memory available to it
• Allow only for Oracle (or application) and Linux
– Size the Oracle System Global Area (memory footprint)
• 31-bit Oracle has maximum 1.5GB SGA for both 31bit and 64bit Linux on zSeries
– Need genksms from 9.2.0.2
• Don’t expect to move a very large SGA into a 256MB Virtual Machine
– Linux may appear hung if it detects memory shortage
• KILLMEM process shuts down processes to free memory
– Paging will occur if memory is over-committed • not a good thing
Paging and Swap Space
• VM paging space– Configure expanded storage for VM to use as first
paging device• Configure 25% of physical memory for
expanded storage• Alter as performance needs are better
understood• Separate paging space from guest data
– Monitor VM paging• VM likes contiguous space on paging DASD
Keep paging space at apprx < 50%
Paging and Swap Space
• Linux Swap– Linux swaps pages when it needs
memory• Could be double paging
– Configure VDISKS• Swap space on VDISKs not utilized until
Linux pages
• Linux completes P.O.– Moves page to
swap device
• Page B paged out by z/VM– Available for P.I.– Linux clueless
• Page B paged in by z/VM– Linux needs to page– Page fault accessing Page
B
Double Paging
DCBA
DCBA
.
.
.
z/VM Expanded Storage
Linux Pagingz/VM Main Memory
Linux Guest
B
B
1 2
3
1
2
3
VM and Linux Guest Setup
• Decisions - z/VM or standalone in an LPAR– Value proposition is to consolidate many servers using the
virtualization capabilities of z/VM
– Linux can be run standalone in an LPAR • Nothing is free - z/VM has its overhead
– VM overhead 0 - 12% on our tests– Minidisk I/O uses more VM processing
Minidisk caching may improve throughput on data re-
use • Think through all the options
VM and Linux Guest Setup
• DASD - sample setup201 root 200-600 CYL
202 swap VDISK (2GB)
203 swap VDISK (2GB)
204 home
205 tmp 200 – 1000 CYL
206 Usr 2700-3000 CYL
Linux Guest Priorities
• Idle Linux guest machine takes resources– Logoff VM after shutting down Linux– Timer patch
• Popup blocker for Linux• Stops Linux from waking up to check for work
– Remove unneeded cron tasks– Stop unnecessary daemons
• LPAR Weights determine CPU for guests in that LPAR
• Great tuning tips at:– http://www.vm.ibm.com/perf/tips/#Tune– http://www.vm.ibm.com/perf/tips/linuxper.html
Scheduling
Dormant ListDormant List
Logoff Logon
Work To DoIdle
Eligible ListDispatch List
Time Slice Expires
Resource Availableor E0
Resource Limit Exceeded
VM Machine ready
VM Machine waiting
VM DefinitionBlock E1
VM DefinitionBlock En
VM DefinitionBlock D1
VM DefinitionBlock Dn
Scaling
• Horizontal scaling– Add new instances of Linux – a Penguin
colony– Have sufficient resource
• Vertical scaling– Increase LPAR weight – Increase number of real CPUs or IFLs– Don’t forget to retune Oracle
Async I/O
• Oracle 9i 31bit for Linux for S/390 was built without support for Asynchronous I/O
• You must put DBWR_IO_SLAVES=4 in init.ora– Number could be higher than 4 for very
intensive I/O workloads
• Should not be an issue in future releases of Oracle
Storage (Disk) Considerations
– ESS 800 should not be treated as a black box
• Layout of files for database can have huge effect on performance
• Our tests showed 3x to 4x improvement in throughput
Storage Configuration
• Best performance will be achieved using:– multiple host adapters, – both caches, – more channels, – more CHPIDs and more paths
• Spread data across multiple arrays (ranks)• FICON Express is better than ESCON
Creating LVMs
• Use Logical Volume Manager to create file systems for best performance
• Striping critical to get more than one channel used for the data transfers – Performance gated by number of CHIPIDs
• File systems in TBytes can be created– Change extent size– Default value is 256M
• RedHat 7.2 does not have LVM – use software raid
Backup
• Backup SW used to manage backup sets created with Oracle RMANFDR UpstreamTivioliCA
• PPRC can be used for backup (ESS remote mirroring) or Oracle Advanced Replication feature
Monitoring Performance
• VM has the only accurate view of what resources each guest is consuming, so any measurements need to start there.
• Goal of VM is to provide each guest with the illusion that it's got an entire machine all to itself.
• A result is that no guest can possibly tell what real resources it is using, unless it asks VM for the information.
Performance Monitoring - Tools
• Used VM measurement tools such as– CP Indicate command (IND)
• Ind user vmlinux2 exp– CP Monitor facility– VM Real Time Monitor (RTM)
• Used Linux facilities such as– VMSTAT– NETSTAT– IOSTAT and SAR (not on SuSE)– TOP
• Total Storage Expert for ESS• CA Unicenter• Velocity Software (ESALPS)
Performance Monitoring – CP Cmds
• INDICATE - Broad overview of how system resources are being used
• LOCK - Lock in real storage selected pages
• SET SHARE - Control percentage of system resources a guest receives
• SET QUICKDSP - Designate guests that don't wait in the eligible list
• SET RESERVED - Set number of pages resident in real storage
• DEDICATE - Allocate a processor to a guest
Oracle is Oracle is Oracle
• Most Oracle performance problems are application issues– Remember to use StatsPack
• Oracle looks the same on Linux for S/390 as it does on other platforms
• Check init.ora parameter and SGA size
Key Success Factors
• Memory for Linux guest – less is better or Linux spends it’s time managing memory
• Swap space and expanded storage are mandatory
• I/O bottlenecks can be avoided byUsing multiple arrays in ESSLVM with striping
Oracle Database 10g Workshop
A new off ering - Dennis Wunder f rom the Poughkeepsie eServer Briefi ng Center has scheduled 2 newworkshops in Poughkeepsie, New York. This workshop is intended to help customers gain an understandingof how to install and manage an Oracle 10g database on Linux on zSeries and z/ VM, by providing the basicskills necessary to launch a Linux guest with Oracle on zSeries.
The workshop is designed with minimal lecture and maximum hands-on, practicing a learning by doingmethodology. Experimentation is encouraged.
Attendees will learn: Overview of VM and Linux Overview of Oracle on Linux on zSeries I nstalling a Linux guest I nstalling Oracle 10g Binaries
****
Measuring the Oracle workloadEtc
The Workshop will be held at the eServer Executive Briefi ng Center located in Poughkeepsie, N.Y.. The dates selected for the Workshop are as f ollows:- J une 22 -24-August 10 - 12
Acknowledgements
• Thanks to the following contributors– Oracle – Barry Perkins, Andres Lubomirsky, Thomas
Niewel– IBM BOE- Martin Kammerer, Juergen Doelle– IBM Storage – Bill Worthington, Pat Blaney, Sandy Abu– IBM/Oracle team- Bruce Frank, Tom Russell, Kathryn
Arrell, Denny Dutcavich