Austin Java Users Group developerWorks article – µActor Library BARRY FEIGENBAUM, PH. D. 02/26/13.
1 Daniel Martin November 19 th 2013 [email protected] Find me on developerWorks.
-
Upload
sydni-laning -
Category
Documents
-
view
215 -
download
1
Transcript of 1 Daniel Martin November 19 th 2013 [email protected] Find me on developerWorks.
1
Daniel MartinNovember 19th [email protected] Find me on developerWorks
© 2013 IBM Corporation
• Porting from Intel x86 to Linux on Power - Thoughts
• IBM POWER Linux Tools Repository
• IBM POWER Linux Tools Repository – Install
• IBM POWER Linux Performance Recommendations
• IBM POWER Linux Java Install and Performance Recommendations.
• Redhat on IBM POWER – Tips
• Obtaining examples - SPEC.org
• Leveraging IBM – developerWorks, VLP, IIC, Webinars
2
© 2013 IBM Corporation
IBM POWER7 and POWER7+ recommended distros -
The following distributions are optimised for POWER7 and POWER7+ processor-based
servers:
SUSE Linux Enterprise Server 11 SP1, any subsequent service packs. SLES 11
SP2 is the latest service pack.
Red Hat Enterprise Linux 6, and subsequent updates. RHEL 6.4 is the latest
update.
PowerLinux Tools –
There are a number of related tools that you should install to get the very best out of your
Linux on Power systems. These are: IBM Linux hardware diagnostic aids and productivity,
IBM Software Development Kit and the IBM Advance Toolchain.
3
© 2013 IBM Corporation
The IBM POWER Linux Tools Repository enables the use of standard Linux package management tools to provide access to the following:
• IBM POWER Linux hardware diagnostic aids and productivity tools
• IBM Software Development Toolkit for POWER Linux servers
• IBM Advance Toolchain for POWER Linux servers
The Tools repository supports the following versions of Linux:
• Red Hat Enterprise Linux 5 (RHEL5)
• Red Hat Enterprise Linux 6 (RHEL6)
• SuSE Linux Enterprise Server 10 (SLES10)
• SuSE Linux Enterprise Server 11 (SLES11)
5
© 2013 IBM Corporation
Installing the IBM POWER Linux hardware diagnostic aids and productivity tools.
Create your repo information and then update the new repo:
# vi /etc/yum.repos.d/ibm-power.repo
[ibm-power-repo]
name=IBM POWER Tool
baseurl=http://public.dhe.ibm.com/software/server/POWER/Linux/yum/IBM/RHEL/6/
ppc64
enabled=1
gpgcheck=0
# yum update
Once the updates are downloaded, create the related libraries:
# yum install ibm-power-repo.ppc
Confirm what repos are now available:
# yum repolist enabled
6
© 2013 IBM Corporation
Update the installed packages to the latest version available, enter:
# yum update
Then choose one of the following to install the recommended packages.
For an HMC or IVM-managed system (in most cases), enter:
# yum install ibm-power-managed-rhel6
OR
For a standalone system, enter:
# yum install ibm-power-nonmanaged-rhel6
I recommend installing these additional packages:Performance Management # yum install ibmPMLinuxNMON– Performance monitoring # yum install nmonPOWER Energy Management # yum install pseries-energyNUMA policy for processes or shared memory # yum install numa*
7
© 2013 IBM Corporation
Environmental Settings –
These are a number of recommended environmental settings based on a system with 16 POWER7+
cores and 128GB RAM.
fs.file-max = 4878674
This relates to the maximum number of open files/file descriptors:
# cat /proc/sys/fs/file-max
201561
This will set the limit to be applied to the system after the next reboot:
vi /etc/sysctl.conf
Adding the line:
fs.file-max = 4878674
Run the following to re-read the sysctl.conf file and apply it to the current system:
# sysctl -p
9
© 2013 IBM Corporation
Setting the maximum number of open file descriptors.
Displaying your current limits:
# ulimit -n
1024
Changing those limits on your current session:
# ulimit -n 1048576
Preserving the change on system reboot:
# vi /etc/security/limits.conf
Add the lines:* hard nofile 1048576* soft nofile 786432
Then repeat the same principles.
This relates to the maximum number of processes available to a single user:
# ulimit -u unlimited
This relates to the maximum stack size:
# ulimit -s 524288
10
© 2013 IBM Corporation
The previous slides cover a few environmental settings, the following are some which relate to the CPU and its features which may help improve performance.
Disabling ‘preempt scheduler’:
# mount -t debugfs debugfs /sys/kernel/debug
# echo NO_WAKEUP_PREEMPT > /sys/kernel/debug/sched_features
# umount /sys/kernel/debug
It can be re-enabled via:
# echo WAKEUP_PREEMPTION > /debug/sched_features
11
© 2013 IBM Corporation
Disable Idle Power Saver - Idle Power Saver can be disabled by accessing your browser and utilising the Advanced System Management Interface (ASMI) console.
From the HMC, navigate to System Configuration -> Power Management Mode Setup -> Disable Power Save Mode. Set the Idle Power Saver value to Disabled, then click on the "Save settings" button at the bottom of the page.
12
© 2013 IBM Corporation
Disable IPv6
IPv6 support can be disabled in the Linux kernel by adding the following options to the command line in the boot loader configuration:
ipv6.disable_ipv6=1 ipv6.disable=1
Disabling IPv6 support in the Linux kernel guarantees that no IPv6 code will ever be run while the system is booted. Always remember that if you run this command then you cannot enable IPv6 until the system is rebooted. A lighter touch is to let the kernel boot with IPv6 support and then use the sysctl facility to dynamically set a kernel variable to disable IPv6:
sysctl -w net.ipv6.conf.all.disable_ipv6=1
The example above disables IPv6 on all interfaces. You can also opt to just disable IPv6 support on specific interfaces:
sysctl -w net.ipv6.conf.eth0.disable_ipv6=1sysctl -w net.ipv6.conf.eth1.disable_ipv6=1
If your workload only uses IPv4 (in order to allow you to disable IPv6 support in the JVM) add the flag:
-Xaggressive
There is no reason for the JVM to spend any time trying to use IPv6.
13
© 2013 IBM Corporation
Red Hat Java install - Firstly here are a couple of binaries that you will need to ensure that the Java ‘installanywhere’ script works:
# yum install libstdc++.ppc64# yum install rpm-build Then you can download the related package (in our case it’s Java7) into the install directory (/usr). Once that is complete, run the *.bin file:
Java 7# ./ibm-java-sdk-7.0-4.0-ppc64-archive.bin
Java 6# ./ibm-java-sdk-6.0-13.0-linux-ppc64.bin
This should enable you to simply follow the prompts from the install script, and as long as you have sufficient space all should configure correctly.
Check the version as follows:
# /usr/<install-path>/bin/java -version
15
© 2013 IBM Corporation
Further Recommendations:
Turn off Hardware data prefetch with the Linux command "ppc64_cpu --dscr=1“.
The default page size is 64 KB page size, this will normally provide good performance gains over 4 KB pages. For further potential performance gains, enable 16 MB page size -
# cat /proc/meminfo
Predominantly the best performance is from 32-bit Java, that is unless the memory requirement of the application requires you to be running in 64-bit mode. Make sure libgcc.ppc and glibc.ppc are installed otherwise the 32bit Java will not include the correct libraries.
/root/.bashrc - set the path in your profile to Java 32 or 64 bit.
Most applications benefit from SMT. However, some applications do not scale with an increased number of logical CPUs on an SMT enabled system. One way to address such an application scalability issue is to change to a lower SMT mode with fewer logical CPUs; for example, changing from SMT4 to SMT2. Details of SMT will follow.
16
© 2013 IBM Corporation
The server is now installed, below are a few commands to assist you in tailoring your environment.
POWER SMT on Red Hat.
How does Linux handle SMT modes, as IBM POWER7 processors support Simultaneous Multi-
Threading (SMT)?
To toggle the CPUs offline/online consistently across the cores:
# ppc64_cpu --smt=x
Linux numbers all of the CPUs sequentially across the cores:
# ppc64_cpu --smt=1
This will "turn off" cpu1,cpu2, cpu3 of the four CPUs associated with each core, leaving cpu0 running.
0,1,2,3 is the example for the first core. This repeats throughout the CPUs:
# ppc64_cpu --smt=2
This ensures that cpu0, cpu1 are "on", and cpu2, cpu3 are "off" for each core:
# ppc64_cpu --smt=4
This ensures that all cpus are "on“
With SMT=1 mode (aka SMT off), the Linux scheduler should shift to the appropriate scheduling mode.
18
© 2013 IBM Corporation
Useful Commands.
If you want to get virtual machine configuration data, then the ’lparcfg’ file contains information relating to
the IBM Power Virtual Machine:[root~]# cat /proc/ppc64/lparcfg
19
The –i flag lists details of the LPAR configuration.
[root ~]# lparstat -iNode Name : server1.iic.hur.cdnPartition Name : server1Partition Number : 6Type : SharedMode : UncappedEntitled Capacity : 0.40Partition Group-ID : 32774Shared Pool ID : 0Online Virtual CPUs : 1Maximum Virtual CPUs : 4Minimum Virtual CPUs : 1Online Memory : 2051328 kBMinimum Memory : 1024Desired Variable Capacity Weight : 128Minimum Capacity : 0.10Maximum Capacity : 4.0Capacity Increment : 0.1Active Physical CPUs in system : 32Active CPUs in Pool : 32Maximum Capacity of Pool : 32.0Entitled Capacity of Pool : 290Unallocated Processor Capacity : 0Physical CPU Percentage : 40Unallocated Weight : 0Memory Mode : SharedTotal I/O Memory Entitlement : 2147483648Variable Memory Capacity Weight : 0Memory Pool ID : 65535Unallocated Variable Memory Capacity Weight : 0Unallocated I/O Memory Entitlement : 0Memory Group ID of LPAR : 32774Desired Variable Capacity Weight : 128
The interactive command version of this is ‘lparstat’ [root~]# # lparstat
System Configuration
type=Shared mode=Uncapped smt=Dedicated lcpu=1
mem=2051328 kB cpus=32 ent=0.40
%user %sys %wait %idle physc %entc lbusy vcsw phint
----- ----- ----- ----- ----- ----- ----- ----- -----
0.13 0.17 0.05 99.65 0.00 0.00 0.30 103475 0
[root]# lparstat 5 2
System Configuration
type=Shared mode=Uncapped smt=Dedicated lcpu=1
mem=2051328 kB cpus=32 ent=0.40
%user %sys %wait %idle physc %entc lbusy vcsw phint
----- ----- ----- ----- ----- ----- ----- ----- -----
0.05 0.00 0.00 99.95 0.00 0.00 0.05 130373 0
0.00 0.00 0.00 100.00 0.00 0.00 0.00 130475 0
© 2013 IBM Corporation
Useful Commands continued:
CPU settings utility
Determine cpu frequency:[root ~]# ppc64_cpu --frequencymin: 4.31 GHz (cpu 2)max: 4.31 GHz (cpu 3)avg: 4.31 GHz
Obtain the number of cores installed:
[root ~]# ppc64_cpu --cores-present
Number of cores present = 1
Obtain the number of cores currently online:
[root ~]# ppc64_cpu --cores-on
Number of cores online = 1
[root@eg02ph05 ~]#
20
Linux Performance Customer Profiler Utility (lpcpu)
This script captures potentially interesting performance data (profile information, system information, and some system configuration information) on a single pass approach. Gathering all of this information at once allows the context to be understood and the performance profiling data to be analyzed.The script will check to ensure you have the basic tools installed on your system.
This script takes advantage of all of the "normal" performance tools used on Linux:
iostatmpstatvmstatperfmeminfotopsaroprofileperf
In addition, relevant system information is gathered with the profiler output into a single tarball saved on your system. By default the file is saved in /tmp.
© 2013 IBM Corporation
RHEL Kernel image too large?
I have seen this problem a few times whilst trying to install (DVD/ISO or Network) RHEL on IBM POWER:
Welcome to yaboot version 1.3.14 (Red Hat 1.3.14-35.el6)Enter "help" to get some basic usage informationboot: linuxPlease wait, loading kernel... Elf64 kernel loaded...Loading ramdisk...Claim failed for initrd memory at 02000000 rc=ffffffff After some searching on Bugzilla I discovered my problem, and from this I did a little testing to resolve the issue which I have detailed below:
Boot the IBM POWER server and enter the ‘OpenFirmware’ prompt with selection 8 in the IBM menu, and run the following command at the prompt:
0 > printenv real-base
If you see the following the I-------------- Partition: common -------- Signature: 0x70 ---------------real-base 2000000 2000000
21
© 2013 IBM Corporation
Continued: RHEL Kernel image too large?
The issue is that firmware is expecting an image of 32MB, and in our example for RHEL6 it is 16MB (or smaller) so this is how to correct this issue:0 > setenv real-base 1000000
This sets the expected image to 16MB, which we can then verify:
0 > printenv real-base -------------- Partition: common -------- Signature: 0x70 ---------------real-base 1000000 2000000
Now perform a reboot:
0 > reset-all
The system should then boot off of the RHEL image and the installs should now work as expected. In this instance it was a virtual server so until the LPAR is removed from the HMC it will continue to work.
Ref:2000000 is 32MB1800000 is 24MB1000000 is 16MBc00000 is 12MB
22
© 2013 IBM Corporation
The previous section covers a few changes that can be made in relation the operating system, and it can be difficult to find reference material and example systems to test comparably too. I would also recommend looking at the www.spec.org website.Searching in the ‘SPECjEnterprise2010’ section will also yield some useful results.
24
© 2013 IBM Corporation
You can view all of the results or search for a direct example:
The website helpfully includes all of the related environmental
information, such as DB2, JVM,
ulimits, and other parameters.
25
© 2013 IBM Corporation
Hardware & Facility Access
Special discountedISV Developer Box
Purchase or lease PowerLinuxservers for development or
in-house use at specialdiscounted rates.
Virtual Loaner Program (VLP)Reserve and provision IBM Power
Systems online for migration,porting, testing and solution
demonstrations.ibm.com/systems/vlp
IBM Innovation Centers andLinux Technology Centers
Get remote and on-site accessto IBM Power Systems with
dedicated support at no charge.Build customized environments,
conduct optimization, or host clientworkshops.
IBM Innovation Centersibm.com/partnerworld/wps/servlet/ContentHandler/isv_com_tsp_iic_overview
IBM Linux Technology Centersibm.com/linux/ltc/index.html
SDK & Tools
IBM Installation Toolkitfor PowerLinux
A simple tool to install andconfigure Red Hat Enterprise Linuxand SUSE Linux Enterprise Server,
open source workloads, as wellas IBM value-added software on
Power Systems.ibm.com/webapp/set2/sas/f/lopdiags/installtools
Software Development Toolkitfor PowerLinux
All-in-one solution integratingLinux and IBM tools such asOProfile, Perf, Valgrind, and
Autotools for Linux and FDPR forIBM. You can even use the EclipseIDE directly on the Power Systems
server or in x86_64 clients forremote development.
ibm.com/support/customercare/sas/f/lopdiags/sdklop.html
Technical Assistance
ISV Application Porting AssistanceGet help to port and enable your
applications to IBM Systemsplatforms.
ibm.com/partnerworld/mem/support/trs_develop_rapidport.html
ChiphopperAssistance to port, test, and
support your existing Linux x86applications on other IBM Systemsand middleware platforms—at no
cost.ibm.com/isv/go/chiphopper
IBM Migration FactoryAddress your customers’ concernswhen migrating from Solaris/HPUX/
x86 to PowerLinux withmigration expertise andmethodologies from IBM
ibm.com/systems/migratetoibm/factory
IBM developerWorks – A Technical Community for PowerLinux
Get access to blogs, wiki, andmessage boards for recent
news, discussions, events andassistance.
ibm.com/developerworks/group/tpl27
© 2013 IBM Corporation
Technical Support
• Face to face or remote
• Proof of concept
• IBM products consulting
• Prototyping
• Testing
• Porting IBM platforms
• Validation
• Integration
• Performance/scalability testing
• Sizing guides
• SmartCloud
Education and events
• Extensive program of technical education on the latest IBM technologies
• LoB events focused on:– Maximising your Relationship with IBM– Industry focused seminars– New business models
Solutions Laboratories
• Smarter Planet
• Smarter Cities
• Smarter Commerce
• Smarter Energy
• Smarter Government
• Intelligent Operations Centre
Infrastructure (onsite and virtual)
• Latest hardware and software
• Industry and cloud solutions
• Technical expertise
28
© 2013 IBM Corporation29
Technology and ServersSystem x, (incl. Nehalem)System i inc i520, i570,
IBM POWER inc POWER6 570, POWER7 740/50/770/80, &
POWER7+ 760/70/80Storage inc SVC, SFS,Tape, SONAS,
V7000 DS8800 & XIVFibre connected
Blades, & SmartAnalyticsAll Middleware engagements
accommodated, including most SWG acquisitions (e.g. iLog, FileNet,
SPSS, Cognos, etc.) PureSystems(PureApplication, PureFlex
& PureData)
World-class facilitiesLocated in Hursley Park, site of IBM’s largest
non-US Development Laboratory6 project / engagement rooms
3 conference rooms2 Dedicated classrooms, + access to 4 more
Hursley Innovation TheatreSmarterPlanet Lab, Retail Lab,
E&U Lab, Local Govt. Lab, Sustainability Demos
InfrastructureHigh bandwidth infrastructure
Private network separate from internal IBM network
VPN Support to allow remote workingDual SAN Fabric to dedicated Storage
Large number of load generation machinesSoft switching KVM system to allow customers
to work anywhere in centre
Skills and PeopleSkilled “development” knowledge and
experienceWide range of technical expertise across
SWG and STGGreat networks inside IBM’s technical
communitiesTechnical events management and
coordination
29
© 2013 IBM Corporation30
42 Globally Networked Locations
Results• Build skills• Build solutions• Close deals and improve your bottom line• Execute marketing tactics
Staff• Full-time IBM technical experts for hands-on
assistance and knowledge transfer• Access to IBM marketing and sales teams
Classrooms• Technical workshops and seminars• Business seminars and collaboration events
Meeting rooms • Customer briefings• Sales closure meetings• Business Partner Demo rooms
Technology supportAccess to the latest IBM hardware and software:• Onsite at the Centers• Remotely via secure Virtual Private Network (VPN)
North America (6)• Austin• Chicago• Dallas• San Mateo• Toronto• WalthamLatin America (2)• Sao Paulo• Mexico City
Europe (13)• Copenhagen• Dublin• Stockholm• Hursley• London• Stuttgart• Zurich• Amsterdam• Barcelona• La Gaude• Tel Aviv• Milan• Paris
CEEMEA (12)• Bratislava• Bucharest• Budapest• Casablanca• Istanbul • Johannesburg• Kiev• Ljubljana• Moscow• Prague• Warsaw• Nairobi
Japan (1)• Tokyo
Asia Pacific (8)• Bangalore• Kuala Lumpur• Manila• Seoul• Shanghai• Sydney• Vietnam• Ho Chi Minh
30
© 2013 IBM Corporation31
The PowerLinux Community(developerWorks)
Power Systems Linux Portal
(Product Information)
www.ibm.com/developerworks/group/tpl/
www.ibm.com/systems/power/software/linux/
@thinkpowerlinux plus.google.com/communities/100156952249293416679
© 2013 IBM Corporation
Sessions:– #1: Exploiting Virtualisation on IBM Power Systems with
PowerVM– #2: VIOS - how to get going – #3: Controlling processor resources in virtualised partitions– #4: Deeper dive into Active Memory Sharing– #5: Virtualisation Best Practices– #6: Deeper dive into Shared Storage Pools and Thin
Provisioning – #7: Virtualisation and the world of 10Gbit Ethernet– #8: Deeper dive into Active Memory Expansion– #9: VIOS maintenance– #10: Active Memory Deduplication– #11: IBM i Suspend/Resume– #12: Dual VIOS Upgrade walk-through– #13: Shared Storage Pools ... from Experience– #14: IBM i Live Partition Mobility– #15: Capture and Deploy partitions– #16: Virtual Partition Manager for IBM i
– #17&18: Updating Power Systems, I/O and HMC– #19&20: Power7 Affinity and Performance– #21: Power Advisors (VIOS, LPAR and Java)– #22: PowerSC – What/Why/How– #23: Dynamic Platform Optimizer (DPO)– #24: Removable Disk Cartridge (RDX) on Removable
Mass Storage (RMS)– #25: Electronic Service Agent– #26: Active and Dynamic Systems Optimizer
(ASO/DSO)– #27: Shared Storage Pools Phase 3– #28: Whole POWER Machine Monitoring– #29: Monitoring POWER/AIX/VIO with IBM Tivoli
Monitoring– #30: Tricks of the Power Masters– #31: External Storage and IBM i – sizing/modelling– #32: Power Systems Configuration Best Practices
32
@JyotiDodhia
A series of technical webinars on Power Systems (including virtualisation):
InformalUsually about an hour eachMany include demos
Aimed at:Technical audience - operators, systems administrators and technical specialistsThose using / planning to use IBM's Power based systemsCustomers / Business Partners / IBMersUK & Ireland audience initially, but spread to other countries
More info, replays, and registration at:http://tinyurl.com/PowerSystemsTechnicalWebinarsor send an email to [email protected]