IBM JVM 1.4.2 64-bit in SAP NetWeaver Application Server Java

Environments

This document can be found on the web, www.ibm.com/support/techdocs

Version Date: March 25, 2010

Andreas Steiner Christoph Langer

Dr. Thomas Obermeier

IBM Boeblingen Lab, Germany SAP on IBM Systems Development @ SAP Copyright 2010, IBM Corporation

http://www.ibm.com/support/techdocs

Copyright 2010, IBM Corporation Version 3/25/2010 http://www.ibm.com/support/techdocs IBM JVM 1.4.2 64-bit in SAP NetWeaver Application Server Java Environments Page 2

Table of contents Table of contents .................................................................................................................... 2 Technology Used.................................................................................................................... 5

GC Policy used for SAP NetWeaver Application Server Java................................................................. 5 Important Java VM Parameters and their defaults .................................................................................. 6 Verbose GC output .................................................................................................................................. 8

GC Problem Determination .................................................................................................. 10 Types and root causes of an Out Of Memory (OOM) ............................................................................ 10 Long GC Pause Times........................................................................................................................... 14

Class unloading times ..................................................................................................... 14 Paging on the system ...................................................................................................... 15

Frequent GC .......................................................................................................................................... 17 Insufficient heap size for the nursery area ...................................................................... 17 Insufficient heap size for the tenure area ........................................................................ 18

Tuning Recommendations based on Known Application Specific Workload .................. 19 Stability versus Performance ................................................................................................................. 19 Overall Heap Size .................................................................................................................................. 19 Application Footprint Tenure Size ....................................................................................................... 19 Short Life Objects - Nursery Size .......................................................................................................... 24 Load balancing Number of Server Nodes versus Larger Heap .......................................................... 25 JVM Memory Usage Outside the Java Heap ..................................................................................... 25

Tools...................................................................................................................................... 27 Memory Problem Analysis with Eclipse Memory Analyzer .................................................................... 27

Preconditions and Installation ......................................................................................... 27 Configuration ................................................................................................................... 27 Prepare Dump Files for MAT ........................................................................................... 29 What to do with the Dump Files ...................................................................................... 29

Verbose GC Analysis ............................................................................................................................. 30 Thread Dump Analysis........................................................................................................................... 31

IBM Thread and Monitor Dump Analyzer for Java .......................................................... 31 Thread Dump Viewer for SAP NetWeaver Application Server Java 6.40/7.0 ................. 32

SAP Solution Manager - End-to-End Root Cause Analysis .................................................................. 34 Summary ............................................................................................................................... 35 Appendix ............................................................................................................................... 36

AIX ......................................................................................................................................................... 36 Native memory svmon .................................................................................................. 36

IBM i ....................................................................................................................................................... 37 Memory pools .................................................................................................................. 37 WRKJVMJOB .................................................................................................................. 37

Linux on System z .................................................................................................................................. 38 Native memory ps......................................................................................................... 38

http://www.ibm.com/support/techdocs

Copyright 2010, IBM Corporation Version 3/25/2010 http://www.ibm.com/support/techdocs IBM JVM 1.4.2 64-bit in SAP NetWeaver Application Server Java Environments Page 3

Linux on AMD64/EM64T ........................................................................................................................ 39

Native memory ps......................................................................................................... 39 Linux on Power Systems ....................................................................................................................... 39

Native memory ps......................................................................................................... 39 Resources ............................................................................................................................. 40 About the authors ................................................................................................................. 41 Acknowledgements .............................................................................................................. 41 Trademarks and special notices ......................................................................................... 42

http://www.ibm.com/support/techdocs

Copyright 2010, IBM Corporation Version 3/25/2010 http://www.ibm.com/support/techdocs IBM JVM 1.4.2 64-bit in SAP NetWeaver Application Server Java Environments Page 4

Abstract

Across the IBM platforms J9 (IT4J: IBM Technology for Java) has become the preferred technology to run Java applications in the SAP NetWeaver stack of releases 2004 and 7.0x.

On the one hand SAP workload may be specific in some respect. On the other hand a huge quantity of general information is available. Therefore, based on our experience in support and development, this paper aims to provide a compact and relevant selection regarding terminology, description of typical problem situations, tuning options and tool usage.

We present selected options and Java parameters and describe some basics of garbage collection. In addition we describe the basic ideas of SAP's default parameterization.

Out Of Memory situations and garbage collection issues are the main focus of a section dedicated to problem determination.

The documented default parameterization is able to support the whole range of different workloads in the SAP context with few exceptions. If it comes to potential improvements it is key to understand the ideas behind it as well as the characteristic workload of your system. We point to the most important aspects like heap layout and workload profiles to discuss our guidelines.

Lastly, we give hands on information regarding tool usage. We describe state of the art tools for analysis of garbage collection and thread dumps. In particular we explain how to prepare your system for use of the Eclipse Memory Analyzer, which stands for a major improvement in memory problem analysis.

http://www.ibm.com/support/techdocs

Copyright 2010, IBM Corporation