© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Justin Bruns , Performance Test Lead, Fireman's Fund Insurance

Robert May , Enterprise Software Tools Administrator

Fireman's Fund Insurance

© Copyright 2012 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice.

HP Tools for Software Performance Testing and Analysis

Justin Bruns, Performance Test Lead - Fireman’s Fund Insurance Company

June 2012

3

Agenda

• Fireman’s Fund Overview

• Problem Statement

• Solution (Sitescope, Performance Center, Diagnostics)

• Performance Analysis Methodology

• Test Execution and Monitoring, Analysis File Review, Exploratory Testing

• Additional Tools for Analysis

• Q & A

4

Fireman’s Fund Overview • Company profile: Fireman's Fund Insurance Company is a premier property and

casualty insurer, providing personal and commercial insurance products nationwide. The company is differentiated from its competition through its product and service excellence, and exceptional claims handling.

• Based in: Novato, California Employees: 3,900

• Company history: Founded in San Francisco in 1863, Fireman's Fund was named for its founding social mission: contributing a portion of its profits to widows and orphans of firefighters. Today the company continues its philanthropic mission, supporting firefighters for safer communities through its Fireman's Fund Heritage SM program.

• Parent company: Allianz SE of Munich, Germany is one of the world's largest providers of insurance and other financial services with more than $1 trillion in assets. It is rated A+ by A.M. Best.

5

Problem Statement • In addition to performance test response times, customers want answers. • Performance bottlenecks sometimes difficult to isolate and correct • Challenges Encountered

– Multiple causes of performance slowdowns – Insufficient communication between performance testing and development teams – Disparate monitoring tools and techniques

6

Solution • Establish integrated and comprehensive monitors

– HP Sitescope for Load Testing 11.10 – HP ALM – Performance Center 11.00 – HP Diagnostics Server 9.12 – HP LoadRunner Analysis Tool 11.03 – HP Virtual User Generator 11.03

• Gather and analyze results

– Performance Analysis Methodology – Inspect information in real-time or after completing tests

• Make recommendations and necessary changes – revalidate results

Application Architecture

8

Sitescope Monitors

9

Sitescope Monitors • Wintel performance counters (CPU, memory, disk, network) • AIX performance counters (CPU, memory, disk, network)

– AIX process level details (CPU, memory size, threads)

• WebSphere MQ counters – Queue details (current depth, open input count, open output count) – Channel details (status, bytes sent/received)

• IIS counters – High level (bytes/min, hits/min) – Web service counters (current connections, method requests/sec) – .NET CLR (SQL client connection Data, garbage collection statistics)

• SQL server counters – Average buffer chain length, % physical I/O used by process – CPU, memory, disk

• Custom solution templates – JBoss, Oracle, Linux, WebLogic

10

Sitescope - Performance Center • Link sitescope monitors in Performance Center • Performance Center test resources section

– Add a Sitescope monitor profile

• Enter the appropriate Sitescope information – Account details not the same as login details

11

Sitescope - Performance Center

• Enter Server and Port Details • Click on Get Counters, Monitors Should Appear • Select the Appropriate Sitescope Monitors • Repeat this Process for all Desired Sitescope Monitors

12

Sitescope - Performance Center

• Add the monitors in Performance Center • Performance test designer window • Select desired monitor(s) from profile tree

13

Performance Center Monitors

• Monitors may be added directly in Performance Center

14

Performance Center Monitors

• Multiple monitor types – Windows Resources, UNIX Resources, SNMP, IIS, SQL Server, Oracle

15

Diagnostics Monitors

• Types of Monitors – .NET Probe – J2EE Probe – SQL Server Collector – Oracle Collector – WebSphere MQ Collector

• Steps Involved – Install Probe or Collector Software – Configure Probe or Collector – Instrument Probe

16

Diagnostics Monitors • .NET Probe

– Standard View (CPU, VM Heap Utilization, Latency, Throughput)

17

Diagnostics Monitors • .NET Probe

– View Server Request Details

18

Diagnostics Monitors • .NET Probe

– View Server Request Details

19

Diagnostics Monitors • WebSphere MQ collector

– Queue information (depth, messages enqueued/dequeued per second)

20

Diagnostics Monitors • SQL Server and Oracle collectors

– View SQL statements, latency, throughput

21

Diagnostics - Performance Center

• In Performance Center’s test designer view – Diagnostics tab • Diagnostics can be enabled for desired % of Vusers • Configure J2EE/.NET application diagnostics

22

Diagnostics - Performance Center

• Select the Desired Probes or Collectors

23

Analysis Methodology

• Start at the web layer • Look deeply into front end application • Examine the database layer • Know the integration layer • Determine the type of problem

24

Test Execution and Monitoring - Analysis File Review -

• Execute the monitored load test • Upon completion, collate and analyze the results • Using the LoadRunner analysis tool, we can

– View transaction response time results – View other Performance Center measurements (hits per second, throughput, etc.) – View sitescope monitoring results – View J2EE/.NET diagnostic information – Overlay graphs and correlate information – Identify trends and hotspots

25

Test Execution and Monitoring - Analysis File Review -

• View transaction response times to identify potential problems

26

Test Execution and Monitoring - Analysis File Review -

• Show J2EE/.NET chain of calls

27

Test Execution and Monitoring - Analysis File Review -

• Overlay graphs to correlate information

28

Test Execution and Monitoring - Analysis File Review -

• Overlay graphs to correlate information

29

Test Execution and Monitoring - Exploratory Testing -

• Profile single user transactions

30

Additional Tools for Analysis

• HttpWatch / Fiddler / Firebug – Inspect browser HTTP traffic

• Database Monitoring Reports – SQL Server Profiler – Oracle AWR Report

• JVM Inspection – Garbage collection logs – Thread dumps – Heap dump analyzer

31

Q & A

32

Contact Details – Justin Bruns, Performance Test Lead Fireman’s Fund Insurance Company

• Work email – jbruns@ffic.com

• Home email – jbruns1@yahoo.com