Performance Tuning You’re doing it wrong!

Number 10:No performance requirements

• No requirements == no problem

Number 9:

Tuning by folklore• Performance happens in a context

• admin manuals are devoid of context

• blogs don’t speak to your context

• Measure don’t guess

• use measurements to guide all decisions

®

Number 8:

Tuning by Google-XX:InitialHeapSize=1610612736 -XX:MaxHeapSize=2147483648

-XX:MaxNewSize=697933824 -XX:NewSize=697933824 -XX:OldSize=1395867648

-XX:OldPLABSize=16 -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution

-XX:+PrintGCApplicationStoppedTime

-XX:SurvivorRatio=1 -XX:TargetSurvivorRatio=90

-XX:-UseAdaptiveSizePolicy -XX:+PrintAdaptiveSizePolicy

-XX:+UseCompressedClassPointers -XX:+UseCompressedOops

-XX:-UseLargePagesIndividualAllocation

-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled

Number 7:That is how my system works

• Often the diagnosis takes 15 minutes

• convincing the client that the implement is a problem can take hours!!!

Number 6:Not listening to the system

• Presented groups of developers with a performance problem

• not one group has solved it in 8 years!!!

• once they learn to read the signals the problem can be diagnosed in minutes!

Number 5:Not being able to profile in prod• Would you let your doctor diagnose you

by running the tests on a newt?

• Profiling is statistical sampling technique

• Performance bottleneck is a statistically significant event

• Profilers dilute the answer with expensive to capture useless data points

Number 4:

Diving into the code• “Know” what the problem is based on local

knowledge

• even if you tell them what the problem is they won’t change course!!!

• In one exercise >97% of developers fail to identify and fix the primary bottleneck

• even when they’ve been emphatically told where they will fail

Number 3:

Poor system visibility• Monitoring pukes a ton of data into your

lap

• often leave people scratching their heads trying to understand where to start!!!

• Often app is only logging

• logs are optimized for development, not ops

Number 2:

Give it more hardware• If you don’t understand your bottleneck…..

• Won’t work if you can’t use the hardware you already have

• Seemingly perfectly parallelizable batch job needed to run in 1 hour

• single server took 4 hours

• 4 servers took more than 8 hours to complete!!!

Number 1:

Setting up a tiger team• Seriously!!! resort to diagnosis by

committee?

• Clear indication that your team is using technology that either don’t understand or can’t manage

• Nice way to trigger “tribal” behavior.

jClarity Illuminate

• Picks up where monitoring stops

• Minimize ambiguity

• uses heuristics to identify performance bottlenecks in live system

• points out casual execution path!

• Profiles with minimal impact on (already poor) performance

Performance Diagnostic Engine