BEYOND AVERAGESDan Kuebrich / @dankosaur

Beyond Averages

• Abstraction: summary statistics for performance data

• Why performance data is hard

• Visualizing data distributions

A few of my favorite abstractions

A few of my favorite abstractions• Abstraction lets us trade information for

actionability

A few of my favorite abstractions• Abstraction lets us trade information for

actionability

• Min, max, average (“mean”), quantiles, stdev

A few of my favorite abstractions• Abstraction lets us trade information for

actionability

• Min, max, average (“mean”), quantiles, stdev

• That’s a great trade! • ... right?

Averages: average at best

Averages: average at best

Averages: average at best

Averages: average at best

Averages vs Percentiles

[1, 16, 17, 19, 13, 5, 20, 3, 10, 14, 8]

Averages vs Percentiles

[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]

Averages vs Percentiles

[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]

Average (Mean): 11.54

Median (50th Percentile): 13 90th Percentile: 19

Percentiles: 1 of 100 slices95%

X

Percentiles: 2 of 100 slices95%

10%

X

Y

Percentiles: 2 of 100 slices95%

10%

Percentiles: 2 of 100 slices95%

10%

Percentiles: 2 of 100 slices95%

10%

Computers are hard• Rarely do we have a single distribution underlying the

data

• Different users, different requests, different resources, different instances, different times

Percentiles vs Distributions

http://en.wikipedia.org/wiki/Percentile

Percentiles vs Distributions

[13, 13, 13, 13, 13, 13, 19, 19, 19, 19, 19]

Median (50th Percentile): 13 90th Percentile: 19

Rarely do we have a single normal distribution underlying the data

Median Mean 90th

Median Mean 90th

The Log-Normal Distribution

(source: http://www.geo.mtu.edu/volcanoes/vc_web/background/S_chem.html)

Log-Normal Distribution

(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)

Log-Normal Distribution

(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)

Log-Normal Distribution

Is there a place between Averageland and “A Beautiful Mind”?

http://now-here-this.timeout.com/2012/10/07/crazy-walls-of-clues-from-tv-film-reviewed-by-carrie-from-homeland/

Histograms

Freq

uenc

y (e

g. #

of c

alls

)

Value (eg. latency)

Populations revisited95%

10%

HistogramsFr

eque

ncy

(eg.

# o

f cal

ls)

Value (eg. latency)

Populations re-revisited95%

10%?

3d Histograms?

Freq

uenc

y (e

g. #

of c

alls

)

Value (eg. latency)

3d Histograms?Fr

eque

ncy

(eg.

# o

f cal

ls)

Value (eg. latency)

Time

Heatmaps

Freq

uenc

y (e

g. #

of c

alls

)

Value (eg. latency)

HeatmapsFr

eque

ncy

(eg.

# o

f cal

ls)

Value (eg. latency)

HeatmapsFr

eque

ncy

(eg.

# o

f cal

ls)

Value (eg. latency)

HeatmapsVa

lue

(eg.

late

ncy)

Time

HeatmapsVa

lue

(eg.

late

ncy)

Time

Latency in the wild…

http://sciencefiction.com/2013/10/24/throwback-thursday-jurassic-park/

Average, or Absolute?

Average, or Absolute?

Multi-modal Data

Multi-modal Data

Multi-modal Drill-down

Multi-modal Drill-down

Multi-modal Drill-down

Long Tails and Outliers

bottom 98%

Long Tails and Outliers

bottom 98%

Long Tails and Outliers

all of it

Long Tails and Outliers

Added Population

Added Population

Added Population

Thanks!

Sign up for a free TraceView account at appneta.com/products/traceview-free-account