Edge 2014: A Modern Approach to Performance Monitoring

Cliff Crocker @cliffcrocker SOASTA VP Product, mPulse

1.  How fast am I? 2.  How fast should I be?

3.  How do I get there?

Synthe'c vs. RUM

Synthetic 101 Synthetic monitoring (for purposes of this discussion) refers to the use of automated agents (bots) to measure your website from different physical locations. •  A set ‘path’ or URL is defined •  Tests are run either adhoc or scheduled and data is collected

RUM 101 Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user.

•  Involves instrumentation of your website via JavaScript

•  Measurements are fired across the network to a collection point through a small request object (beacon)

<JS> <beacon>

RUM Cast a wide net •  Identify key areas of concern •  Understand real user impact •  Map performance to human behavior & $$

Synthetic Diagnostic tool •  Identify issues in a ‘lab’/remove variables •  Reproduce a problem found with RUM

h0p://www.flickr.com/photos/84338444@N00/with/3780079044/

h0p://www.flickr.com/photos/ezioman/

The Early Days of RUM

•  Round-trip time •  Start/stop timers via JavaScript •  Early contributors:

•  Steve Souders/Episodes •  Philip Tellis/Boomerang.js •  Both widely in use today

Navigation Timing

Browser Support for Navigation Timing

!

10 © 2014 SOASTA CONFIDENTIAL - All rights reserved.

Navigation Timing

DNS: Domain Lookup Time

function getPerfStats() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; }


Navigation Timing

TCP: Connection Time to Server

function getPerfStats() { var timing = window.performance.timing; return { connect: timing.connectEnd - timing.connectStart}; }


Navigation Timing

TTFB: Time to First Byte

function getPerfStats() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; }


Navigation Timing

Base Page

function getPerfStats() { var timing = window.performance.timing; return { basePage: timing.responseEnd - timing.responseStart}; }


Navigation Timing

Front-end Time

function getPerfStats() { var timing = window.performance.timing; return { frontEnd: timing.loadEventStart - timing.responseEnd}; }


Navigation Timing

Page Load Time

function getPerfStats() { var timing = window.performance.timing; return { load: timing.loadEventStart - timing.navigationStart}; }

Measuring Assets

•  Strength of synthetic •  Full visibility into asset performance

•  Images •  JavaScript •  CSS •  HTML

•  A lot of which is served by third-parties •  CDN!

Object Level RUM

Browser Support for Resource Timing

CORS: Cross-Origin Resource Sharing

Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"

Start/End time only unless Timing-Allow-Origin HTTP Request Header defined

Resource Timing

var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’; var me = performance.getEntriesByName(url)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart }

Resource Timing

•  Slowest resources •  Time to first image •  Response time by domain •  Time a group of assets •  Response time by initiator type (element type) •  Cache-hit ratio for resources

For examples see: http://www.slideshare.net/bluesmoon/beyond-page-level-metrics

Resource Timing

•  PerfMap - https://github.com/zeman/perfmap •  Mark Zeman

•  Waterfall.js - https://github.com/andydavies/waterfall •  Andy Davies

Picking a Number

•  Industry benchmarks? •  Apdex? •  Analyst report? •  Case studies?

“Synthetic monitoring shows you how you relate to your competitors, RUM shows you how you relate to your customers.”

– Buddy Brewer

Benchmarking

Benchmarking

•  http://soasta.com/mpulseUX

Benchmarking

•  Page construction •  Requests •  Images •  Size

•  Other important metrics •  Speedindex •  Start Render •  PageSpeed/Yslow scoring

Performance is a Business Problem"

"

Yahoo! - 2008

Increase of 400ms causes 5-9% increase in user abandonment

http://www.slideshare.net/stubbornella/designing-fast-websites-presentation

Shopzilla - 2009

A reduction in Page Load time of 5s increased site conversion 7-12%!

http://assets.en.oreilly.com/1/event/29/Shopzilla%27s%20Site%20Redo%20-%20You%20Get%20What%20You%20Measure%20Presentation.ppt

Walmart - 2012

http://minus.com/msM8y8nyh#1e

SF WebPerf – 2012 Up to 2% conversion drop for every additional second

So What?"""

SIMULATION

How Fast Should You Be?

•  Use synthetic measurement for benchmarking your competitors

•  Understand how fast your site needs to be to reach business goals/objectives with RUM

•  You must look at your own data

Real users are not normal

Page Load Times

2.76s – Median

10.45s – p95

17.26s – p98

Platform Median 95th Percentile

98th Percentile

Mobile 3.62s 12.53s 20.04s

Desktop 2.44s 9.31s 15.86s

Page Load Times

Page Load Times

OS Median 95th Percentile

98th Percentile

Windows 7 2.41s 9.29s 15.89s

Mac OS X/10 2.30s 8.11s 13.45s

iOS7 3.27s 10.64s 15.79s

Android 4 4.06s 14.30s 27.93s

iOS8 3.53s 11.54s 19.72s

Windows 8 2.67s 10.75s 18,74s

Other Factors

•  Geography •  User Agent •  Connection Type •  Carrier/ISP •  Device Type

Not All Pages are Created Equal

For a typical eCommerce site, conversion rate

drops by up to 50% when “browse” pages increase

from 1 to 6 seconds

Not All Pages are Created Equal

However, there is much less impact

to conversion when “checkout” pages degrade

How Do I Get There?

•  Focus on the highest value opportunities/demographics •  Identify key pages that have the most impact on your

KPIs •  Prioritize based on reducing friction within the funnel or

critical path

Thank You!"""

Images

1.  Modern (Title Slide): https://www.flickr.com/photos/looking4poetry/3477854720/in/photolist-6ijV3q-9t3sjb-cj8VWE-MNjPA-4yBqug-2rk5he-4nZVzJ-4yE45J-9t3Fvj-2rptr5-69ymdr-6XVTH-6QZ4dg-9t3HN5-8geUkB-6oEXs5-eH9zS-H2XTt-cKdJvb-7GNKWx-cKdKZd-5Rw3qt-jfT5Dx-Jfiuy-bTop6R-2m8kAB-Jfiuu-7E8eMf-9jqD6-9hYAvd-Jfius-gsqr7U-7KiNAZ-8kzG9V-euXdb-44qJNN-47jBuY-nsxZrZ-7cGjQL-4cfHKq-cGzbSC-aQtWPR-8y3sR3-6okjAW-5A7BSC-6aRsvh-eSMHkX-kxvdyK-9t3udu-c8EkYu

2.  Man vs. Machine: https://www.flickr.com/photos/eogez/3289851965/in/photolist-61HmnB-9nkQu3-dfinF1-4GjK24-4GjL6Z-4GoV6C-dibVCU-4GoTXo-4GjF3k-4GjH7Z-4GoR6h-ek6eT9-ek6eUU-4fADUx-4fEDV9-65JA6W-65Jsxu-65JDnj-4fAE12-65JDTh-65JBZu-65Ek9M-65JtDu-65JuiN-65JzFS-65EfiH-65Jv63-65JxX7-65Enyc-65Eh4c-65Jxjm-65JwmU-65EhUP-65JCML-65EjxP-65JBQf-65Ee4Z-65JD4b-65JBF1-65Ec1r-65EkPB-65Jy4o-65Eg96-65JAjb-65Emfe-65Efoz-65JBam-65Eniv-65JCrU-65Eed2

3.  https://www.flickr.com/photos/bradybd/2818154005/

4.  https://www.flickr.com/photos/perspective/149321089/

5.  Dartboard: http://en.wikipedia.org/wiki/Darts#mediaviewer/File:Darts_in_a_dartboard.jpg

References:

1.  http://calendar.perfplanet.com/2011/a-practical-guide-to-the-navigation-timing-api/

2.  http://www.slideshare.net/bluesmoon/beyond-page-level-metrics

Edge 2014: A Modern Approach to Performance Monitoring

Technology

Transcript of Edge 2014: A Modern Approach to Performance Monitoring