Cloud Performance: Guide to Tackling Cloud Latency [Cloud Connect - Chicago 2012]

Web Acceleration and Front End Optimization

Cloud Connect 2012 Chicago

Hooman Beheshti VP Technology, Strangeloop

hooman@strangeloopnetworks.com

Web Application Acceleration

•  Means lots of things to lots of people –  TCP optimization

–  Caching –  HTTP protocol optimization

–  Compression –  Etc

•  We’ll focus on “front-end” issues –  Front-end Optimization (FEO) –  Sometimes called WCO or WPO

Better Performance = Better Business

Impact of page load time on average daily searches per user

-0.70%

-0.60%

-0.50%

-0.40%

-0.30%

-0.20%

-0.10%

50ms pre-header

100ms pre-header

200ms post-header

200ms post-ads

400ms post-header

Impact of additional delay on business metrics

-5.00% -4.50% -4.00% -3.50% -3.00% -2.50% -2.00% -1.50% -1.00% -0.50% 0.00%

50 200 500 1000 2000

Added delay

Queries per visitor Query refinement Revenue per visitor Any clicks Satisfaction

Shopzilla had another angle

� Big, high-traffic site ◦ 100M impressions a day ◦ 8,000 searches a

second ◦ 20-29M unique visitors

a month ◦ 100M products

�  16 month re-engineering ◦  Page load from 6 seconds

to 1.2 ◦  Uptime from 99.65% to

99.97% ◦  10% of previous hardware

http://en.oreilly.com/velocity2009/public/schedule/detail/7709

5-12% increase in revenue

Mobile Case Study

Customer Pro"le

•  Top 200 Internet retailer, US based •  Target geography: US and Europe •  $3B in revenue

•  30,000 employees

Page Views by Mobile Device

Experiment

•  Induce delay on HTML –  200msec, 500msec, 1000msec

•  Apply to small percentage of tra#c

•  12 weeks

•  Monitor impact on key business metrics

Results

Impact on Bounce Rate

Impact on Returning Visitors

More Details

•  http://www.webperformancetoday.com/2011/11/23/case-study-slow-page-load-mobile-business-metrics/

•  http://velocityconf.com/velocityeu/public/schedule/detail/21632

What Is FEO?

0 6 12

What Is FEO?

0 6 12

DNS TTFB

TCP Connection

Download

What Is FEO?

0 6 12

Back End: The time from when the request is made by the browser to last byte of the HTML response

Front End: Everything after the HTML arrives

Important Timers: Start Render

onload (Document Complete)

Google’s Waterfall Chart

Measurement/Tools

Waterfall Analysis

•  Best way to address front-end problems is to diagnose your site/application through waterfall analysis

Waterfall Tools: webpagetest.org

Waterfall Tools: HTTPWatch

Waterfall Tools: Firebug

Waterfall Tools: WebKit Dev Tools

Waterfall Tools: PCAP2HAR

Measurement

•  Performance measurement is still a challenge •  Synthetic performance “proxies”

–  Backbone testing services –  Desktop tools and browser plugins –  Browser-based tests

•  Real User Monitoring (RUM) –  Using real user beacons –  Services available –  Can build your own –  Now a part of Google Analytics –  Caveat: need lots of data!

Front End Problems

Front End Performance Problems

•  Latency: –  every round trip incurs a latency

penalty

•  Payload:

–  last mile bandwidth isn’t in"nite

penalty

•  Payload:

•  Caching: –  coming back to the page must be

much faster

penalty

•  Payload:

•  Caching: –  coming back to the page must be

much faster

•  Rendering: –  browser work takes time

Client Platforms

•  Desktop vs Mobile –  Desktop browsers have more access to compute

resources –  Larger screens

–  Faster networks (lower latency)

•  The problems are often similar –  Addressing them is often di$erent

Addressing Front End Problems

Latency

•  TTFB (Time To First Byte) •  Every fetch incurs the latency penalty •  Two ways to address the problem:

–  Reduce latency –  Get rid of round trips

•  Global network of caching proxies that gets content closer to all your users

•  The closer the asset, the lower the latency

•  Lots of vendors to choose from

Per object TTFB savings of ~50%

CDN: because physics!!

•  Well understood way to leverage science

•  Mitigates the latency problem by reducing it for the majority of your users

•  Less e$ective when it comes to mobile users

Mobile Networks (3G example)

Resource Consolidation

•  Eliminating round trips altogether also "ghts the latency problem, often more e$ectively

Combine images into fewer "packages"

Resource Consolidation

•  A number of consolidation techniques –  Images (sprites)

–  JavaScript/CSS consolidation/concatenation –  Inlining (DataURI for images)

–  MHTML (IE only)

•  Browser makes one request for the “package”

•  HTML is marked up so the browser can get individual resources from inside the package

Payload Reduction

•  Bytes still need to get from server (cloud or otherwise) to client

•  Ways to reduce bytes: –  HTTP compression

–  JS/CSS mini"cation –  Image compression (lossless or lossy)

Payload Reduction

•  Any reduction in bytes will make pages load faster

•  This is particularly important with mobile clients

Browser Caching

•  The browser cache is a resource seldom used optimally

•  Reasons why we generally don’t do good browser caching –  Caching rules are often complicated

–  We never want to server stale content

Browser Caching

•  Use long expiry on static objects –  Served from origin

–  Served from CDN

•  Invalidation framework is a must –  Protect against serving stale content

–  Example: versioning

/images/image.jpg à /images/image.jpg?v=00001

Browser Caching on Mobile Platforms

•  Di$erent than desktop

•  The cache available to the browsers is relatively small

•  Use localStorage instead of browser cache –  A programmable cache, unlike HTTP object caches

–  Limited size (~2.5MB per domain) –  Good for caching CSS/JS, and small images

Rendering Issues

•  More complicated

•  The order of events in the browser make a di$erence to how fast a page looks –  Put things in the optimal order for rendering

•  Deferral –  CSS and JS (sometimes images) –  Asynchronous JavaScript

•  Above the fold vs below the fold

Some Examples

Before and After Waterfalls

58 roundtrips à 5 roundtrips

4.2 seconds à 1.1 seconds

57 roundtrips à 4 roundtrips

3.2 seconds à 0.7 seconds

First Time Visitor

Repeat Visitor

Before FEO After FEO

Mobile Example

•  Velocity Conference 2012

•  Step-wise acceleration of O’reilly’s site (for a mobile phone on 3G) –  Start with the original site (purposely made worse)

–  Add CDN –  Add resource consolidation and payload reduction

–  Add deferrals

•  Examine impact of each

What Does it Look Like?

http://youtu.be/iPtbU1KvLjM

Original Site + CDN

15.29 sec 13.7 sec

Add Consolidation and Payload Reduction

13.7 sec

9.47 sec

To onload Before After

# of resources 92 28

KBytes 727 417

Add Deferral

9.47 sec

3.6 sec

5.56 sec

2.2 sec

What Does It Look Like?

MobileFEO = +Consolidation and payload reduction MobileFEO2 = +Deferrals

http://youtu.be/zTTxdAtbhsg

Summing up…

Sounds Really Easy!

•  It’s not!

•  Some techniques are just di#cult to implement

•  Optimizing for performance sometimes requires signi"cant dev resources –  Mortal companies can’t a$ord to sacri"ce new

feature cycles

•  Maintenance and upkeep is a constant problem –  Every version to roll out will need optimization

FEO Automation Industry

•  Solutions available to automatically do this stu$

•  Multiple deployment options –  Software/Hardware/Service

–  Cloud apps will use either service or software

•  The goal is to “"x the code” for performance, automatically

It Gets Complicated

•  Rewriting HTML can break pages

•  You have to do this stu$ based on browser –  Play to the strength of each browser (supported

techniques, etc) –  Stay away from their weaknesses (bugs, undocumented

issues, etc) –  Mobile is its own beast

•  Optimizing once per page isn’t enough –  First view (cold cache) –  Repeat view (warm cache) –  User "ow

When Looking For FEO Automation

•  Do your research, and understand your needs

•  Understand the deployment model and how disruptive it will be to you, if at all

•  Are there provisions in place for breaking pages

•  Granularity in functionality: –  Browser-based optimization –  mobile –  "rst/repeat views –  transaction %ows

•  Choose what’s right for you, based on your needs

Thank You

hooman@strangeloopnetworks.com

Cloud Performance: Guide to Tackling Cloud Latency [Cloud Connect - Chicago 2012]

Technology

Transcript of Cloud Performance: Guide to Tackling Cloud Latency [Cloud Connect - Chicago 2012]

Tackling Cloud Security Concerns

IEEE TRANSCTIONS ON SUSTAINABLE COMPUTING 1 Latency …repository.essex.ac.uk/24274/1/Latency-based... · that Cloud workload behaviours cannot be generalised, and should be treated

Latency Optimization for Resource Allocation in Cloud Computing ...

Multidimensional Cloud Latency Monitoring and Evaluationrdc.fel.cvut.cz/download/publications/ComNet16.pdf · Multidimensional Cloud Latency Monitoring and Evaluation Ondrej Tomanek,

Understanding the Latency Beneﬁts of Multi-Cloud ...

Oracle Cloud Infrastructure...Oracle Cloud Infrastructure Compute Bare Metal and VMs supporting traditional, HPC AI/ML and cloud-native workloads Networking Predictable, low latency,

Latency–based Benchmarking of Cloud Service Providersrdc.fel.cvut.cz/download/publications/UCC16.pdfperformance parameters, such as for example latency, is in-su cient. In this work,

Low Latency Geo-distributed Data Analytics · · 2018-01-04Low Latency Geo-distributed Data Analytics Qifan Pu1;2, Ganesh Ananthanarayanan1, ... Large scale cloud organizations

Characterizing the impact of network latency on cloud ... · network latency measurements performed across six data centers from three diﬀerent cloud operators. Section 3 outlines

Low Latency Data Center Interconnect Using Infinera Cloud ... · Low Latency Data Center Interconnect sing Infinera Cloud Xpress Optical Transport and Arista Data Center Switches

Low Latency via Redundancy - Peopleradhika/low-latency...Achieving consistent low latency is challenging. Modern applications are highly distributed, and likely to get more so as cloud

Architecting a Hybrid VMware Cloud€¦ · Designing the Solution ..... 11 6.1 Distance and Latency Considerations ...

ISSN 2364-7108 Latency Optimization in Large-Scale Cloud ......Open Journal of Internet of Things (OJIOT), Volume 3, Issue 1, 2017 18 Latency Optimization in Large-Scale Cloud-Sensor

Network Latency and Cloud Games: A Study Using ...claypool/iqp/game-anywhere/...network latency and both quality of experience and player performance while using cloud gaming systems.

Distributed Data Centers - BICSI · • A Distributed Cloud is a micro-cloud located outside the centralized cloud • It performs computation, storage, ... Latency of Data Transactions

LOW LATENCY SCALABLE POINT CLOUD COMMUNICATION …a.web.umkc.edu/aa95b/doc9.pdfLOW LATENCY SCALABLE POINT CLOUD COMMUNICATION Anique Akhtar, Birendra Kathariya, Zhu Li University of

Predicting Response Latency Percentiles for Cloud Object ... · of the cloud object storage system for the following reasons. First, the response latency is a key performance metric

Latency in Cloud Computing and Recent Research on IDC in Xenjain/cs770-13/ftp/sisu_xen.pdf · Latency in Cloud Computing and Recent Research on IDC in Xen Sisu Xi Network Seminar

MULTI-ACCESS EDGE CLOUD - Open Networking Foundation...Jul 22, 2019 · Necessary Resources for Low Latency, Mission Critical Services, Localized Traffic,… PUBLIC CLOUD TELCO CLOUD

Edge Cloud System for Low -latency Processing Using ...