Optimizing WordPress for Speed and Conversions by Ben Metcalfe
Optimizing your A/B Test Installation for Speed
26
Optimizing Your A/B Tests Installation for Speed & Conversion Rate Brought to you by:
Transcript of Optimizing your A/B Test Installation for Speed
Optimizing Your A/B TestsInstallation for Speed & Conversion Rate
Brought to you by:
Dennis van der HeijdenFounder & CEO of Convert.com
@Convert #ConvertAcademy
Learn How Speed Impacts A/B Tests and Overall Conversions&
Minimize the Risk of Impact
Purpose of Webinar
Packed Presentation, Let’s Get StartedThe Trust About Async/Sync Speed and How to Test Well
● How Tools Increase Load○ Any Script Lowers Speed○ Lower Speed Lowers CR○ When Scripts are loaded○ Async/Sync○ What to Do?
● Inner workings of A/B testing tools○ Async and sync codes○ Avoid seeing the blink, what
hack do A/B testing tools use?
www.convert.com
Script in Header
Being the first to load allows A/B testing tool to manipulate the content following and make the changes without blinking.
It is blocking and yes it is hurting site-speed, later on why that hardly matters...
www.convert.com
Manage Expectations of ClientsAt moment new JS is live baseline changes slightly
www.convert.com
What is really winning?● Are tools wrong?● What is common sense testing?
Manage Expectations of ClientsAt moment new JS is live baseline changes slightly
www.convert.com
Installing Anything Extra Lowers Conversion Rate● Installing any Javascript increases speed
and lowers conversion rate● Test results should offset that loss
Workings of A/B Testing ScriptsLayers of testing
www.convert.com
Where and when do we make changes?
Manage Expectations of ClientsAt moment new JS is live baseline changes slightly
www.convert.com
Expect Average Drop 1%
1. Installing A/B testing tool lowers load speedadding ~200 ms / ~500 ms on first load per CDN hub
2. + ~100 ms / 200 ms first load page 3. + ~50 ms / ~100 ms on follow up loads
Average CR drop will be in scenario 1/2/32% / 1% / 0.5%
Workings of A/B Testing ScriptsLayers of testing
www.convert.com
Where and when do we make changes?
The Inner Workings of A/B Testing Tools
psst, please don’t tell...
“The Dirty Secrets and Misdirections”
Understanding the terminology is super important when selecting a new A/B testing tool. There is a reason smart agencies go deep on sync/async parts of the script and how tools hide their true technology from you?
Let's talk about:
● Do you really understand sync and async?● Rethink your current A/B testing tools impact...● How do we trick your eyes and make SmartInsert© possible?
Don’t Copy Low Quality A/B Testing ArticlesHow do we really affect site speed and why to some tools “blink”?
www.convert.com
Understand How Page LoadsThanx New Relic, for image
www.convert.com
A/B Testing Tools TechnologyHow does Convert Patent Pending SmartInsert© Work?
www.convert.com
● Click to add text.○ Click to add text.
■ Click to add text.● Click to add text.
● 0.30 Sec.Nothing...
● 0.35 Sec.Convert Starts○ Hide Body for 1 Sec.○ Check Every 0.05 Sec.○ Unhide Body
● 1.50 Sec.Convert Long Done
● 0.30 Sec.DOMloading
● 0.35 Sec.DOMcontent Loaded Start
● 1.50 Sec.DOMcontent Loaded End
● 1.80 Sec.DOMcomplete
Fast Website + Convert Experiments
A/B Testing Tools TechnologyHow other A/B testing Tools Work… example #1
www.convert.com
● Click to add text.○ Click to add text.
■ Click to add text.● Click to add text.
● 0.30 Sec.Nothing…
● 0.35 Sec.Tool Starts○ Makes All Changes○ Visible Changes
Swapping / Blinking● 1.50 Sec.
Tool #1 Done
● 0.30 Sec.DOMloading
● 0.35 Sec.DOMcontent Loaded Start
● 1.50 Sec.DOMcontent Loaded End
● 1.80 Sec.DOMcomplete
Fast Website + Other A/B Testing Tool #1
A/B Testing Tools TechnologyHow other A/B testing Tools Work… example #2
www.convert.com
● Click to add text.○ Click to add text.
■ Click to add text.● Click to add text.
● 0.30 Sec.Nothing...
● 0.35 Sec.Nothing…
● 1.50 Sec.A/B Testing Tool Loads○ Hide Body for 1 Sec.○ Make All Changes.
● 1.80 Sec.Body Still Hidden
● 2.50 Sec. + 0.7 Sec. WhiteBody Unhide
● 0.30 Sec.DOMloading
● 0.35 Sec.DOMcontent Loaded Start
● 1.50 Sec.DOMcontent Loaded End
● 1.80 Sec.DOMcomplete
Fast Website + Other A/B Testing Tool #2
Synchronous and Asynchronous Scripts
Calling #BS on 35kb Code Size
Comparing Sync and AsyncSo ehh … so how much load time does a 35kb script take in seconds?
Does it matter how big the initial script is?
www.convert.com
Client Small Script http://cdn.fast.com
CDN.Fast.com35kb
jQuery X280kb
Geo-Check120ms
New Image Variation400kb
715kb+
120ms
Comparing Sync and AsyncNow what is it really?
www.convert.com
100ms 200ms 400ms
In a Sync-World ≠ Convert Experiments
100ms
200ms
In a Async-World ≠ Convert Experiments
400ms
Total Time = ~700ms
Total Time = ~450ms
Convert Experiments ScriptWhat do we really do and when?
www.convert.com
100ms 100ms 400ms
Sync and Async = + 100ms extra
200ms
200ms
100ms
Async Goals & Cookie SettingSmartInsert© Sync
Competitor #2 Async ScriptOthers might do it different… and why that matters
www.convert.com
100ms
100ms
400ms
Sync and Async
200ms
200ms 100ms
Async Goals & Cookie Setting
Async Script Variations = Blinking*
*When changes are loaded in areas A or B
A
B
C
Where can you find most important speed and debug settings in Convert?
● jQuery reduction● Measure jQuery Code Send● Chrome Plugin
Demo Where to Find Settings
Convert ExperimentsAuthentic A/B Testing Software
http://www.convert.com
Brought to you by:
