Download - The what, why and how of web analytics testing

The What, Why & How of

Web Analy3cs Tes3ng

Anand Bagmar So#ware Quality Evangelist

Principal Consultant, ThoughtWorks

Why do we do Tes3ng?

What is Web Analy3cs?

Why do we need it?

Web Analy*cs Solu*on Providers

Adobe Marke*ng Cloud / Omniture

SiteCatalyst

WebTrends

AWStats

Google Analy*cs

StatCounter

…

How to test Web Analy3cs?

The end report

Pros

•  Ensure report is setup correctly

Cons •  Licensing •  Reports not yet setup •  Validate all requests are sent / captured

Tes3ng at the report level

At the origin!

The problem

Manual

The challenge

Repea3ng over-‐and-‐over again

The solu3on

What does WAAT do for me?

• Minimal changes to exisFng test framework •  Does not depend on junit / testng / … • Web AnalyFc tool independent •  UI Driver framework independent •  Browser independent

Flavors

Architecture

•  Original flavor •  Source code + jar •  Supports 3 plugins •  Omniture Debugger

•  HUpSniffer •  JsSniffer

•  Available on github

WAAT -‐ Java

•  Ruby gem implemented over WAAT-‐Java

•  Uses RJB – Ruby-‐Java-‐Bridge •  Supports 2 plugins •  HUpSniffer •  JsSniffer

•  Available on github and rubygems.org

WAAT -‐ Ruby

Omniture Debugger

Omniture Debugger How does it work? Pros •  OS independent •  Run using the regular test-‐user Cons •  Browser dependent – need to implement

ScriptRunner for the UI-‐driver in use •  Web-‐AnalyFc soluFon dependent – Adobe MarkeFng

Cloud / Omniture SiteCatalyst

HHpSniffer How does it work? Pros •  Web-‐analyFc soluFon independent •  Browser independent •  UI-‐driver independent Cons •  3rd party libraries are OS Dependent •  HTTPs not supported out-‐of-‐the-‐box •  Run tests as ‘root’

JsSniffer How does it work? Pros •  Web-‐analyFc soluFon independent •  Browser independent •  HTTPs supported out-‐of-‐the-‐box •  No dependency on any 3rd party library Cons •  Need to write javascript to get the URL from the

browser •  UI-‐driver dependent

•  Download zip from github / rubygems.org

•  Setup

•  jpcap / ScriptRunner setup

•  Update exis*ng tests

How to use WAAT?

Update exis3ng Tests •  Import relevant packages / gems

•  Define & provide parameter values

•  Ini*alize Engine

•  Enable Web Analy*cs Tes*ng

•  Define Test Data

•  Verify Web Analy*cs repor*ng using WAAT

•  Valida*ons

•  Disable Web Analy*cs Tes*ng

<Section !!!actionName="OpenUpcomingPage_HttpSniffer"!!numberOfEventsTriggered="1"!

!tagList="pageName=upcoming:index||!! ! !cc=USD||!! ! !ch=upcoming||!! ! !events=event2||"/>!

Define Test Data

Ini3alize Engine

!engine = getInstance(!! ! !webAnalyticTool, !! ! !inputFileType, !! ! !keepLoadedFileInMemory, !!! ! !log4jPropertiesAbsoluteFilePath!! ! !);!

Enable Web Analy3cs Tes3ng

engine.enableWebAnalyticsTesting();!

Verify Web Analy3cs Repor3ng

!Result verificationResult = !!! !engine.verifyWebAnalyticsData (!! ! !inputDataFileName, !! ! !actionName, !! ! !urlPatterns, !! ! !minimumNumberOfPackets);!

Cart.checkout();!

Sample Valida3ons assertNotNull(verificationResult.getVerificationStatus(), "Verification status should NOT be NULL"); !!assertNotNull(verificationResult.getListOfErrors(), "Failure details should NOT be NULL"); logVerificationErrors(verificationResult); !!assertEquals(verificationResult.getVerificationStatus(), Status.PASS, "Verification status should be PASS"); !!assertEquals(verificationResult.getListOfErrors().size(), 0, "Failure details should be empty");!

Disable Web Analy3cs Tes3ng

engine.disableWebAnalyticsTesting();!

hUp://essenceo#esFng.blogspot.com/search/label/waat

hUps://github.com/anandbagmar/

hUp://rubygems.org/gems/WAAT

[email protected] [email protected]

@BagmarAnand