The What, Why and How of (Web) Analytics Testing (Web, IoT, Big Data)
The what, why and how of web analytics testing
-
Upload
anand-bagmar -
Category
Technology
-
view
2.824 -
download
1
description
Transcript of The what, why and how of web analytics testing
The What, Why & How of
Web Analy3cs Tes3ng
Anand Bagmar So#ware Quality Evangelist
Principal Consultant, ThoughtWorks
Why do we do Tes3ng?
What is Web Analy3cs?
Why do we need it?
Web Analy*cs Solu*on Providers
Adobe Marke*ng Cloud / Omniture
SiteCatalyst
WebTrends
AWStats
Google Analy*cs
StatCounter
…
How to test Web Analy3cs?
The end report
Pros
• Ensure report is setup correctly
Cons • Licensing • Reports not yet setup • Validate all requests are sent / captured
Tes3ng at the report level
At the origin!
The problem
Manual
The challenge
Repea3ng over-‐and-‐over again
The solu3on
What does WAAT do for me?
• Minimal changes to exisFng test framework • Does not depend on junit / testng / … • Web AnalyFc tool independent • UI Driver framework independent • Browser independent
Flavors
Architecture
• Original flavor • Source code + jar • Supports 3 plugins • Omniture Debugger
• HUpSniffer • JsSniffer
• Available on github
WAAT -‐ Java
• Ruby gem implemented over WAAT-‐Java
• Uses RJB – Ruby-‐Java-‐Bridge • Supports 2 plugins • HUpSniffer • JsSniffer
• Available on github and rubygems.org
WAAT -‐ Ruby
Omniture Debugger
Omniture Debugger How does it work? Pros • OS independent • Run using the regular test-‐user Cons • Browser dependent – need to implement
ScriptRunner for the UI-‐driver in use • Web-‐AnalyFc soluFon dependent – Adobe MarkeFng
Cloud / Omniture SiteCatalyst
HHpSniffer How does it work? Pros • Web-‐analyFc soluFon independent • Browser independent • UI-‐driver independent Cons • 3rd party libraries are OS Dependent • HTTPs not supported out-‐of-‐the-‐box • Run tests as ‘root’
JsSniffer How does it work? Pros • Web-‐analyFc soluFon independent • Browser independent • HTTPs supported out-‐of-‐the-‐box • No dependency on any 3rd party library Cons • Need to write javascript to get the URL from the
browser • UI-‐driver dependent
• Download zip from github / rubygems.org
• Setup
• jpcap / ScriptRunner setup
• Update exis*ng tests
How to use WAAT?
Update exis3ng Tests • Import relevant packages / gems
• Define & provide parameter values
• Ini*alize Engine
• Enable Web Analy*cs Tes*ng
• Define Test Data
• Verify Web Analy*cs repor*ng using WAAT
• Valida*ons
• Disable Web Analy*cs Tes*ng
<Section !!!actionName="OpenUpcomingPage_HttpSniffer"!!numberOfEventsTriggered="1"!
!tagList="pageName=upcoming:index||!! ! !cc=USD||!! ! !ch=upcoming||!! ! !events=event2||"/>!
Define Test Data
Ini3alize Engine
!engine = getInstance(!! ! !webAnalyticTool, !! ! !inputFileType, !! ! !keepLoadedFileInMemory, !!! ! !log4jPropertiesAbsoluteFilePath!! ! !);!
Enable Web Analy3cs Tes3ng
engine.enableWebAnalyticsTesting();!
Verify Web Analy3cs Repor3ng
!Result verificationResult = !!! !engine.verifyWebAnalyticsData (!! ! !inputDataFileName, !! ! !actionName, !! ! !urlPatterns, !! ! !minimumNumberOfPackets);!
Cart.checkout();!
Sample Valida3ons assertNotNull(verificationResult.getVerificationStatus(), "Verification status should NOT be NULL"); !!assertNotNull(verificationResult.getListOfErrors(), "Failure details should NOT be NULL"); logVerificationErrors(verificationResult); !!assertEquals(verificationResult.getVerificationStatus(), Status.PASS, "Verification status should be PASS"); !!assertEquals(verificationResult.getListOfErrors().size(), 0, "Failure details should be empty");!
Disable Web Analy3cs Tes3ng
engine.disableWebAnalyticsTesting();!
hUp://essenceo#esFng.blogspot.com/search/label/waat
hUps://github.com/anandbagmar/
hUp://rubygems.org/gems/WAAT
[email protected] [email protected]
@BagmarAnand