THE WHAT, WHY & HOW OF (WEB) ANALYTICS TESTING

Anand Bagmar Software Quality Evangelist

@BagmarAnand about.me/anand.bagmar

ABOUT ME

What do you expect from this session?

Why do we do Testing?

HOW DO WE VALIDATE?

What are Heuristics?

Web Analytics

WHAT IS WEB ANALYTICS?

WHY DO WE NEED WEB ANALYTICS?

WEB ANALYTICS SOLUTIONS

Google Analytics

SiteCatalyst

AWStats

WebTrends

….

Testing Web Analytics

LEARN & OPTIMIZE

TESTING AT THE REPORT LEVEL

Pros

¨ Ensure report is setup correctly

Cons

¨ Licensing

¨ Reports not yet setup

¨ Validate all requests are sent / captured

Biggest Problem

It is TOO LATE!

The Solution

Web Analytics Testing Challenges

MANUAL

REPEATING OVER-AND-OVER AGAIN

WAAT - Web Analytics Automation Testing

WHAT DOES WAAT DO FOR ME?

q Plugs into existing test framework

q With minimal changes

q Web Analytic tool independent

q UI Driver framework independent

q Browser independent

FLAVORS

ARCHITECTURE

WAAT-JAVA

q Original flavor

q Supports 3 plugins

q Omniture Debugger

q HttpSniffer

q  JsSniffer

q Available on github

WAAT-RUBY

q Ruby gem implemented over WAAT-Java

q Uses RJB – Ruby-Java-Bridge

q Supports 2 plugins

q HttpSniffer

q  JsSniffer

q Available on github and rubygems.org

OMNITURE DEBUGGER

OMNITURE DEBUGGER

Pros

q OS independent

q Run using the regular test-user

Cons

q Browser dependent – need to implement ScriptRunner for the UI-driver in use

q Web-Analytic solution dependent – Adobe Marketing Cloud / Omniture SiteCatalyst

HTTPSNIFFER

Pros

q Web-analytic solution independent

q Browser independent

q UI-driver independent

Cons

q 3rd party libraries are OS Dependent

q HTTPs not supported out-of-the-box

q Run tests as ‘root’

JSSNIFFER

Pros q Web-analytic solution independent

q Browser independent

q HTTPs supported out-of-the-box

q No dependency on any 3rd party library

Cons q Need to write JavaScript to get the URL from the browser

q UI-driver dependent

Is that all to

Web Analytics?

The new “kids” in town

IoT & Big Data

Some popular use cases

IOT – INTERNET OF THINGS

Opportunity to

¨ Create new value propositions

¨ Be Innovative

¨ Be Creative

IOT – HOW TO DELIVER VALUE?

¨ Automate the manual processes

¨  Integrate data capabilities

¨ Collect – integrate from various sources

¨ Repeat collection – automate the collection

¨ Analyze – manual & machine learning

¨ Optimize / Pivot

¨ Repeat

IOT – CHALLENGES

¨ Federated devices

¨ Different types of networks

¨ Different communication channels

¨ Physical (hardware) & Virtual (software)

IOT – BIGGER CHALLENGES

¨ Too many devices

¨ Lots of data

BIG DATA …. IS GETTING BIGGER

¨ Volumes of data generated ¨  A jet engine generates 1TB of data per flight.

¨  A large refinery generates 1TB of raw data per day.

¨  As cars get smarter, the number of sensors is projected to reach as many as 200 per car.

¨  Sensors of all types will generate immense amounts of data. In fact, analysts estimate that by 2020, 40 percent of all data will come from sensors.

¨  IoT leads to massive volumes of data

http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper

IoT is about Data!

TO GET VALUE FROM IOT …

¨ Collect

¨ Analyze

¨ Predict

¨ Plan

What does this mean for Testing?

TESTING OPPORTUNITIES

We need to build capabilities to validate –

¨ Data collection is working well

¨ From all sources

¨ Analyzing data, capturing patterns and trends

¨ Optimize business value

¨ Create new opportunities and value propositions

RESOURCES

¨ https://github.com/anandbagmar/waat

¨ https://github.com/anandbagmar/waat-ruby

¨ http://essenceoftesting.blogspot.in/search/label/waat

¨ http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper

¨ http://www.dezyre.com/article/how-big-data-analysis-helped-increase-walmart-s-sales-turnover/109

52

@BagmarAnand about.me/anand.bagmar

THANK YOU