Mobile Testing Center of Excellence GroupPerfectomobile

Real Device Mobile Monitoring

The fundamentals of true synthetic mobile monitoring

Presenters

Eran Kinsbruner

Mobile Quality Evangelist

Director Product Marketing

HP Discover,

Vegas

June 11-13/2013

Silver Sponsors

Amir Rozenberg

Director of Product

Management,

Mobile Monitoring &

Performance Testing

Santa Clara,

June 19-20/2013

Silver Sponsors

Mobile Application Qualityacross the mobile application lifecycle

TestingTesting AutomationAutomation PerformancePerformance

MonitoringMonitoringALM

IntegrationALM

IntegrationContinuous IntegrationContinuous Integration

Public Cloud Private Cloud

∙ Founded 2006

∙ 90+ employees

∙ Office in Boston, London, Israel

∙ Over 600 customers

∙ Fortune 500, Telcos, Developers

∙ MobileCloud Platform + Mobile Quality Product Suite

Mobilizing your Existing Software ALM

Solution - The MobileCloud™ Platform

Agenda

Background

• The market need

• Approaches for monitoring

Mobile Mobile Monitoring

• Challenges in mobile monitoring

• The need for real insight

Solutions

• Requirements for an applicable solution

� Richness

� Immediacy

� Simplicity

� Relevancy

Ability to offer targeted

services

• Context

• Time

• Location

• Proximity

• Behavioral

preferences

• Past choices

What makes mobile such a unique channel?

� Improve ability to target end users, achieve business success criteria

� IT $$ flowing to mobile

Mobile Strategy increasingly critical

• Mobile commerce sales will reach $12 billion this year. Sales will increase exponentially

over the next few years and reach $27 billion by 2016 (Forrester Research, May 2013)

• 72% of consumers now use banking apps at least once a week, while 56% report using

a credit card app (or mobile Web site) every week (Mojiva, May 2013).

• Mobile influences 45% of in-store shopping (inMobi, May 2013)

Mobile Leading as Preferred Channel for Banking

Mobile Becoming Close Companion

• 91% report their mobile device within 1 meter reach 24X7

• 95% of cell phone owners never leave home without carrying their phone*

End User Have Demanding

Expectations, Failure is Expensive

Source: Compuware

Delivering Optimized Experience is Challenging

• Introduction of Agile methodologies

• Accelerated continuous delivery cycle

• The rise of the DevOps team

• Complexity of quality control

• Features, devices, scenarios

• Appearance of availability and

response time issues in the field

• Reliance on 3rd party technology &

content

Challenges across the organization

LoB

Carl Howe, director, consumer research for Yankee Group

Operations

Dev & QA

• Meet business criteria (RoI, brand, adoption,

productivity)

• Leverage the mobile opportunity

• Ability to scale the solution

• Ability to leverage existing investment into ALM

• Insight into solution health across all users,

devices, locations and carriers

• Avoid expensive recurring operational incidents

• Apply existing operations tools, policies &

processes

• Ensure solution ability to handle load

• Address production issues quickly

• Leverage same scripts through lifecycle

• Share scripts, devices, screenshots etc. globally

• Reduce risk from rapid agile development

cycles

"The explosion of smartphones and

mobile applications has created a

performance monitoring nightmare for

businesses developing or delivering

mobile content and applications.”

Mobile Performance Impacts Business Success

Walmart reported 1 second

performance improvement in their

online presence can contribute up to

2% in conversions1

Amazon reported every 100ms

latency could cost them 1% revenue2

Poll- Service level expectations

Major US Bank:

“We have support staff for web issues. On mobile, users rip us apart on social

networks”

“If you get bad reviews, don’t bother fixing your app, build a new one!”

Poll- Service level expectations

Meeting Users’ Expectations

Performance Testing Monitoring

Monitoring: Different Perspectives

More 3rd parties

Server

RUMClient

RUM

Synth

etic

Synth

etic

Real User Monitoring and Synthetic Monitoring are complementary

Monitoring Methodologies

• Apps RUM Insight, incl. crashes, screenshots etc.

• Instrumentation

• ”One Web” RUM

• Instrumentation

• True insight

• End-user experience

• Coverage, maintenance

• Scalability

• Locations

• Root cause

• Execution of thick clients

• True Insight Browser Monitoring: FF/webkitover LAN

Browser Monitoring: FF/webkitover LAN

Real device MonitoringReal device Monitoring

Real device RUM:

instrumented apps

Real device RUM:

instrumented apps

Browser RUM (JS)Browser RUM (JS)

RUM

Synthetic

Real

Devices

Browser

• Proactive

• Pre & post

Production

• Sampled

• Clean room

• Root-cause

• Reactive

• Business

impact

• Real world

• Coverage

• Analytics

• W3C strategy

The need for real devices throughout the mobile ALM

Mobile App Ex.:

Insurance

Feature Real device Browser

Location, time � �

Address book,

camera� �

Local storage � �

Mobile Web Ex.:

Weather

Real Device Examples

HTML5

storage

Rendering

issues

Rendering

and

functionality

issues

Navigator

API

redirection

“ 2FA authentication is a regulatory requirement for logging into

Internet banking (Mobile phone OTP). …we have no way to

monitor or get past this stage.

Since the industry is heading this way, we are at a serious

disadvantage of not being able to monitor many critical

functions. I’m sure we’re not the only bank who is interested in

monitoring this.”

Director of Service Assurance

Leading Global Bank: Interoperability use case

�Monitor using real devices

� Measure the metrics mobile users care about

� Ensure the monitoring solution SLA is better than your service

target

� Enable mobile monitoring integration into proven enterprise-

grade solutions

�Drive mutual language between Ops and Dev

� Common tools and workflows are key to reduce MTTR (mean time to

resolution)

4 Key Requirements for Mobile Monitoring

What is the most important requirement from a

synthetic mobile monitoring solution?

Utilize real devices on real networks

Integrate into proven monitoring solutions

Provide integrated insight with RUM

Controlled maintenance overhead

High solution SLA

Poll

Poll- Synthetic mobile monitoring requirements

Applicable Mobile Monitoring

Leverage

Real Devices

Reduce

solution

Overhead

Leverage

Proven

Platforms

Address

Solution SLA

Perfecto Mobile Monitoring Product Introduction

“Introduction to MobileCloud Monitoring - Real Device Monitoring Solution”

June 26th 2013, 11 AM EST

Hosted by Perfecto Mobile and SkyIT

Register at:

https://www3.gotomeeting.com/register/208623254

Upcoming Mobile Monitoring Webinar

In this white paper, you will be able to gain insights

into the following mobile monitoring aspects:

Key building blocks for a mobile monitoring solution

Best practices and process to build a mobile monitoring

solution

Best practice solution design overview

LINK to download (Registration required):

http://help.perfectomobile.com/article/AA-01854/

Mobile Monitoring - White paper

amirr@perfectomobile.com

Twitter: @Amir_RozenbergErank@perfectomobile.com

Info@perfectomobile.com