Cigniti & Experitest Co-webinar:

Mobile Banking Testing – Enabling Financial

Institutions to Succeed in the Digital Age

Jan 2015

Jan 2015

Guy Arieli

CTO

Experitest

Sachin Srivastava

BFSI COE Head

Cigniti Technologies

Speakers:

Agenda

Mobile banking industry: trends

Mobile banking industry: application type

Challenges in mobile banking testing

Experitest portfolio of tools for mobile banking

Cigniti framework for mobile banks

Case study

Demo

1

2

3

4

5

6

7

Trends in The Mobile Banking Industry

Banking – So Far

Branch

Banking

ATM

IVR

Internet Banking

Mobile Banking

Wearable

Gadgets

Changes leading the Banking Industry

Moore’s LawInternet of ThingsPredictive Banking

Image BankingOmni channel BankingAdoption of Social MediaCustomer Empowerment

Mobile Banking – The Evolution and Go Forward

7

115M Europeans will

bank using tablets in

2018, up from 19M in

2013

19% of US consumers

prefer to check their

bank balances via

mobile devices

70% of US consumers

are comfortable with

the increasing use of

virtual

communications in

banking in addition to

in-person

conversations

By 2017, more than

1B mobile subscribers

(15% of global

mobile subscribers)

will use mobile

banking

SMS Alerts

Account Queries

Funds

Transfer, Bill

Payment

Payment

Services

Loan request,

Service request

Service Convergence via Mobile Banking

ATM CALL CENTRE WEB CHANNEL

Mobile

• Cash withdrawal

• Account balance

• Transfers

• Deposit

• Bill payment

• Account balance

• Credit card balance

• Recent transactions

• Bill payment

• Lost credit card

• Account balance

• Bill payment

• Transfers

• Credit card balance

• Pay credit card

• Cash withdrawal

• Account balance

• Transfers

• Deposit

• Bill payment

• Credit card balance

• Recent transactions

• Pay credit card

Mobile converging most of the

services via technological

evolutions

Top Five Transactions via various Channels

Mobile Banking Functions

9

Account Information • Mini-statements and checking of account history, Balance Enquiry, Alerts on account activity or passing of pre- set thresholds and Monitoring of term deposits

• Access to Loan statements, Card Statements, Mutual funds/ equity statements and Insurance policy managementi

Bill Pay and Recharge

Services Utility bills, mobile bills, air ticket, bus ticket, DTH Recharge, Data Card Recharge, Movie Ticket etc.

and Bill Payment History

Transfer Funds

Via IMPS, Via NEFT and RTGS, Intra Bank transfers and VISA money transfers

Value added servicesCheque Book Request, Statement Request, Cheque Status Enquiry, IMPS Requests, Stop Cheque, Open a

Fixed Deposit, Open a Recurring Deposit , Apply Loans - Home, Car, Personal etc. and assisting customers

in location of ATM & Branches and various other Service Requests

Type of Application

Type of Application in the Banking industry

Customer Applications 1

Bank representative Applications 2

Web Applications3

Customer Applications

Focus on usability

Execution environment cannot be

controlled:

Different mobile OS

Different OS versions

Different device capabilities

Different network conditions.

1 Customer Applications

Bank Representative application

Bank Representative application

2

Usually a single device application

(usually iPad).

It's a rich in data

Requires longer training time

Web Applications

All the major bank have Web sites that

are adjust to mobile devices.

Web applications should be tested on

both Chrome and Safari with the

newest browsers versions.

Web Applications3

Challenges in Mobile Banking Apps Testing

Security

The Challenge

Strict security regulation on mobile application development and testing

Zero tolerant for security breaches

.

The Solution

Testing solution must be found inside the bank’s VPN, thus providing the same top-

security level as the bank’s other IT systems

Testing tool that does not require jail breaking or rooting of the devices under test

Long Tail of Customers

The Challenge Big banks service many users and need to support all OS , all OS version and devices

All the operating system in all versions need to be tested

include Blackberry and WP8-because also a small portion is a big number of users

.

The Solution

• Testing tool that support all OS: iOS, Android, Windows Phone and Blackberry

• Same tests should run – WITHOUT MODIFICATION – on all mobile OS

• Object repository that support native and web properties

• Hybrid app can be an amazing shortcut, in single development effort the bank can release an

application that will be supported on the majority of the OS, OS versions and devices.

• But as the user experience is extremely important, thy hybrid approach is only been used by the

small / mid banks.

Complex Data

.

The Solution

The Challenge the system under test configuration the user account configuration

How do I know that the information presented by the mobile application reflect

the bank backend databases?

Is the balance shown to the user is the actual balance?

Having a pool of accounts that can be used by the automation tests

Having a mechanism to lock a user for a specific test

Writing relative tests, the test will first learn the account status (like balance)

Closing the loop

.

The Solution

The Challenge How to close the automation loop?

To perform a test a user account details required

Usually creating new user is a very complex procedure that cannot be

performed as part of the automation flow

Terminal Emulation API that will enable the automation test to perform

query on the bank backend Mainframe.

Early Adoption

The Challenge Industry that constantly adopts new technologies.

Usage of new UI technologies and frameworks.

Support for new devices, new versions and new mobile OS

Short release cycles

.

The SolutionHighly spread agile development methodology

Continues integration for mobile

Mobile testing tool that can integrate to all major ALM assets

Continuous Integration For Mobile

Developers

Source Code

Repository

CI Server

YOURAPPExecuter Agent

Execute Test On Devices

x

Test Results

Need to scale

The Challenge Need to run regression and to execute the tests on 10's of devices

The number of tests and the execution time of single execution suite can take 10 Hrs.+

Number of engineers (automation and manual) can scale to hundreds.

.

The Solution Parallel/serial execution on numerous devices

Testing tool that enables:

High scalability

High Portability of test

Low maintenance of test

Top 10 banks span large continents

.

The Solution

On premise mobile device Lab that reside inside the organization VPN

The Challenge Testing team are spread globally

Ability to run tests on devices that are physically located in a different location

Due to security issues all devices and tests need to reside inside the bank’s VPN.

Fast speed response time when testing on a remote device.

Suitable Tools and Services

End-to-End Mobile Testing

App2Database

Legacy

App

Consumer Layer

Web services Web services

Middleware

App1

Apps, DB, Legacy

Layer

The Solution blueprint for end to end testing for Mobile Applications

• Integration testing of

Mobile Apps with

backend systems

(remove all stubbed

components)

• Validation of all the

Mobile UI and

Functionality with real

systems

2E2E Testing

• Web Services testing for project specific

functionality

• Web Services testing for all possible

positive and negative scenarios based on

the Interface Contract documents

• Web Services Testing at Unit level with

schema validation, data formats,

connectivity tests

1b Web Services Testing

• Mobile Application navigation testing with

wireframes documents and backend

stubbed

• Mobile Application UI testing with available

wireframes

1a Mobile Testing

Sample view of IT Architecture Proposed Test Model

Cigniti Enablers

Smart Tools-

Web Service

Validator

Automation

Centre of

Excellence

Mobile Solution

Centre on Cloud

Mobile Test

accelerators

(reusable test cases,

mind map)

Mobile Specific Tests

Non-Functional

Tests

Functional Tests

Mobile Specific Tests

Interruption Test Installation Test

Accessories Test Functional Testing

Upgrade Test

Drive Test

Memory Test

Performance Test

Network Type Test

Battery Performance Test

Carrier Testing

Geo-Fence Test

Details

Need for Mobile Test Automation

Challenges

• Ever shrinking timelines and minimal resources

• Testing of critical functionality on all the devices/OS is limited at the time of defect fixes or upgrades

• Defect leakage to production due to lack of sufficient

testing in Testing Phase

Attributes

• Manual Test suite development cost

• Automation Test suite development cost

• Cycle wise test execution cost

• Test suite maintenance cost

• Infrastructure cost

Influencing Factors

• Project schedules

• Processes

• Product / Application tech stack

• People expertise and skill set

• Price –investment on licenses etc.

Benefits

• Reduced Cost of Testing

• Reduced Time to Market

• Improved Product Quality

• Improved Test Coverage

• Improved Execution Cycles

End-to-End Product Suite - Manual, Automation, Monitoring, Load, Performer, Manager

• Manual testing tool

• Mini-recorder for automating short, repetitive, manual scenarios

• Report with screenshots of each step

• Manual to Automation Script Convertor

• Device Lab located onsite (in the customer VPN)

• Devices can be reserved and accessed by any remote user

• Used in conjunction with SeeTestAutomation/Manual for testing of remote devices

SeeTestManual

SeeTestCloud

• Automation tool

• Same test runs on different devices

• Plugs into all testing & CI environments:

SeeTestAutomation

SeeTest Network Virtualization

• Device performance under varying network conditions

• Simulating mobile network bandwidth latency and reception while on the move

SeeTestLoad• Real device generated mobile load

• Full integration to LoadRunner, Jmeter, etc.

Mobile Add-On for UFT• Full & complete integration into HP UFT:

• Record a test, edit it, run it, and view results

• Integration of object repository and more…

SeeTest ManagerFree Tool!

N

L

G

U

Demo

Case Study

• Client is a leading community bank in the US

• Key Requirement: Improve overall quality of mobile testing

and establish formal test processes

• Existing Application Environment: Applications built for iOS

and Android devices

• Engagement Model: Onsite – Offshore

Tools

• HP ALM

• SeeTest (from Experitest)

• Charles Proxy

• Cigniti Device Cloud

About the Client

Team Size and Duration

• Team Size: 8

• Duration: ongoing (expected 24 months +)

• Functionality Testing

• UI /Compatibility testing

• Usability testing

• Basic Application Performance Testing

• Drive Test

• Network Throttling tests

• Carrier Testing Type

Testing Types

Mobile Case Study - Leading Mobile Security ISV

Functional (UI/Compatibility) Testing Framework

Network

Real devices

Real User Perception via Test Automation Tool

Test Execution report

Graphs Reports Metrics

• Execution Reports and Dashboards for

making informed decisions

• Test application on carriers and Wi-Fi

networks

• Various types of UI, Compatibility,

Navigation, Usability Specific tests

• Customized Cigniti’s Test Automation

Framework

• Automated tests for any change on the build

• Best practices, entry and exit criteria-

improved test processes

Application Under Test

Challenges and Solution Provided

• Lack of visibility of product quality ahead of the release

into the production

• No template, guidelines, checklists, best practices to

ensure high levels of testing productivity and

effectiveness

• High testing costs and long test cycles with defects

slippage to production

• Long test cycles due to lack of test automation of

regression cycles

• Identified the application user scenarios and built a comprehensive

end to end mobile test strategy

• Selected Charles proxy to capture logs and perform Network

throttling tests

• Conducted tool feasibility analysis and selected Experitest SeeTest as

the automation tool for their native mobile application

• Leverage real devices from Cigniti’s Device Cloud to perform testing

• Built a framework with portability, re-usability, easy maintainability,

customized reporting engine and test data from various sources

Key Challenges Solutions Provided

Key Benefits Delivered

• Leveraged Cigniti mobile automation framework to optimize the total effort and reduced total effort by 40%

• Reduction in test cycle execution time by 60%

• Built scripts that could be leveraged across platforms (iOS and Android)

• 40% of carrier specific testing cost was reduced cost by leveraging Cigniti’s device cloud

Q&A

Thank you