Mobile App Testing Strategy by RapidValue Solutions
-
Upload
rapidvalue-solutions -
Category
Software
-
view
540 -
download
0
description
Transcript of Mobile App Testing Strategy by RapidValue Solutions
© RapidValue Solutions
Mobile Application
Testing Strategy
© RapidValue Solutions
Mobile App Testing Strategy – Adopted to Deliver Quality Service
and Provide Utmost Consumer Satisfaction with the Perfect App !
There has been an increase in the adoption of smartphones, tablets and several mobile
devices with the passage of time. And it has led to an enormous growth of mobile
applications in recent years. Mobile device is considered to be the primary medium of
interaction for the customers and also, businesses worldwide. And mobile applications are,
actually, driving the communication.
People, generally, do not give much importance to mobile application testing because of its
expensive nature. But it is very much essential to ensure that the consumers have a great
experience, every time they use the application.
The aim of mobile application testing should be to acquire knowledge about the quality of the
service that you are offering. Does it work properly or not? Will it provide services as per the
expectation of the customer? These questions need to be answered to ensure that the
customer comes back to you, for your service again.
Mobile testing is becoming more and more complicated and complex with each passing day.
Strategies are invented and used to simplify the mobile application testing.
This presentation primarily addresses, the strategy to be adopted in mobile app testing, the
types of mobile app testing and the stages to be followed, before the application is set live.
© RapidValue Solutions
The main objective of mobile application testing is to help customers to gain assurance that the
software applications are performing and functioning as expected with the devices or with the
several other applications of the vital players in the mobility ecosystem.
Objectives of Mobile Application Testing
Parameters Details
Functionality UI responsiveness, Peak load performance, Operational times
Performance Network type (Wi-Fi, 2G, 3G, 4G), Impact of connectivity issues
Network Security of data in motion and data at rest
Security Mobile platform compatibility, Device model compatibility,
Backward compatibility with previous app version
Compatibility Marketplace guidelines’ compliance for Apple and Google stores
Conformance Compare the developed product with user stories
Usability User Experience
Installation and
Provisioning
Installation process, Un-installation process, User provisioning and de-
provisioning
Service Testing Check that the services are up and running and also measure the
performance
Resource Testing Battery performance, Memory usage, Local database growth and Garbage in
memory, come under this process
Load Testing on Mobile With heavy loads of images, videos and also loading the app with more and
heavy functions, use third party application like blaze meter to generate load
Localization With this, the QA team checks the application’s multilingual capability
© RapidValue Solutions
Emulators Cloud Testing Solution Physical Device
o All applications can be
deployed and tested on
emulator without
investing in mobile
handset for various OS
o Emulators are mostly
available free, and we
can also perform UI,
Stress and performance
testing on that
o 30-40% test
o The Mobile devices can be
accessed through web
interface i.e. the browser
o Application can be
deployed, tested and
managed
o Automation module is
available and the solution
is secure, if private cloud is
used with no maintenance
o Up to 100% test
coverage can be
achieved
o Real device testing will
give most realistic view
of test results
o All possible types of
testing activities can be
performed including
those, that are
dependent on hardware
Mobile Testing Strategy is an essential element to ensure the smooth functioning of any
application. Emulators are considered to be vital testing platforms while Cloud testing solution is
very much secure. Real Device Testing is said to yield the most realistic and practical results.
Mobile Testing Strategy
© RapidValue Solutions
• Different Form Factors
• Wide Variety of devices from different manufacturers, incase of Android
• Different versions of Mobile OS
• Different Screen Density
• Infrastructure and Connectivity
• Mobile testing tool availability – still on nascent stage
• Cloud testing platform solves some of these issues, however:
o It is very expensive
o Lock – in with the platform
o Scripts cannot be used outside of the platform
o OCR based platform. Because of automation, it is time consuming
Challenges in Mobile Testing
Mobile applications are considered to be “game-changing” technologies, which have further led to
the development of various unique and beneficial testing challenges. The growth of the application
market has increased the customer’s expectation of obtaining quality service. Companies are also
coming up with new apps in the marketplace. Hence, challenges of mobile testing are also
increasing each day. Whether it is an enterprise app or a consumer app, companies are finding
out ample opportunities to boost their revenues and reaching out to a vast range of consumers.
This ambition leads to a greater development of mobile testing strategies which face certain
challenges:
© RapidValue Solutions
• iOS 7 and above
• iPhone 4, iPhone 4s
• iPhone 5, iPhone 5s, iPhone 5c
• iPhone 6
• iPad Air, iPad2
• iPod touch 4th generation
• iPod 5th generation
• Android 4.x and above
• Samsung galaxy s4, s5
• Samsung Note 3
• Nexus 5
• HTC desire 816,one
• Moto X
• Sony Xperia Z
Popular Mobile Devices for Testing Applications
A great application certainly needs a greater customer service solution. And it is recommended
for the application to support all the iOS and Android devices. The following are some of the
devices and versions, on which its good to test your application.
© RapidValue Solutions
The following statistical chart gives a clear picture of the screen sizes and densities for Android:
Screen Sizes and Densities for Android
© RapidValue Solutions
Simulator
Testing
Device Testing
(On Wi-Fi Networks +
Network Simulation
Tools)
Device Testing
(In-Network)
• Supports only device dependent use cases
like camera, accelerometer, GPS etc.
Device Testing (In –Network)
• Ideal during the development phase
• Best and economical option for functionality
testing
Simulator Testing
Device Testing ( Wi-Fi )
• Use proxy tool Charles proxy to monitor the
load and performance of the application
There are certain distinctive features of Simulator and Device Testing. They are summarized
below:
Simulator vs. Device Testing
© RapidValue Solutions
Advantages of Cloud Testing:
• Variety of plans available, such as hourly, monthly or yearly subscriptions
• Test incoming calls and sms without having any cellphone connection
• Wide variety of devices
• Automated test execution, can be played as videos
• Test can run on several devices, in parallel
• Build integration
The performance testing is conducted on the applications and services provided via cloud
computing. Cloud testing ensures optimal performance and scalability under a wide variety of
conditions. Cloud testing monitors and reports on the real-world user traffic conditions.
Cloud Testing
User interacts with the device through studio or web front
provided by the cloud provider
Server receives the request and
delegates that to the actual device
Device sends back the result and that would be displayed
on the studio or web front
© RapidValue Solutions
The Application testing commences with Unit Testing and then proceeds with Functional Testing,
Regression Testing and UI Testing. This is further followed by Automation Testing, Performance
& Load Testing and the final stage is the Security Testing.
Application Testing Flow
Unit Testing
Functional Testing
Regression Testing
UI Testing
Automation Testing
Performance & Load Testing
Security Testing
Performed by developers before releasing the build
QA team perform the functional testing based on the test cases that are created using user stories
Regression testing is a suite of testing, that include smoke test and execution of regression test cases
UI testing is a usability testing, focusing on the UI design of the application, carried out by UI & QA teams
Once the development is in shape, all the regression test cases should be automated
QA team should check the UI response, peak load, transaction time as part of the performance testing. This should be conducted after every 3 sprints
Security testing should be considered for data security, authentication and authorization
© RapidValue Solutions
App Store Play Store
Terms & Conditions
Functionality/Reliability
Metadata/Ratings & Location
Intellectual Property Rights
UI conformance & Contents
Spam & Placement
Ads & System Interference
Contents
Personnel & Confidential Information
Content Ratings
Conformance testing is considered to be critical. Conformance is essential and crucial to
determine whether the product or the service meets the specified standards. Validation of the
App Store and provision of the Google Store Guidelines are the integral part of the effective
process, which plays a pivotal role.
Conformance – Validate App Store & Google Play Store Guidelines
© RapidValue Solutions
• The network testing should be carried out in physical device by the QA team to ensure the
network performance under various technologies like GSM and CDMA
• Cloud testing should also be leveraged to test it in the real environment
• Separate test case suite should be designed to test the various signal strength under tele-
signals using 4G,3G,2G etc. and the Wi-Fi signals
• Charles proxy, a third-party tool can be used to control the network bandwidth by creating
device and Charles proxy operate under one network
• Applications should be tested under multiple signal strength and quality performance under
live environment is ensured
The most important thing is to ensure smooth and seamless network performance which can be
evaluated with the help of efficient Network Testing. Performance is evaluated under
technologies like CDMA and GSM. The applications are tried and tested under live environment
to ensure utmost customer satisfaction.
Network Testing
© RapidValue Solutions
• Usability testing should be carried out by the following testing:
o Validation of gray scale wire frames
o Testing the compatibility with all devices and browsers
o UI testing that focuses on user acceptance
o UI comparison testing with benchmarked applications & standards
o Review testing before every UAT by customer
• Usability test should be conducted by UI/UX and QA team
Another important testing procedure is the Usability Testing which should be ideally conducted
by UI/UX team and the QA team. It is essential to test the compatibility factor with the various
devices and browsers. UI Comparison Testing is also needed to know whether the benchmarked
App is in conformity with the existing standards.
Usability
© RapidValue Solutions
• SOAP UI is a free testing tool, based on open-source standards as XML, XPATH and
GROOVY (Java). It is stable and robust too.
• Using SOAP UI, tester are able to perform
o Web service regression testing
o Web service test automation
o Web service load testing
• SOAP, REST service will be covered as a part of service and all the responses will be
evaluated using tools like GET, POST etc.
Service testing will enable the providers to evaluate the performance of the application and work
on it, according to the test results. It is essential to deliver a comprehensive testing as a service
platform. There has been a rapid growth in the demand for effective application testing tools and
services that will leverage mobility services. Service testing is performed to deliver effective and
high-quality applications.
Service Testing
© RapidValue Solutions
• Resource utilization like memory, CPU, etc. is measured while doing Performance
Testing.
• Set criteria for Application/Device performance, Server Performance and Network
performance
o Quadrant and Linpack is used for measuring performance
o Lint (come with Android SDK), static code analyzer is used for performance and
optimization
o OC Lint (IOS), Static code analyzer is used for performance and optimization
o iOS instrumentation tool is used to find the leaks and memory foot print in iOS
application
Testing is, indeed, an important aspect of development. It is crucial and critical for both, creating
new applications and modifying the existing ones. Utilization of resources is measured
effectively while going through the process of Performance Testing. Linpack and Quadrant prove
to be effective in measuring performance of the application.
Resource Testing
© RapidValue Solutions
There are two methods to perform load testing - Manual & Automation
Automation Testing:
o Load Testing is trying to simulate the work load of various user scenarios to the
applications with third Jmeter
o Testing the load within a specific pagination by monkey testing
Manual Testing:
o The application under test are loaded with large images (high resolution) and videos
to evaluate the peak point where the application can handle the maximum load
o Load testing are performed under various signal bandwidth to understand the
minimum network signal strength to run the application with standard usage
Load Testing
© RapidValue Solutions
Automation testing is the key testing strategy to automate Regression testing which include all
functionalities:
o Appium is a good tool for Automation testing. And automation should start after the
development is in the right stage
o Appium is an open source, cross-platform test automation tool for native, hybrid and
mobile web apps. It is tested on simulators (iOS, FirefoxOS), emulators (Android),
and real devices (iOS, Android, FirefoxOS)
o Automation testing should be a part of every sprint to reduce the effort of Regression
testing and Smoke testing
o Automation frame should be designed after couple of sprints
One of the best characteristic of Automation Testing is that it does not require any manual
intervention while executing the particular test. The test has certain benefits like greater
consistency and the reduced nature of the repetitive work. Automation testing can be relied upon
to provide the ease of accessibility to information about the tests.
Automation Testing
© RapidValue Solutions
• Automation of regression suite can be incremental starting from 1st sprint
• Automated regression suite to check the impact of latest sprint deliverables on the
previous sprint(s) deliverables, for continuous integration
• Re-use the same (with additional manual\automated tests) for hardening sprint before
application is launched
Regression Testing is another essential procedure which deals in rerunning test cases just to
ensure that the software changes do not have any side-effects. Regression Testing has become
more effective and efficient with time. This testing is critical to check or detect whether new
issues have cropped up after the change has been applied. Regression Testing is critical for
large applications as it is a little difficult to figure out whether any part of the application has
given rise to any issue or problem due to the change.
Regression Testing
© RapidValue Solutions
• Supporting types: Text, Images, Videos etc.
• Size: Maximum size supported for each category
• Size: Beyond maximum supported size for each category to see the stability / Large volume
of data and measure performance
• Mix of different data types with maximum size
• Negative Data (Invalid data): The application is not intented to support
• Data and network (2G,3G,Wifi) combination
• Data and Device type/OS combinations
The following gives a detailed description of the data types, data sizes, blend of different data
types and sizes, supporting formats etc. It also provides information regarding the data and
device/network combinations. One obtains knowledge about negative data also, which is a
particular data type or data size that the application does not support.
Test Data – Type and Size
© RapidValue Solutions
• The app diagnostics and crashes on iOS and Android can be monitored using Crittercism
• Google Analytics tool helps to monitor the exit points and identify the reason. It collects
user interaction data, manages how the data is processed and provides a report.
The post-production stage is equally important as the pre-production and the production stage.
A report has to be provided after collecting user-interaction data and managing the data
procedure. Crittercism and Google Analytics provide immense support for monitoring the
application.
Post Production
© RapidValue Solutions
• Hardening sprint(s) - QA only sprint(s) for
Go Live
• On actual environment and configurations
• Regression suite to focus on system level,
Integration, Impacted (changes) areas,
performance and security
• Real-time data, text, videos and images
with maximum supporting size
• Go Live check lists based on the
sprint level confidence
o Entry Criteria
o List of devices and configurations
o Actual environment &
Infrastructure, Test Data
o Regression Suite and Use cases
o Installation/ Uninstallation
o Exit criteria
Testing stage holds utmost importance. It gives the clear picture of the task that has been carried
out with such great effort. Testing is done in actual environment and configurations to know
whether the application supports it or not.
Go Live – Testing
© RapidValue Solutions
The growth of the application market has increased the customer’s expectation of obtaining
quality service. Companies are also coming up with new applications in the marketplace.
Hence, challenges of mobile testing are also increasing day-by-day. Mobile application
testing is, indeed, an integral and essential part of the production process. It is a procedure
by which the application software, that is developed for the mobile devices, is tested for its
supportive feature, its functionality, usability and for its consistency. The main objective of
mobile application testing is to help and assist customers to gain assurance that the software
applications are performing and functioning as expected. Mobile testing strategy is an
essential element to ensure the smooth functioning of any application.
• The most important thing is to ensure smooth and seamless network performance which
can be evaluated with the help of efficient Network Testing.
• Service testing enables the providers to evaluate the performance of the application and
work on it, according to the test results.
• The performance testing is conducted on the applications and services provided via
cloud computing.
• The Usability testing is equally essential to test the compatibility factor with the various
devices and browsers.
• Service testing, Load testing, Regression testing and Automation Testing - all the testing
procedures form the significant part of the application testing strategy which is crucial
and vital for the production of an application.
Conclusion
© RapidValue Solutions
Thank You
© RapidValue Solutions
About RapidValue
RapidValue is a leading provider of end-to-end mobility solutions to enterprises worldwide.
Armed with a team of 250+ experts in mobility consulting and application development,
along with experience delivering over 400 mobility projects, we offer a range of mobility
services across industry verticals. RapidValue delivers its services to the world’s top
brands and Fortune 1000 companies, and has offices in the United States and India.
www.rapidvaluesolutions.com www.rapidvaluesolutions.com/blog
+1 877.690.4844 [email protected]
This document contains information that is confidential and proprietary to RapidValue Solutions Inc. No part of it may be used, circulated, quoted, or reproduced for distribution
outside RapidValue. If you are not the intended recipient of this report, you are hereby notified that the use, circulation, quoting, or reproducing of this report is strictly prohibited and
may be unlawful.
Disclaimer:
Presentation by
Rinish KN
CTO, RapidValue Solutions