| 2
1. What is Mobile?
2. Software Testing Overview
3. Mobile App Testing
4. Mobile Platforms
5. iOS vs. Android
6. Wrap Up & Questions
Agenda
|
What is Mobile?
4
• Mobile Device – portable device powered by a battery that is carried
around by the user and uses wireless communications.
• Mobile activity primarily consists of two categories
– Web: Customized websites intended for viewing on smartphones and tablets over a cellular or wireless internet connection.
– Mobile/Native apps: Software programs developed specifically for
smartphones and tablets. They access data over a cellular network
or wireless internet connection.
|
Mobile Web vs. Native App
5
Some Companies use a Hybrid of Native App and Web for Mobile
– ESPN ScoreCenter
- A mobile app with updated scores within the app
- Directs users to its mobile website for further articles
– Pandora
- A native app for listening to music, with small ads
- HTML5-powered website to stream music and
eventually deliver more content
– For use on mobile and other devices
| 6
Smartphones have full operating
systems, that can be updated,
making them almost multi-purpose
computers whereas features
phones are more limited. Today’s
smartphones boast a multitude of
operating systems. The most
popular ones in the US are
Google’s Android operating
system, Apple’s iOS, and RIM's
Blackberry
Smart Phone vs. Computer
|
Mobile Market Trends
7
• Web and native apps were developed due to the
growth of the Smartphone and tablet market and
their ability to keep users connected more
consistently
• Smartphone applications are ubiquitous and
mainstream (banking/finance, games,
productivity, fitness)
• Abundance of free apps
Opportunities for testing “…because your app is free and has a built-in market doesn’t
mean you can lower your standards or ignore what end users
like. .. You need to fully understand how the program can be
useful on-the-go and focus solely on the features that would be
handy and increase productivity in a mobile, untethered setting.”
http://blog.utest.com/why-testing-your-business-app-is-
important/2012/05/
Mobile Market
| 8
Mobile App Success
To succeed Mobile should provide value to the
user based on the following types of services:
Games: Most downloaded app category -65%
(Nielsen Wire) – Ux/entertainment and
performance
Social: latest news and share content with
friends
Retail: location intelligent, m-commerce
Media: news stream, offline access to content
Travel: mobile bookings, check-ins, maps,
deals
http://www.razorianfly.com/2012/03/07/heres-the-itunes-app-stores-100-all-time-top-apps-direct-links/
Mobile Success
|
Why Test Mobile Apps?
9
Mobile Apps and Usage are a Big Deal 49.7% of US mobile subscribers owned Smartphones in
February 2012 (http://www.newmediatrendwatch.com/markets-by-country/17-usa/855-
mobile-devices)
• By 2015, International Data Corporation (IDC) predicts
that 182.7 billion mobile apps will be downloaded, close to a
1600% increase from the 10.7 billion apps downloaded in
2010
• Diverse user base, multiple devices, manufactures,
platforms and fragmentation
Our focus and goals
• First Impression for the brand/product is important in the
app stores
• Identify technical, design and functional challenges early
to help reduce the cost of failure and repair
• Evaluate readiness for release
|
Definitions & Terminology
11
• What is an App?
– Container for a product and/or service that is of value it its users.
• What is an App’s Purpose?
– It should provide value to the user based on a type of a service
(e.g., email, social media, game or productivity app)
• What is Quality?
– Value to some person (Jerry Weinberg)
Different users have different opinions about the app quality
Ratings
User feedback
• What is a Bug?
– Any threat to the value of the product to any stakeholder who
matters (James Bach)
|
What is Testing?
12
Testing - An empirical, technical investigation conducted to
provide stakeholders with information about the quality of
the product or service under test. - Cem Kaner Testers design and run tests in order to gain useful information
about the products quality.
http://www.testingeducation.org/BBST/exploratory/
Exploratory Testing - An interactive process of simultaneous
learning, test design, and test execution. - James Bach
|
What is Testing?
13
Different objectives require
different testing tools and
strategies and will yield
different tests, test
documentation and test results. Black Box Software Testing –
Foundations
Cem Kaner & James Bach
http://www.satisfice.com/tools/satisfice-tsm-4p.pdf
|
Quality & Mobile Context Driven App Testing
15
•Quality - is Value to Some Person (Jerry Weinberg)
– The value being tested may not be of value to you
– As a Test professional, you test for that value(s)
•Testing
– Whether it’s mobile, desktop or other types of testing the core approaches are
the same
•Context Driven Testing (CDT)
– There are no "best practices" in testing, but rather testing is a set of skills that
allow the tester to select or invent testing practices to suit each unique situation.
•Mobile Context Driven Testing (MCDT) - applies the mobile world
- Project Scope (Functional, Showstoppers/Crashes, Ux, Battery, etc.)
- Devices (phones, tablets)
- Tools (DDMS & Jing, LogCat)
- Mindset, difficulty, challenges
| 16
http://www.kohl.ca/articles/ISLICEDUPFUN.pdf http://moolya.com/blog/wp-
content/uploads/2012/04/COP-FLUNG-GUN1.png
Quality & Mobile Context Driven App Testing
| 17
Quality & Mobile Context Driven Testing
http://unifiedtestinginitiative.org/files/utc_android_v1.pdf
| 21
http://www.engadget.com/2010/12/07/visualized-the-real-android-fragmentation/
•15-month-old Gingerbread operating
system (63.7%)
•2nd most popular version - 23-month-old
Froyo OS (23.1%).
•1st unveiled in October 2009 and currently
at 6%, Android 2.1 Eclair is still found on
nearly as many devices as Honeycomb and
Ice Cream Sandwich combined.
Android Fragmentation
|
Android Fragmentation
Android fragmentation highlights
include:
• 5 OS versions on devices
(Donut, Eclair, Froyo,
Gingerbread, and Ice Cream
Sandwich)
• Devices in various sizes, from
small phones to TVs
• Device manufacturers branching their own versions
of Android
22
|
iOS Fragmentation
24
http://www.marco.org/2011/11/30/more-ios-device-and-os-version-stats-from-instapaper
http://www.redmondpie.com/study-reveals-that-ios-apps-crash-more-frequently-than-android/
|
Which one is easier to test – Android or iOS?
25
http://411droid.com/humor-surrender-iphone-have-surrounded/
| 27
iOS Android
I
n
s
t
a
l
l
a
t
i
o
n
1. iTunes or iPhone Configuration Utility 2. With UDID provisional and executable or just executable - download and drag them to your iTunes
library (1st provisional, then the executable file) or iPhone Config Utility 3. iDevice test apps can also be downloaded Over-The-Air (OTA) via a secure link to authorized devices.
1. Device: "Settings -> Applications -> Unknown sources" 2. Download -email attachments, Dropbox, a barcode scanner program, etc.
3. Accept the permissions and install the application. 4. App can be launched via shortcut or widget (needs to be added - Settings>Applications>Manage Applications available). After initially launching a test app, it can be called by pressing and holding the native device Home button and then tapping on the
respective icon under Recent Apps list.
S
c
r
e
e
n
s
h
o
t
s
Hold down the home button and press the power button at the top right. Either email or sync your device with either iTunes or a photo management program of the user's choice. Digital cameras can be used for videocaptureing.
Digital cameras can be used to take a photo or video of the Android device's screen. Also Dalvik Debug Monitor Server (DDMS), Droid Explorer Android and AtScreen programs and TechSmith's Jing to capture the steps to reproduce a problem.
L
o
g
s
Console logs - iPhone Configuration Utility; Crash Reports - sync up with iTunes (https://developer.apple.com/library/ios/#technotes/tn2151/_index.html)
Log Collector or CatLog, DDMS/Eclipse(http://developer.android.com/sdk/index.html)
O
t
h
e
r
Speedtest - for easy connection testing (Download, Upload, and Ping)
Which one is easier to test – Android or iOS?
| 28
Few Tips and Recommendations
•“Software testers who are assigned to test applications on mobile devices need to go
beyond thinking “if I tap on the icon, does the application start” or “if I press the cancel
button, does the operation/function get cancelled.”
•Functional application testing is important but, the application is a part of an entire system
and not just a singular entity. The application is completely dependent on the entire system,
to function within the system, is a part of the system.
•Temperature, memory, CPU Speed, Database searches are all affected when a device is
charging and these tests are continually not considered.
•A mobile app tester’s responsibility is not simply to find the bugs, their goal must be to
increase the quality of the app, finding bugs, is just a small part of this. The testing
performed must also take into consideration, the goal of the app, the target audience etc.
•By collaborating with the app developer, a mobile app tester can not only significantly
increase the app’s quality but also their involvement can drive forward the project with
alternative design suggestions and usability feedback. http://www.testing4success.com/t4sblog/mobile-device-testing-vs-mobile-applications-testing/
Wrap Up
Top Related