Software Testing of Mobile Applications: Challenges and Future Research Directions
-
Upload
henry-muccini -
Category
Education
-
view
4.977 -
download
4
description
Transcript of Software Testing of Mobile Applications: Challenges and Future Research Directions
![Page 1: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/1.jpg)
Università degli Studi dell’Aquila
1
Università degli Studi dell’Aquila
Software Testing of Mobile Applications:Challenges and Future Research Directions
@AST 2012, the 7th International Workshop on Automation of Software Test, Zurich, June 02 2012
Henry Muccini, Antonio Di Francesco, Patrizio Esposito Dipartimento di Ingegneria, Scienze dell’Informazione e Matematica,
University of L’Aquila www.henrymuccini.com, @muccinihenry
![Page 2: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/2.jpg)
2
Goa
l & R
Qs
Goal and RQs
Goal of this presentation is to convey two messages to the audience:
testing applications running on mobile devices is an important task
those applications require dedicated testing techniques
![Page 3: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/3.jpg)
3
This work is conducted in the context of a series of activities conducted by my Department on mobile applications:
Testing of (context-aware) applications Engineering of mobile applications A course we launches one year ago on
“applications for mobile devices) A lab we are planning to build on those topics
![Page 4: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/4.jpg)
# of downloads:
29 billions of mobile apps downloaded in 2011. abiresearch
185 billions of downloaded apps by 2014. Gartner
Market:
$ 128 billions (2011).
$ 150 billions (2012).Strategy Analytics
The mobile apps market is growing…
![Page 5: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/5.jpg)
… but failures are growing as well…
http://code.google.com/p/android/issues/list
![Page 6: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/6.jpg)
… and mobile applications are being used in safety/business domains
M-health
Navatar
M-Payments
6
![Page 7: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/7.jpg)
Mobile Computing & Context-Awareness
In mobile computing an application is considered to be mobile if it runs on an electronic device that may move (e.g., mp3 readers, digital camera, mobile phones).
In context-aware computing, an application is aware of the computing environment in which it runs, and adapts/reacts according to its computing, user, physical, or time context [Schilit1994,Chen2000].
7
![Page 8: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/8.jpg)
Native applicationsWeb applicationsHibrid applications
Type of Mobile Apps
![Page 9: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/9.jpg)
Why differentiating Context-aware apps from Apps4Mobile?
From a testing perspective: The type and variaty of inputs are unpredictable Low programming support
From a technological perspective: Sensors-based applications are becoming pervasive
From a market perspective: $ 96 billions in 2015 (gartner) “context-aware mobile applications will change the business world…”
[Beth Schultz] ”creating applications for use in the mobile context will ultimately create
better applications that scale across connected devices” [Eric Schmidt] Projects ongoing
![Page 10: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/10.jpg)
Peculiarities of mobile applications10
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
![Page 11: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/11.jpg)
Peculiarities and Implications on testing11
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
![Page 12: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/12.jpg)
Peculiarities and Implications on testing12
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
512 MB of RAM,64 GB of disk space, and 1 Ghz dual core
CPU
![Page 13: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/13.jpg)
Peculiarities and Implications on testing13
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
iPhone 4S:200 hours autonomy in stand-by …drops to9 hours when Wi-Fi is enabled … drops to 6 hours when a 3G connection is active
New OS releases to increase autonomy
![Page 14: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/14.jpg)
Peculiarities and Implications on testing14
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
Huge effort
![Page 15: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/15.jpg)
Peculiarities and Implications on testing15
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
Is the app going to work in any environment and under any unpredictable scenarios?
![Page 16: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/16.jpg)
Peculiarities and Implications on testing16
Type of Mobile
Application
Peculiarity Implications on Testing
Mobile Applications
MobileApps
Apps4Mobile
Mobile Connectivity Reliability, performance, security, and functional testing through different networks
Limited Resources Performance and functional monitoring
Autonomy Monitoring for energy consumption
User Interface GUI Testing
Context Awareness Context-dependent functional and extra functional testing
Adaptation Adaptation correctness
New programming languages New white box and black-box testing, byte-code analysis
New O.S.s Compatibility and O.S. testing
Diversity of phones and O.S.s Diversity coverage
Touch Screens Usability, and response to screen touch
1,800 different hw/sw configurations
![Page 17: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/17.jpg)
Challenges
Test Selection for context-aware mobile applications
Challenges: unpredictabile and high variable inputs
Potentials: Create higher level libraries for developers Coverage criteria Scenario-based approaches
17
![Page 18: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/18.jpg)
public class SensorActivity extends Activity, implements SensorEventListener { private final SensorManager mSensorManager; private final Sensor mAccelerometer;
public SensorActivity() { mSensorManager = (SensorManager)getSystemService(SENSOR_SERVICE); mAccelerometer = mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER); }
protected void onResume() { super.onResume(); mSensorManager.registerListener(mAccelerometer, SensorManager.SENSOR_DELAY_NORMAL); }
protected void onPause() { super.onPause(); mSensorManager.unregisterListener(this); }
public void onAccuracyChanged(Sensor sensor, int accuracy) { }
public void onSensorChanged(SensorEvent event) { }}
Context-aware programming
![Page 19: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/19.jpg)
19
![Page 20: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/20.jpg)
Challenges
Security testingChallenge:
the mobility of the device into networks with different security levels increases security.
a trojan might have access to personal date, private networks, and credit card information
the rich contextual information presents real privacy concerns.
the democratization of mobile application publishers also increases the number of apps store the final user does not know anything about.
20
![Page 21: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/21.jpg)
Challenges
Performance and Reliability testingChallenge:
Performance and reliability of mobile applications strongly depends on the mobile device resources, on the device operational mode, on the connectivity quality and variability, and other contextual information.
Potentials: new techniques to explicitly consider characteristics related
to (changing) contexts and different devices. Run-time analysis techniques to prevent performance
degradation.
21
![Page 22: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/22.jpg)
Challenges
Product LineChallenge:
130 devices running Android, with 1,300 hw/sw configurations
Potentials new systematic combinatorial/product line testing
techniques run-time analysis of instrumented beta versions of the
application testing as a service
22
![Page 23: Software Testing of Mobile Applications: Challenges and Future Research Directions](https://reader033.fdocuments.us/reader033/viewer/2022051616/554d2b6cb4c905ca208b52a7/html5/thumbnails/23.jpg)
Conclusions
Test automation to decrease the cost of testing mobile applications must be cheap outsourcing, cloud- and crow-based testing
Testing automation towards all the different layers current bugs are due to interoperability problems that exist
today among the application, application framework, operating system, and hardware (sensoring) layers
25