Large-scaleAnalysis of Framework-specificExceptionsinAndroidAppsLingling Fan, Ting Su, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu, Zhendong Su

ICSE 2018Gothenburg, Sweden

ACM SIGSOFT Distinguished Paper Award

2

One of the key priority is to prevent fail-stop errors, e.g., crash

Mobile app is continuously increasing

Time

#Apps

[statista.com]

However…...

3

Apps still suffer from crashes.

Customer complaints

4

Framework-specific Crash for Android Apps

5

java.lang.RuntimeException: Unable to resume activity {*}:java.lang.NumberFormatException: Invalid double: “”

at android.app.ActivityThread.performResumeActivity(…)….

Caused by: java.lang.NumberFormatException: Invalid double:“” at java.lang.StringToReal.invalidReal(StringToReal.java:63)at java.lang.StringToReal.parseDouble(StringToReal.java:248)….

Root exception

Crash signaler

An example of exception trace

➢ App➢ Framework➢ Lib

Crash Signaler

NOTE: We do not consider exceptions caused by the bugs of framework itself.

6

Developers: avoid and fix crashes

Researchers: improve bug detection tools

However, existing studies on functional bugs analysis:

• Small scale (AST’11, ICST’14)

• Different goals (ICST’14, MSR’15)

(1) generate testing oracles

(2) investigate bug hazards of exception-handling code

With the understanding of framework crashes

Analyzing (framework-specific) crashes is challenging

7

● Lack of comprehensive dataset○ No publicly available data○ Only 16% issues contain exception traces on Github and

Google Code

● Lack of tool support○ Crash reproducing tools○ Failure localization tools

● Substantial human effort○ Require understanding of Android framework

Contribution

● First empirical study to characterize Android framework-specific exceptions○ 11 fault categories

● Evaluate the state-of-the-art bug detection techniques○ Static & dynamic tools

● Prototype tools to demonstrate the usefulness of findings○ Stoat+ & Exlocator

● Publicly available dataset

8

9

Data Collection

9

ü Monkeyü Sapienzü Stoat

● 6,588 unique traces from issues

and comments of 583 apps

● 613 fixing commits

● 9,722 unique traces

(Github & Google code apps)

Each app runs for 3h4 months in total

2174

Crawling Testing

Research Questions

10

RQ1: Are frameworkexceptions recurring?

RQ2: Fault Patterns?

RQ3: Detected by

current techniques?

RQ4: Fixing patterns and

effort?

RQ1: Are framework exceptions recurring?

11

Yes, framework exceptions are more recurring and pervasive

Based on 6,588 unique exceptions from Github and Google code

Affected projects occurrences Fixing rate

RQ2: Taxonomy of Framework Exceptions

12

.

.

.

2,016

Framework exceptions

Top 200 buckets occupy over 80% exceptions

Bucket: repository for exceptions that are thrown from the same location of Android framework

RQ2: Taxonomy of Framework Exceptions

13

Bucket: repository for exceptions that are thrown from the same location of Android framework

Android doc. & Java specifications

Project src

Fixing commit repo

Q&As 11 fault categories covering 84.6% of all framework exceptions

14

RQ2: Taxonomy of Framework Exceptions

Developers make more mistakes on Lifecycle Error, Framework Constraint Error and Memory/Hardware Error.

Category Occurrence #S.O. posts

API Updates and Compatibility 68 60

XML Layout Error 122 246

API Parameter Error 820 819

Framework Constraint Error 383 1726

Index Error 950 218

Database Management Error 128 61

Resource-Not-Found Error 1303 7178

UI Update Error 327 666

Concurrency Error 372 263

Component Lifecycle Error 608 1065

Memory/Hardware Error 414 792

RQ3: Auditing bug detection tools

15

Static Tools 75 different exception instances from 11 categories

Tools Android support # Detected (out of 75 exceptions) # Rules for Android

Lint 4 281

FindBugs 0 0

PMD 0 3

SonarQube 0 0

● Existing static analysis tools are ineffective in detecting

framework exceptions

RQ3: Auditing bug detection tools

16

Dynamic Tools

Tools Approach # unique framework exceptions

Monkey Random 1842

Sapienz Search-based 2342

Stoat Model-based 1438

● 2104 apps (4560 versions)● Each runs for 3h

Detection time & OccurrenceMetrics:

17

RQ3: Auditing bug detection tools

Detection time: The time of detecting an exception for the first time Occurrence: The times of an exception detected during 3 hours

● Dynamic testing tools are still far from effective in detecting

database, framework constraint and concurrency errors

RQ4: Fixing Patterns

1. Refine Conditional Checks

2. Move Code into Correct Thread

3. Work in Right Callbacks

4. Adjust Implementation Choices

18

+ if(…){…...

+ }

Worker thread Main thread

UI update UI update

onCreate() onstop() onStart() onStop()register unregister register unregister

Code refactoring

RQ4: Fixing Efforts

19

Category Closing Rate

API Updates and Compatibility 93.9%

XML Layout Error 93.2%

API Parameter Error 88.5%

Framework Constraint Error 87.7%

Index Error 84.1%

Database Management Error 76.8%

Resource-Not-Found Error 75.3%

UI Update Error 75.0%

Concurrency Error 73.5%

Component Lifecycle Error 58.8%

Memory/Hardware Error 51.6%

● Lifecyle, Concurrency, UI update and memory errors are more difficult to fix

Ø Issue duration: The time cost to fix the issue (day)Ø Changed line: Exclude “//...”, “@Override”, “import *.*”Ø Closing rate: The percent of issues being closed

Applications

(1) Improving Bug Detection

20

Stoat+

➢ Meaningful corner cases○ e.g., “"” and “%”

➢ Enforce environment interplay○ Screen rotation

○ Start an activity and quickly back

○ Put the app at background for a

long time and navigate to it again

3 previously unknown bugs

● Parameter error

● UI update error

● Lifecycle error

https://github.com/tingsu/Stoat

Applications

(2) Enabling Exception Localization

21

java.lang.RuntimeException: Unable to resume activity {*}:java.lang.NumberFormatException: Invalid double: “”

at android.app.ActivityThread.performResumeActivity(…)….

Caused by: java.lang.NumberFormatException: Invalid double:“” at java.lang.StringToReal.invalidReal(StringToReal.java:63)at java.lang.StringToReal.parseDouble(StringToReal.jav a:248)….

ExLocator1. Fault category2. Root cause3. Explanations4. Fixing solutions

Report

● Built on Soot● Implemented 5 categories

25 out of 27 exceptions (92% precision) are correctly located by comparing the patches from the developers.

Conclusions

22

RQ1: Are frameworkexceptions recurring?

RQ2:Fault Patterns?

RQ3: Detected by current

techniques?

RQ4: Fixing practices and

effort?

Yes! Need improvementLifecycle Error,Constraint Error…

Lifecycle,Concurrency,UI update

➢ First large-scale analysis of Android framework-specific exceptions➢ Supporting follow-up research on bug detection, fault localization and

patch generation➢ Large-scale and reusable dataset available on

https://crashanalysis.github.io/Dataset-CrashAnalysis

NTU is still hiring PhDs and post docs

23

24

Contact: ecnujanefan@gmail.com

Homepage: http://www.sqslab.com/llfan/