PhD Dissertation Defense (April 2015)

Cross-Platform Testing and Maintenance of

Web and Mobile ApplicationsPhD Thesis Defense

Shauvik Roy Choudhary

Advisor: Dr. Alessandro Orso School of Computer Science, Georgia Tech

Committee: Dr. Mayur Naik, Dr. Milos Prvulovic Dr. Russ Clark, Dr. Mukul Prasad

1

Web and Mobile Apps

Web MobileWeb Mobile

2

Web Developer

3

Mobile Developer

LollipopAndroid 5.0

4

Cross-Platform Testing and Maintenance

Issues

5

1.Identifying InconsistenciesGoogle&Chrome+

Apple%Safari+

Mozilla'Firefox-

6

1.Identifying InconsistenciesMozilla'Firefox-

Internet&Explorer,

7

2. Finding Missing FeaturesDesktop Mobile

8

3. Automated Test Migration

App 1

Mobile Platform 1

Mobile Platform 2

App 2

Test Suite

Test Suite

9

Identifying Inconsistencies Cross Browser Testing

Finding Missing FeaturesFeature Mapping

Test-suite Migration(for mobile applications)

Problem Space

10

Thesis• Key Insight: Analyze cross-platform application

behavior to address the problems

• Challenge: Significant difference in cross-platform behavior. Match behavior despite legitimate differences.

• Thesis: Approximate matching algorithms with the right abstractions can be used to automate cross-platform testing and maintenance.

11

BehaviorCapture

BehaviorMatching

InconsistencyDetection

Test Migrationa1

a2

Cross-PlatformInconsistencies

(e.g., XBIs)

MissingFeatures onone platform

RetargetedTests for

new platform

Feature Analysis

1

2

3

a1.b

a2.b

MatchedBehavior

a1.b1 = a2.b1a1.b3 = a2.b3

UnmatchedBehavior

a1.b2, a2.b2

Applicationson two platforms

Captured Behavior of each application

MatchingResult

Cross-Platform Applications

Overall Approach

12

BehaviorCapture

BehaviorMatching

InconsistencyDetection

Test Migrationa1

a2

Cross-PlatformInconsistencies

(e.g., XBIs)

MissingFeatures onone platform

RetargetedTests for

new platform

Feature Analysis

1

2

3

a1.b

a2.b

MatchedBehavior

a1.b1 = a2.b1a1.b3 = a2.b3

UnmatchedBehavior

a1.b2, a2.b2

Applicationson two platforms

Captured Behavior of each application

MatchingResult

Cross-Platform Applications

Overall Approach

13

Cross Browser TestingWebDiff [ICSM’10],

CrossCheck [ICST’12], X-PERT [ICSE’13]

Feature MappingFMAP [ISSTA’14]

Test-suite MigrationCurrent work

Overview

14

X-PERT: Accurate Identification of Cross-Browser Issues

in Web Applications WebDiff [ICSM’ 10], CrossCheck [ICST’12], X-PERT [ICSE’13]

Tools [ICSM’10, ISSTA’14]

15

Overall approach

12

My Awesome Site δδ

WebApplication

ErrorReport

Model Generation Model Comparison

16

ModelScreen Model!

Screen image !+ geometry!

DOM Tree!

bodyheader

menu

hr1

main

hr2

footer

link1

link2

subHeading

countText

paperListpaper1

paper2

paper3

paper4

button1

button2

above

above

above

above

above

leftOf

above

above

aboveleft-alignright-align




Alignment Graph!

17

Comprehensive Cross-Browser Issue Detection

Type Detection Technique

Behavioral Graph Isomorphism

ContentText String Equality

Visual Image Matching (Computer Vision)

Structural Alignment Graph Matching

18

Evaluation Subjects

19

Name Type URL States TransitionsOrganizer Productivity http://localhost/organizer 13 99

GrantaBooks Publisher http://grantabooks.com 9 8DesignTrust Business http://designtrust.org 10 20DivineLife Spiritual http://sivanandaonline.org 10 9SaiBaba Religious http://shrisaibabasansthan.org 13 20

Breakaway Sport http://breakaway-adventures.com 19 18

Conference Information http://localhost/conference 3 12Fisherman Restaurant http://fishermanslodge.co.uk 15 17Valleyforge Lodge http://valleyforgeinn.net 4 12

UniMelb University http://www.economics.unimelb.edu.au/ACT/ 9 8

Konqueror Software http://www.konqueror.org 5 4UBC Club http://www.ubcsororities.com 7 7

BMVBS Ministry http://m.bmvbs.de 5 20StarWars Movie http://www.starwarsholidayspecial.com 10 9

Prior Art (6)

Example & Survey (4)

Random (4)

Evaluation

• RQ1: Can X-PERT find Cross-Browser Issues in real web applications?

• RQ2: How does X-PERT compare to the state-of-art?

45%↑ Precision and 14%↑ Recall

Yes!77% Precision and 95% Recall

20

Feature Mapping Across Platforms

(for web applications)FMAP [ISSTA’14]

Cross-Platform Feature Matching

for Web Applications

Shauvik Roy Choudhary*, Mukul R. Prasad†, Alessandro Orso*

*Georgia Institute of Technology †Fujitsu Laboratories of AmericaAtlanta, GA, USA Sunnyvale, CA, USA

{shauvik | orso}@cc.gatech.edu [email protected]

ABSTRACTWith the emergence of new computing platforms, softwarewritten for traditional platforms is being re-targeted to reachthe users on these new platforms. In particular, due tothe proliferation of mobile computing devices, it is commonpractice for companies to build mobile-specific versions oftheir existing web applications to provide mobile users witha better experience. Because the di↵erences between desk-top and mobile versions of a web application are not onlycosmetic, but can also include substantial rewrites of keycomponents, it is not uncommon for these di↵erent versionsto provide di↵erent sets of features. Whereas some of thesedi↵erences are intentional, such as the addition of location-based features on mobile devices, others are not and cannegatively a↵ect the user experience, as confirmed by nu-merous user reports and complaints. Unfortunately, check-ing and maintaining the consistency of di↵erent versions ofan application by hand is not only time consuming, but alsoerror prone. To address this problem, and help developersin this di�cult task, we propose an automated techniquefor matching features across di↵erent versions of a multi-platform web application. We implemented our techniquein a tool, called FMAP, and used it to perform a prelimi-nary empirical evaluation on nine real-world multi-platformweb applications. The results of our evaluation are promis-ing. FMAP was able to correctly identify missing featuresbetween desktop and mobile versions of a set of web appli-cations, as confirmed by our analysis of user reports andsoftware fixes for these applications.

Categories and Subject Descriptors: D.2.7 [SoftwareEngineering]: Distribution, Maintenance, and Enhancement—portability, reverse engineering

General Terms: Software Maintenance, Software Testing

Keywords: Cross-Platform, Mobile Web

1. INTRODUCTIONToday’s users run software on a variety of platforms, in-

cluding desktop computers, mobile devices such as smart-

Permission to make digital or hard copies of all or part of this work for

personal or classroom use is granted without fee provided that copies are

not made or distributed for profit or commercial advantage and that copies

bear this notice and the full citation on the first page. To copy otherwise, to

republish, to post on servers or to redistribute to lists, requires prior specific

permission and/or a fee.

ISSTA ’14, July 21-25, 2014, San Jose, CA, USA

Copyright 2014 ACM 978-1-4503-2645-2/14/07 ...$15.00.

phones and tablets, and even wearable embedded comput-ing devices [11, 8]. In fact, desktop computers are rapidlybeing supplanted by mobile devices as the preferred meansof accessing Internet content. Case in point, the market re-search firm IDC predicts that, by 2015, more users will beaccessing the Internet from mobile devices than from theirpersonal computers [32]. This move to mobile platforms hasbeen fueled, in part, by the increasing computing power ofmodern mobile devices, coupled with their rich interactiveuser interface, portability, and convenience.Because of this increasing prevalence of mobile devices

and platforms, most companies whose business largely de-pends on web presence, build versions of their existing webapplications customized for mobile devices, so as to providemobile users with a better experience. This customizationis necessary, despite the inherently multi-platform natureof web applications, due to the unique features of mobiledevices, such as their form factor, user interface, and user-interaction model [33]. Therefore, developers commonly re-target their web applications, sometimes substantially, tomake them more suitable for mobile platforms [9].In spite of the inherent di↵erences between desktop and

mobile platforms, and the resulting di↵erences between desk-top and mobile versions of a web application, end users ex-pect some level of consistency in the feature set o↵ered byan application across all platforms. The World Wide WebConsortium (W3C) standards committee, for instance, rec-ommends the “One Web” principle for web browsing plat-forms [37], which stipulates that web application users shouldbe provided with the same information and services irrespec-tive of the device on which they are operating. Prominentweb service providers, such as Google [6] and Twitter [29],follow this guideline. Figure 1 provides an illustrative exam-ple involving the desktop and mobile versions of the populardeveloper discussion forum stackoverflow.com. Althoughthere are substantial di↵erences in the look and feel of thewebsite in the two versions, both versions share the samecore functionality: clicking on a question shows detailed in-formation for that particular question in both versions, bothversions allow the user to sort the questions according todi↵erent criteria (using tabs in one case and the order bydrop-down menu in the other), and so on.In this context, the challenge for web developers is to de-

velop di↵erent versions of their applications that are cus-tomized to suit the specific characteristics of the di↵erentplatforms, yet provide a consistent set of features and ser-vices across all versions. To do this, one common strategyused by developers is to create separate front-end compo-

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for profit or commercial advantage and that copies bear this notice and the full citation

on the first page. Copyrights for components of this work owned by others than ACM

must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specific permission and/or a

fee. Request permissions from [email protected].

ISSTA’14, July 21–25, 2014, San Jose, CA, USA

Copyright 2014 ACM 978-1-4503-2645-2/14/07...$15.00

http://dx.doi.org/10.1145/2610384.2610409

��

��

��

��

��

��

��

��

��

��

82

21

Problem Finding Missing Features

Desktop version Mobile version

22

Desktop version Mobile version

Missing Feature on MobileMissing Feature on Desktop

Problem Finding Missing Features

23

Overall Approach

Platforms

TraceExtraction

Traces

1 1. REQUEST: GET /index.php2. RESPONSE: 200 OK, 'text/html'3. REQUEST: GET /style.css4. REPONSE: 200 OK, 'text/css'5. REQUEST: GET /logo.png6. REPONSE: 200 OK, 'image/png'7. REQUEST: GET /script.js8. REPONSE: 200 OK, 'text/javascript'9. REQUEST: POST /login.php user=user1&pass=..&sid=w2s3110. RESPONSE: 200 OK, 'text/html'....11. REQUEST: POST /create_blog.php title=..&content=..12. RESPONSE: 200 OK, 'text/html'

24

Overall Approach

Platforms

TraceExtraction

Traces

1

ActionRecognition

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

Labeled Actions

ABCD

AQRS


Domain specific vocabulary to identify keywords in requests

Ignore requests to stylistic resources

25

1. REQUEST: GET /index.php2. RESPONSE: 200 OK, 'text/html'3. REQUEST: GET /style.css4. REPONSE: 200 OK, 'text/css'5. REQUEST: GET /logo.png6. REPONSE: 200 OK, 'image/png'7. REQUEST: GET /script.js8. REPONSE: 200 OK, 'text/javascript'9. REQUEST: POST /login.php user=user1&pass=..&sid=w2s3110. RESPONSE: 200 OK, 'text/html'....11. REQUEST: POST /create_blog.php title=..&content=..12. RESPONSE: 200 OK, 'text/html'

Overall Approach

Platforms

TraceExtraction

Traces

1

ActionRecognition

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

Labeled Actions

ABCD

AQRS


(‘index’),

(‘script’),

(‘login’, ‘user’, ‘pass’, ‘sid’),

(‘create’, ‘blog’, ‘title’, ‘content’)

Keywords

26

Overall Approach

Platforms

TraceExtraction

Traces

1

ActionRecognition

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

Labeled Actions

ABCD

AQRS

2

indexlogin, ..

create, ..script

27

Overall Approach

Platforms

TraceExtraction

Traces

1

ActionRecognition

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

Labeled Actions

ABCD

AQRS

2

Features

Trace SetCanonicalization

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

ABCD

AQR

S

3 login

create post

preview post

publish post

create post

preview post

login

create post

preview post

publish post

28

Overall Approach

Platforms

TraceExtraction

Traces

1

ActionRecognition

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

Labeled Actions

ABCD

AQRS

2

Features

Trace SetCanonicalization

AXBYCD

ABCD

AQRS

AQRQRS

MNOP

UVW

ABCD

AQR

S

3

FeatureMatching

AXBYCD

A

B

CD

AQRS

AQRS

MNOP

UVW

MatchedFeatures

UnmatchedFeatures

4

29

Feature MatchingMaximum Weighted Bipartite Matching (MWBM) problem Hungarian Algorithm [Kuhn 1955]

1

2

3

4

5

1

2

3

4

Fd Fm

0.9

0.20.8

0.30.7

0.20.3

0.7

1

2

3

4

5

1

2

3

4

Fd Fm

0.9

0.8

0.7

0.7

30

Evaluation• Tool: FMAP (Feature Mapping Across Platforms)

• Subjects: Popular web apps (6 open source + 3 public)

Subject Type Mobile plug-inwordpress Blogging wordpress mobile pack

drupal Content Management nokia mobile themephpbb Forum artodia mobile style

roundcube Email mobilecube themeelgg Social Networking elgg mobile module

gallery Photo Management imobile themewikipedia.org Wiki -

stackoverflow.com Q&A -twitter.com Social Networking -

31

Evaluation• RQ1: How effective is FMAP in recognizing

web application actions?

• RQ2: How effective is FMAP in matching features between the desktop and mobile versions of real web applications?

Accuracy: 86.3% (FMAP) 51.5% (Baseline)

Accuracy: 97.8% (Desktop)99.6% (Mobile)

32

Evaluation• RQ1: How effective is FMAP in recognizing

web application actions?

• RQ2: How effective is FMAP in matching features between the desktop and mobile versions of real web applications?

Accuracy: 86.3% (FMAP) 51.5% (Baseline)

Accuracy: 97.8% (Desktop)99.6% (Mobile)

4050

Missing features reported by FMAPconfirmed from user reports & software fixes

33

Cross-Platform Test Migration

(for mobile applications)Current Work

34

Mobile Test

<action1, action2, …, actionn, Oracle>

{actionType, element, actionParams}

{assertionType, element, expectedValue}TAP “Submit”

TYPE “Name” “Shauvik”AssertExists “Success”

AssertText cell[0] “Alex”

GUI

^

36

MyList App - Platform 1

MyList Item >

MyList +

TAP “MyList Item”

MyList Item

MyList +

Delete ItemStar Item

TAP “Star Item”

MyList Item >

MyList +

AssertExists “Star”

37


View Items

Add Item

MyList

38


View Items

Add Item

MyListItems

MyList Item

MyList

TAP “MyList Item”TAP “View Items”

Items

Delete

MyList Item

Star

MyList

TAP “Star”

Items

MyList Item

MyList


39

MyList App - Test Migration

View Items

Add Item

MyListItems

MyList Item

MyList

TAP “MyList Item”TAP “View Items”

Items

Delete

MyList Item

Star

MyList

TAP “Star”

Items

MyList Item

MyList


MyList Item >

MyList +


MyList Item

MyList +

Delete ItemStar Item

TAP “Star Item”

MyList Item >

MyList +


40

Challenges

Independently developed

(In different languages& frameworks)

Posts

tap "New Post"

tap"Refresh"

tap"More Options"

tap "Hello World!"

Menu

tap "Open drawer"

tap "Posts"

tap "Comments"

tap "Media"

tap "Pages"

tap "Stats"

tap "Quick Photo"

tap "Quick Video"

tap "View Site"

tap "Navigate up"

tap "Upload" type "Title"

tap "Content"tap

"Post Settings"type "Content"

BACK button

tap"Navigate up"

tap "Navigate up"

tap "Delete Post"

tap"Preview"

tap "Preview Post"

tap "Share URL"

tap "Add Comment"tap "Edit Post"

tap "Sign out"

tap "Settings"

Automated Behavior

Exploration

Same actionsDifferent Widgets

No structural similarity Partial Consider all

possibilities

41

Assumptions

• Action correspondence: 1-1 correspondence between actions

• Action ordering: Is preserved in matched use cases

• Functionality under test: present on both platforms

42

IntuitionTest Case Model Generation

InitState

State A

State B

State C

Action 1

Action 2

Action 3

Oracle

InitState

State 1

State 5

State 6

State 2

State 3

State 4

State 7

InitState

State 1

State 5

State 6

State 2

State 3

State 4

State 7

PLATFORM 1 PLATFORM 2

43

Model Definition

Labeled Transition System (LTS)(S, s0, A, δ)

Transition FunctionInitial state Actions

States

δ(scurrent,a) = snext

44

View Items

Add Item

MyList

Items

MyList Item

MyList

Solving for maximizing matching

Given:

Constraint: Where:•

• Subject to:

•

Model for Platform 2:Test Trace for Platform 1:

a & b are from platforms p1 & p2

45


Given:

Constraint: Subject to:•

•

•


Model for Platform 2:Test Trace for Platform 1:Ordering Constraint

on matched actions

ai aj

bx by

46


Given:

Constraint: Subject to:•

•

•


Model for Platform 2:Test Trace for Platform 1:No Invalid Ordering

between matched actions

ai aj

bx bybz

ak

47


Given:

Constraint: Where:•

• Subject to:

•

Model for Platform 2:Test Trace for Platform 1:


48

Technique Overview

Platform 1App

Platform 2App

Test Trace Generation

Platform 1Tests

IncrementalModel

Generation

Platform 1Test Trace

Test Generation

Platform 2Tests

Model-Trace Mapping

1 2 3

49

Technique Overview

Platform 1App

Platform 2App


Platform 1Tests

IncrementalModel

Generation


Test Generation

Platform 2Tests

Model-Trace Mapping

1 250

Technique Overview

Platform 1App

Platform 2App


Platform 1Tests

IncrementalModel

Generation


Test Generation

Platform 2Tests

Model-Trace Mapping

1 2 3

51

1. Trace Generation

MyList Item >

MyList +

TAP “MyList Item”TAP

<“cell”, “”, “”, “MyList Item”, 0>

Action Widget Information:• Class Name • ID • Name • Value • Position in Parent

52

1. Trace GenerationTAP “MyList Item”

TAP “Star Item”AssertExists “Star”

Test

TAP <“cell”, “”, “”, “MyList Item”, 0>TAP <“button”, “”, “btn1”, “Star Item”, 0>AssertExists <“icon”, “Star”, “”, “”, 0>

Test Trace

53

2. Incremental Model Generation

App 2

TestTraces

Crawler Matcher

Controller

Mapping

Model

54

General Model ExplorationINPUT: Application under Test ALGO:

Model ← (S, s0, A, δ) | S = {s0}, A=ϵ, δ=ϵ Worklist ← s0.actions() While (Worklist.size > 0) and time() < TIMEOUT

(s, a) ← getNextAction(Worklist) snext ← doAction(s, a) if snext ∉ S Worklist ← snext.actions() Model.add(s, a, snext) Return Model

View Items

Add Item

MyList

Items

MyList Item

MyList

55

MigraTest Model ExplorationINPUT: Application under Test Test Traceset = {<a1, a2, …, an>, …}ALGO:

Model ← (S, s0, A, δ) | S = {s0}, A=ϵ, δ=ϵ Worklist ← s0.actions() While (Worklist.size > 0) and time() < TIMEOUT Sol ← MaxMatchSolve(Model, Traceset)

(s, a) ← Sol.getNextAction(Worklist) snext ← doAction(s, a) if snext ∉ S Worklist ← snext.actions() Model.add(s, a, snext) Return Model, Sol.Map

MigraTest decides what to explore

56

getNextAction

• Intuition: Prioritize clicks based on action similarity and scope of new potential assignments

Cross-platform action similarity (widget similarity & action compatibility)

Number of unassigned dependent actions

57

Solving for Matching

a1

a2

a3

b1 b2

～

～

～～

～

～


TAP “Star Item”

TAP “Delete Item”

TAP “V

iew Ite

ms”TA

P “Add Ite

m”

?

?✔

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

58

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

Add

TAP"MyList Item"

TAP"Add"

Type"Item Name"

b3

b4

b5

59


a1

a2

a3

b1 b2 b3 b4 b5

＝

～

～

～

～～

～

～～


TAP “Star Item”


TAP “M

yList

Item”

TAP “V

iew Ite

ms”TA

P “Add Ite

m”

TYPE “It

em N

ame”

TAP “A

dd”

✔

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

Add

TAP"MyList Item"

TAP"Add"

Type"Item Name"

b3

b4

b5

60

? ?

? ?

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

AddTAP

"Add"

Type"Item Name"

b3

b4

b5

Items

Delete

MyList Item

Star

MyListTAP

"MyList Item"

TAP"Star"

TAP"Delete"

b6 b7

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

Add

TAP"MyList Item"

TAP"Add"

Type"Item Name"

b3

b4

b5

61


a1

a2

a3

b1 b2 b3 b4 b5 b6 b7

＝

～～

～

～

～

～～

～

～～

✔

✔


TAP “Star Item”


TAP “M

yList

Item”

TAP “V

iew Ite

ms”TA

P “Add Ite

m”

TYPE “It

em N

ame”

TAP “A

dd”TA

P “Star

”TA

P “Dele

te”

✔

62

? ?

? ?

Trace Action

Model Action

a1 b3

a2 b6

a3 b7

Trace-Model Action Mapping

3. Test Generation

Model

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

AddTAP

"Add"

Type"Item Name"

b3

b4

b5

Items

Delete

MyList Item

Star

MyListTAP

"MyList Item"

TAP"Star"

TAP"Delete"

b6 b7

Tracea2a1

a1 a3

Trace Action

Model Action

a1 b3

a2 b6

a3 b7

Trace-Model Action Mapping63

3. Test GenerationTrace

a2a1

a1 a3

Model

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

AddTAP

"Add"

Type"Item Name"

b3

b4

b5

Items

Delete

MyList Item

Star

MyListTAP

"MyList Item"

TAP"Star"

TAP"Delete"

b6 b7

b3b1 b6

b3b1 b7Trace Action

Model Action

a1 b3

a2 b6

a3 b7

Trace-Model Action Mapping64

3. Test Generation

Model

TAP"View Items"

View Items

Add Item

MyList

TAP"Add Item"

b1

b2

Items

MyList Item

MyList

Add Item

MyList

Item Name

AddTAP

"Add"

Type"Item Name"

b3

b4

b5

Items

Delete

MyList Item

Star

MyListTAP

"MyList Item"

TAP"Star"

TAP"Delete"

b6 b7

b3b1 b6

b3b1 b7

Tap “View Items” Tap “MyList Item”

Tap “Star”

Tap “View Items” Tap “MyList Item”

Tap “Delete”

65

3. Test Generation (Oracle Migration)

AssertExists selector

AssertText selector

ExpectedValue

AssertText new_selector

ExpectedValue

AssertExists new_selector

new_selector := best_of(class, label, name, value, position)

66

3. Test Generation (Oracle Migration)

AssertExists selector

AssertText selector ExpectedValue

AssertText new_selector ExpectedValue

AssertExists new_selector

new_selector := best_of(class, label, name, value, position)

Extra Checksto catch potential migration issues

assertExists✔

Multiple elements exist

Element doesn’t exist yet (Flaky?)

Element exists but with different value

67

Case Study: wikiHow

68

Case Study: wikiHow

69

Case Study: wikiHow

70

wikiHow: Test CasesTEST CASE DESCRIPTION

Items loaded When user selects category, check if items in category are loaded

Page title When user selects category, check if page title matches category name

Back button When user selects category & item, & presses button with label “Back”, check if page title matches category name

Pop-up shown When user selects category & item, & presses button with label “Add”, check if pop-up is displayed to user

WebView loaded When user selects category & item, check if a WebView is added on the screen

71

wikiHow: Test MigrationTEST CASE MIGRATED CHALLENGE ISSUES FOUND

Items loaded Yes

Page title Yes Duplicates found

Back button Partial Platform specific functionality

Pop-up shown Partial Missing functionality

Missing functionality

WebView loaded Yes

72

Case Study: WhiteHouse

73


74


75

iOSTest cases

AndroidModel

Generation

WhiteHouse.govNetworkCapture-Replay

Proxy

WhiteHouse: Test Cases

76

TEST CASE DESCRIPTION

Menu Loaded When the user taps on Menu, check if “BLOG” is present

Photo Loaded When the user taps on Menu, and selects “PHOTOS” check if the first photo is loaded

Video Loaded When the user taps on Menu, and selects “VIDEOS” check if the first photo is loaded

Search When the user taps on Menu, and enters “A” in the search bar, check if page contains “No Results”

WhiteHouse: Test Migration

77

TEST CASE MIGRATED CHALLENGE POTENTIALISSUES FOUND

Menu Loaded Yes

Photo Loaded Yes Empty label on Android

Video Loaded Yes Empty label on Android

Search Yes Text Input

Future Work

Platform-specific Functionality

AssertTrue

AssertNotExists

Complex or Null Oracle Migration

Sand-boxing (Mocking, Reset)

78

Plans• Submit unpublished work to ISSTA’15

• Capture-Replay project for “Testing Everywhere”

• NSF i-Corps Program to Commercialize Capture-Replay technology for Business Intelligence

79

Summary

80

PhD Dissertation Defense (April 2015)

Education

Transcript of PhD Dissertation Defense (April 2015)