Test-Driven Development of Graphical User Interfaces: A Pilot Evaluation Thedore D. Hellmann, Ali...
-
Upload
arleen-lester -
Category
Documents
-
view
213 -
download
0
Transcript of Test-Driven Development of Graphical User Interfaces: A Pilot Evaluation Thedore D. Hellmann, Ali...
Test-Driven Development of Graphical User Interfaces:A Pilot Evaluation
Thedore D. Hellmann , Ali Hosseini-Khayat, Frank Maurer
XP 2011
Outline
Capture and Replay Tools
Introduction
Difficulties
A Prototype-Driven Approach to UITDD
Pilot Evaluation
Conclusion
Capture and Replay Tools
Capture and Replay Tools(CRTs) Tools that record users’ interactions with an application.
Can be used to record test scripts.
Introduction
Test-driven development (TDD) has proven benefits for the software engineering process increased developer confidence
increased software quality
In the approach, user interface prototypes are created in such a way as to allow capture/replay tools to record interactions with them.
These recordings can then be replayed on the actual GUI as it is being developed in a test-driven fashion.
TDD of GUI-based applications (UITDD)
Does having tests for a GUI written before the functionality of the GUI is implemented help developers?
Difficulty
What Makes GUI Testing and UITDD Difficult?
1. Complexity
2. Verification
3. Finding Widgets from Test code
Difficulty - Complexity
Modern GUIs are composed of a set of user interface elements, or widgets. Like most classes in object-oriented systems, widgets tend to have a long chain of inheritances.
Even a simple GUI has a huge number of possible user interactions, GUI states, and events.
For Example:
Difficulty - Verification
A test is separated into two parts: test procedure
test oracle
The test procedure interacts with part or all of an application in order to generate a state that is interesting from a testing perspective.
The test oracle verifies that the system behaves as expected in response to this interaction.
The effectiveness of automated tests is directly limited by the difficulty of writing useful test oracles .
Difficulty - Finding Widgets from Test code
In GUI testing, it is necessary to look up widgets when a test is run based on information about the widget that is recorded when a test is created.
This process of look up widgets is in large part responsible for the fragility of GUI tests.
There are two ways of performing this search: testing with object maps
keyword-based testing
A Prototype-Driven Approach to UITDD
ExpenseManager ExpenseManager is a GUI-based
application.
Vies Totals
New Report
Modify Report
A Prototype-Driven Approach to UITDD(2)
After defining the features of ExpenseManager, a prototype of the application was created. This prototype was created using SketchFlow .
SketchFlow Prototypes are created by dragging-and-dropping widgets onto a canvas.
Widgets can also be assigned behaviors such as transitioning to a different state.
we used LEET (LEET Enhances Exploratory Testing) to record events from interactions with prototypes of ExpenseManager.
LEET It is based on the Automation API
Using keyword-based identification to determine which widget to interact with for each step in the test script.
Pilot Evaluation
We designed a pilot evaluation to determine whether our approach is actually useful to developers.
Based on 3 particepants:
1. Researchers completed the development of ExpenseManager, including its GUI and event handlers, based on the tests.
2. The researchers removed all of the code contained in ExpenseManager’s 7 event handlers.
Pilot Evaluation(2)
Interestingly, all three participants entered into the same development cycle during their development of ExpenseManager.
Participants used the provided automated GUI tests as the sole indication of completeness of functionality.
Observations Collected during the Study
Pilot Evaluation(3)
The participants asked to fill out a survey, which perception of the usefulness and usability of this approach.
Observations Collected from Post-Study Surveys
The first and second questions were ranked from A (Very Useful) to E (Useless).
These responses imply that the participants saw the potential of this approach to be of some benefit in their own work.
Pilot Evaluation(4)
Participants were also asked to rank the resources they were provided with in terms of decreasing usefulness for their development task.
Instead of ranking features by frequency of use, participants seem to have ranked resources based on the value they provided to the development effort.
Observations Collected from Post-Study Surveys
This implies that one way to immediately make this approach to UITDD provide higher value to users :
Making technical details of widgets being tested more visible so that users can understand the expected details of the system they are building.
Study Limitations
1. Only three participants took part in this study.
2. The time allotted for each study session was limited to one hour in order to avoid inconveniencing participants.
3. The application used for this study was, based on Table 2, too complicated for a one-hour study.
Conclusion
In this paper we present a pilot evaluation of an approach to test-driven development of GUI-based applications.
we hope to test our approach to UITDD on multi-touch and gesture-based interfaces.
we were able to determine that participants seemed to feel that UITDD had benefits, and would be a useful practice.
Question Time