The Value of USAP in Software Architecture Design

Presentation by:

David Grizzanti

Introduction

• Usability is an important quality attribute of interactive systems.

• Since the 1980’s, usability has been treated as a subset of user interface functionality.

• Since usability is not considered early in the design process, problems found in user testing are very costly.

Introduction

• The authors research between usability and software architecture has lead to the development of Usability-Supporting Architectural Patterns (USAPs).

• Each of these address a usability concern not addressed by separation alone.

Introduction

• Each USAP consists of:• An architecturally sensitive usability

scenario.• A list of general responsibilities derived

from forces inherent in the task and environment, human capabilities and desires, and software state.

• A sample solution implemented in a larger separation-based design pattern.

Introduction

• In their paper, the authors described a controlled experiment design to assess the value of using USAP components in modifying a software architecture design.

• Participants were asked to redesign an existing architecture to allow for users to cancel a long-running command.

• This experiment measured whether the architectural solutions produced using all the USAP components was more beneficial than using only certain subsets.

Experiment - Participants

• 18 computer science graduate students at Carnegie Mellon participated in the experiment.

• The 15 male and 3 female participants ranged in age from 23 to 30, and all had completed work for a master’s degree.

• Participants also reported spending an average of 22.9 hours per work programming.

Experiment Design & Materials

• Participants were randomly assigned to one of 3 teams.

• Participants in each group received a different version of a “Training Document.”

• All received the same architecture redesign task.

Groups

• Group 1 received:• A usability scenario describing user

circumstances.

• Group 2 received:• The usability scenario• A list of responsibilities to consider with a cancel

command.

• Group 3 received:• The usability scenario• List of responsibilities• Sample solution for adding cancellation function.

Experiment

• The task given to participants was to redesign an existing architecture that did not support cancellation, to support cancellation.

• Chose the Plug-in Architecture for Mobile Devices (PAMD), developed by students at Carnegie Mellon.

• Developed for the Palm OS4.

Experiment

• Task Instructions included 7 elements:• General description of PAMD architecture• Example PAMD scenario• List of responsibilities for PAMD operations• List of Component Interaction Steps detailing PAMD run-

time operation• A Component Interaction Diagram (Fig. 1)• A Sequence Diagram of PAMD of run-time component

interaction (Fig. 2)• Final page instructed participant to add the ability to cancel a

plug-in to the PAMD architecture design.

Fig. 1

Fig. 2

Experiment

• Answer Paper was designed so participants could easily and efficiently record the information relevant to their design.

• Component diagrams were identical to those provided to the Task Instructions, except that run-time steps and assignments were removed.

• Participants were instructed to use the diagrams as a bases for their designs.

Procedure

• Participants were randomly assigned to 3 groups and given unlimited time to complete their task.

• They were told that they were participating in a study about fixing one kind of usability problem in a specific software architecture design.

• Participants were then given the Training Document and after reading it, received the task instructions.

Results

• The time to complete the redesign task ranged from 39 to 138 minutes.

• Participants given only the USAP scenario considered 2-4 cancellation responsibilities.

• Those who received both the USAP and list of 19 responsibilities considered 4-15.

• Participants who were given all 3, considered 5-15 cancellation responsibilities.

Results - Overview

• Average for 3 groups:• 1st group considered an average of 3.17• 2nd group considered an average of 7.7• 3rd group considered an average of 9.5

• An analysis of variance between groups showed that the time on task had no significant effect on the number of cancellation responsibilities considered.

Discussion

• Experiment showed a strong main effect of providing the full USAP on the number of responsibilities considered.

• Participants who received only the cancellation scenario considered, on average, a third of the responsibilities of the participants who received the full USAP.

• This indicates that the full USAP helped the participants in remembering which responsibilities to consider.

Discussion

• The absence of a correlation between the time on task and performance showed no speed/accuracy tradeoff.

• However, administering the full USAP allowed participants to create a more complete solution in nearly the same amount of time.

• Though the group was small, factors such as self-reported experience or familiarity with technology did not interact with experiment conditions.

Discussion

• For 15 of the 19 Cancellation Responsibilities (CRs), Chi-squared tests revealed no significant difference in training conditions, however the following was observed:• CR1, CR5, and CR10 were considered by the majority of

participants.• CR16 was not considered by a single participant.• CR4, CR13, CR17, and CR18 were considered very

differently between those who received the cancellation scenario only and those who received the full USAP.

Conclusions

• Using a full USAP increased the number of responsibilities considered.

• Participants who used all 3 parts of the USAP were able to identify 3 times as many cancellation responsibilities, in the same amount of time and without having more work experience or formal training prior to the task.

Conclusions

• Assumption in SE world that programmers know how to implement whatever requirements are given to them.

• Mostly true, however, not the case for usability.• Usability issues have seen to comprise more than

60% of requirements-related defects in some professional software projects.

• Usability heuristics for software design are not obvious and should be made more explicit to software designers as well as SE students.