Post on 12-Jan-2016
description
The Value of USAP in Software Architecture Design
Presentation by:
David Grizzanti
Introduction
• Usability is an important quality attribute of interactive systems.
• Since the 1980’s, usability has been treated as a subset of user interface functionality.
• Since usability is not considered early in the design process, problems found in user testing are very costly.
Introduction
• The authors research between usability and software architecture has lead to the development of Usability-Supporting Architectural Patterns (USAPs).
• Each of these address a usability concern not addressed by separation alone.
Introduction
• Each USAP consists of:• An architecturally sensitive usability
scenario.• A list of general responsibilities derived
from forces inherent in the task and environment, human capabilities and desires, and software state.
• A sample solution implemented in a larger separation-based design pattern.
Introduction
• In their paper, the authors described a controlled experiment design to assess the value of using USAP components in modifying a software architecture design.
• Participants were asked to redesign an existing architecture to allow for users to cancel a long-running command.
• This experiment measured whether the architectural solutions produced using all the USAP components was more beneficial than using only certain subsets.
Experiment - Participants
• 18 computer science graduate students at Carnegie Mellon participated in the experiment.
• The 15 male and 3 female participants ranged in age from 23 to 30, and all had completed work for a master’s degree.
• Participants also reported spending an average of 22.9 hours per work programming.
Experiment Design & Materials
• Participants were randomly assigned to one of 3 teams.
• Participants in each group received a different version of a “Training Document.”
• All received the same architecture redesign task.
Groups
• Group 1 received:• A usability scenario describing user
circumstances.
• Group 2 received:• The usability scenario• A list of responsibilities to consider with a cancel
command.
• Group 3 received:• The usability scenario• List of responsibilities• Sample solution for adding cancellation function.
Experiment
• The task given to participants was to redesign an existing architecture that did not support cancellation, to support cancellation.
• Chose the Plug-in Architecture for Mobile Devices (PAMD), developed by students at Carnegie Mellon.
• Developed for the Palm OS4.
Experiment
• Task Instructions included 7 elements:• General description of PAMD architecture• Example PAMD scenario• List of responsibilities for PAMD operations• List of Component Interaction Steps detailing PAMD run-
time operation• A Component Interaction Diagram (Fig. 1)• A Sequence Diagram of PAMD of run-time component
interaction (Fig. 2)• Final page instructed participant to add the ability to cancel a
plug-in to the PAMD architecture design.
Fig. 1
Fig. 2
Experiment
• Answer Paper was designed so participants could easily and efficiently record the information relevant to their design.
• Component diagrams were identical to those provided to the Task Instructions, except that run-time steps and assignments were removed.
• Participants were instructed to use the diagrams as a bases for their designs.
Procedure
• Participants were randomly assigned to 3 groups and given unlimited time to complete their task.
• They were told that they were participating in a study about fixing one kind of usability problem in a specific software architecture design.
• Participants were then given the Training Document and after reading it, received the task instructions.
Results
• The time to complete the redesign task ranged from 39 to 138 minutes.
• Participants given only the USAP scenario considered 2-4 cancellation responsibilities.
• Those who received both the USAP and list of 19 responsibilities considered 4-15.
• Participants who were given all 3, considered 5-15 cancellation responsibilities.
Results - Overview
• Average for 3 groups:• 1st group considered an average of 3.17• 2nd group considered an average of 7.7• 3rd group considered an average of 9.5
• An analysis of variance between groups showed that the time on task had no significant effect on the number of cancellation responsibilities considered.
Discussion
• Experiment showed a strong main effect of providing the full USAP on the number of responsibilities considered.
• Participants who received only the cancellation scenario considered, on average, a third of the responsibilities of the participants who received the full USAP.
• This indicates that the full USAP helped the participants in remembering which responsibilities to consider.
Discussion
• The absence of a correlation between the time on task and performance showed no speed/accuracy tradeoff.
• However, administering the full USAP allowed participants to create a more complete solution in nearly the same amount of time.
• Though the group was small, factors such as self-reported experience or familiarity with technology did not interact with experiment conditions.
Discussion
• For 15 of the 19 Cancellation Responsibilities (CRs), Chi-squared tests revealed no significant difference in training conditions, however the following was observed:• CR1, CR5, and CR10 were considered by the majority of
participants.• CR16 was not considered by a single participant.• CR4, CR13, CR17, and CR18 were considered very
differently between those who received the cancellation scenario only and those who received the full USAP.
Conclusions
• Using a full USAP increased the number of responsibilities considered.
• Participants who used all 3 parts of the USAP were able to identify 3 times as many cancellation responsibilities, in the same amount of time and without having more work experience or formal training prior to the task.
Conclusions
• Assumption in SE world that programmers know how to implement whatever requirements are given to them.
• Mostly true, however, not the case for usability.• Usability issues have seen to comprise more than
60% of requirements-related defects in some professional software projects.
• Usability heuristics for software design are not obvious and should be made more explicit to software designers as well as SE students.