1 Early Usability Evaluation in Model-Driven Architecture Environments 6 th INTERNATIONAL CONFERENCE...
-
Upload
brittany-young -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Early Usability Evaluation in Model-Driven Architecture Environments 6 th INTERNATIONAL CONFERENCE...
1
Early Usability Evaluation in Model-Driven Architecture
Environments
6th INTERNATIONAL CONFERENCE ON QUALITY SOFTWARE (QSIC
2OO6)Beijing, China - October 27-28, 2006
Silvia Abrahão and Emilio Insfrán
Departmento de Sistemas Informáticos y Computación (DSIC)Universidad Politécnica de Valencia{sabrahao, einsfran}@dsic.upv.es
2QSIC 2006, Beijing, China - October 27-28
Contents
Introduction A Framework for Usability Evaluation
Usability at the Platform Independent Model (PIM) Usability at the Platform Specific Model (PSM)
Usability Model Instantiating the Usability Framework in a MDA
tool Results Conclusions and Further Work
3QSIC 2006, Beijing, China - October 27-28
Introduction Software development companies recognize the usability
as a key factor for the success of their products. However, software engineering practices don’t consider
usability aspects properly. Several studies show that 80% of total maintenance costs
are related to problems of users with the system. Among these costs, 64% are related to usability problems! (Landauer95)
Many approaches to evaluate the software usability have been proposed in the last years. Most of them focus on defining a set of attributes that explains usability and on developing guidelines and heuristics for testing it. They mainly focus on artifacts used in the late stages of
software development without connecting usability evaluation results with changes in software design.
4QSIC 2006, Beijing, China - October 27-28
Introduction Model-Driven Software Development (MDSD) appears with
its main characteristics: Separation of business functionality from technology
implementation The use of models along the complete software development
process by means of transformations (until the final source code)
This is a new challenge for the usability evaluation!
There are many tools and approaches for MDA (including User Interface specification) but none of these incorporate the usability explicitly There are no approaches which systematically evaluate the
usability of the applications obtained as a result of a process of successive transformations.
5QSIC 2006, Beijing, China - October 27-28
Contents
Introduction A Framework for Usability Evaluation
Usability at the Platform Independent Model Usability at the Platform Specific Model
Usability Model Instantiating the Usability Framework in a MDA
tool Results Conclusions and Further Work
6QSIC 2006, Beijing, China - October 27-28
A Framework for Usability EvaluationGoal A generic framework that incorporates usability in MDA
processes, where: The usability of a software application is evaluated and
improved at the platform independent model (PIM) using a Usability Model.
As outcome, a list of usability problems is produced. Instead of relate a given usability problem with changes in the
interface tier (code model level), we relate it to the elements of the PIM that are affected by it.
The evaluation also gives a feedback to the transformation rules itself (PSM).
7QSIC 2006, Beijing, China - October 27-28
A Framework for Usability Evaluation
An iterative process that combines model development and usability evaluation. It is applied until the PIM has the required level of usability.
The evaluation is made without users.
This allows very early usability evaluation by verifying the PIM properties that are related to usability.
MODEL COMPILATION
SYSTEM ANALYSIS
APPLICATION MODEL
EVALUATION WITHOUT USERS
Usability Model
PLATFORM INDEPENDENT
MODEL
PLATFORM SPECIFIC MODEL
CODE MODEL
1 A
REQUIREMENTSMODEL
REQUIREMENTS ELICITATION
COMPUTATION INDEPENDENT
MODEL
1 B
PLATFORM INDEPENDENT
USABILITY REPORT
CONCEPTUAL MODELS
OBJECT MODEL
TASKS MODEL
DIALOG MODEL
PRESENTATION MODEL
SOURCE CODE
INTERFACE TIER
APPLICATION TIER
PERSISTENCE TIER
MDA proposalMDA-based
development processEarly Usability evaluation
process
...
8QSIC 2006, Beijing, China - October 27-28
Contents
Introduction A Framework for Usability Evaluation
Usability at the Platform Independent Model Usability at the Platform Specific Model
Usability Model Instantiating the Usability Framework in a MDA
tool Results Conclusions and Further Work
9QSIC 2006, Beijing, China - October 27-28
The development of the model followed several activities.
According to the Goal-Question-Metric (GQM), the goal of our usability model is:
Analyze PIM (task, dialog, presentation models, etc.) and/or the CM (final UI) For the purpose of Evaluate with respect to its Usability from the viewpoint of a set of evaluators In the context of Software applications obtained with a
MDA development process
Usability ModelGoal
10QSIC 2006, Beijing, China - October 27-28
Attractiveness
Compliance
This activity is concerned with the decomposition of Usability into a set of sub-characteristics and measurable attributes.
The ISO/IEC 9126-1 quality model is used as the basis for building the proposed usability model:
Usability ModelSpecify Sub-Characteristics and Attributes
Operability
ISO 9126
FunctionalityReliabilityUsabilityEfficiencyMaintainabilityPortability
Understandability
Learnability
Concepts with high level of abstractionConcepts with high level of abstraction
11QSIC 2006, Beijing, China - October 27-28
Usability ModelSpecify Sub-Characteristics and Attributes
The decomposition is done to support usability inspection methods and to identify and fix usability problems.
1-Learnability refers to the attributes of a software product that facilitate learning. In our model it is defined in terms of: Help facilities (pre-defined keywords, wizards, on-line help, and documentation) Predictability which refers to the ease with which a user can determine the result of his/her future actions, Informative feedback in response to user actions Memorability as a measure of how quickly and accurately users can remember how to use an application that
they have used before.
12QSIC 2006, Beijing, China - October 27-28
Usability ModelSpecify Sub-Characteristics and Attributes
2-Understandability refers to the attributes that facilitate understanding : We differentiate the own optical legibility of texts and images from
readability that refers to the information grouping cohesiveness and density.
Familiarity is the degree to which a user recognizes UI components and views their interaction as natural; UI can be familiar by imitating the visual appearance of real-world objects, by relying on standardized commands, or by following other common metaphors.
Brevity is related to the reduction of user cognitive effort (i.e., limiting the reading and input workload and the number of action steps).
User guidance is related to the message quality and navigability to guide the user by providing mechanisms such as path and current position.
13QSIC 2006, Beijing, China - October 27-28
Usability ModelSpecify Sub-Characteristics and Attributes
3-Operability refers to the attributes that facilitate user control and operation: Capacity of the system to provide instalability facilities (e.g., user’s
help, documentation and installation procedures) Data validation (data validity), the user’s degree of control to services
execution (controlability), Capability of adaptation of the UI or services. It refers to the system
capacity to behave contextually and according to the users’ needs and preferences. We distinguish between
Adaptability, which is the system capacity to be adapted by the user, and Adaptivity, which is the system capacity to adapt to the users’ needs.
Operational consistency in the execution of services and controls (consistency), the prevention and error management, and the capacity to monitor the state of the system (state system monitoring).
14QSIC 2006, Beijing, China - October 27-28
Usability ModelSpecify Sub-Characteristics and Attributes
The other two sub-characteristics are related to the perception of the end-user (4-Attractiveness) or evaluator (5-Compliance) using the system. It can be mainly measured using subjective measures (e.g., surveys).
Attractiveness: some aspects of attractiveness related to aesthetic design can also be quantified by measuring the UI uniformity in terms of font color, font style, font size and elements position. Example: Some studies suggest that designs should not use
more than 2-3 fully saturated intense colors.
15QSIC 2006, Beijing, China - October 27-28
Usability ModelSpecify Sub-Characteristics and Attributes
Compliance: can be measured by assessing the agreement of the proposed usability model with respect to the following standards: ISO/IEC 9126-1 (from SE field) ISO 9241-11 (from HCI field) Microsoft and Java style guides …
The first two assessments can be performed at PIM or CM levels, whereas the last one can only be done at the CM, as it analyses the fulfillment of the UI widgets to the prescriptions of the style guides.
16QSIC 2006, Beijing, China - October 27-28
Usability Model
Learnability Help Facilities
Documentation Completeness
Multi-user Documentation Capability Profiled Role Profiled
Predictability Icon Significance Icon/Link Title Significance Action Determination
Informative Feedback Memorabitity
Time to Remember Accuracy
Understandability Legibility
Font Size Contrasting Text Disposition
Readability Information Grouping
Cohesiveness Location Grouping Format Grouping
Information Density Familiarity
Labeling Significance Internationalization Metaphor
Workload Reduction Brevity
Values Inizialization Initial Values Completion Initial Values Modifiability
Actions Minimization Self-descriptiveness
User Guidance Message Quality Navigability
17QSIC 2006, Beijing, China - October 27-28
Usability Model Operability
Installability Ease of Installation Multiplicity of Installation Updateability Update Transparency
Data Validity Controlability
Edition Deferral Cancel Support Explicit Execution Interruption Support Undo Support Redo Support
Capability of Adaptation Adaptability Adaptivity
Consistency Steady Behavior of Controls Permanence of Controls Stability of Controls Order Consistency Label Consistency
Error Management Error Prevention Error Recovery
State System Monitoring Attractiveness
Background Color Uniformity Font Color Uniformity Font Style Uniformity Font Size Uniformity UI Position Uniformity Subjective Appealing
Compliance Degree of Fulfillment with
the ISO 9241-10 / ISO 9241-11
Degree of Fulfillment with the Microsoft style guide
Degree of Fulfillment with the Java style guide
18QSIC 2006, Beijing, China - October 27-28
Contents
Introduction A Framework for Usability Evaluation
Usability at the Platform Independent Model Usability at the Platform Specific Model
Usability Model Operationalizing the Usability Model into a MDA
tool Results Conclusions and Further Work
19QSIC 2006, Beijing, China - October 27-28
Operationalizing the Usability Model into a MDA tool The Usability framwork was
instantiate in a specific MDA tool: OlivaNova Model Execution (ONME)
This tool is based on:
Conceptual Model (PIM): Object Model, Dynamic Model, Functional Model and Presentation Model
Model Compilers that establish the specific to a given platform (PSM) and generates the code (CM) applying transformation rules
Hierarchical Action Tree
Service IU
Instance IU
Population IU
Master/Detail IU
Introduction
Defined Selection
Argument Grouping
Supplementary Information
Order Criterium
Display Set
Filter
Dependency
Actions
Navigation
A uses B
A B
Key
20QSIC 2006, Beijing, China - October 27-28
Operationalizing the Usability Model into a MDA tool
Association of one or more elements of the PIM / CM models to each identified attribute.
The attributes in our Usability Model are inherent to UI modeling and make a product usable.
One or more metrics are associated to each model attribute.
ISO/IEC 9126
Usability
Learnability Compliance
AttractivenessOperability Understandaility
Platform Independent Model (PIM)
UI de maestro/detalle UI de poblaciónUI de instanciaInstance IU UI de servicioService IU AJAHAT
Display set
N A ActionNavegationOrder criteria
Filter
Usability definitions
Abstract interface patterns
Ergonomic criteria
Concrete interface
components
Explicit ControlInformation Significance
Platform Specific Model (PSM)
Interface standards
- IBM CUA - Microsoft
Poblation IU
Explore Bar
Control Filter
Generic component INSTANCE
Radio Button
CIO (Concrete Interface Object) VB
Prompting
Menu
Grid
Choose of platform
Legibility
ONME Presentation patterns
F
. . .
Computing Platforms
Visual Basic
Java/Swing ColdFusion
JSP
ASPPocketPC
Error Management
Consistency
Master-Detail IU
explicit control
Workload
A/Z
AIO (Abstract Interface Objects)
21QSIC 2006, Beijing, China - October 27-28
Contents
Introduction A Framework for Usability Evaluation
Usability at the Platform Independent Model Usability at the Platform Specific Model
Usability Model Instantiating the Usability Framework in a MDA
tool Results Conclusions and Further Work
22QSIC 2006, Beijing, China - October 27-28
Results Examples of sub-characteristics, attributes, PIM elements
affected and potential metrics.
23QSIC 2006, Beijing, China - October 27-28
The Usability Model was applied across several usability inspections carried out in ONME interface models (PIM) and generated applications (CM).
This allowed us to validate the model experimentally. Some identified Usability Problems (UPs):
Lack of descriptive labels Difficult navigation Inflexible/invalid search capabilities Inconsistent menu item and window labels No feedback for user selection Meaningless error messages
Results Obtained and their Effects on the PIM and PSM
24QSIC 2006, Beijing, China - October 27-28
Once identified the usability problems, its source must be determined (PIM or PSM) to define a Recommendations Plan.
e.g. the problem Meaningless error messages affects the quality of error messages
Although the message informs how to solve the problem, they must be meaningful for users!
In this case, the source of the problem is in the PIM (Presentation Model) and relates to the property “Validation message” of the Introduction Pattern of a Service Interaction Unit (Service IU).
Usability Problems that affect the PIM
25QSIC 2006, Beijing, China - October 27-28
Recommending a change in the PIM
Identifier: 17
Usability Problem: Lack of descriptive labels
Context: Instance Interaction Unit (IIU) and Supplementary Information pattern
Related ergonomic criteria:
Guidance
Usability attribute: Labeling significance
Example: Note that the display set of the IIU (indicated by the row) has no label.
Recommendation: Identify the patterns in the Presentation Model without alias and define them.Apply the proposed metric (proportion of elements with meaningful names).
26QSIC 2006, Beijing, China - October 27-28
Information grouping cohesiveness and information density attributes were poor evaluated because of the lack of grouping mechanisms. This problem and recommendation is as follows:
Usability Problems that affect the PSM
Identifier 10
Usability Problem The users’ understanding of a display screen depends on the grouping and density of the objects (images, texts, etc.) that are presented. Currently, in ONME, all the attributes are shown in an abstract interaction unit without any grouping mechanism.
Context Instance Interaction Units, Master-Detail Interaction Units
Related ergonomic criteria:
User explicit control, Predictability
Usability attribute: Information group cohesiveness and Information density
Recommendation: Change the transformation rules (PSM) to incorporate different representations to the grouping mechanisms such as boxes, tabs, eye fishing, etc.
27QSIC 2006, Beijing, China - October 27-28
In a MDA environment, some usability attributes are garantee by the transformation process User guidance attributes
Visual feedback in menus and dialogs boxes where choices can be selected or when cursor is pointing (tooltips)
Visual feedback when elements (objects) are selected …
Legibility attributes: Prompts and error messages always appear in the same place(s) Error messages displayed in pop-up windows with the same
structure: what-where-howToSolveIt …
These give us empirical evidence of the intrinsic quality of UIs generated by a MDA tool.
MDA Benefits
28QSIC 2006, Beijing, China - October 27-28
Contents
Introduction A Framework for Usability Evaluation
Usability at the Platform Independent Model Usability at the Platform Specific Model
Usability Model Instantiating the Usability Framework in a MDA
tool Results Conclusions and Further Work
29QSIC 2006, Beijing, China - October 27-28
We have presented a framework for early usability evaluation that fits in MDA development processes.
Unlike existing proposals, the usability is evaluated and improved at the PIM level instead of in its final user interface (CM).
This framework has been operationalized by means of a Usability Model which represents the relationships among the ISO/IEC 9126 and the usability patterns and criteria from the HCI field.
This model was applied in the inspection of several PIMs (presentation models) specified using a MDA tool.
Conclusions
30QSIC 2006, Beijing, China - October 27-28
The framework was found to be very useful not only to evaluate PIMs but also to discover deficiencies and/or limitations of the PIM expressiveness and the transformation rules (giving feedback for their improvement!).
The contribution of the work is to provide a set of attributes (independent of any method or technology) that can be verified in the PIM/PSM/CM defining the corresponding relationships and metrics.
Conclusions
31QSIC 2006, Beijing, China - October 27-28
Further Work The application of the usability framework to other MDA
tools specifically oriented to Web development (i.e., VisualWade).
We are concerned, in particular, to explore and evaluate to which extend our usability model can improve the development of Web applications.
The operationalization of the Usability Model using a set of usability metrics from the Web Quality Model (Calero et al., 2005). The first project will be the evaluation of the usability of a Spanish museum (Thyssen-Bornemisza Museum) from its code model.
This will allow to measure the level of usability of a model or a implemented UI.
32QSIC 2006, Beijing, China - October 27-28
Thanks for your attention!