Presented by : Souleymane ABDALLAH Economic Affairs Officer, ACGS
Usability Assessment Framework for ACGs using AHP USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS...
-
Upload
austin-marshall -
Category
Documents
-
view
236 -
download
5
Transcript of Usability Assessment Framework for ACGs using AHP USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS...
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS USING
ANALYTICAL HIERARCHY PROCESS
Joshua R. Dolecal
Dahai Liu
Remzi Seker
Andrew Kornecki
Department of Human Factors and Systems
Embry-Riddle Aeronautical University
Daytona Beach, Florida
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
2
Outline
• Usability
• Code Generation
• Rationale
• Framework
• AHP
• Results
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
3
Usability
• Also called: Usability Engineering, Human Factors, Ergonomics, and User Centered Design.
• UCD is defined as “the practice of designing products so that users can perform required use, operation, service, and supportive tasks with a minimum of stress and maximum of efficiency. (Woodson,1981)
• Nielson (1993) defines Usability as the overall acceptability of a system.
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
4
Usability Criteria
• Learnability
– How easy something is to learn
• Efficiency
– Of the product (how well it aids users in performing tasks)
• Memorability
– How easy it is to remember skills learned/used
• Errors
– Error prevention, correction, and identification
• Satisfaction
– How well users are satisfied with the product
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
5
Process of Generating Code
• Select template
• Create a model/chart/diagram
• Select output language (C/C++, Java, Ada…)
• Generate Code
(Stephens, 2002)
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
6
Rationale
• ACG is relatively new
• ACGs will change the way companies work and SW is developed
• Usability need to be addressed, for special users– Experts (Computer Programmers)
• Little research done
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
7
Framework
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
8
Framework
• General Usability Criteria– (Nielson, 1993) Basic well accepted usability
• Specific Usability Criteria– (Stephens, 2002 & Whalen, and Heimdahl, 1998 &
Maclay, 2000, etc.) – Elements that are unique to ACGs according to the
current literature
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
9
Method
• AHP framework constructed– Usability criteria and– ACG functional analysis
• 8 Graduate students gave pairwise comparison. (~6 months exposure with ACGs)
• AHP weight
• Consistency
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
10
Analytical Hierarchy Process
• 3 Main steps
– Establish a hierarchy– Calculate pair-wise comparison weights– Check for consistency– Apply weights to gathered data (objective/subjective)
• Process can be Automated using Expert Choice– http://www.expertchoice.com/
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
11
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
12
AHP Values
Thomas Saaty (1982)
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
13• Thomas Saaty (1982)
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
14
4 Step Consistency Process
• Thomas Saaty (1982)
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
15
Consistency
• When dealing with many criteria/variables, people are not going to be perfectly consistent. It is up to you to review their decisions with them such that the “decision maker” understand transitivity.– E.g. If A > B and B > C then A > C
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
16
Results
AHP Weights
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
Frequency
Length of use
Content
Complexity
Time spent
Feedback
Time to modify
Log Maintenance
Expertise required
Time to compile
Semantic consistency
Flexibility
Code size
Code range
Meaningfulness
Percentages %
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
17
Results
AHP Weights
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
Support
Efficiency
Satisfaction
Source Code Modification
Translator
Output
Percentage %
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
18
Results
AHP Weights
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
General Usability Criteria
Specific Usability Criteria
Percentage %
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
19
Results
• Semantic consistency and User satisfaction received the highest weights, but traversing up the hierarchy we see the Translator and then Specific Usability Criteria received the highest weight
• A customized answer was produced– Best product for your situation– Does not label products as better than others– Allows a combination of subjective & objective data
Usability Assessment Framework for ACGs using AHPUsability Assessment Framework for ACGs using AHP
20
• Questions, Comments?