Usability evaluation – checklist/guideline

Lecture notes

Evaluation with Checklist/Guideline

• More specific guidelines than principles

• Types of Guidelines– What sources of “wisdom” or design parameters are available?

• User Interface Guidelines• Commercial Platform Guidelines • External Standards• Customized Style Guides

Form-Fillin Design Guidelines

• Meaningful title • Comprehensible instructions • Logical grouping and sequencing of fields • Visually appealing layout of the form • Familiar field labels • Consistent terminology and abbreviations • Visible space and boundaries for data-entry fields • Convenient cursor movement • Error correction for individual characters and entire fields • Error prevention • Error messages for unacceptable values • Optional fields clearly marked • Explanatory messages for fields • Completion signal

Nielsen의 Home Page Guidelines

• Make the Site's Purpose Clear: Explain Who You Are and What You Do– 1. Include a One-Sentence Tagline

– 2. Write a Window Title with Good Visibility in Search Engines and Bookmark Lists

– 3. Group all Corporate Information in One Distinct Area

• Help Users Find What They Need– 4. Emphasize the Site's Top High-Priority Tasks

– 5. Include a Search Input Box

www.nngroup.com

Nielsen의 Home Page Guidelines

• Reveal Site Content– 6. Show Examples of Real Site Content– 7. Begin Link Names with the Most Important Keyword– 8. Offer Easy Access to Recent Homepage Features

• Use Visual Design to Enhance, not Define, Interaction Design– 9. Don't Over-Format Critical Content, Such as Navigation

Areas– 10. Use Meaningful Graphics

www.nngroup.com

Checklist for Navigation

Navigation Rating Explanation for Rating

Current location within the site is shown clearly

Link to the site's main page is clearly identified

Major/important parts of the site are directly accessible from the main page

Site map is provided for a large, complex site

Easy to use Search function is provided, as needed

Checklist for Architecture and navigation

• Can you tell where you are in the site? • Can you tell where you can go in the site? • Is there a reasonable number of navigation choices? • Do link names match page names? • Do all link phrases make sense when read out of

context? No links with the dreaded "click here" or "more" used as text.

• Is there a link back to the home page? • Do all links lead to the proper destination? No broken

links or link rot. • Is there an option to search for information?

General User Interface Guidelines

• These are publicly available wisdom or suggestions.• They are based on research and practice.• They require skill and training to apply.

Commercial Platform Guidelines

• A vendor defines the guidelines and makes them commercially available.

• Applications can then advertise that they adhere to the vendor’s standard for user interface.

• These guidelines are more concrete and are a more useful framework for design than general guidelines or standards.

Commercial Platform Guidelines Advantages

• Commercial platform guidelines tend to be specific.

• Commercial platform guidelines are often based on

research.

• Commercial platform guidelines often have a

commercial toolkit or development environment.

• Commercial platform guidelines promote consistency

and standardization across applications

– Transitions across hardware platforms should be seamless

when the platforms conform to a specific set of commercial

platform guidelines.

Commercial Platform Guidelines Disadvantages

• Commercial platform guidelines alone do not insure interface usability.

• Commercial style guides often describe the look and feel of parts but do not tell you how to put them together for the best result.– For example, commercial platform guidelines may proscribe

the appearance of individual buttons but do not tell the designer where to put buttons or how to group them.

• An application may be faster, more efficient, or have better task match if it is not compliant with the commercial platform guidelines.

External Standards

• An external standard is an official document.– They are often published by a government agency.– They are publicly available.– Their purpose is to give requirements for a particular agency

or official organization. • External standards may be enforceable by law.

Project Guidelines

• User interface development teams should always produce a customized set of project guidelines.– The guidelines should be evaluated and validated.– They should be customized for the project or projects that

they team is working on.

• Project guidelines can be used to maintain a record of decisions.

various UI standards or guidelines

Standard or Guideline PagesDIN 66234 part 8 standard [1988] (note: dense pages) 6The Data Company's standard 57Apple Human Interface Guidelines [1987] 166Motif™ style guide [OSF 1990] 167

"Advanced CUA" (for graphical interfaces) [IBM 1990a] 209

"Basic CUA" (for traditional terminals) [IBM 1990b] 288Original CUA [IBM 1987] 340OPEN LOOK™ [Sun Microsystems 1990] 404Smith and Mosier [1986] guidelines 485ISO 9241 Ergonomics of human-system interaction-- Part 151: Guidance on World Wide Web user interfaces 532

5-15

Pros and Cons of checklist/guideline methodPros and Cons of checklist/guideline method

• Pros– Less time and less expensive– Can evaluate various areas if guidelines exist – Used before and after implementation

• Cons– Heterogeneous guidelines: width, depth – Lacks overall evaluation: specific evaluation items– Quality of guidelines varies

5-16

Usability evaluation- cognitive walkthrough

Lecture notes

5-17

Cognitive Walkthrough

• Simulating step-by-step user behavior for a given task using a prototype interface– Review actions needed for task,

• cognitive & operational activities– Attempt to predict how users would behave and – What problems they’ll encounter

• Focus on learnability of the technical system• Become familiar with interaction• Essentially a usability “thought experiment”• Like code walkthrough in software engineering

Four steps of CW

1. Convene the analysts and create the scenarios2. Perform the walkthrough3. Record the usability problems found4. Fix the usability problems

1. Convene Analysts and Create Scenarios• An indication of

– who the users are and – what kind of experience and knowledge they possess

• the evaluators can assume about them.

• A description of the prototype of the IT system. – It doesn't have to be complete, – but it should be fairly detailed.

• Details such as the location and wording for a menu can make a big difference.

• A specific description of the task the user is to perform on the IT system.– This should be a representative task that most users will want to do.– Construct carefully designed tasks from IT system spec or screen

mock-up• A complete, written list of the correct actions

– needed to complete the task with the given prototype.

Scenario

• a story about how a person would use a IT system to achieve a goal

• Overview– a textual description which describes

• the user, • what they are trying to do, • their motivation and • how they achieve their goal.

• Task Description– a more formal description of the user’s activity needed to

solve their task than presented in the above description

• Screenshots– The overview and task description will refer to screenshots

of the system.

5-20

Scenario - Overview

• Who: – describe the person, knowledge, role and activities etc.

• What and Why: – describe the user’s goal and their motivation.

• With: – what tools, documents and resources will the person have

available?

• How: – describe how the user could use the tech system to achieve

their goal.

5-21

Scenario 1- OverviewScenario 1- Overview

• Who– Kim maintain a previously

written price list which exists as a MS word document.

• What and Why– given the price list– decides to reformat the table so

all the textbooks will take one line.

• With – MS Word 2003

• How– Select the column– set the correct width of the

column

품목 번호 품목 명 가 격 담당자

101Advances in Hci

5,000 홍길동

102 Hci Models 12,000 김철수

103

Inclusive Design Guidelines for Hci

5,000 박갑식

104Perspectives on Hci

15,000 이태백

105Hci International 2003

86,000 허기완

106 Hci Theories 98,500 이진희

Task Description

• Goal hierarchy– a full decomposition of the main goal (what and why) into

envisioned subgoals which result in a set of methods for the simplest subgoals.

• Method Description– a set of actions which combine to satisfy the subgoal.

Task Description-Goal hierarchy• examples

A) Make a copy of the text

B) Copy text to clipboard

C) Select text to copy

D) Issue copy command

E) Select point to copy text to

F) Issue paste command

Task Description - Method Description

Method Description Table

Task Description – example

Method to achieve subgoal B: Select ColumnMain goal and active subgoals:A) Set the width of the column to ensure all entries are on one line

B) Select ColumnComments: None

User Action System Feedback comments

B1 Move mouse pointer to top left of the left column of the table and hold mouse button down.

B2 Move mouse pointer to bottom right hand corner of left column of table, and release mouse button.

Selected text isHighlighted.

Figure 1

Task Description - screenshots

Figure 1: The Column is highlighted

Task Description – exampleMethod to achieve subgoal C: Set Column Size

Main goal and active subgoals:A) Set the width of the column to ensure all entries are on one line

C) Set size of columnComments: Assumes a column has been selected

User Action System Feedback comments

C1 Move mouse pointer to ‘표’ Menu and click

‘표’ menu appears Figure 2

C2 Select ‘표속성’ menu item ‘표’ menu disappears‘표속성’ dialog box with ‘표’ tab appears

Figure 3

C3 Select “셀” tab ‘표속성’ dialog box with ‘셀’ tab appears Figure 4

C4 Select text field right of ‘나비지정’ ‘3.00’ in the text field is highlighted Figure 5

C5 enter ‘5.00’ ‘5.00’ appears in the text field Figure 6

C6 Select ‘확인’ button Dialogue box disappearsColumn size now adjusted

Figure 7

Task Description- screenshots

Figure 2: ‘표’ menu appears

Task Description- screenshots

Figure 3: ‘표속성’ dialog with ‘표’ tab selected appears

Task Description- screenshots

Figure 4: ‘표속성’ dialog with ‘셀’ tab selected appears

Task Description- screenshots

Figure 5: the text field is highlighted

Task Description- screenshots

Figure 6: ‘5.00’ is input in the text field

Task Description- screenshots

Figure 7: column size adjusted

2. Perform the walkthrough

• The user sets a goal to be accomplished with the system (for example, "check spelling of this document").

• The user searches the interface for currently available actions (menu items, buttons, command-line inputs, etc.).

• The user selects the action that seems likely to make progress toward the goal.

• The user performs the selected action and evaluates the system's feedback for evidence that progress is being made toward the current goal.

Cognitive Walkthrough Questions

• Examine the sub-goals– 1) Will the user try to achieve the right sub-goal?– 2) What knowledge is needed to achieve the right sub-goal?

Will the user have this knowledge?

• Analyze each action – 3) Will the user notice that the correct action is available?– 4) Will the user associate the correct action with the sub-goal

they are trying to achieve?

• Assess feedback, If the correct action is performed– 5) Will the user perceive the feedback?– 6) Will the user understand the feedback?– 7) Will the user see that progress is being made towards

solution of their task in relation to their main goal and current sub-goals?

Example

Examine the sub-goal B: Select Column1) Will the user try to achieve the right subgoal?

– Yes, the user will know from experience using the Windows, that objects such as text must be selected before applying a command.

2) What knowledge is needed to achieve the right subgoal? Will the user have this knowledge?– The user requires knowledge of how to achieve this subgoal. This

will be assessed when the method is analyzed.

Example - Analyze each action for achieving subgoal B

Analysis of B13) Will the user notice that the correct action is available?

– There is no representation of the action on screen. However, experienced users of the Windows will be aware of it. However, this is an action which must be learned.

4) Will the user associate the correct action with the subgoal they are trying to achieve?– There is no representation of the action to associate with, but an

experienced user will know that is the correct action.5) Will the user perceive the feedback?

– No visible or audio feedback for the user to see, but the user will be aware that they are in a mode, as they have held the mouse button down!

6) Will the user understand the feedback?– There is no feedback to understand.

7) Will the user see that progress is being made towards solution of their task in relation to their main goal and current subgoals?– Although there is no feedback, the user is aware of progress by

experience.

Example - Analyze each action for achieving subgoal B

Analysis of B23) Will the user notice that the correct action is available?

– There is no representation of the action on screen. However, experienced users of the Windows will be aware of it. However, this is an action which must be learned.

4) Will the user associate the correct action with the subgoal they are trying to achieve?– There is no representation of the action to associate with, but an

experienced user will know that is the correct action.5) Will the user perceive the feedback?

– The column will be highlighted as they move the cursor, so they should see the feedback.

6) Will the user understand the feedback?– They will understand the feedback by experience of using the Windows.

7) Will the user see that progress is being made towards solution of their task in relation to their main goal and current subgoals?– Assuming the user understood the feedback, they will see a direct

connection between the subgoal “B” (the selection of the column), and the feedback. There is no feedback to suggest that goal “A” has been achieved.

Example

Examine the sub-goal C: Set Column Size1) Will the user try to achieve the right subgoal?

– Assuming they have reached this point of the interaction, they will have selected the column and now will need to apply a command to set the width of that column.

2) What knowledge is needed to achieve the right subgoal? Will the user have this knowledge?– The user requires knowledge of how to achieve this subgoal. This

will be assessed when the method is analyzed.– The user may also need to know the correct width for the column.

The user may be able to obtain the knowledge from the system later.

Example - Analyze each action for achieving subgoal C

Analysis of C13) Will the user notice that the correct action is available?

– Yes, the action is the selection of a drop down menu which is a standard feature of the Windows user interface.

4) Will the user associate the correct action with the subgoal they are trying to achieve?– Possibly. Their current subgoal is to set the correct width given that they have already selected the

column. They are looking for a command and the only obvious source of commands are the drop down menus.

– ‘서식’ is a good match with their current goal, but so is ‘표’ or ‘편집’.– The ‘표’ menu has the command ‘표속성’ which is a very good match. It’s possible the user would

try to look for a command under these menus, and try the ‘표속성’ command. The user would lose time exploring these menus and possibly get confused.

– This will be recorded as a potential usability problem.5) Will the user perceive the feedback?

– Yes. The user will be looking at the mouse pointer and will see the menu appear at that position.6) Will the user understand the feedback?

– Yes. The drop down menu is a standard feature of the Windows and it is assumed that the user has some experience using the Windows.

7) Will the user see that progress is being made towards solution of their task in relation to their main goal and current subgoals?

– The user will see some progress to applying a command, but subgoal “C” and goal “A” remain unsatisfied

Example - Analyze each action for achieving subgoal C

Analysis of C23) Will the user notice that the correct action is available?

– The correct action is the selection of a command from the drop down menu. This is a standard action and should be very familiar to anyone who has used the Windows.

4) Will the user associate the correct action with the subgoal they are trying to achieve?

– Probably. “표속성” is a possible match to the current subgoal of setting the width of column 2 of the table. The user may try this command, or may try ‘표자동서식’. Otherwise, they will probably try other menus such ‘편집’ or ‘서식’ (see analysis of C1).

– This will be recorded as a potential usability problem.5) Will the user perceive the feedback?

– Assuming they perform the correct action, they will see the menu disappear and see the dialogue box as they will be focusing on the screen.

6) Will the user understand the feedback?– The dialogue box is a standard Windows component. Assuming they see the tab ‘표’

activated, they will recognize the dialogue box.7) Will the user see that progress is being made towards solution of their task in

relation to their main goal and current subgoals?– The user will not see any progress, and subgoal “C” and goal “A” remain unsatisfied.

3. Record the usability problems

• Problem Number• Interaction Point (Scenario/Subgoal/Step)• Description of problem and why it is a problem• Cause of Problem• How did you find the problem

3. Record the usability problems - Example

• 1. S1/ Subgoal C/C1– Possible to select ‘서식’ menu instead of ‘표’ menu

• Hard to expect where the command to set the column width would belong.– Cause of Problem?

• The ‘편집’ and ‘서식’ are good matches to the current subgoal.– How did you find the problem?

• CW4

• 2. S1/ Subgoal C/C2– Possible to select ‘표자동서식’ command instead of ‘표속성’ command

• Hard to expect where the command to set the column width would belong.– Cause of Problem?

• The ‘표자동서식’ and ‘표속성’ are good matches to the current subgoal.– How did you find the problem?

• CW4

4. Fix the usability problems

• 1. regrouping menus?• 2. regrouping commands?• 3. better label for menus or commands?