User Interface Guideline
-
Upload
joanna-may -
Category
Documents
-
view
219 -
download
0
Transcript of User Interface Guideline
-
8/2/2019 User Interface Guideline
1/24
User interface guidelines and standards: progress,issues, and prospects
P. Reeda,*, K. Holdawayb, S. Isenseec, E. Buied, J. Foxe, J. Williams f,A. Lundg
a Lucent Technologies, 7282 Silverhorn Drive, Evergreen, CO 80439, USAbUIS Consulting Inc., 3112 Fox Hollow, Round Rock, TX 78681, USA
cBMC Software, 10415 Morado Circle, Austin, TX 78759, USAdComputer Sciences Corporation, 15245 Shady Grove Rd., Rockville, MD 20850, USA
ePacific Northwest National Laboratory, 901 D Street, SW, STE 900, Washington, DC 20024, USAfBellcore, 6 Corporate Place, PYA 1M186, Piscataway, NJ 08854, USA
gUS WEST Advanced Technologies, 4001 Discovery Drive, Suite 340, Boulder, CO 80303, USA
Abstract
This article reviews progress in the development of standards and guidelines for humancomputer
interaction, including those developed within international and US standards bodies. Guidance for
incorporating software ergonomics standards and guidelines into software design and development
processes is discussed. Several different techniques that have been defined for assessing the confor-
mance of a product to guidelines are reviewed. In addition, the strategies employed by formally
approved standards developed in ISO and ANSI for determining conformance are discussed. Finally,
we discuss the prospects and challenges for software ergonomics standards and guidelines that must
be addressed as the pace of technological change continues to accelerate. 1999 Elsevier Science
B.V. All rights reserved.
Keywords: Software ergonomics standards; User interface design; User interface guidelines; Software usability
standards; Software ergonomics in design and development; Usability conformance assessment; Tools for work-ing with guidelines
1. Introduction
Since the mid-1980s, formal standards and published guidelines for software user
interfaces and HumanComputer Interaction (HCI) have increased in importance as the
Interacting with Computers 12 (1999) 119142
INTCOM 1143
0953-5438/99/$ - see front matter 1999 Elsevier Science B.V. All rights reserved.
PII: S0953-5438(99)00008-9
www.elsevier.com/locate/intcom
* Corresponding author. Tel.: 1-303-670-6955; fax: 1-303-670-6955.
E-mail addresses: [email protected] (P. Reed), [email protected] (K. Holdaway), [email protected] (S. Isensee), [email protected] (E. Buie), [email protected] (J. Fox), [email protected]
(J. Williams), [email protected] (A. Lund)
-
8/2/2019 User Interface Guideline
2/24
use of computers has become pervasive in work settings and other environments. Substan-tial benefits for both end-users and employers can result from the use of software user
interface standards and guidelines including: increased productivity, reduced mental and
physical stress, reduced training expense, improved usersystem interoperability across
applications, and improved overall product quality and aesthetics.
The purpose of this paper is to: (1) provide an overview of standards and guidelines for
humancomputer interaction, including the International Organization for Standardiza-
tion (ISO) and American National Standards Institute (ANSI) standards development
activities in the area of software ergonomics, (2) discuss how software ergonomics stan-
dards and guidelines may be incorporated into software design and development
processes, (3) review the techniques that have been defined for assessing the conformanceof a product to guidelines and how the formally approved standards developed in ISO and
ANSI address conformance, and (4) examine the prospects and challenges for software
ergonomics standards and guidelines that must be addressed as the pace of technological
change continues to accelerate.
There are a wide range of resources available to system developers seeking guidance
regarding the design of software user interfaces. Numerous books, e.g. Nielsen [1], prin-
ciples, e.g. Mayhew [2], style guides e.g. Apple Computer [3], standards e.g. ISO [4], and
guidelines, e.g. Brown [5] have been published in the extant literature. Depending on the
requirements for a particular project, some or all of these sources may be consulted.
The least formal of these sources are books (often containing principles) and guide-
lines, which usually do not undergo any systematic consensus-building process, and which
usually provide general guidance that may be applied to most user interfaces. Style guides
are generally published by software producers (or consortium of software producers) that
wish to define a specific look and feel, to a fairly high degree of detail, for applications
developed on a defined hardware/OS/middleware platform. Style guides are formal to the
extent that there is usually a systematic review process conducted with representatives
from consortia members, application developers, and other directly concerned interests,
but there is no formal, due process defined for achieving consensus among all potentially-
affected interests (e.g. end-users).Standards such as those in ISO and ANSI are distinguished by documented standards
development procedures that have been designed to ensure that all companies and
P. Reed et al. / Interacting with Computers 12 (1999) 119142120
Fig. 1. First documented use of the shall statement to create a mandatory standard.
-
8/2/2019 User Interface Guideline
3/24
individuals that are materially- or directly-effected by a particular standard will have an
opportunity to represent their interest and participate in the standards development
process. These procedures require consensus-building processes to be utilized before
any standard is approved. Each individual design guidance statement in a formal standardis either a requirement or a recommendation. A requirement must include the word
shall its statement providing design guidance, and anyone who claims conformance
with the overall standard must conform with each and every requirement (i.e. shall
statement) contained in the standard. In contrast, a recommendation must use the
word should in its statement of design guidance, and a software product does NOT
have to conform with each and every recommendation in order to claim conformance with
the overall standard. As shown in Fig. 1, humans have been challenged to comply with
standards, with varying degrees of success, for many centuries!
2. History and organization of ergonomics standards and committees
As recently as a dozen years ago there were few if any standards to be found which
could be directly applied to the software design of user interfaces. There were some
guidelines available, such as The Mitre Corporation Guidelines on Designing User Inter-
face Software published in 19831986, but no national or international efforts by formal
standards bodies were underway. In the 19851986 timeframe however, several events
occurred which became the catalyst to the formation of groups of interested professionals
who felt the time was right to consider whether or not work should begin on standards for
software user interfaces. Some of these events were:
The rapid development and delivery of different graphical user interfaces for personal
computers. While these new GUIs promoted ease of use and increased productivity,
they brought inconsistencies, incompatibilities, and questions concerning the sound
scientific basis for some of the techniques used;
The emergence of standards for the hardware or physical aspects of the user interfaces
such as VDT character legibility, screen refresh rates, or keyboard characteristics, but
no attempts at standards for the cognitive aspects of the interface. The software issues
seemed to be overlooked;
Widespread concerns over health and safety issues relative to emissions from VDTscaused others to be equally concerned about mental stress resulting from poorly
designed software. While the concerns over emissions have largely gone away, the
concerns about mental workload and mental stress have remained.
These and other similar events prompted human factors and user interface design
professionals, working through the auspices of recognized standards organizations, to
create subgroups within the standards organizations that would specifically address the
software ergonomic issues. Two of these groups, one a US national group and the other an
international group (described below), are active today and are producing practical stan-
dards which can guide developers in developing human centered user interfaces.Today there are approved or draft standards for many aspects of software user inter-
faces. For example; how should menus be constructed, how and when should color be
P. Reed et al. / Interacting with Computers 12 (1999) 119142 121
-
8/2/2019 User Interface Guideline
4/24
used, what factors should be considered when using multi-media in the interface? Standards
on these and many other issues dealing with the design of user interfaces are available or
under development within formal standards organizations throughout the world.
2.1. International software ergonomics standards
International Standards Organization (ISO), Technical Committee 159 (TC159-Ergo-
nomics), Sub Committee 4 (SC4-Ergonomics of Human System Interaction), Working
Group 5 (WG5-Software Ergonomics and Human Computer Dialogs). WG5 first met in
1985 to decide what could or should be done in developing software user interface
standards. Other Working Groups within the parent organization SC4 were busy devel-
oping standards for hardware user interface issues such as front of screen characteristics or
keyboard design, but no one was working on software concerns. So, WG5 became an
official ISO group chartered to develop ergonomic software standards for the use of VDTs
doing office work. Over the next several meetings they outlined 8 additional software parts
(see Appendix A) to be added to the 9 hardware parts already defined and underway within
the overall SC4 standard ISO 9241Ergonomic Requirements for Office Work with
Visual Display Terminals (VDTs).
WG5 is comprised of professionals from government, academia, industry, and user
groups representing approximately 12 different countries. The standards are developed
within the committee using consensus as the method for agreement on proposed material.
Once the committee believes the draft standard is ready for public review, it is sent to the
member bodies of ISO where it is reviewed, voted, and commented upon three separate
times. After each vote and comment period, the draft is returned to WG5 where theymodify the document to accommodate as many of the comments as can be agreed upon.
Finally the standard is approved as an ISO international standard and is available for use
by anyone. Although ISO standards are intended to be used on a voluntary basis, some-
times they are cited in legislation or in a procurement process. When this happens the
standards cease to be voluntary and instead become a mandatory condition of doing
business.
A summary of the titles and content of the parts of the standard being developed by
WG5 can be found in Appendices A and B. Information about the availability of ISO
standards can be found at www.iso.ch or www.ansi.org or in the source column of
Appendix A, B and C. Also, Smith [6] provides an excellent treatment of ISO andANSI ergonomic standards for computer products.
2.2. U.S. national software ergonomics standards
The Human Factors and Ergonomics Society (HFES) Human Computer Interaction
(HCI) committee was formed in 1985 by a group of interested User Interface (UI) profes-
sionals who were concerned with the lack of empirical and other reliable evidence support-
ing proposed software user interface standards. Because of the concentration of user
interface design expertise in the US, it was believed that the US should have some
mechanism for contributing to the ISO work. This group initially consisted of approxi-mately 15 people from government, academia, and industry, all members of the HFES.
They banded together in the common interest of promoting UI standards based upon
P. Reed et al. / Interacting with Computers 12 (1999) 119142122
-
8/2/2019 User Interface Guideline
5/24
scientific research and empirical evidence rather than abstract principles and ad hoc
current implementations. For the first several years the focus was on reviewing the ISO
documents and providing technical reports to ISO to aid them in their developing stan-
dards. These technical reports subsequently became the base material for several parts ofthe 9241 standard being developed by ISO WG5.
After years of contributing to the ISO activity, the HFES HCI committee felt the work
by ISO needed to be extended in several areas and updated to include more recent
technologies and research. In addition, they felt that a US standard was needed to represent
the United States perspective since the US is a major world wide developer of user inter-
face software. The HFES-200 committee was accepted as an accredited Standards Devel-
oping Organization (SDO) by the American National Standards Institute (ANSI). This
standard has been given the title HFES 200Ergonomic Requirements for Software User
Interfaces.
In 1994, the Human Factors and Ergonomics Society (HFES) obtained approval fromthe American National Standards Institute (ANSI) to develop a national standard that
addresses Ergonomic Requirements for Software User Interfaces. HFES-200 is the
title designating this project as well as the committee (formerly known as the HFES
HCI Committee) formed to develop the standard. Both ISO and the HFES also have
committees working on standards which address hardware user interface topics (HFES-
100see Appendix C).
HFES 200 will adapt parts 1217 of the 9241 ISO standard by updating them to
reflect current research, correcting any information considered to be in error, or
modifying recommendations that may have a uniquely U.S. requirement. In addition,
the standard will contain new material which was felt to be lacking in the ISOstandard. It will add major sections which deal with: use of color, computer acces-
sibility, and voice input/output. In its formative years, the HCI committee decided
that UI recommendations must be based upon a critical minimum level of sound
evidence before they could be included in their standard. A three level test of
evidence was developed which is applied to each proposed recommendation. Is
there documented scientific research? Is there documented empirical evidence? Is there
general agreement among the committee members and in documented the state of the
practice? One or more of these criteria must be affirmed before a recommendation can be
accepted.
The HCI committee works (like ISO) on a consensus method of seeking agree-
ment on the content of the draft standard. The committee will release the proposed
national standard for public review through a canvass process. This ensures that
interested and materially affected individuals can vote, comment and suggest
changes to the document. The HCI committee will review the votes and comments
and attempt to accept as many of the suggestions as can be agreed upon. If the negative
votes were in the majority, a second draft will be prepared and second canvass will be
conducted. This process continues until general agreement is secured. It is expected the
standard will be ready for the first canvass public review around late 1999. For a listing of
the content of HFES 200 see Appendices A and B. Information about the availability ofHFES standards can be found at www.hfes.org or in the source column of Appendices B
and C.
P. Reed et al. / Interacting with Computers 12 (1999) 119142 123
-
8/2/2019 User Interface Guideline
6/24
3. Using HCI standards in product/system development
Whether a projects software life cycle process is waterfall, spiral, or something else
altogether, it will accommodate and benefit from the application of HCI standards. Thissection provides some guidance for using standards during the software design and devel-
opment process. Additional guidance on humancomputer interaction design in product
development can be found in Karat [7] and Hix and Hartson [8]. Standards use involves
eight steps:
Define project objectives
Manage expectations
Select standard(s) to use
Tailor the standard(s) to your project
Determine compliance approach
Apply the tailored standard in designing the interface
Assess compliance of interface design
Conduct usability testing.
In addition, some specific success strategies can help projects get the most out of using
standards.
3.1. Define project objectives
The first step is to identify what the project aims to achieve in using HCI standards. Here
are some possibilities: Provide a familiar look and feel. Standardizing the look and feel of software allows
users to transfer the skills they learn on one piece of software to another. Training costs
are minimized.
Provide consistency. Standardization may occur in varying scopes. Examples include
the various components of an application, applications that will be used together, and
an operating system and all the software that runs on it. The more broadly a standard
can be applied, the greater the benefits.
Use human factors findings. Standards take advantage of the large body of human
factors research and accepted practice. The authors of the standards assimilate and
interpret the research, turning it into guidelines (best practice) for designers to follow.
Streamline development. Standards make many design decisions routine. This frees
designers to spend time on decisions that are more difficult or critical.
Evaluate usability. Standards provide one basis for judging the usability of products.
All else being equal, a product that meets an HCI standard should be more usable than
one that does not.
Comply with requirements. Standards compliance for the software may be required
by the buyer (e.g., in many US government contracts) or by law (e.g., in Europe).
3.2. Manage expectations
The second step is to manage project expectations, as many organizations may have
P. Reed et al. / Interacting with Computers 12 (1999) 119142124
-
8/2/2019 User Interface Guideline
7/24
inflated ideas of what standards can do. First, be aware of some general limitations of
standards:
No guarantees: Following an HCI standard will not guarantee that a product or system
is usable. Standards cannot address sufficiently the information structure and content,often the hard part of HCI design.
Evolution: User interface knowledge and technology are continually changing and
improving, and so are standards. Although many of their guidelines are based on
consistent human characteristics and are enduring, others need to be replaced by
more current advice when new research and practices suggest better techniques.
There are times when a designer should try innovative solutions before the standards
have caught up. (See Section 5 for a more in depth discussion of this issue).
Generality: Standards provide guidance to meet a broad range of circumstances, not to
help design for specific intended users and their tasks. It is usually beneficial to tailor
the general guidelines into task- or system-specific design rules.
Leading edge? Standards provide guidance that is stable and well accepted. They are
not intended to be leading edge.
State-of-the-art? Long development times can put standards somewhat out of date
before they are released. It is not unusual for an HCI standard to spend several years or
more in development.
Strong? The consensus process involved in standards development ensures that
guidance is well thought out and meets the needs of many parties. However, many
proposed guidelines are controversial and are eliminated, made optional (should
rather than shall), or qualified. Platform-specific? HCI standards are designed to be platform independent. For exam-
ple, ISO 9241 applies to all GUIs and character based interfaces. Platform indepen-
dence ensures that standards are nonproprietary and can be applied widely; however,
the guidance must be general and can be difficult to interpret.
Universal? Design situations vary and a given recommendation is unlikely to be
optimum in all cases. Applying a standard requires the designers judgment.
3.3. Select standard(s) to use
Which standards should be followed, and whether they are required or optional, depend
on project application domain and location, and on project requirements. First, determine
whether you are looking to start with high-level principles, more specific guidelines, or
organizational conventions.
Principles specify high-level goals a user interface should meet (e.g. minimize memory
load). Guidelines specify techniques that can be used to accomplish these goals and that
have been shown to have appropriate usability characteristics (e.g., provide a keyboard
accelerator for every important menu item). Standards sometimes provide principles (e.g.,
ISO 9241 Part 10) and usually provide guidelines.In contrast, conventions specify which of a set of acceptable design techniques or
elements an organization is going to adopt and how it is going to apply them to its
P. Reed et al. / Interacting with Computers 12 (1999) 119142 125
-
8/2/2019 User Interface Guideline
8/24
Fig. 2. Factors concerning the users, their tasks and environment, and the hardware and software implementation drive th
project.
-
8/2/2019 User Interface Guideline
9/24
product(s) (e.g., use the letter T as the accelerator for the Templates menu item). Orga-
nizations typically develop their own design conventions.
Second, determine which type(s) of standard applies to your project:
International (e.g., ISO, ITU) standards are applicable worldwide. Regional (e.g., CEN) standards are applicable within a given region such as Europe.
National standards (e.g., ANSI, DIN, AFNOR, SIS, BSI, CSA) are applicable only
within a given country.
Military or government standards (e.g., DoD, MIL, NASA, ESA) are applicable to
products and systems being developed for specific military or government organiza-
tions.
Platform (e.g., Mac OS, CUA, Windows, Motif) style guides are applicable to all
applications developed for a given operating environment.
Independent (e.g., books by individual authors, company-specific) standards are either
very general or are applicable to the domain for which they are written.
3.4. Tailor the standard to your project
Tailoring is the process of creating a detailed, project-specific HCI specification from
the set of design guidelines in the selected standard(s).
First, select the guidelines that apply to your project. If, for example, the project team
selects ISO 9241 and the user interface is going to include menus but not direct manip-
ulation, Part 14 will directly apply and Part 16 will be irrelevant. Within Part 14, the
project team should identify the specific shalls and shoulds that apply to your users,their tasks and environment(s), and relevant implementation constraints. Fig. 2 shows the
factors that you should consider when determining which guidelines apply to your HCI.
See Section 4 for detailed information on determining applicability.
The other factor that will drive guidelines selection is compliance. You may need to
follow the shalls but not the shoulds; you may have to follow both shoulds and
shalls; or you may not be required to comply with any specific guidelines in the stan-
dard.
Next, translate the selected guidelines into specific design rules for your project. Review
the selected guidelines in the context of the users, their tasks and environment, your
implementation constraints, and the overall requirements for the product or system. Useprototyping as necessary to refine and validate the design rules.
3.5. Determine compliance approach
Standards compliance is most effectively applied during product design and implemen-
tation. If the product or system is not evaluated for compliance until the end of the
development cycle, changes will be difficult to make.
Compliance with some guidelines may be difficult to determine by examining the
interface. They may require knowledge of the users, the designers intent, and the devel-
opment and testing procedures. Examine the tailored standard to identify the best compli-ance approach for each guideline.
For each guideline in the tailored standard, determine the following:
P. Reed et al. / Interacting with Computers 12 (1999) 119142 127
-
8/2/2019 User Interface Guideline
10/24
Level of compliance required: What is the level of compliance that is desired or
required for your project? As mentioned earlier, you may have to follow just the
shalls, or both shoulds and shalls; or compliance may be at your discretion.
Method of assessment appropriate for that guideline: Is the guideline best suited forcompliance assessment by content analysis, documented evidence, observation, analy-
tical evaluation, or empirical evaluation?
The degree of compliance documentation also varies. Detailed information about
compliance with each guideline and the method used to judge compliance may be
required; a letter stating that the product is compliant may be required; or no statement
may be required.
As the standard is tailored, develop and maintain a checklist to help in assessing
compliance. If a checklist was provided with the standard, start with a copy of that and
refine it, or create a checklist for the selected standard(s). (See Section 4 for detailed
information on evaluating compliance).
3.6. Apply the tailored standard in designing the interface
The application of standards to HCI design is most effective when it is part of an overall
usability engineering process for the product or system. Standards may guide this process
as well as the characteristics of the interface itself. For specific information, see
ISO 13407, Human Centered Design of Interactive Systems.
First, ensure that everyone who will be involved in making HCI design decisions is
familiar with the tailored standard and the checklist.
As you make, modify, and implement HCI design decisions, consult your tailored
standard and your checklist to ensure that your design follows the standard.
Issues will probably arise during implementation that will necessitate revising the
tailored standard. Hardware and/or software configurations may change; functions may
need to be added or removed. User interactions (e.g., messages about the status of the
system or the software) will need to be designed that could not have been foreseen or
designed in advance. A process for revising the tailored standard should be established, in
which all parties affected by a change are informed about it and (where appropriate) are
involved in determining design decisions.
3.7. Assess compliance of interface design
As stated earlier, compliance with some guidelines may not be assessable from an
examination of the interface, but may require knowledge of the users, the designers intent,
and the development and testing procedures. (See Section 4 for detailed information on
assessing compliance).
3.8. Conduct usability testing
Standards can promote usability but by themselves cannot guarantee it. Even with theassistance of organizations (such as test houses) or tools (such as checklists or software
products) that can verify or even certify standards compliance, you still cannot be certain
P. Reed et al. / Interacting with Computers 12 (1999) 119142128
-
8/2/2019 User Interface Guideline
11/24
that your product or system is adequately usable until you have subjected it to usability
testing.
After assessing standards compliance, conduct usability testing to evaluate how well the
design, prototype, or built product/system supports the effectiveness, efficiency, and satis-faction of the intended user population as they perform their tasks in the intended context
of use. Have a representative sample of users perform realistic tasks, and collect two types
of information:
Qualitative: The problems and sources of confusion that the users have in doing the
assigned tasks, and an identification (if possible) of HCI design changes that could
alleviate the problems.
Quantitative: Measures of the effectiveness (completeness and accuracy) and effi-
ciency (resources expended with respect to completeness and accuracy) of the user
tasks performed, and of the users satisfaction in using the product or system to perform
these tasks.
Use this information to identify and prioritize the areas of the HCI that need further
change, and to drive the design of the necessary changes. For more information on
usability and usability evaluation, see ISO 9241, Part 11.
3.9. Success strategies
Here are some strategies for success when applying standards:
Follow the guidelines in the standard throughout product design and development;
dont wait until the end to think about standards Make sure all appropriate members of the design team are aware of the standard. It
seldom works well to depend on a standards czar to achieve compliance.
Use tools whenever possible. For example, some GUI builders take care of setting up
the menu bar, adding keyboard accelerators, etc.
Do not follow standards blindly. Conduct usability testing to verify that the guidelines
are correct for your situation and that the interface is usable.
4. Compliance and conformance evaluation techniques
A number of methods have been defined in the field of software ergonomics for deter-
mining the applicability of a standard or guideline and for determining whether the
requirement or recommendation has been met. The particular choice of a method depends
on various project drivers and constraints. Established methods (extracted from the human
factors literature on evaluation techniques) include:
Content analysis This refers to the analysis of any documents which may describe
the general and specific properties of user interface and the system. Such documents
may include design documents containing system and user requirements, manuals, user
guides, etc. Content analysis is used to determine whether a standard or guideline isapplicable on the basis of its relevancy to the particular system under development or
being evaluated.
P. Reed et al. / Interacting with Computers 12 (1999) 119142 129
-
8/2/2019 User Interface Guideline
12/24
Documented evidence This refers to any relevant documented information about the
task requirements or characteristics, flow of work, user skills, user aptitudes, existing
user conventions or biases, test data from the design of similar systems, etc. Such
information may be used to determine whether a given recommendation is applicableas well as whether a standard or guideline has been met due to previously document
supporting evidence as to the appropriateness of a particular solution.
Observation This consists of the examination or inspection of a user interface
element for the presence of a particular observable property (e.g., applicability is
based on the observation that a source document is used for input or a recommendation
has been met because it is observed that instructions are provided on the form). Obser-
vations can be made by anyone who has the necessary skill to systematically check the
interface and determine if it has the particular properties associated with the applic-
ability of, or compliance with, a given conditional standard or recommendation. Due to
their obvious nature, such observations readily can be confirmed by another person. Analytical evaluation Refers to Informed judgments concerning the properties of
a user interface by a relevant expert (i.e., of those properties). This method is typically
used for the evaluation of properties which can be judged only in the context of other
information or knowledge. In addition, analytical evaluation may be appropriate when
the system exists only in terms of design documents, user populations are not available
for empirical evaluation, or time and resources are constrained. Analytical evaluation
can be used to determine whether a particular recommendation is applicable, e.g., to
determine if a temporary save function would be appropriate to the users task. In
addition, analytical evaluation can be used to determine if a particular standard or
recommendation has been met, e.g., analytical evaluation might be used to determinewhether a form has distinctive labels. In the above case, distinctive is the judgmental
aspect. Analytical evaluation can be performed by any suitably qualified person who
has the necessary skill and experience to judge the relevant property of the interface.
Where these properties concern the application of ergonomic principles, the expert
should possess appropriate skills in software ergonomics. If the properties concern
the work environment, system characteristic, or other aspects of the design, the
judge needs to be an expert in the particular relevant domain. It is important to note
that analytical evaluation can verify the tenability of a design, but cannot validate the
design. Validation can be accomplished only by using empirical evaluation.
Empirical evaluation Is the application of test procedures using representative end
users to determine the applicability of a standard or recommendation. This method is
most appropriate when a prototype or the actual system is available and potential or
actual user population representatives are available. Many kinds of test procedures
could be used, but in each case the test subjects must be representative of the end
user population and be of sufficient number that the results can be generalized to the
user population as a whole. For example, empirical evaluation to determine the applic-
ability of whether error feedback should be provided as soon as the user completes a
field in a form could be done by having typical users enter and correct field values under
both the condition of providing the feedback immediately after the field is completedand providing the feedback prior to transmitting the completed form. The task perfor-
mance of end users using a software application could be analyzed to determine
P. Reed et al. / Interacting with Computers 12 (1999) 119142130
-
8/2/2019 User Interface Guideline
13/24
whether the various standards or recommendations have been met. Such tests could be
performed both during the development process (e.g. by prototyping) and after the
design and implementation of the system (e.g. by system evaluation techniques) and
could be based on both objective and subjective user data. Although empirical evalua-tions typically are used to determine whether standards/guidelines have been met by
comparing the test results against specific software ergonomic standards or recommen-
dations, it also may be necessary to evaluate test results in terms of effectiveness. That
is, the dialogue supports the user in his/her task in a manner which leads to improved
performance, results in a difficult task being performed with less difficulty, or enables
the user to accomplish a task that he/she would not have been able to otherwise). It
should be noted that empirical evaluation should be conducted by individuals posses-
sing appropriate skills in testing methodology and evaluation techniques.
The above methods for determining conformance can be applied to any evaluation of
software ergonomic standards or guidelines. Although shalls are given more importance
than shoulds in most standards, to optimize the quality of the user interface it still is
important to evaluate whether the applicable shoulds have been met. In addition, some
software procuring organizations may require that all of the recommendations as well as
the requirements be met.
4.1. Evolution of the conditional approach for specifying software ergonomic
standards/guidelines
The conditional approach to stating ergonomic guidelines for software applications
was initially developed by the Human Factors Society Human Computer Interaction (HCI)Standards Committee (see Williams [8]). During the process of defining guidelines rele-
vant to user interface design from various literature sources, the HCI Standards Committee
noted that a major problem with most proposed guidelines was that the applicability of a
particular guideline depended on a number of variables. The most common variables were:
the type of user (e.g. expert vs. novice), the task activity being accomplished, the system
configuration being used, and the environment. As a result, the HCI Standards Committee
began to use an ifthen structure to state the conditions relevant to the applicability of a
guideline where ever sufficient evidence allowed. Since a number of the initial drafts of the
dialogue parts of ISO 9241 were submitted by the HCI Standards Committee as a US
contribution, this approach carried over into the software parts of ISO 9241.
The conditional approach is also used to determine whether or not a major section/
clause of a standard is applicable. For example, if the application does not using pointing
devices, recommendations concerning the use of pointing devices would not apply. The
following provides some examples of conditional statements extracted from some of the
ISO 9241 software parts:
User characteristics:
If casual or intermittent users may enter data on the form, instructions should be
provided on the display (or easily accessible through a help facility).If users are experienced and/or need fast access to specific menu options, one or both
of the following methods should be applied:
P. Reed et al. / Interacting with Computers 12 (1999) 119142 131
-
8/2/2019 User Interface Guideline
14/24
Task characteristics:
If the task requires error management by the user, the dialogue should provide a
means (information and/or function) of enabling the user to continue.
If information concerning unavailable options is not required for the task or othersupporting activities (e.g., training), only options available to the user.
System capabilities:
If a system or application has modes (i.e. a specific user action has different results
depending on the state of the system), users should be able to discriminate the current
mode.
If the system response to option execution will be delayed (more than three seconds
after initiation), an indication should be provided to the user of the system.
Interface design characteristics:
If options are placed in columns, options and groups of options should be visually
distinct from one another and should be arranged to minimize search time.
If brief error messages are displayed, users should be able to request more detailed
on-line information or should be referred to additional off-line information.
4.2. Traditional view of conformance to a standard and problems for software
Most standards, particularly hardware standards, specify attributes that can be objec-
tively measured or directly observed in some way (e.g., voltage, dimensions, pressure).Conformance to such standards can be demonstrated by measuring the product or device to
determine whether it complies with the requirements stated in the standard. In addition,
these standards can establish those attributes that are required (shall statements) from
those that are desirable, but not required (should statements).
Unfortunately, in the domain of software user interface standards, it is rarely possible to
unambiguously state what is required and what is desirable. The conditional nature of
many software ergonomic standards statements requires the applicability of a particular
statement to depend on a number of variables. Due to these often-complex dependencies,
most software ergonomic experts are unwilling to designate such statements as a require-
ment because of the difficulty of precisely determining the dependencies. An additionalconcern is that developers will only pay attention to requirements and ignore recommen-
dations. It is also important to note that organizations that utilize a consensus process for
developing standards (e.g., ISO and ANSI) have a much more difficult time obtaining
agreement on shalls than do organizations which follow a less open review process (e.g.,
ITU, government agencies).
4.3. A pragmatic approach to conformance with software user interface guidelines
Utilizing the conditional approach and the evaluation methods described above, a two
step process for determining conformance to software ergonomic standards/guidelineswas developed based on extensive work in ISO. The first step of this process is to deter-
mine the applicability of the standard or guideline and the second step is to assess
P. Reed et al. / Interacting with Computers 12 (1999) 119142132
-
8/2/2019 User Interface Guideline
15/24
Fig. 3. ISO 9241 conformance evaluation process flow chart.
-
8/2/2019 User Interface Guideline
16/24
whether the requirement/recommendation has been met by the developer, i.e. compli-
ance (see Fig. 3 for a flow diagram of the process). Applicability can be determined both
as part of the early design process when the designer is specifying elements of the inter-
face, and later by both the designer and evaluators as part of any heuristic evaluation of thedesign. It is strongly recommended that the evaluation be made using the most detailed
method which is appropriate to the requirement or recommendations. For example, it
would not be necessary to do usability testing (empirical evaluation) to determine that
menu options keyboard equivalents (underlined character) are shown when it can be
readily observed. The intent of the pragmatic approach is to minimize the time and effort
required of an organization to determine whether they comply with a standard or guide-
line. The pragmatic approach to conformance has been incorporated in the software parts
of ISO 9241 and is planned to be utilized in the HFES 200 standard as well.
4.4. Conformance to the ISO 9241 software standard
All of the dialogue parts of ISO 9241 now use a common approach to conformance as
was agreed by the national bodies on Part 14. This approach includes the two step process
(Applicability and Adherence) as a sample procedure, and allows users of the standard to
use other processes. It should be noted that the term adherence was agreed because
compliance was thought to be too related to requirements and would not apply to
recommendations. The only actual statement of conformance in 9241 Part 12 through
17 is: If a product is claimed to have met the applicable recommendations in ISO 9241-
(14), the procedure used in establishing requirements for, developing, and/or evaluating
(menu dialogues) shall be specified. The level of specification of the procedure is a matterof negotiation between the involved particles.
The two step process recommends that the user of the standard first determine which of
the recommendations are applicable and then determine whether those recommendations
that are deemed applicable have been met. It should be noted that the approach described
in ISO 9241 suggests applying the method most relevant and cost effective to determining
the applicability and adherence for a particular recommendations. For example, if the
recommendation specified an attribute that could be directly observed (e.g. left justifica-
tion of a menu option list), the method indicated as the most relevant would be observa-
tion. Thus the sample procedure is intended to be as cost effective as possible.
Several other means for determining conformance also have been devised. Some of the
test houses in Europe (e.g. TUV Rheinland) have been evaluating products for their
conformance to the software as well as the hardware parts of ISO 9241. The actual method
used for these evaluations is not known since the test houses have not revealed the exact
procedure used. A somewhat different approach has been used by Lloyds Register who
has developed a procedure to assess the application of the ISO 9241 software parts as part
of the overall software quality assessment procedure. Another approach to evaluation of
the software ergonomic parts is the 9241 Evaluator developed in Germany (Oppermann,
R and Reiterer, H. [9]). In their paper, Oppermann and Reiterer provide an overview of
different evaluation techniques and describe the 9241 Evaluator as an expert-basedmethod for conformance testing of the 9241 standard. While an important step towards
the development of a performance support system for standards evaluation, the 9241
P. Reed et al. / Interacting with Computers 12 (1999) 119142134
-
8/2/2019 User Interface Guideline
17/24
Evaluator is intended to be used only by experts after the system has been developed (at
least to a prototype stage). However, the ISO 9241 software parts state that the intended
users include designers as well as evaluators and that the standards should be used during
all stages of system design and development.All of the approaches to conformance described above satisfy the conformance clause
stated in the software parts of ISO 9241. The HFES 200 Committees current direction is
to use the same basic approach to conformance as stated in ISO 9241. However, it is
important to note that many organizations (particularly in the United States) are stating the
same recommendations as can be found in the ISO and emerging ANSI software ergo-
nomic standards as requirements (i.e., shall statements). For example, many of the
statements found in the recently released Federal Aviation Administration The Human
Factors Design Guide section on human computer interfaces contain shalls.
4.5. Utilizing the pragmatic conformance approach for evaluating other softwareergonomic standards or guidelines
As noted above, the two step process and the methods used to determine applicability
and adherence can be used to evaluate many types of standards and guidelines. The
process is particularly relevant where standards or guidelines have been qualified in
terms of the conditions in which they apply (e.g., conditional recommendations as
described above). Currently many style guides require users to complete a check list
indicating both which style guide requirements and recommendations apply and which
have been met. Although most style guide requirements and recommendations are look
and feel type statements which usually can be evaluated for compliance by observation,many recommendations concern design strategies which could be evaluated by means of
many of the other applicability and adherence methods described above. The pragmatic
conformance approach, particularly when a checklist is incorporated as is used in Annex A
of the 9241 software parts, also provides an audit trail which can be used to satisfy
ISO 9000 and ISO 9001 requirements.
5. Future challenges and the evolution of HCI standards
Two issues commonly identified in discussions of standards are: (1) Standards are not
useful., and (2) Standards stifle creativity. A fundamental issue underlying each ofthese statements is the belief that technology is changing so rapidly and standards take so
long to create that they either do not address the current set of technologies in which design
is taking place or they should not address the technologies. The argument that they should
not address the current environment is based on the assumption that the behavior that we
need to measure in order to provide empirical justification for standards is only now
emerging for these technologies. Standardizing prematurely may result in standardization
of the wrong things. Further, it would follow that creating standards that apply to future
technologies should be impossible.
5.1. Moores law, the Internet, and HCI standards
There is a great deal of face validity to these arguments. Many human computer
P. Reed et al. / Interacting with Computers 12 (1999) 119142 135
-
8/2/2019 User Interface Guideline
18/24
interface designers have moved from programming with punch cards and paper tape to
designing animations, collaborative video conferencing, and virtual reality interfaces,
within the time it takes a child to complete their schooling. The explosive growth of the
Internet and novel technologies based on the Internet is well known. A recent conferencemarking the first 50 years of computing invited pioneers and seminal thinkers in the
computing field to use the last 50 years as a foundation for predicting the next 50. Bell
and Gray [10] pointed out that with processor speed, storage capacity, and transmission
rates doubling every 18 months consistent with Moores Law, the clear trend is the emer-
gence of essentially zero-cost, specialized, fully networked computers embedded in
virtually everything around us, and attached to our bodies. Bell and Gray state The
only limits will be our ability to interface computers with the physical world-that is, to
design the interface between cyberspace and physical space. It would appear that
virtually any interface that can be conceived will be possible to build.
Nathan Myhrvold [11], at the same conference, pointed out that software is also follow-ing Moores Law. He stated So we are talking 20 years hence about computers being able
to do in 30 seconds what one of todays computers at a comparable price would take a year
to do. But 40 years hence we are talking about a computer doing in 30 seconds what one of
todays computers would take a million years to do. So what will we do with all that
power? Well, the answer is software. The increasing pace of software evolution will drive
corresponding changes in HCI design. Those looking at a standards process that attempts
to integrate years of design guidance about what works and does not work in user inter-
faces with a given technology may be justifiably skeptical, especially given that it may
take years for the standard to be approved once it is defined. We already know that in the
time it has taken to approve standards addressing personal computer technology, we havemoved into designing for virtual reality. As we move forward, we will be designing the
next generation of interfaces without the benefit of standards utilizing behavioral research
focused on human interaction with next-generation technology. Fortunately for designers,
human capabilities and limitations in perception and performance do not evolve as rapidly
as modern computing technology!
Practically speaking, therefore, can standards be written in an environment driven by
Moores Law? Can standards be written that will be useful for designers working with
technologies emerging for the Internet, computers appearing in novel forms such as tele-
phones and televisions, and technologies with which we have had even less experience?
Should they be written? We believe they can and should be written. For useful standards to
be written for the next generation of interfaces, however, we will need to continue to
improve the process of writing and maintaining the standards and the sophistication of our
taxonomy of standards will need to increase.
5.2. Lessons we have learned
In the last few years, several important lessons have been learned that will shape
effective standards activities in the future. First, a key assumption that we reached in
developing the standards for ANSI and the work for ISO is the assumption of humility.For standards to accommodate rapidly changing circumstances, they need to build on what
should be done more than what shall be done. In other words, they need to provide
P. Reed et al. / Interacting with Computers 12 (1999) 119142136
-
8/2/2019 User Interface Guideline
19/24
guidance and flexibility, such that the final arbiter of a design is the user. Users adapt
effectively to the situation in which they find themselves. The combination of the design
elements and the structure of the design as it supports the users task can have as much
influence on user satisfaction and performance as individual aspects of the design that havebeen designed according to a standard. Applying the guidelines may even require tradeoffs
between guidelines, such that a given guideline must violated in order to apply a guideline
that is more important in the circumstance. The conditional approach to specifying
design guidance must be extended to provide new guidelines that better address the
context of application will be needed, and we will need to continue to write guidelines
that can be adapted to and provide guidance in changing circumstances.
A second lesson that will shape standards in the future is that it is important to continue
to improve our ability to define standards that are useful and yet that are as technology
independent as possible. Fundamentally, human beings do not change as rapidly as the
technologies they are using. At a very basic level, our needs remain the same. The humanperceptual system works the same now as it did 15 years ago. The properties of the
cognitive system remain the same. This, I believe, is the key to standards in the future.
If we can define standards in terms of the reliable phenomena of human behavior, and
eventually in terms of any theoretical consensus, we are more likely to define standards
that are useful when applied to emerging technologies. This implies that a more tightly
integrated dialogue between academia and industry will be needed, with research direc-
tions being set at least in part as a result of direction from those writing or requiring
standards.
A third lesson is that the best standards emerge from data rather than politics. While we
began the standards effort attempting to build standards largely from well controlledlaboratory studies, over time we have found that data and conclusions emerging from
practice are also important in shaping a useful standard. Harnessing the wealth of data that
is automatically generated in the current environment of design mutation and evolution
on the Internet and in the world of consumer products would be expected to further inform
the standards process. It would be reasonable to predict that a day of activity on the
Internet results in more people being exposed to more design alternatives than all the
labs where testing is currently taking place could produce in a year.
5.3. Improving our approaches to user interface guidelines and standards
Standards efforts, like usability work in general, may need to harness the trend of trying
a design and rapidly fixing it in response to feedback from the field in order to respond
more quickly to emerging technologies. Indeed, one approach to increasing the ability of
standards to reflect emerging technologies is to explore new ways of developing standards,
new processes. Perhaps the solution is in finding new ways to involve the stakeholders in
the development process, finding ways to access emerging data being collected in other-
wise inaccessible corporate labs, and shortening cycles of approval. The challenge, of
course, is that national and International standards involve consensus building. It is not
clear that we have identified the processes yet that meet the needs of technical standardsdevelopment as well as the political and cultural realities of standards approval.
Another approach for developing useful standards in the future is to define standards
P. Reed et al. / Interacting with Computers 12 (1999) 119142 137
-
8/2/2019 User Interface Guideline
20/24
that are virtually technology independent. These standards would define how task proper-
ties should influence design primitives, and their interactions. An example of this would be
a set of standards that recommends list length for the task of selecting a category from a list
presented auditorially, when the contents and names of the categories are not known aheadof time. The approach that would be most effective to these standards would be the
integration of behavioral science findings and theory focusing on the capabilities and
limitations of human beings, and applied research identifying the fundamental elements
of tasks as shaped by the properties of the user.
The next level of standards would be standards applying to combinations of these
primitives associated with generations of technologies. An example of this kind of stan-
dard would be standards that describe the design of a form that supports pointing and
clicking, or the category of window currently typically used in Macintosh and Windows95
applications. This is the level of standard that is most typically defined today. These kinds
of standards currently are defined based on research using the technologies for which thestandards are being written (which means the standards appear well after they are needed
by designers), and based on the years of experience and logical thinking of those writing
the standards. In the future, this level of standard could initially be shaped in part by
applying the higher level standards to relevant properties of the specific technology
(coupled with that same experience and logical thinking that is currently being applied).
The resulting standards could be validated and extended (where the higher levels do not
sufficiently address the design options) through usability studies of specific implementa-
tions. With a sound base of technology-independent standards, it should be easier to
produce flexible, useful standards for emerging technologies in a timely way.
This structure depends on a richer understanding of human behavior in applied settingsthan we currently have, and it has the potential to provide a mechanism for useful design
guidance to precede technology-dependent data collection. A process governor will have
to be added in order to restrain standards bodies from spinning too far from the empirical
base underlying the two levels. While lower level standards should be consistent with
higher level standards, they would also be expected to be inherently technology depen-
dent. These standards would include standards that attempt to address consistency of
applications built for a given operating system and standards that attempt to define unique
brands within such environments.
An area that is missing from even this view of standards, however, is the relationship
between design for usability and emotion. Increasingly it is being recognized that even a
successful business application needs to be attractive and often fun. Applications built
using emerging multi-media technologies increasingly explicitly engage the total person.
When the aesthetic elements of the design result in an application being learned more
completely and used more often, these elements should be included in the definition of
usability and should be instantiated in standards. Macroergonomic considerations of when
applications are adopted and product usability as it interacts with brand identity also
suggest that the definition of usability will be broadened in the future. To the extent
that standards serve as an instantiation of accumulated knowledge about how to create
usable design, the standards will also address a wider range of design considerations.Extending our knowledge of the relationship between aesthetic elements, design that
impacts enjoyment, the creative design process itself and performance, and design as it
P. Reed et al. / Interacting with Computers 12 (1999) 119142138
-
8/2/2019 User Interface Guideline
21/24
relates to the diffusion of technology is needed. Combined with the standards approaches
that we have just defined, these information sources will enrich the body of available
design guidance, and produce a set of standards that are technology independent and that
can be effectively applied to technologies that will emerge in the future.
Appendix A. Process standards
P. Reed et al. / Interacting with Computers 12 (1999) 119142 139
Process standards (HCI/GUI design, evaluation, and verification)
Standard Title Contents Source
ISO 9241 (Part 11) Ergonomic
requirements for
office work withvisual display
terminals
9241 Part 11: guidance on
usability
International Standardization
Organization, 1 Rue de Varembe
Case Postale 5b, CH-1211, Geneva20, Switzerland;
Tel.: 41-22-734-1240;
URL: www.iso.ch
MIL-STD-1801 Usercomputer
interface
(Various requirements for
verification)
Wright-Patterson AFB ATTN:
ASD/ENES Wright-Patterson AFB,
OH 45433
ISO/CD/13407 Human-centred
design for
interactive systems
Assessing the benefits of
human-centred design.
Interactive system
development processes and
human-centred design.
Planning the human-centred
design process. Human-
centred design activities.
Managing and documenting
human-centred design.
International Standardization
Organization, 1 Rue de Varembe
Case Postale 5b, CH-1211, Geneva
20, Switzerland;
Tel.: 41-22-734-1240;
URL: www.iso.ch
-
8/2/2019 User Interface Guideline
22/24
Appendix B. Software user interface standards
Software standards and guidelines (HCI/GUI appearance and behavior)
Standard Title Contents S
ISO 9241 (Parts 10-17) Ergonomic requirementsfor office work with
visual display terminals
Part 10: Dialogue principles.Part 11: Guidance on usability.
Part 12: Presentation of
information. Part 13: User
guidance. Part 14: Menu
dialogues. Part 15: Command
dialogues. Part 16: Direct
manipulation dialogues. Part 17:
Form filling dialogues.
IO
V
C
S
T
U
HFES-200 ANSI Project (under
development; canvass draft due
in late 1998)
Software user interfaces Same as ISO 9241, Parts 10-17,
plus color accessibility for
people with disabilities voice
input/output
H
E
9
T
U
ESD-TR-86-278 Guidelines for designing
user interface software
Data entry, data display,
sequence control, user
guidance, data protection, data
transmission
E
A
C
M
MIL-STD-1472D (Section 5.15) Human engineering
design criteria for
military systems,
equipment and facilities
Data entry, data display,
interactive control, feedback,
prompts, defaults, error
management
C
m
A
A
P1201.2 This failed two ballotsand was withdrawn. It is here
because other documents may
reference it.
Graphical user interfacedrivability Keyboards, pointing devices,menus, controls, windows, user
guidance, common user actions
IH
P
T
f
-
8/2/2019 User Interface Guideline
23/24
Appendix C. Hardware user interface standardsHardware standards and guidelines
Standard Title Contents Source
ANSI/HFES-100-1988
(currently under revision)
Human factors
engineering of visual
display, terminal
workstations
Working environment visual
display design, workstation
design, keyboard design,
measurement techniques
Human
Society
Monica
Tel.:
Fax:
URL: w
ISO 9241, Parts 3-9 Ergonomic requirements
for office work withvisual display terminals
Visual display requirements,
keyboard requirements,workplace requirements,
environmental requirements,
display requirements with
reflections, requirements for
displayed colours, requirements
for non-keyboard input devices
Internat
OrganizCase Po
Geneva
Tel.:
URL: w
MIL-STD-1472D
(selected parts)
Human engineering
design criteria for
military systems,
equipment and facilities
(various) Comma
comma
Redston
NASA-STD-3000
(selected parts)
Mansystems
integration standard
(various) Johnson
SP34 N
-
8/2/2019 User Interface Guideline
24/24
References
[1] J. Nielsen, Usability Engineering, Academic Press, New York, 1993.
[2] D. Mayhew, Principles and Guidelines in Software User Interface Design, Prentice-Hall, Englewood Cliffs,
NJ, 1992.
[3] Apple Computer, Macintosh Human Interface Guidelines, Addison Wesley, Cambridge, MA, 1995.
[4] ISO 9241-14: 1997 (E), Ergonomic requirements for office work with visual display terminals (VDTs):
Menu dialogues, International Organization for Standardization, Geneva, Switzerland, 1997.
[5] C.M. Brown, Human-Computer Interface Design Guidelines, 1988.
[6] W. Smith, ISO and ANSI Ergonomic Standards for Computer Products, Prentice-Hall, Upper Saddle River,
NJ, 1996.
[7] J. Karat (Ed.), Taking Software Design Seriously: Practical Techniques for HumanComputer Interaction
Design Academic Press, New York, 1991.
[8] J.R. Williams, in: G. Salvendy, M. Smith (Eds.), Guidelines for dialogue design, in Designing and Using
HumanComputer Interfaces and Knowledge Based Systems, Elsevier Science, Amsterdam, 1989.
[9] R. Oppermann, H. Reiterer, Software evaluation using the 9241 Evaluator, Behaviour and InformationTechnology 16 (4/5) (1997) 232245.
[10] G. Bell, J.N. Gray, The revolution yet to happen, in: P.J. Denning, R.M. Metcalfe (Eds.), Beyond Calcula-
tion: The Next Fifty Years of Computing, Copernicus, New York, 1997, p. 3.
[11] N. Myhrvold, The future of software, the software industry, and possibly, the features of Windows 47,
Speech given at ACM97, The Next 50 Years of Computing, March 3, 1997. Viewable at www.vxtreme.
com/live/acm97/archived/nathanmyhrvold/nmyhrvold.html.
P. Reed et al. / Interacting with Computers 12 (1999) 119142142