Usability Testing - 4imprint Learning...

4imprint.com

Usabi l i ty Test ing

© 2013 4imprint, Inc. All rights reserved

Usabi l i ty test ing: Does your product make the grade? What is usability testing? More importantly, should you care? If you create

products, tools or applications that are used by customers, then usability testing is

something you should not only care about, but something you should consider.

Research shows that successful companies rely on usability testing to create

effective products, tools and websites. Did you know, for example, that at

any given time, Amazon.com® has about 200 usability tests underway?1 Brian

Eisenberg, co-founder and chairman emeritus of the Web Analytics Association,

explained it more plainly: “Alec Baldwin says in the movie Glengarry Glen Ross

to ‘always be closing.’ Online you have to ‘always be testing.’”2 And in the case

of usability testing, website design is only one of the areas where it is valuable; it

can also be used for any product or tool that requires human interaction.

Let’s begin with a basic definition. Usability testing is a method applied

to user-centered designs and products to evaluate how someone actually

uses a product. For example, you could observe a customer trying to

assemble something you designed with printed instructions and see if the

task is completed as intended. Or, you could watch a user navigate your

website to find product information or services. Specifically, it measures

a human-made product’s capacity to meet its intended purpose.3

Usability testing is commonly used for consumer products, websites, Web

applications, computer interfaces and devices. It measures the ease of use

of a specific object or set of objects, and shows a company how people

use its product or application in real life scenarios.

Although the concept of usability testing is fairly clear-cut, there’s a lot that goes

into the process. It needs to be done properly to illicit value and it should follow

a defined process. This Blue Paper® looks at the evolution of usability testing and

reviews the processes and methods that are commonly used. There are many

helpful resources that can help you understand the basics, and these are explored

throughout the paper. If you haven’t conducted usability testing before, consider

this paper your introduction. Rest assured, however, there will not be a test at the

conclusion. Although, once you understand the benefits, you might be tempted

to start testing your customers.

1 Goliger, Sarah. “Why Internet Marketers Must Always Be Testing.” HubSpot, 27 June 2011. Web. 02 July 2013. <http://blog.hubspot.com/blog/tabid/6307/bid/17695/Why-Internet-Marketers-Must-Always-Be-Testing.aspx>.

2 Ibid.3 “Usability Testing.” Wikipedia. Wikimedia Foundation, 20 June 2013. Web. 01 July 2013. <http://en.wikipedia.

org/wiki/Usability_testing>.


The evolut ion of usabi l i ty test ing

The idea of usability testing first appeared during World War II when John

Flanagan, an industrial psychologist, discovered that reducing the number

of buttons, knobs, switches and controls in a fighter aircraft dramatically

improved pilot performance. Soon thereafter, Flanagan developed the

Critical Incident Technique (or CIT) that outlined procedures for collecting

direct observations of human behavior. The observations were tracked as

incidents to solve practical problems.4

The work of Flanagan and others created the early foundation for usability

testing. The concept was simple, whenever a customer interacted with a company,

brand or product through an informational display, it was helpful to observe users

performing tasks in order to identify opportunities for improvement. By viewing

how pilots interacted with fighter control panels in simulated scenarios, the

design of the jet control panel was greatly improved to promote ease of use and

reduce in-flight errors.

Usability research increased during the 60s as computer interfaces and micro-

electronics appeared. During this time, early pioneers such as Doug Engelbart,

Ivan Sutherland and Alan Kay developed many of the user interface hardware

and software designs that are still used today.5 Often called the “pioneers of

cultural software,” these individuals invented concepts that made it possible for

computers to be used for media creation and manipulation.6

In the 1970s, innovation in usability testing was propelled to a new level, largely

due to the efforts of Xerox®. It’s Palo Alto Research Center® (PARC®) created

many of the elements found in corporate usability research. You probably heard

of the acronym “WIMP,” which is the abbreviation for Windows, Icons, Menus,

Pulldowns. These concepts were developed by Xerox PARC, and as the 1980s

began, usability testing was used primarily for academic or corporate research in

companies like Apple®, Sun Microsystems®, AT&T®, Microsoft® and others.

After the dot-com boom in the 90s, usability testing took on a new format. For

the first time, usability testing was used on a wide-scale basis for commercial

purposes. It also was the first time usability, or the “customer experience,” was

used to influence the decision making process at companies like Amazon.com®

and Google®. Amazon.com founder Jeff Bezos, for example, spent a significant

4 “Usability Testing Central.” Usability Testing Central. N.p., 27 Feb. 2007. Web. 01 July 2013. <http://www.usabilitytestingcentral.com/2007/02/the_history_of_.html>.

5 Ibid.6 “1967-69--The FLEX Machine.” TheEarlyHistoryOfSmalltalk. SmallTalk.org, n.d. Web. 03 July 2013. <http://www.

smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_II.html>.


amount of time using the basic principles of usability testing to improve the

Amazon.com experience. He said: “In our first year we didn’t spend a single dollar

on advertising ... the best dollars spent are those we use to improve the customer

experience.”7 It was during this period that usability testing flourished and played

an influential role in helping companies improve website design.

Dr. Jakob Nie lsen

“ It takes only five users to uncover 80 percent of high-level

usability problems.”

—Jakob Nielsen

You can’t discuss usability testing without multiple nods to Jakob Nielsen, PhD.,

who is largely credited for developing many of the principles of modern usability

testing. Not only did he prompt the “discount usability engineering” movement,

but he also developed “Nielsen’s Law of Internet Bandwidth,” which evaluated

the design implications from annual bandwidth growth.8 According to Fortune®,

he is the “reigning guru of Web usability.” He’s also named “the world’s leading

expert on Web usability” by U.S. News & World Report® and the world’s foremost

expert in Web usability by Business Week®.9 As a result of his work, Dr. Nielsen

holds 79 U.S. patents, most of which focus on methods of Web efficacy.

Nielsen holds a Ph.D. in human-computer interaction (HCI) from the Technical

University of Denmark. From 1994-1998, Dr. Nielsen worked at Sun Microsystems

as an engineer. He was hired to improve the user experience in complex

enterprise software. It was during this time that Dr. Nielsen started exploring the

emerging field of Web usability; he later became the lead designer for SunWeb®,

the corporate website and intranet.

This is when Nielsen had a major breakthrough. As he was

testing the website, it occurred to him that small groups of

test subjects were more effective during the development

stages. His rationale was simple: If it was too difficult for two or

three people to figure out a site, there was little to be gained

by watching others suffer through the same flawed design.

Accordingly, he favored small and frequent usability testing at

various stages of development. He found that the best results

7 Ibid.8 “Nielsen Norman Group.” Nielsen’s Law of Internet Bandwidth. N.p., n.d. Web. 02 July 2013. <http://www.

nngroup.com/articles/law-of-bandwidth/>.9 “Nielsen Norman Group.” Jakob Nielsen, Ph.D. and Principal. N.p., n.d. Web. 02 July 2013. <http://www.

nngroup.com/people/jakob-nielsen/>.


came from testing no more than five users at a time and conducting as many

tests as possible. He also defined the five components of usability goals (which

we detail later), which are:

•learnability,

•efficiency,

•memorability,

•errors (as in low error rate), and

•satisfaction.

In 1998, Dr. Nielsen partnered with Don Norman, Ph.D., a well-known academic

and expert on user-centered design to establish the Nielsen Norman Group

(NN/g). The organization provides consulting services and evaluates user

interfaces. Since its creation, NN/g has consulted on projects at well-known firms,

including General Electric®, General Motors®, UPS, Hallmark® and WSJ.com®.

Each year, the company recognizes the “Ten Best Intranets” across industries and

publishes highly respected research on interface design and testing. Nielsen also

publishes an online column, the “Alertbox,” that addresses current issues and

trends in the field. You can access Neilsen’s current research and blogs on his

website. (www.nngroup.com/articles).

Let’s explore the basic components of usability testing in detail and gain a holistic

understanding of current practices and methods.

Usabi l i ty test ing 101

There’s an article that was published in 2012 by the Nielsen Norman Group (NN/g)

on usability testing titled “Usability 101: Introduction to Usability.”10 It claims

to be the perfect introduction for anyone that needs to understand the basics

of usability testing in a short amount of time. It’s a great starting point for any

organization interested in the idea and can point you to other resources that can

help you get started.

If you were to simplify the idea, usability testing begins with creating

a testing scenario, a user and an observer. Scenarios provide realistic

settings where a person performs a list of tasks while observers take notes.

Testing instruments such as scripted instructions, paper prototypes and

questionnaires may be used to gather additional feedback on the product

being tested. It’s a lot like a test you’d get in the classroom, only no

advanced studying is required of testers.

10 “Nielsen Norman Group.” Usability 101: Introduction to Usability. N.p., 4 Jan. 2012. Web. 01 July 2013. <http://www.nngroup.com/articles/usability-101-introduction-to-usability/>.


Let’s assume, for example, that you want to test how a customer would send

a complaint through your website. You would create a real-life scenario that

asks a participant to do this task and provide a space that is equipped with the

necessary tools. You might give a written script to the user that describes the

assignment in detail and ask them to complete the task. The goal is to observe

how users function in a real life scenario so that developers can identify areas

for improvement. In short, it’s an opportunity see exactly how your customers

react to your products in a setting that mirrors real life. Often, usability tests

are videotaped so that actions and reactions can be viewed by others in the

organization to improve product design.

The common perception is that testing is conducted in a quiet room and

observed through a one-sided mirror. In truth, testing can be conducted

anywhere—you don’t need a usability lab or expensive equipment.

Conference rooms or general work spaces are sufficient as long as they are

equipped with portable recording devices and a place to take notes. Testing

can even be conducted remotely when participants are separated by space

and time. Remote testing and other approaches are discussed in detail in

the following section on usability methods.

As previously mentioned the word “usability” refers to methods for improving

ease-of-use and is defined by five key components. The first component is

“learnability.” This refers to how easy it is for users to accomplish basic tasks the

first time they use the design. “Efficiency” is the second component. This looks at

how efficient a user is when performing program tasks once the design is learned.

If a user continues to struggle trying to complete easy tasks, you’d probably get

low marks in this category. The third component evaluates “memorability.” In

other words, how long does it take for a user to reestablish proficiency after a

period of inactivity? Will users remember how to do something quickly or will

they need to relearn basic features? The idea is to have a design that is easily

remembered, so users recall basic functions.

The fourth feature examines errors users make. Do users make a lot of errors

using the product? This component also examines the severity of errors and how

easily a user can recover. As you probably heard before, in life, falling down is

inevitable, what matters is how quickly you get back up. The same concept applies

to errors in testing, users are bound to make (or find) mistakes, it’s important to

see if they can recover.

The final component of usability testing is satisfaction. That is, how pleasant is it

for users to use the program or design? Do users enjoy the program or activity,

or is there a high level of frustration and confusion in the process? Naturally, a


pleasant experience is the goal, but you can learn a lot from users that find your

program or product frustrating during testing. There are a number of different

testing methods that can examine these components.

At a minimum, usability testing is about capturing information on these

components in a formal and defined setting. There are different methods

and approaches that can be used to achieve these goals. Some of these are

summarized in the following section, however, as described later in the paper,

the tools and approaches that are used vary by product, objectives and other

factors. You may choose to use a combination of methods, so understanding the

full range of options is a good starting point. Some of the methods you might use

include Hall Intercept Testing, remote usability testing, and expert reviews . Other

techniques like think-aloud protocols, co-discovery learning and eye tracking

are often used during testing sessions. Each of these methods and techniques is

defined in the following section.

Usabi l i ty test ing methods

In general, there are three widely recognized methods for usability testing. Hall

Intercept Testing, or Hallway Testing, is the method that uses 5-6 participants for

a testing session. Hallway testing includes random users that you might pass

in a hallway, hence the name. It probably is an obvious point, but participants

should not be designers or programmers, but typical users of your product or

service. Hallway testing can be effective in the early stages of a new design

when product managers or designers need to uncover serious challenges.

Remote usability testing is aptly named—it refers to conducting testing

when participants and observers are separated by space and/or time. You might

use remote testing if participants are located in different countries, for example.

Remote testing can be synchronous or asynchronous. Synchronous testing uses

video conferencing or remote accessible tools like WebEx® or GoToMeeting®. Even

though it’s a big name, the idea is simple: users participate at the same time even

if they aren’t in the same place.

Asynchronous testing, on the other hand, applies to websites or mobile device

testing and refers to the systematic collection of a user’s activities while

interacting with an application. For example, asynchronous testing might collect

click streams or user logs of critical incidents. It might also collect task times

and other data associated with how a user navigates a website or mobile site.

Asynchronous tests are carried out in the user’s own environment rather than labs

to further simulate real-life testing. Unlike synchronous testing, it may not involve

two parties interacting together but collect data on the actions by users.


Expert reviews are another method found in usability testing. As the name

suggests, it involves experienced experts in the testing process. Experts can be

leaders in the field or an organization that specializes in usability testing. There

are a number of organizations that offer expert reviews or testing services. An

automated expert review provides usability testing using technical programs

designed to analyze design and heuristics. In plain English, that means either an

expert or an application will conduct testing to provide detailed analytics on your

site or product. Automated reviews may not provide as much detail and insight

as real people, but they are popular because they are cost and time efficient.

ClickTale®, Inspectlet and TecEd are companies that provide automated review.

There are even some services (e.g. UserTesting.com) that will conduct usability

testing for your organization from start to finish.

Not only can you use different methods in usability testing, but you can also

use different techniques. Think-aloud protocols, co-discovery learning and eye

tracking are three of the techniques commonly used during scenario testing to

capture information on the user experience.

Think-aloud protocols, also known as talk-aloud protocols (or TAP), ask

participants to talk as they are performing a set of specified tasks. Users are

asked to share what they are doing, thinking and feeling as they complete an

assignment. Observers take notes as they gain a better understanding of the user

experience. Often, these sessions are recorded so that developers can revisit how

participants reacted to specific tasks or challenges.

Co-discovery learning (CL) relies on two test users to perform a task together

while they are being observed. Specifically, users are encouraged to talk aloud

while they complete a common task. CL is similar to TAP in the sense that both

encourage users to communicate feelings, frustrations and thoughts during the

testing process. However, some believe that CL makes it more natural for users to

verbalize thoughts during a test because they are working in tandem on the same

tasks. It can be difficult to vocalize thoughts and feelings in an isolated room,

but CL sometimes makes it easier to share thoughts because there’s someone else

working on the same assignment.

Eye tracking research uses special equipment to detect where people’s eyes

are focused when they look at a computer screen. This form of usability

research helps organizations understand what users find attractive in a

design and what they tend to overlook. You can download a free copy of

“How to Conduct Eye Tracking Studies” from NN/g that has 65 tips to help

with usability tests. While it may seem like it is a science fiction tool, it’s

actually fairly common in usability testing, and there are a lot of tools that

can help implement this approach.


Any of these methods or techniques can be used to find out how (and why)

customers use your products. The methods you choose largely depend on your

product, website or scenario. Before you get started, it’s important to have a clear

understanding of the objectives you want to achieve. In a sense, you have to act

like a professor and develop a comprehensive plan for testing and review to make

sure that you are testing the right things.

An or ientat ion to usabi l i ty test ing processes

If you plan to conduct usability testing internally, there’s a lot more to it

than sitting a user in front of a product and taking notes. Whether you’re

testing an application, software or product, you need a plan that clearly

articulates what you will test and how it will be tested. You also need a

strategy to analyze and report results that can be implemented to improve

product design. In short, testing requires extensive design and planning in

order to be effective, and it’s a lot more work than expected.

There are generally three phases of usability testing implementation. And, within

each phase, there are a number of tasks and deliverables that must be completed.

These phases are:

1. Planning

2. Testing

3. Analysis and evaluation

This three-step phase should not be mistaken for simplicity, there’s actually a lot

of activity within each phase. In fact, there are entire books written on each of

the phases, or a specific part of the phase. For example, NN/g has a free, 147-page

report that outlines 234 guidelines on how to recruit participants for usability

studies. The Handbook of Usability Testing is a book that gives a broader view

of the processes required in usability testing. The next section summarizes each

phase at a beginner’s level.

To get started, you need to clearly outline the study. This is where you define

what you are testing and why. You will also develop the how in this phase, in

other words, exactly how will you test users on your product? This phase generally

begins with the creation of a scenario. If you recall, scenarios provide realistic

settings where a person performs a list of tasks while observers take notes. Testing

instruments such as scripted instructions, paper prototypes and questionnaires

are used to gather additional feedback on the product being tested. These items

need to be developed in the first phase.


In the planning stage, you also identify the methods and tools you might use

during testing. For example, will you use eye tracking, expert reviews or Hallway

Testing? Testing observers need to be trained on the types of questions to ask

users during testing, if any. Think of the first stage as the opportunity to plan

every detail leading up to testing. Once the plan is developed, it’s a good idea to

share it with stakeholders or project sponsors to make sure the planned testing

meets the expectations of all involved parties. It’s also a good idea to do a dry

run to check for flow, consistency and clarity. A director wouldn’t open a play

without a dress rehearsal; similarly, you shouldn’t unveil a testing scenario until

it’s been rehearsed.

In the next phase, it’s time to recruit participants and administer testing.

Although it might be easier to use someone in house, participants (or volunteers)

should not be designers, developers, account executives, clients or employees.

They should be people that represent the target audience of the product or

service. Experts recommend testing 4-8 users at different times for each testing

scenario. Some believe that recruiting the right participants is the key ingredient

to successful usability testing. As Clara Teoh from the UK-based firm Webcredible

said: “Recruitment is one of the most important things to get right. If your

participants aren’t representative of your target audience then you won’t be able

to trust your findings.”11

In order to identify the right participants you might consider developing

a recruitment profile that clearly articulates who should participate. You

might select people with a range of experiences to make sure there is a

diverse skill set. For instance, if you’re testing how someone might use

a camera, it might be a good idea to test a professional photographer

as well as a recreational user to get a range of perspectives and input.

Don’t avoid testers that may dislike your product, either. Patrick

Neeman, the Director of Product Design at nPario®, said that effective

tests incorporate users that are likely to hate the product. He said: “The

best suggestions I’ve gotten have been from users that provide more

negative feedback.”12

Once you’ve selected users, it’s time to conduct testing. The session should

be observed and recorded so it can be reviewed at later stages by designers,

programmers, or any resources that have a stake in the product. It’s important

to stick to the testing plan that was developed in phase one and follow the

11 Teoh, Clara. “6-step Process for Planning a User Test.” Web log post. Webcredible. N.p., 1 Oct. 2012. Web. 10 July 2013. <http://www.webcredible.co.uk/user-friendly-resources/web-usability/testing-plan.shtml>.

12 Neeman, Patrick. “Four Steps to Great Usability Testing (Without Breaking the Bank).” Usability Counts RSS. N.p., 4 Jan. 2011. Web. 10 July 2013. <http://www.usabilitycounts.com/2011/01/04/four-steps-to-great-usability-testing-without-breaking-the-bank/>.


script and testing scenario exactly as intended. Again, if you were on stage,

you wouldn’t encourage actors to be spontaneous or impulsive, and the same

approach applies to usability testing.

Testing itself should take 15-30 minutes, depending on the scenario. Do not guide

the user during testing; it’s important that the user is given a task and observed

without interference. Although it’s tempting to offer suggestions and advice, the

user needs to figure out the problem independently. It’s okay to ask questions

during testing, as long as they are outside of the test. For example, asking open-

ended questions like “What were you expecting to happen at that phase?” can

help clarify the user’s thought process. You can also ask questions that elicit

feelings or frustrations the user might experience during testing.

The last phase in usability testing is evaluation and testing. Evaluation and

analysis is where you make sense of what users told you during testing, and

translate results into product improvements or process efficiencies. Although this

is the last phase in the testing process, it’s also the hardest. According to testing

expert Michael Summers, “Analysis of usability data is the hardest step in the

process because it is easy to jump to the wrong conclusions.”13 Evaluating the user

experience relies on subjective observations, but the key to success is to translate

agreed upon interpretations to fix problems and improve products.

In this phase, it’s a good idea to assemble the development team

and watch testing videos as a group. Since usability testing is

largely a qualitative process, there’s a lot of room for interpretation

of results. It’s unlikely that two observers will always reach the

same conclusion. For this reason it’s important to review the data

together, so the team can reach a consensus on observations and

brainstorm solutions on how to address problem areas to improve

the product.

During analysis, you might look for patterns in testing. Were there areas that

produced challenges for every tester? It’s also when you review any comments or

concerns that were raised during testing and discuss methods for improvement.

The next step is to develop a plan to make changes to the product.

Usability testing is not a one-time thing, if you recall, experts say companies

should “always be testing.” Once you’ve made the product improvements from

the first round of testing, testing begins again. You might test to see if you fixed

the challenges that were uncovered in previous scenarios. The more you test, the

13 Summers, Michael. “Usability Testing: Critical Success Factor #3, Effective Usability Analysis.” ‘UserResearch.com’ N.p., n.d. Web. 10 July 2013. <http://www.userresearch.com/my-blog/2010/05/usability-testing-critical-success-factor-3-effective-usability-analysis.html>.


more you refine your product. This is why companies like Amazon.com have more

than 200 usability tests underway at any given moment. There is always room for

improvement, and that means there’s always room for testing.14

You may find that usability testing is too difficult to manage internally, but

you’re convinced of the value and importance. Relax, there are a lot of options

available for outsourcing. If you want to use an outside party for testing, there

are a number of companies that provide a full range of usability testing services.

Companies like UserZoom®, Centralis, and TechEd® can help you define, create,

and implement usability testing from start to finish. If you want help with parts

of testing, there are quite a few tools and resources that can be downloaded and

catered to your testing needs.15

The valedictor ian is … usabi l i ty test ing

There’s no question about it, if done correctly, usability testing can yield

significant results and measurable outcomes. Consultant Michael Summers

claims that usability testing led to significant website improvements for his

clients. Summers and his team observed user behaviors in different scenarios

and used the information to improve user interfaces on specific websites. In

one case study, after multiple testing sessions that led to design improvements,

the customer experienced a 54 percent improvement in successful task

completion, a 164 percent reduction in time-on-task, and a 23 percent

improvement in subjective satisfaction.16

In another example, the same firm did a usability study of a retail website.

The website had recently been redesigned and the completion rates

at the checkout process were lower than anticipated. After adopting

recommendations from usability testing, the retailer measured fewer

users abandoning their carts and an increased number of completed

purchases.17 Stories like these make a compelling case for the benefits of

usability testing when it translates observations into action.

There are other benefits usability testing provides. If done correctly, it can

increase sales, improve performance, and decrease costs associated with design

14 Goliger, Sarah. “Why Internet Marketers Must Always Be Testing.” HubSpot, 27 June 2011. Web. 02 July 2013. <http://blog.hubspot.com/blog/tabid/6307/bid/17695/Why-Internet-Marketers-Must-Always-Be-Testing.aspx>.

15 “ Basics of Website Usability Testing.” Web log post. Template Monster Blog. N.p., n.d. Web. 03 July 2013. <http://blog.templatemonster.com/2011/09/14/usability-testing-basics/>.




and development. Usability testing can also fast-track design and development

timelines and help influence more accurate designs while reducing errors. Not

to mention, by seeing how customers interact with your product first hand, you

might obtain a competitive edge in the market. If you use testing to improve

a website, you’ll probably see a reduction in your bounce rate (i.e. people that

leave your website immediately) and an increase in sales.

Testing can also provide valuable insight to your internal design team. It provides

a way to capture feedback from a target audience that can be filtered directly to

the project team. That can be immensely beneficial to product designers tasked

with improving the product. It can also help settle internal debates on design

issues; different groups can test options that are being discussed to resolve what

method works best. Finally, it gives an opportunity to identify issues and potential

problems before a product is launched. For these reasons and many others, the

popularity of usability testing has continued to increase.

Jacob Creech from IntuitionHQ published an article on the top 10 reasons

you should initiate usability testing.18 Most of the reasons are similar to those

previously mentioned, like the ability to improve usability, user experiences and

accessibility. Creech also claims it can increase customer satisfaction and improve

the reputation of your brand. He notes that: “Developing a reputation for

producing usable, enjoyable websites will help set you apart from others. Happy

clients who recommend you to others will help you stand out. Even offering a

usability testing service in the first place will help you stand out.”19

Does usabi l i ty test ing real ly make the grade?

Not everyone gives usability testing high marks. Critics claim that usability testing

doesn’t always live up to its full potential, especially if it’s conducted poorly

without a well-planned methodology. Some even challenge Nielsen’s assertion

that five users are adequate to find 80 percent of usability errors in almost any

product. Rolf Molic from .net magazine, for example, claims that this is a myth.

After capturing information from the Comparative Usability Evaluation (CUE)

study with teams of experienced usability professionals, there is some evidence

that suggests that more users uncover greater percentages of errors in a website.

The study found that small groups catch about 30 out of 200 usability problems,

but teams of 15 or more found 60 percent of total errors.20 The CUE also showed

that expert reviews found the largest percentage of errors in a website and were

more effective than randomly selected users.

18 Creech, Jacob. “10 Great Reasons to Usability Test.” IntuitionHQ, 13 May 2011. Web. 11 July 2013. <http://www.intuitionhq.com/blog/2011/05/10-great-reasons-to-usability-test/>.

19 Ibid.20 Molich, Rolf. “Usability Testing Myths.”.net Magazine. N.p., 27 Feb. 2013. Web. 02 July 2013. <http://www.

netmagazine.com/features/usability-testing-myths>.


Another criticism of usability testing is that it is never 100 percent

representative of real life scenarios. Although the goal is to

create scenarios that mirror common use, it’s impossible to fully

replicate how a customer uses your product in a home or a business

setting. For example, one of your testers may be a mother of two,

and unless her children are running around the testing lab, the

scenario is not fully representative of her true life. Likewise, a sales

representative may need to access your site while talking to a client

on the phone, which is unlike a testing situation where he or she is

completely focused on the testing at hand.

Usability testing can be expensive and time consuming, too. As noted, to be

effective, it requires a significant amount of resources for planning and execution.

Another complaint is that test validity depends heavily on identifying the right

target group. You can jeopardize the entire session if you pick the wrong testers.

This happens when people select testers that are friends, family members

or coworkers. While it may seem like these individuals are representative of

potential users, in actuality they are most likely from the same demographic, and

have similar education and economic levels. Suddenly, the study does not have a

wide variety of users.

Dana Chisnell summarized it nicely in a usability-focused blog when she said that:

“for many of us, usability testing is a necessary evil. For others, it’s too much work,

or it’s too disruptive to the development process.” Not every company has the

resources and time available to conduct usability testing, and even when they

do, if it’s not done correctly it won’t produce results. That’s why if you plan to

implement usability testing studies it’s important to get it right, or hire someone

that can do it for you.

The f inal grade

In the final analysis, the pros of usability testing far outweigh the cons. Canoo,

a Swiss firm that claims to specialize in “end-user happiness,” claims that it can

be done on limited budgets and doesn’t necessarily require extensive training

and tools. According to a consultant at Canoo, “The key to getting value from

[usability testing] is to keep investment and costs to a minimum. Train yourself,

don’t commit to fancy tools, resist the allure of pricey consultants, and just get on

with practicing.”21

21 “Canoo RIA Blog.” Canoo RIA Blog. N.p., n.d. Web. 01 July 2013. <http://www.canoo.com/blog/2010/03/19/10-lessons-learned-from-usability-testing/>.


4imprint serves more than 100,000 businesses with innovative promotional items throughout the United States,

Canada, United Kingdom and Ireland. Its product offerings include giveaways, business gifts, personalized gifts,

embroidered apparel, promotional pens, travel mugs, tote bags, water bottles, Post-it Notes, custom calendars,

and many other promotional items. For additional information, log on to www.4imprint.com.

Whatever (if any) testing method you pursue, the key is to develop a plan

and stick to it. Overall, testing can be a valuable tool to make sure your

product receives a passing grade in the market.

Usability Testing - 4imprint Learning...

Documents

Transcript of Usability Testing - 4imprint Learning...