Download - Fair Witness

Contents

Preface.......................................................................................................................................2

Introduction..............................................................................................................................3

Planning for a Usability Test with Morae ..............................................................................4 Establish Usability Testing Goals ......................................................................................................4 Define Test Tasks...............................................................................................................................4 How Morae Expands Task Definition................................................................................................5

Setting up the Usability Test ...................................................................................................6 How Morae Simplifies Test Setup .....................................................................................................6 The Test Team....................................................................................................................................6

Conducting the Test.................................................................................................................7 The Pilot Test .....................................................................................................................................7 The Actual Test ..................................................................................................................................7 How Morae Automates Data Logging during Testing .......................................................................7

Analyzing the Results ..............................................................................................................9 How Morae Simplifies Analysis ........................................................................................................9 Making Design Changes Based on Results ........................................................................................9

Presenting the Results............................................................................................................10 How Morae Streamlines the Presentation of Results .......................................................................10

Conclusion ..............................................................................................................................11

Overview of Usability Testing with Morae UIanswers

Preface

About this Overview This overview is an excerpt of the comprehensive guide, Usability Testing with Morae, which will be available soon from TechSmith.

The complete guide represents a collaboration between Dr. Piyusha Paradkar, of UIanswers, and TechSmith Corporation, developers of Morae. The guide presents Dr. Paradkar’s method for usability testing, updated to highlight the rich functionality offered by Morae for usability testing. This proven methodology has been used by Dr. Paradkar for over five years. It focuses specifically on the usability testing of software, Web sites, and online documentation.

The complete guide also includes a CD-ROM, which features:

• An interactive PDF version of the entire book • Sample templates of common usability testing forms • A sample PowerPoint presentation that you can customize with your own results • Videos illustrating the real-life usability testing example presented in the book

About UIanswers and the Author UIanswers offers state-of-the-art user experience solutions for improving the usability and accessibility of products.

Piyusha Paradkar, owner and Principal of UIanswers, has a Ph.D. in Industrial Engineering from the University of Wisconsin-Madison. She specializes in the field of Human Computer Interaction, focusing on interface design and analysis. She has led the design and usability testing of graphical user interfaces, voice response systems, touch-screen interfaces, and networked communication systems for organizations, including e-mail and voicemail. She helps organizations define usability effort on cross-functional teams in all stages of product development, including product concept, initial design analysis, requirements definition, prototype testing, and online help and user guide development.

Dr. Paradkar has a strong background and expertise in Section 508 accessibility and has guided product teams on accessibility evaluations and design solutions. She also conducts mentoring and training workshops on usability evaluation techniques, and assists organizations in setting up usability testing labs and programs that succeed.

Dr. Paradkar can be contacted at [email protected].

2 • www.techsmith.com Preface

mailto:[email protected]

UIanswers Overview of Usability Testing with Morae

Introduction

What is Usability Testing? Usability testing is a powerful and exciting technique that lets you determine how well a product can be used by its intended users. It properly identifies and prioritizes usability issues and highlights areas that need change, thus eliminating wasteful speculation about where the problem areas are. This predictive power ultimately permits critical design changes early in development and prior to product release, which is cost-effective and also enhances customer product acceptance.

Identifying What Works and What Doesn’t Usability testing has two primary goals, which are to identify:

1. Areas of a product or product prototype where users have difficulty. Once identified, you can then either redesign those areas so the user will not have difficulty, or provide appropriate assistance.

2. Areas of a product that are marketable — that really work! Once identified, you can provide promotional ideas for the product that extend beyond product features and the solution the product provides.

Translating Results into Improvements Once the critical areas needing redesign are identified, these design changes can be made during development. This process can have the following positive results.

Increased: Decreased:

• Sales revenues

• Productivity

• Web site traffic

• Customer satisfaction and loyalty

• Human error

• Development time and costs

• Technical support calls

How Morae Simplifies and Improves Usability Testing Morae is an innovative software-based solution for usability testing that enhances data collection and speeds up analysis. Its dramatic effectiveness lies in the fact that it actually reduces the amount of work needed, not only at setup but also during the data logging, analysis and presentation stages. Its design works well for users new to the field of usability testing, as well as seasoned specialists with existing usability labs. It is simple to set-up, use, and maintain.

Morae fits well with existing methodologies and processes of usability testing. It improves the usability testing experience without compromising any established procedures. We present here an overview of the UIanswers usability testing methodology, updated to make full use of the rich functionality offered by Morae.

www.techsmith.com Introduction • 3


Planning for a Usability Test with Morae

Successful usability testing involves several people performing multiple steps with dependencies and contingencies. Each usability test needs to be considered a project. The planning phase is the key to maintaining proper timing, budget, and ultimate success.

This section covers the major planning steps for a successful usability test, describes how Morae fits into the planning process, and highlights new ways to enhance usability testing using Morae.

Establish Usability Testing Goals The first step in planning your usability test is to define what you want to know about your product and establish clear goals for the testing.

Create a vision statement for the usability of your product or site At the highest level, you need to create a vision statement for the usability of your product or site if one does not exist. This vision may not be measurable, but it should be related to the business objective and should help focus the specific measurable aspects of your product or site.

Example vision statement:

“Novice users will be able to complete their taxes in a reasonable time and without major errors.”

Establish specific, measurable test objectives The objective of a usability test has to be something you can measure. "Is my product usable?" is not a good objective. Neither is, “customers will perceive the product or Web site to be easy-to-learn and efficient-to-use.” Why? Because they cannot be measured. Test objectives must be expressed in terms of measurable usability attributes.

Example test objective:

“The overall goal is to find out if novice users make any critical errors when using this software for their taxes.”

Define Test Tasks The usability test requires understanding the real tasks that real users will be attempting to accomplish on the Web site or with the software application being tested, and building your usability test procedure based on this understanding/knowledge.

Choose appropriate tasks Select test tasks based on the following:

• Tasks that users will perform most frequently in typical usage • Tasks that users have had difficulty in current or past versions, as indicated through support calls

or previous testing • Tasks based on design areas that have been particularly challenging during design • Tasks for which you have more than one design option • Tasks that users perform infrequently but are critical (e.g., software downloads, monthly reports,

troubleshooting routines)

4 • www.techsmith.com Planning for a Usability Test with Morae


Provide context for tasks through scenarios UIanswers recommends that you script your tasks for the usability test by including them within the context of one or more usage scenarios. This will provide perspective for each participant as they read the instructions and perform the tasks using the product under test.

Select what to measure for each task Standard task measures include: time on task, errors, task completion, and access of help during tasks.

How Morae Expands Task Definition During task definition, it is often frustrating to discover that what you want to measure cannot be measured because of equipment limitations. For example, the number of clicks taken to complete a task is an important measure of user success in many applications. However, these measures are often not included in the usability test because:

• Manually logging this data during live-testing is tedious and error prone. • Reviewing the video after testing is complete in order to log the data is cumbersome and time-

consuming.

Morae’s Rich Recording Technology gives you more flexibility in what you can measure right at the task design stage. It dramatically reduces the effort involved and makes this data both easy to capture and accurate. The following table compares example traditional task measures with example expanded measures possible using Morae:

Traditional task measure Task measures possible with Morae Time on task

Number of steps required to complete the task

Time on task

Number of steps required to complete the task, with the screens for each step

Number of clicks

For participants who took more than the estimated number of clicks, exact spots where the clicks happened on the screen.

Number of times user accessed help Number of times user accessed help

Exact places where user accessed help

Time spent in help

Contents accessed in help

www.techsmith.com Planning for a Usability Test with Morae • 5


Setting up the Usability Test

Proper equipment makes the usability test easier, more effective and more meaningful, especially during analysis. UIanswers recommends that, at a minimum, you record the participant interacting with the product. While testing software products and Web sites, it is also critical to record the screen activity of the participant along with the participant video.

How Morae Simplifies Test Setup Morae is an all-digital, software-based solution for usability testing. An additional microphone and Web camera are optional (both are inexpensive and often built in). Because Morae is truly portable, it is perfect for mobile testing. Morae eliminates the need for special usability testing equipment and dramatically reduces both the complexity and the time involved in setup.

Morae allows you to set up a configuration file, which defines what you want to record, in advance of the test. You can then put this configuration file on the testing computer ahead of time so that you are not burdened with setup tasks immediately before or during testing.

Morae offers an easy way to record audio, video and screen activity. By automatically creating a log of this activity, Morae minimizes the data logging during testing.

Morae’s features include:

Audio and video recording of screen and participant

Picture-in-picture capabilities to show participant video and screen

Synchronized screen recording

Live data logging capabilities

Automatic and manual data logging capabilities

Remote observation of screen activity from another computer on the network

Software-based, integrated editing capabilities to create and produce highlight video

Using Morae, you will increase your testing and analysis capabilities, increase the overall productivity of your testing efforts, and eliminate the following complex and expensive hardware:

Scan converter

A video splitter device

Cables

Tapes

Picture-in-picture generator

Mixer

Editing equipment for highlight video

Data logging equipment

The Test Team The critical/essential people in a usability test are the Test Participant, the Facilitator, and the Primary Data Collector, but there are additional roles that may be involved as well, such as the Stakeholder Observers.

By facilitating setup and data logging, and by eliminating equipment handling, Morae greatly reduces the effort required by the test team. It makes the “one-person testing team” more feasible and less burdensome than it has ever been before. In situations when the Facilitator is the only person available, Morae permits collection of data that either would not be collected, or would only be collected during data analysis at a much greater effort.

6 • www.techsmith.com Setting up the Usability Test


Conducting the Test

The Pilot Test The pilot test is a practice run through the test protocol prior to the actual test. Pilot testing allows you to verify that the tasks work as intended, that the test length is appropriate, and that the correct information is being properly captured. Run one or two participants through the protocol before conducting the actual test. Conduct the pilot test exactly as you would conduct the actual testing. Allow time before the actual test to make any needed changes to the protocol.

The Actual Test The Test Participant and the application or Web site under study are the primary focus during testing. The following is a summary of the chronological steps the Facilitator should follow during the actual test.

Pre-Test Steps

• Greet the participant and establish rapport. Have participant complete the necessary forms. • Reinforce their right to take a break or withdraw from the test for any reason if they wish to do

so. Explain how the data will be used. Explain how the video and audio will be used. • Explain the participant’s role, explain your role as the Facilitator and how you will interact during

testing, and inform the participant of any audience or observers for the test and ask consent. • Provide an introduction and demonstration of the product if it is a part of your test design.

Provide or simulate the testing scenario. Ask the participant if (s)he has any questions.

Test Steps

Hand the first task to the participant and have him/her begin the task. Position your seat 3 to 4 feet away from, and behind, the participant. Continue until all tasks are complete. Post-Test Steps

Administer the post-test questionnaire. Debrief the participant. Ask clarifying questions about the participant's interactions with the interface. Solicit any final comments. Thank the participant, handle reimbursement arrangements, and say good-bye.

How Morae Automates Data Logging during Testing During a usability test, you can use the automatic data capture capabilities of Morae to record a majority of the quantitative data. In addition, a primary data logging person or observers may log qualitative data manually using Morae.

Synchronized Data Logging Morae creates a recording of the system activity synchronized with participant audio and video. In addition, Morae automates much of the data that is typically either logged manually during testing, or logged after testing with time-consuming video reviews.

The automatic data capture:

• Improves accuracy of the data collection • Reduces the amount of data that needs to be manually logged during live testing • Frees the facilitator and observer’s time for other activities

www.techsmith.com Conducting the Test • 7


Screen video is always captured automatically by Morae. The following table describes the additional capture options that may be selected within Morae:

This Capture Option: Records: Camera Video of the user from an attached camera.

Microphone Audio data from an attached microphone.

Keyboard Data related to the activity on the source computer’s keyboard. These data include all the numeric and alphabetic keys pressed on the keyboard.

Screen text Text being displayed on the source computer’s screen. Screen text includes every word shown on the screen except for those that are part of an image.

Mouse clicks Mouse clicks (left and right) occurring during a recording.

Application events Windows events and Web page changes that occur with a specified application during a recording.

Flexibility in Data Logging When conducting a usability test, unexpected behaviors (e.g., user accessing help, user taking a different path to complete a task, etc.), which are very important to record, can suddenly change the data collection and logging requirements. Morae is flexible, and easily accommodates the extemporaneous events and unexpected paths that are not only common, but also especially valuable and meaningful in usability testing.

Integrated Live Data Logging In addition to the task data we plan to capture, it is also important to be able to log points of interest in the data while testing is in progress. For example, it is important to note areas where the test participant had difficulty completing a task or where the participant had a particularly positive experience. Morae allows the Primary Data Collector or Observers using Remote Viewer to directly add categorized markers during live testing. These are saved and synchronized directly with the recording for easy access during analysis. Morae also allows association of audio and text based notes with markers to add more meaning to the markers during analysis.

8 • www.techsmith.com Conducting the Test


Analyzing the Results

Usability test data are more than likely comprised of a combination of quantitative and qualitative data. Quantitative data include measures such as time required to complete a task, number of errors, and number of clicks. Qualitative data include measures such as user feedback on surveys, participant comments during and after testing, participants’ facial expressions, as well as their opinions and 'feelings' about the product. In most cases, the quantitative data are more effectively understood when combined with the qualitative data and observations. Usually, for a particular usability problem, the supporting data will have both quantitative and qualitative elements.

How Morae Simplifies Analysis Data analysis carries the reputation of being the most difficult and time-consuming part of the usability test. Testing with Morae significantly reduces the effort involved in data analysis. Morae allows you to access, view and search for data in new and innovative ways while at the same time reducing the number of steps required for traditional data analysis.

• Add Markers. You can easily add markers in the recording, both during and after testing, to mark points of interest, including participant behavior and expressions.

• Create Segments. Using Morae, test tasks performed by the participant can be segmented by start and end points and saved for analysis and presentation.

• Search for Text or Events. Morae’s powerful search capabilities allow you to quickly get to the data you need.

• See the Data in Context. Search results are automatically synchronized with the video recordings, placing the data in context with when they occurred. Simply select an event in the search results list and the video will move to that point in time, highlighting on the screen where the event occurred:

• Easily Calculate. Typical data calculations, such as time on task and number of clicks, are readily available through built-in calculators.

Making Design Changes Based on Results Usability testing data, typically, does not show what actions to take with the user interface, but rather where actions need to be taken. UIanswers recommends that you always follow through with specific recommendations and design improvement suggestions after your test.

www.techsmith.com Analyzing the Results • 9


Presenting the Results

While a detailed written report is nice for future reference, UIanswers recommends presenting the results directly to the audience. This should be done verbally through a presentation, before or along with any report that is distributed. A report with no presentation/follow-up is more open to misinterpretation.

Set up a presentation, accompanied by a highlight video, for the stakeholders. Follow this up with a detailed report to the main stakeholder on the management team, to distribute as appropriate.

There are four key things that need to be communicated to the development teams after a usability test:

1. An overall “how it went” factor 2. What worked 3. What did not work 4. What can be done

Summarize findings by priority. Include screenshots where applicable. Group the findings by general problem areas or by the different areas of the product. Do include positive findings.

UIanswers also recommends coming up with estimates on “time-to-fix” or “degree of difficulty of fixing” and impact. These should never be made independently, but in collaboration with the development team lead or appropriate development team members.

How Morae Streamlines the Presentation of Results A highlight video is a collection of short and meaningful segments of the testing video. Highlight clips are compelling and effective during presentations. They provide a positive and potentially eye-opening experience for the audience and the development teams. UIanswers recommends using several short highlight clips rather than a complete highlight video. Use the video clips along with the other quantitative data to report a finding in its entirety.

Morae offers a streamlined process for the creation of highlight video clips, with great savings in the time and effort involved. It carries forward all the segments of interest that you have flagged in the analysis stage into the presentation stage, allowing you to consider those for highlight video clips directly. Morae also places the entire recording at your fingertips, to explore and clip.

With Morae, there are several options for presenting your results. You can create a screenshot of any frame from either the screen or camera video portion of the recording, or a screenshot of a frame that contains both. You can produce single video clips as AVIs to embed in your presentation. Or, you can use Morae to produce a polished highlight video with titles.

You can easily order, organize, and preview your video clips on the Storyboard for a better fit with your presentation.

Flexible Picture-in-Picture Another compelling feature of Morae is the flexibility you have with the participant video. You can choose whether or not you wish to display participant video with the screen recording as picture-in-picture (PIP). In addition, Morae lets you choose where to position the participant image. You can position it anywhere and even change the location for the different tasks performed after testing. You can also adjust the opacity (transparency) of the PIP image over the screen video. Because Morae lets you do this after the recording, you never run into the problem of participant video obscuring critical portions of the screen video image.

10 • www.techsmith.com Presenting the Results


Conclusion

Usability testing is a highly effective technique to measure and validate the user experience. It can provide specific and timely direction to ensure an appropriate and compelling user experience. It also saves time and money and ensures a better product.

If you are still on the fence trying to decide how you are going to justify a usability testing solution or justify upgrading your solution, consider how Morae solves the most common hurdles to justification:

• Cost of the usability testing equipment and setup. Morae has successfully removed this hurdle by coming up with a cost-effective, easy to set up and use, ‘complete’ solution.

• Location for usability testing. Morae’s portability and ease of set-up on any computer dramatically reduces this hurdle by expanding the locations where usability testing may be conducted.

• “People” resources and skills to conduct the testing. With Morae, a smaller (or even a one-person) usability test team is now a reality. In addition, the complete UIanswers methodology and templates are available with Morae to assist you in conducting your test. Together, you have the resources and step-by-step guidance necessary to conduct a successful test.

• Speed of analysis and presentation. Morae gives you the tools to get to your results faster and to present your conclusions quickly.

Whether you have access to an existing usability lab or are just in the market to purchase the minimum to get started, Morae is a solution that will work for your needs. Its design works well for users new to the field as well as seasoned specialists with existing usability labs. It is simple to set up, run, use and maintain. It enhances and expands data capture capabilities. Its dramatic effectiveness lies in the fact that it actually reduces the amount of work needed, not only at set-up, but also during the data logging, analysis and presentation stages.

It fits in well with existing processes for usability testing as demonstrated with the UIanswers methodology overview. In addition, Morae offers one of the most portable solutions for user testing on the market today.

For more information on usability testing, usability process, and establishing a program at your company, contact [email protected].

To Learn More about Morae For more information about Morae, please visit the TechSmith Web site (www.techsmith.com) or contact a Morae representative directly:

Phone: US and Canada (Toll free) +1 888.750.0685

International +1 517.381.2300

E-mail: [email protected]

Mail: TechSmith Corporation 2405 Woodlake Drive Okemos, MI 48864 USA

www.techsmith.com Conclusion • 11


http://www.techsmith.com/
