Documentation Usability TechniquesChauncey E. WilsonChaunsee@aol.com

October 30, 1999

Method Focus Notes

Read and Locate Test General sense of ease of navigation and organization of information

Need a fairly large sample and must spend time on representative questions.

Laboratory Testing Use to verify unique features of all types of Online Help. If performance isn’t an issue, participants are generally ask to “think aloud”

Finding participants is the biggest issue. Can be used for performance testing (how fast, how many errors)

Summary Test Use to see if users grasp key concepts (e.g., security)

Users read a section of documentation and reflect back what they think it means. Scoring may not be easy.

Usability Edit Detailed edit of instructions. User is asked to read through text and mark up things that are hard to understand, wordy, inconsistent, etc.

Useful for procedural help to see if language is clear, anything is missing, terminology is consistent

User Interface Inspections Formal inspection of document with focus on usability, readability, and consistency issues. A common UI design technique.

Catches many details. Should have writers from different groups in on the inspection to improve consistency. Requires some minimal training. Should not be the only method used.

Diaries of Documentation Usage Users keep a diary of their experience with the documentation (and product).

Requires dedicated users.

Audio Diaries Use audio computers found on most computers to create an audio diary.

Need a microphone. Users can email their audio file to you. Easy to turn on and off. Can be used for surveys and diaries.

Pop-up Web Surveys A window pops up once for each user asking if the user will fill out a short survey. If the person says, “Yes”, the survey appears. A user only gets one instance of this.

See www.surveysite.com for an example.

Logging of Online Help Usage Wire up some customers so that their use of Help is logged. Have them send longs back once a week.

Security issues. Must develop or purchase logging software that will log all forms of help usage and point to the topic of interest.

Chauncey E. Wilson WilDesign Consulting 1

Mark Up of Documentation Ask users to mark up the hard copy with comments and send it to you.

Requires some dedication and some mark-up training..

Critical Incident Questionnaire Ask users about critical incidents involving any form of documentation.

Critical incident questionnaires allow the gathering of criticality and frequency data. Critical incidents can be successes or failures.

Telephone Survey of Documentation Usage

Get general feedback about documentation issues.

Hard for users to remember things if cold-called. Best technique is to contact users, get them to commit to a study of a particular length, send them questions of interest and ask that they jot down notes before the phone interview.

Tape Recorder with Big Button Detailed, real-time feedback. User hits button to record a comment; hits button again to stop recording.

Requires OK from users and may involve security issues. Collection of tapes can be a problem. Analysis is detailed, but time consuming.

Documentation Survey (Mail Questionnaire)

A good questionnaire requires design and testing; focus on key questions;

Some incentive is needed here or dedicated customers.

Beta Questionnaire Information on product and documentation; requires careful design.

Many customers don’t use the documentation much during beta so this method often disappoints the beta team.

Summaries of Product Reviews What is said about your documentation or your competitor’s documentation in product reviews?

Magazines, journals and Web sites have reviews of product documentation. Some reviews rate documentation on specific usability criteria.

Email Surveys Put out a survey through an email list. Problem here is that the sample may be biased somewhat.

Get names of potential recipients from support, marketing, or user group lists.

Web Survey Put out a survey on the web or send the survey to selected users. Problem here is that the sample may be biased somewhat.

Get names of potential recipients from support.

Training Classes Sit in on training classes. Work with the instructor to provide some training on the documentation.

Ask trainees to note problems with documentation and let you know about them during exercises or at breaks.

Information Race Give users a set of questions and tell them that they are racing each other to find the answer in the documentation.

More for expert users who have strong egos.

Support Debriefings Meet with Support once a month Good information on places where the UI

Chauncey E. Wilson WilDesign Consulting 2

or so and review any usability or documentation problems that are coming up.

or doc could be better.

Screen Movies Ask users to do screen movies of their work or documentation usage. You can use screen-recording tools like Lotus ScreenCam. Users can start recording when they are using documentation or want to show how they use a product. They can stop recording until the next documentation “incident”.

Takes lots of disk space. Most screen recording software can handle voice so you users can think aloud and give you a screen movie.

Remote Usability Testing Use software like NetMeeting or PCAnywhere to view the screen of a user at a distant site. You can hear the user over a regular phone line while observing what the user is doing on the screen.

Takes some practice. Useful internally, but more difficult working with clients because of firewalls and security issues.

Competitive Testing Check out features of other products to see how usable they are.

Often done informally, but results hard to interpret.

Paper and Pencil Testing of Individual Elements of Publications

Early feedback on document structure, language and graphics.

Create samples of proposed publications and have users walk through scenarios. Check out things like terminology, heading levels, task sequences, use of illustrations and examples, and index entries.

User Group Feedback Meet with user groups during conferences, shows, etc. Prepare a list of questions to ask

General feedback, but good way to make contact for more detailed information.

On-Line Focus Groups Participants are selected according to a user profile. Participants are given passwords to prototype products and can use them before the focus group

Cheaper than regular focus groups and people can use products to get more detailed data than face-to-face focus groups.

Coordination with Product Management When Customers Visit

Work with Product Management/Marketing to get some time with customers who are visiting your site.

Prepare a list of questions to have at the ready.

Field Visits Field visits can be used to gather a lot of information, including documentation use.

Must focus on the whole product. Trips to get info on “just documentation” are not all that successful.

Chauncey E. Wilson WilDesign Consulting 3