Post on 17-Jan-2017
8 Usability Lessons for Freddie Macfrom the 2004 UPA Conference
Prepared for the Usability Community of PracticeBy Mark D. AlvesAugust 11, 2004
2
Overview of UPA Conference
The 13th annual Usability Professionals’ Association conference, Connecting Communities, attracted 470 attendees to Minneapolis, Minnesota from June 7-11, 2004.
Some events were major presentations for all attendees while at other times smaller, concurrent sessions were offered. Usability vendors were onsite as well.
This year’s materials are at: http://upassoc.org/conferences_and_events/upa_conference/2004/
Next year's conference is in Montreal, Canada from June 27 - July 2. Some of the most interesting information comes out of informal
discussions in between sessions or after hours. Freddie Mac’s usability experience and challenges are not unique.
Some companies are ahead of us, some behind.
3
Examples of Common Items That Could Use Improved Usability
Minneapolis Sculpture Garden
4
Examples of Common Items That Could Use Improved Usability...
Mini-golf hole designed by an artist.
5
Examples of Common Items That Could Use Improved Usability...
Mini-golf hole designed by an artist.
6
8 Usability Lessons for Freddie Mac
1. If your users are split 50-50 on a design choice, let them decide.2. Your lab doesn’t have to be sterile.3. Join UTEST now and participate later in the community.4. Speed up design and testing, but recognize there’ll be false positives and missed errors.5. Prepare for the difference between an intranet and an enterprise portal.6. Use TechSmith’s Morae for reviewing test participants’ actions and results, http://techsmith.com/products/morae/.
7. Get good card sorting results with helpful tools and a handful of testers.8. Create a successful online “live” event, even when most of it is pre-produced.
Microsoft games and joysticks: down & out or up & away? If your users are split 50-50, then make them pick. For Freddie Mac’s website, offer information both ways. Perhaps the forms are listed numerically and by category; or the products are listed alphabetically and by need.
Bose speakers: their usability lab is set up like a living room. Make your test environment similar to what your users’ areas are like if you can’t actually test them in their natural settings. Perform some Homefront tests at users’ desks.
UTEST is a private, non-archived listserv for discussing budding usability ideas and solving usability problems. To request membership, contact Dr. Tharon Howard at tharon@clemson.edu after reading about the special policies and culture of the community: http://people.clemson.edu/~tharon/utest/policies.html
Speed up design and testing. Add an enterprise portal with care. TechSmith’s Morae software captures test participants’ screen behaviors without the obtrusiveness and intimidation factor of a video camera.
http://techsmith.com/products/morae/ Get good card sorting results with helpful tools and a handful of testers. Create a successful online “live” event, even when most of it is pre-produced.
7
Lessons From Other ParticipantsLesson 1: If your users are split 50-50, let them decide.
When it comes to video games, Microsoft employs usability principles while striving for “appropriate challenges” instead of minimal challenges. Their testing showed that gamers were split 50-50 on certain games as to whether pushing the joystick up should make a character’s gun go down and away (like a pilot) or move up (like a map). Their solution was to force the user to pick a preference before the game can begin.
For Freddie Mac websites: offer information both ways. Perhaps your forms are listed numerically and by category; or your products are listed alphabetically and by customer need.
Lesson 2: Your lab doesn’t have to be sterile. Bose makes high-end audio speakers so their usability lab is set up like a living
room. Make your test environment similar to what your users’ areas are like if you can’t actually test them in their natural settings.
For Homefront: perform some tests at users’ desks.
Lesson 3: Join UTEST and later participate in the community. UTEST is a private, non-archived listserv for discussing budding usability ideas and
solving usability problems. To request membership, contact Dr. Tharon Howard at tharon@clemson.edu after reading about the special policies and culture of the community: http://people.clemson.edu/~tharon/utest/policies.html
8
Speed Up Design and TestingLesson 4: Speed up design and testing, but recognize there’ll be false positives and missed errors.
Inspection evaluations are efficient, but not effective– Miss 25% of problems– Find more non-problems than problems (for every 2 problems found you’ll
have 1 that you missed) User testing is effective, but not efficient Mind Design Systems, http://mindd.com, is working on software to improve the
efficiency of human testing, but this won’t be commercially available this year. One way the software improves efficiency is by stopping a task after a certain number of clicks; you don’t waste time if a user gets really lost.
Ask true/false questions if testing comprehension. This cuts down on the user agonizing over the answer.
For designing intranets, relying on content analysis can speed up the process.– Use paired-comparison tests to see preference (if any) between content– User selects preference or indicates none– User reviews the hierarchy of preferences that results– Don’t focus on employee task analysis, but on what content the staff
needs.
9
Add an Enterprise Portal to Your Intranetwith Care
Lesson 5: Prepare for the difference between an intranet and an enterprise portal.
Boeing is working on adding customizable enterprise portals to its 8 million page intranet.
– Keep user feedback forms simple by skipping the user-selected categories. (25% picked the wrong category, 25% picked “other” so 50% of the data was useless.
– New users typically “test” a portal by searching for applications that they know already exist on the old site: make sure the search engine picks them up.
– Users are receptive to “my pages” and related customization.– Users are resistant to “subscribing” to a community rather than just
looking at a website: “I don’t want to join a club.”
10
Use TechSmith’s Morae for Reviewing Test Particpants’ Actions and Results
Lesson 6: Use TechSmith’s Morae for reviewing test participants’ actions and results.
TechSmith’s Morae software, http://techsmith.com/products/morae/, captures test participants’ desktop behaviors and can integrate a log of activity with any sound and video you record. Captured behaviors include:
– Mouse clicks (left and right)– New windows opened– Entered text– Accessed menus– Any markers you insert
You can capture all activity or specify when the recording starts, such as upon clicking on a certain link or opening a particular application.
Remote viewer option lets you see the subject’s desktop over a LAN connection so you can be in another room. No one-way mirrors required.
11
Card Sorting: Use It Quickly and Easily
Lesson 7: Get good card sorting results with helpful tools and a handful of testers.
How Many Users Are Enough for a Card-Sorting Exercise?Poster presented by Larry Wood of BYU, who developed Web Sorthttp://www.websort.net
Correlations:– 5 users=0.75– 15 = 0.90– 20 = 0.93– 30 = 0.95 – 60 = 0.98
12
Card Sorting: Use It Quickly and Easily (cont.) Card Sorting: How Many Users to Test?
Jakob Nielsen co-opts a UPA poster http://www.useit.com/alertbox/20040719.html
A 13-step Approach to Card Sortingby Gerry McGovernhttp://www.gerrymcgovern.com/nt/2002/nt_2002_09_23_card_sorting.htm
Card sorting tools and technology– Analyzing Card Sort Results with a Spreadsheet Template
by Joe Lamantia http://www.boxesandarrows.com/archives/analyzing_card_sort_results_with_a_spreadsheet_template.php or http://tinyurl.com/lcuj
– uzCardSort for Uzilla http://uzilla.mozdev.org/cardsort.html– IBM’s EZSort http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410– CardZort by Jorge A. Toro
http://condor.depaul.edu/%7Ejtoro/cardzort/cardzort.htm– NIST WebCAT web category analysis tool
http://zing.ncsl.nist.gov/WebTools/WebCAT/overview.html– “Classified” from Information & Design (Australia)
http://www.infodesign.com.au/usabilityresources/classified/
13
Sponsor a Live (Enough) Event OnlineLesson 8: Is it Live or is it Freddie Mac? “We now bring in enough people to fill a stadium for the cost of a barn
dance.” Scott Anderson, director of e-marketing, Sun Microsystem. Sun’s online “live” events replace the hodgepodge of tactics they formerly employed for their quarterly R&D announcements.
Take the best of online news sites, websites for temporary museum exhibits, ESPN-type sports updates, live chats with celebrities/newsmakers, and online learning sites.
Combine them to create the urgency and excitement of a “live” event, with the certainty of pre-recorded commercials and the shelf-life of important product support materials.
It’s cheaper than doing the event in person; it’s more interesting than a regular website; and the results are measurable.
14
Sponsor a Live (Enough) Event Online cont.
Use the Tivo philosophy experience: what’s happening now is central, the past is accessible and the future is only glimpsed at.
The design implications?– The past: Content that has already been released is available for
viewing at any time.– The present: Content is being released linearly.– The future: Promote future content and encourage users to come
back; whether the content is prepared or not, it is not released until the pre-determined time.
Organize prepared content around “live” events. Break a usability code in exchange for business results: everything is
not available all of the time.
15
Sponsor a Live (Enough) Event Online cont.
Implications of using video:– Promoting “live” aspect draws crowds– Users will view content for a longer period of time when they are
watching a live event.– Focus on quality, not quantity.– People want accompanying text to be scannable, not full
transcripts. Have a backup plan
– Users who opt not to install plugins or players– Prepare your help desk for extra volume and more technical
questions– Allow extra time for QA to test multiple browsers and versions of
the same browser
16
Sponsor a Live (Enough) Event Online cont.
Examples of tightly integrated video and website.
17
Sponsor a Live (Enough) Event Online cont.
Post-event, you default to playing the first video (not “now playing”).
18
Sponsor a Live (Enough) Event Online cont.
Options for applying this at Freddie Mac externally.– In conjunction with our own customer conferences or analyst
meetings – Product announcements– As part of regularly scheduled mission updates– In support of a job fair– Integrated with a national event, like homeownership month.
Internal options– Division events where staff is distributed across multiple locations– Diversity events to maximize employee participation– “All”-employee events where not everyone can attend
19
Websites Are Never Finished, But Keep Evolving
Good design always happens.
20
Websites Are Never Finished, But Keep Evolving
Good design always happens...just like the patient always stops bleeding.
21
Available Documents from UPA ConferenceWednesday, June 9, 2004
Making an Enterprise Portal Useful for EmployeesKen Becker, The Boeing Company
Voice Recording Tips for the Non-AudiophileTimothy Keirnan, Tec-Ed, Inc.
The Blind Leading the Blind: Theorizing a Web for the Visually ImpairedJessica D. Moore, Joseph Matthews, AARP Services
Panel: HCI and Human Factors Master's Degree ProgramsKirsten Robinson, Bentley College; Jessica Webster, Tufts University; Amanda Nance, Georgia Tech; Dana Gelman, Carnegie Mellon University; Rebecca Tremaglio, University of Michigan
Expediting the Usability Testing ProcessRobert Bailey, Computer Psychology; Kent Bailey, Mind Design Systems
How to Predict the Future: Becoming an Agent for ChangeSusan Fowler, FAST Consulting
Architecting Time: Designing Online EventsKatrina Friedman, Hot Studio
Site-Level Webapp Navigation Using Breadcrumb and Sibling MenusJames Snell, Concur Technologies
Making a Better Web FormCaroline Jarrett, Effortmark, LTD; Christopher Minott, LoanBright.com
When Your Group Can't Do It All: Investing UCD Resources WiselyLisa Battle, Lockheed Martin
User Experience Issues of IP Telephony: Not Your Plain Old Telephone AnymoreYihsiu Chen, PhD, AT&T Labs
An Alternative Method for Intranet Design Based on Content AnalysisRegis Magyar, Brian M. Anderson, Nancy Babiarz, James Marion, Panasonic Wireless
Accessibility at an Enterprise Software Company: From Guidelines to Accessible InterfacesGeorge Hackman, Oracle
How to Ensure Usability Offshore When Doing UI DevelopmentLiam Friedland, Westbridge Technology
22
Available Documents from UPA ConferenceThursday, June 10, 2004
The Value of Student Internships in Usability: Connecting Students with Professional Usability CommunitiesLee-Ann Kastman Breuch, Alice de la Cova, University of MinnesotaTask Analysis Tune-Up Paul McInerny, IBM; Interaction Guidelines for Japanese Web Forms Manabu Ueno, SociomediaMeeting the Challenges Posed by International Usability Testing: Two Case StudiesJenny Blackburn, Julianne Bryant, Getty ImagesDemographic Differences in Preferred Web Site Content Ann Chadwick-Dias, Donna Tedesco, Thomas Tullis, Fidelity Inv.A Structured Template for Writing Scenarios Paul McInerny, IBMUsability Guidelines for 3G Mobile Internet: A Korean Case Se-Hoon Kim, Young-Hee Kim, Mi-Jin Kim, Hey-Jin Chung...Too Much Usability? Steps and Missteps Ron Vutpakdi, Landmark Graphics CorporationAchieving Experience Equity and Universal Usable Access: Research Results and an Innovative FutureMary Theofanos, National Cancer Institute; Ginny Redish, Redish and AssociatesBalancing Rigor, Adaptation and Mentoring: Field Study with Customers to Initiate a Usability ProgramLaurie Kantner, Tec-Ed, Inc.; Rebecca Shaufl, Manatron, Inc.Going in Blind Doesn't Help: Cues for Navigation Sarah J. Swierenga, PhD, Laura G. Militello, Megan E. Gorman...Advanced Topic Seminar: Progress Indication and Responsiveness: User Needs, Design, and ImplementationJeff Johnson, UI Wizards; Paul McInerney, IBMUsability of Health Websites: What Have We Learned? American Institutes for ResearchWebsite Usability Questionnaires Thomas Tullis, Fidelity Investments; Jacqueline N. Stetson, Fidelity InvestmentsHow Standards Could Benefit Your Business Nigel Bevan, Serco Usability ServicesAdvanced Topic Seminar: Using Sounds & Graphics in Application Interfaces Alice Preston; Susan Fowler, FAST ConsultingPanel: Building an Online "Community“ Tharon Howard, Clemson University; Caroline Jarrett, Effortmark... The Usability Imperative Inherent in the Semantic Web Duane Degler, IP Gems; Renee Lewis, Pensare GroupEvaluating the Usability of Mobile Handheld Devices Regis Magyar, Brian M. Anderson, Nancy A. BabiarzAchieving Global Competitive Advantage: The Business Benefits of Designing for UsabilitySteve Jacobs, Information Technology Technical Assistance & Training CenterPeer Reviewed Paper: Integrating Accessibility into User-Centered Design Process: A Case StudyKay Lewis, Accessiblity Institute, University of Texas at AustinPeer Reviewed Paper: Freehand Interactive Design Offline (FIDO): A New Method for Participatory DesignDonna Tedesco, Ann Chadwick-Dias, Thomas Tullis, Fidelity Investments
23
Available Documents from UPA ConferenceFriday, June 11, 2004
Invited Speaker: Landscapes of Memory: History for Communities to Live ByKurt F. Anschuetz, Rio Grande Foundation for Communities and Cultural LandscapesDesign Patterns and Guidelines for Usable and Accessible Web ApplicationsDavid Hoffman, Lisa Battle, Lockheed MartinJustifying Funding and Space for a Usability LabAlice de la Cova, University of MinnesotaPeer Reviewed Paper: Participatory Techniques for the Design of a New Input DeviceEvelyne Millien, Camille Roux, Centre MultiCom Laboratoire CLIPS-IMAGWe Bombed in New Haven: Lessons from the TheaterWhitney Quesenbery, Whitney Interactive Design, LLCPeer Reviewed Paper: Participatory Design in Cognitive WalkthroughsSusan Mings, PhD, Scott Ottaway, PhD, Microsoft Corp.
PostersViSA: Video Segmentation and Annotation Afzal Godil, National Inst. of Standards and TechnologyNew Techniques for Understanding Communities Erika Darling, Emily Leventhal, The MITRE CorportationAdapting User-Centered Design to the Consumer Electronics Research and Development ProcessJoonhwan Kim, SungWoo Kim, Younghwan Pan, Samsung ElectronicsEye-tracking in Usability Testing: What Questions Can It Help Us Answer?Judith Ramey, Mary Deaton, Rich DeSantis, Elisabeth Cuddihy, Cory King, Quan Zhou, Jana Jones, University of WashingtonBecause Sometimes More is Less Julie Bzostek, Brett Walters, Micro Analysis and DesignRules of the Road: A Practical Guide to International Research Kathi Kaiser, Lyman Casey, Centralis Partners, Inc.Taskflow Charting: An Interactive Validation Method for Task Analysis Kraig Finstad, Intel CorporationHow Many Users Are Enough for a Card-Sorting Study? Larry Wood, Brigham Young University; Thomas Tullis, Fidelity InvestmentsRecruit More Test Participants Faster, Cheaper and Happier by Piggybacking on Existing Customer ConferencesMark Alves, Freddie Mac