Cengage LearningOpposing Viewpoints In

ContextUsability TestHao Zeng, Mikko Tuomela, Josh Keyes, Travis LaFleur

SI 622 Group 3April 14, 2011

Cengage Learning

Publisher of print and digital information services for academic, professional and library markets

Delivers customized learning solutions for colleges, universities, professors, students, and libraries

Opposing Viewpoints In Context (OVIC)

OVIC is an online resource covering today’s hottest social issues

It helps students research, analyze, and organize broad variety of data

Used for conducting research, writing assignments, debates and more

Test Recruiting

• Sent email to SI Informatics email list• Gave general information about study along with

how much time it would take• Promised them a $25 Amazon gift card after their

participation• We aimed for 5-7 subjects• Scheduled 7, 2 canceled so we tested 5 subjects

Usability Test Tasks

4 Tasks15 minutesClient: focus on search and multimedia content+

Try to simulate normal operations that a general user will usually do in the system

Usability Test Tasks

1. Find two articles about nuclear power, one in favor and one opposed. Translate the second article into French and send it to your French friend (hao.hci@gmail.com).

2. Find 2 radio broadcasts about the 2011 earthquake and tsunami in Japan, save them to your personal favorites in the system, and then start playing one of them.

3. Find a magazine article about unemployment and download the article’s APA citation.

4. Find an article related to stem cell research from a peer reviewed journal that was published in 2005.

Test Logging

• Logging sheet for each tasko From 2 to 4 subtasks

Completed?  Y / No General questions

Surprised?    Y / NStuck?          Y / NConfused?    Y / N

o Space for comments and notes

• To be filled either by pen or by computer (Word document) 

Moderation & Debriefing

• Introduction script• Sign consent form• Pre-test questionnaire 

o Background, demographics, experience• Begin tasks• Encourage talk aloud

• Post-test questionnaireo Reflect on experienceo Likes, dislikes, improvements

• Ask for questions• Thank and dismiss 

Recording

Location and Setup

• 1150B/1150C at North Quad• 3-4 researchers present • Audio recording by computer

Usability Test Findings and Recommendations

Usability Test Findings: #1

The “Listen” appears on top of the content of every document for people with visual impairment;But it causes significant confusion for general users.

Usability Test Findings: #1

Usability Test Findings: #1

Usability Test Recommendations: #1Recommendation: Rename to "Read the Screen" and move to Tools section.

Usability Test Findings: #2

Pro and Con Indicators Are Needed In Search Results

• System is designed to give users arguments from "both sides" of an issue

• Sides of articles aren't readily identifiable in search results

Usability Testing Recommendation #2• Implement short summary to let user

know if articles is "Pro" or "Con" for the subject that they are searching about

• "Pro" or "Con" summary could be anywhere from one sentence up to a short paragraph.

• Summary text could be placed to left of search results, just make column with results in it a bit skinnier (which would be ok)

Usability Testing Recommendation #2

Usability Test Findings: #3

Basic Search is used often, but is not fully satisfactory.

Usability Test Findings: #3

• Does not handle Google-style queries

        "stem cell research 2005"         "Japan tsunami audio"

• Recommendation: improve search algorithm to accept more parameters (not just keyword)

    

Usability Test Findings: #3

Recommendation: add filtering options from advanced search into results page for basic search    

Usability Test Findings: #4Finding 4: Options in Advanced Search Are Ambiguous   

• Users were confused about what "Content Level" means

• Because they don't know what content level means, they can't make sense of the options that they can select

• It frustrated users when they weren't able to tailor the advanced search options clearly

Usability Test Recommendations: #4

• Cengage should name "Content Level" something that is more descriptive of what the feature actually filters

• Cengage could add a help section that describes all of the features that are offered in the Advanced Search

Thank you!Questions?