SI 622 - First Class Presentation

8/3/2019 SI 622 - First Class Presentation

http://slidepdf.com/reader/full/si-622-first-class-presentation 1/33

ANN ARBOR.COM USABILITY

TESTING REPORT

Team Newsies:

Danlu Fu, Garima Garg

Catherine Le, Caroline MoranApril 2010

SI 622: Evaluation of Systems and Services

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



• Who is …

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Why Usability Testing?

• To understand real users

– How they interact with the system

– How they feel about the system

• Experts know too much

• Too expensive to fix problem after launch

Adapted From lecture slidesDa

nlu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



METHODS

Usability Testing

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Participant Selection

• Target Participants

– Age range: 20 - 30 years old

– Novice users

• Recruiting

– 5 participants

– Our personal networks, our colleagues and graduate

students at other schools

– Incentives: $25 gift cards

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Process

• When, Where, Who and What

– April 6 and April 7, 2010

– Ann Arbor.com office conference room

– 1 moderator, 1 note-taker and 1 tester – Laptops with a trial version of Camtasia Studio

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Process

• During the testing

– Introduction

– Pre-questionnaire

– 5.5 Tasks – Debriefing

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Process

• Tasks

– Task 1: Register a new account

– Task 2: Find a specific article and share it on Facebook

– Task 3: Find a crime report in your area• 3a: [if user does not use the Neighborhoods section]

Find the crime report in the Neighborhoods section

– Task 4: Find and choose a concert

– Task 5: Find some promotions

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Results and Analysis

Jakob Nielsen’s (1993) rating scale for usabilityproblems to rank the severity of each keyfinding:

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



KEY FINDINGS

Usability Testing

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Finding 1

• Neighborhoods Section does not displaycorrectly in all browsers

Firefox 3.6.2 browser on

the same computer

IE 8.0 browser

4

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Finding 2

• Users have difficulty navigating within theEvents Calendar

– 3 out of 5 users failed the task to find an event

– Most users do not notice that there is an actualcalendar

– Most users did not navigate using the links at the top

of the Events Calendar section

– Users often found an article about an event throughthe Entertainment section

4

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Finding 2

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Finding 3

• It was equally intuitive for users to seek theneighborhood crime information under theNews sub-section of the drop-down menu.

– Unfortunately, it shows an Ann Arbor-wide crimereport rather than the neighborhood-specific crime

report.

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Finding 4

• Hard to differentiate and find Ann Arbor city-wide news versus neighborhood-specificnews

– While searching for neighborhood-specific crime news,some users found the Ann Arbor-wide crime report

whereas others found a more neighborhood-specific

crime report.

–There is no way to access the neighborhood-specificcrime report from the Crime/News section and there is

no way to access the Ann Arbor-wide crime report

from each Neighborhoods section.

2

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Finding 5

• Users have difficulty finding the EventsCalendar

– Users often went to the Entertainment section first to

look for local events such as concerts – The link is actually located on the right side of the

drop-down menu, but not very visible.

1

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



RECOMMENDATIONS

Usability Testing

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Recommendation 1

• Run browser compatibility testing

– Further testing should be done to ensure that all

features and are compatible with and consistent across

all common browsers and platforms.

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Recommendation 2

• Increase visibility of the map of Ann Arborarea for the Neighborhoods section

– A map of Ann Arbor overlaid with the corresponding

geographic labels is displayed once the Neighborhoodssection is reached, but more feedback should be

provided at the onset.

– Either a zip code query or map in the drop-down menu

would alleviate the confusion users face when trying to

determine which geographic label link is the most

relevant for them to click.

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Recommendation 3

• Include an Events Calendar link in theEntertainment section

– There seems to be a strong and intuitive association of

Events (such as concerts) with the Entertainmentsection

– One such example being Boston's

alternative news source, the Weekly

Dig (http://www.weeklydig.com)

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Recommendation 4

• To focus attention, organize events by theweek's most highly featured recommendations

– In LA Weekly’s calendar, only a handful of events are

considered staff picks each week.

© Dan

lu Fu

, Garim

a Garg,

Catherin

e Le,

Caroline

Moran



Recommendation 5

• Add more cues to describe events at a glance

– Icons to the left describe the event type, but no legend

is given.

–Describe the event type in tags, such as "Concert”,“Folk”, or “the Ark", for those who are not able to

determine the event type from titles.

© Dan

lu Fu

, Garim

a Garg,

Cathe

rine L

e, Ca

roline M

oran



Recommendation 6

• Increase consistency of access to articlesacross site

– It is equally intuitive to choose one route or another,

such as choosing the Crime tab or the Neighborhoodstab when looking to find reports about neighborhood

crime.

– If articles are cross-linked in multiple related sub-

sections, initial mistakes in information-seeking can be

minimized since there will be a safety net of multiple

modes and routes to access the same information.

© Dan

lu Fu

, Garim

a Garg,

Cathe

rine L

e, Ca

roline M

oran



New Users

• 5 participants who have never visitedAnnArbor.com – In real cases new users probably won’t use these

specific functions in our tasks on their very first visit

– During their navigation through news articles, theymight notice some non-news related sections such asthe Events Calendar, which might further indicate whatelse they can do on AnnArbor.com.

• Further Research – Recruiting some regular users who never used

functions other than reading the news

© Dan

lu Fu

, Garim

a Garg,

Cathe

rine L

e, Ca

roline M

oran



Unfamiliarity with the city of

Ann Arbor• All of the participants came to Ann Arbor

within the past year – 2 participants said that they are not familiar with the

area distribution of Ann Arbor because they arenewcomers here

• Further Research – Recruit several local young participants so as to

evaluate the problem of area location forNeighborhoods section

© Dan

lu Fu

, Garim

a Garg,

Cathe

rine L

e, Ca

roline

Moran



Task Wording

• Many participants clicked Entertainmentinstead of the Events Calendar to find concertevents

– It might be wise to test other kinds of events which arenot related to any tab on top menu so as to eliminate

the impact of other options as much as possible.

• Further Research – Example tasks such as finding a workshop can be used

to replace the concert task

© Dan

lu Fu

, Garim

a Garg,

Cathe

rine L

e, Ca

roline

Moran



• Findings: – Participants have difficulty finding their neighborhood

within Ann Arbor.

– They also have difficulty with understanding whether

they should go to the Entertainment or the EventsCalendar section for concert information.

– Navigation within the Events Calendar section didn’t

work as users expected.

– Ann Arbor-wide news is only in the main Newscategory, while related neighborhood-specific news is

only available in Neighborhood sections.

© Dan

lu Fu, Ga

rima G

arg,

Cathe

rine L

e, Ca

roline

Moran



•Recommendations: – Unambiguous geographic identification in

neighborhood sections.

– The Events Calendar should be more visible to usersand integrated with other sections such as

Entertainment. – Better and clearer navigation within Events Calendar

should also be supported so as to provide flexiblepaths to desired events.

– Increased consistency across the general News sectionsand Neighborhood sections should be considered forfuture improvements.

© Dan

lu Fu, Ga

rima G

arg,

Cathe

rine L

e, Ca

roline

Moran

SI 622 - First Class Presentation

Documents

Transcript of SI 622 - First Class Presentation