User Research Delivers for the U.S. Postal Service: The Impact of Customer Inputs on the Enhancement of USPS.comUXDC 2017, Washington DCMark Becker, UserWorks Inc.

Background• Started in 2013, still ongoing with 2-3 tests a year on

average• Part of an effort by USPS to update and modernize

their site and web presence• Multiple studies, various research methods• Many areas of USPS.com and a mobile app

What’s the IMPACT?• Influential impact• USPS improving using an incremental UCD approach may

inspire other large organizations• Business impact• An improved user experience can be a competitive

advantage• Innovation impact• USPS gains knowledge about its customers’ needs and

wants, and can develop new technology to serve that need• Visual impact• The changes to the site as a result of testing are highly

visible

METHODS

Methods• Moderated• Usability testing• Card sorting

• Unmoderated• Tree testing• First-click testing

Source: https://blog.prototypr.io/

Usability Testing• Moderated, one-on-

one• Participants perform

representative tasks• Note behaviors, ask

follow-up questions

A participant attempting a task during a session

Usability Testing: Pros And Cons• Pros• Directly observe participants• Easier to control the course of the session• Immediately follow up with questions or probes

• Cons• Unless willing and able to travel, participants from limited

geographic area • Less convenient for participants• Some may be uncomfortable in a lab setting

Remote Usability Testing• Can get same insights as in-person sessions• Remote participant controls our computer• Audio via phone or VoIP

• Wider geographic area• More convenient for participants• Technical considerations

(which software to use, Internet speed, etc.)

Card Sorting• Organize cards representing

site content into logical groups• Either physical cards

or online• In-person allows duplicates,

omissions, sub-groups, online does not

• Open• No limit to number of groups• Can name the groups

• Closed• Limited number of pre-named

groups

A card sort with physical cards An online card sort

Card Sorting: Pros And Cons• Pros• Develop a content organization scheme• See what content belongs together vs. what should be kept

separate• Group labels can inform how to label menu items in IA

• Cons• No insight into layout, visual design• Participants only see descriptions of content• Have to be careful how to label the cards

Unmoderated Methods• Tree testing, first-click testing• Researcher sends email with link to access the tool• Tool displays instructions, tasks to participant and

collects data

Tree Testing• Participants see only the hierarchy, or “tree” of topics

and subtopics that make up the site• Tasks based around finding items in the hierarchy• Can’t rely on visual design• Provides data about how many:• Successfully identified the correct location • Went to the correct location immediately versus spent time

exploring(direct success vs. indirect success)• Visited the correct location but didn’t select it• Etc.

Tree Testing

First-click Testing• Participants shown an image of a page, asked to click

where they would expect to find information• Only the first click is recorded• See heatmap of where participants clicked

First-click Testing

Unmoderated Pros And Cons•Pros•Larger sample size•Asynchronous, participant does on own time•Can get more data in less time•Usually less expensive

•Cons•Participant behavior may be ambiguous •Rushers, cheaters, the unmotivated •Help from others

•Cannot easily ask follow up questions

Testing On Mobile• More USPS customers

using mobile devices to access USPS.com and make purchases• USPS wants to ensure their

mobile site has the same functionality as their full site

A participant exploring the MyUSPS tool during a test session

Mobile Testing: Pros And Cons• Pros• Interact with the site on the device it was designed for• Test responsiveness of the design• Compare between mobile and desktop versions

• Cons• Additional technical considerations (stream screen or use

overhead cameras?)• Privacy concerns

STUDIES

Participants• Users of USPS.com, users of competitors, novices/first-

time/never used before• Those who mail/ship from desktop, tablet, phone• General public, business owners, mail marketers• Frequency, recency of visiting USPS.com• Reasons for visiting USPS.com (make purchases, print

labels, find information, etc.)• Age (“millennials”), gender, Internet proficiency, etc.

Research Focus Areas• Four main areas of USPS.com• Homepage

Homepage Usability Test (2016)

Homepage prototype for desktop

Homepage prototype for mobile

USPS.com Homepage

2014 2017

Research Focus Areas• Four main areas of USPS.com• Homepage• Global Navigation

Global Navigation Baseline Usability Test (2013)

Recommended changes to the content organization

Global Navigation Card Sort (2013)

Online card sort example Recommended top-level changes

Business Card Sort (2014)

Example of subgrouping,

in-person card sortRecommended changes to the second-level

content

Tree And First Click Tests (2014)

Examples of the recommended optimized IA

Tree And First Click Tests (2014)

Example results from a first-click test

Global Navigation

2013 2017

Research Focus Areas• Four main areas of USPS.com• Homepage• Global Navigation• Mailing, Shipping, and Tracking

Shipping Usability Test (2013)

Excerpt from the test report

Tracking Usability Test (2013)

Comparison between the 2013 tracking page and the redesign

Click-N-Ship (2014)

Excerpts from the test report

Pre-shipment Usability Test (2016)

Excerpts from the test report

Mailing, Shipping, And Tracking

2013 2017

Research Focus Areas• Four main areas of USPS.com• Homepage• Global Navigation• Mailing, Shipping, and Tracking• Postal Store

Postal Store Baseline Usability Test (2013)

Excerpt from the test report

Postal Store Card Sort (2013)

Recommended categorization and sub-

categorization

Postal Store Usability Test (2015)

Excerpts from the test report

Redesigned Postal Store Usability Test (2016)

Example results from testing the iconography

Redesigned Postal Store Usability Test (2016)

Excerpt from the test report

Postal Store

2014 2017

Research Focus Areas• Four main areas of USPS.com• Homepage• Global Navigation• Mailing, Shipping, and Tracking• Postal Store

• My USPS

MY USPS Prototype Usability Test (2016)

My USPS prototype on desktop

My USPS prototype on mobile

MY USPS Registration Process Usability Test (2016)

Stages in the My USPS registration process

Mocked up My USPS data used for testing Informed

Delivery

CONCLUSIONS

Conclusions• Many changes over the years• Today’s USPS more modern, better user experience• Challenges remain• Hopefully this work will inspire others

• UserWorks, Inc. performed the work described in this paper while acting as a subcontractor to Aquilent. We gratefully acknowledge their support and collaboration.

• Questions?

Thank You!

Mark BeckerUserWorks, Inc.mbecker@userworks.com301-431-0500

(Now Booz Allen Hamilton)