Usability Within Limits

Jackie ZajacDirector, Strategic Communications

@jackiesauter

• This is not uber technical• $500 over 18 months• Colleagues not familiar with UX• Significant improvements AND mixed results

Ground Rules

Inspiration

• PSU Web Conference• User Interactions with Information Systems• Declining graduate enrollment• Bickering over web strategy

AU’s Kogod School of Business

• 1,200 undergraduate, 600 graduate students

• 120 Faculty & Staff• 8 Graduate Programs• 40,000+ visits/month

“1,000 More Grad Students in 10 Years”

Chronology

First study done

Changes implemented

Presented at PSUWeb 13

Follow-up study done

Changes implemented

06/12 09/12 06/13 07/13 10/13

The Players

8 Faculty Program Directors

6-person Admissions Team

Dean, Provost, University Marketing Advisory Council, Web Steering Committee...

Study Parameters• Time (< 3 weeks)• $250 budget • Silverback ($69.95)• Gift cards

• Technical limitations due to CMS• Only tested American.Edu/Kogod

Magic Numbers

6 Tasks, 5 Users*

*Jakob Nielsen said it was OK!

Magic Numbers

Source: NIELSEN NORMAN GROUP

Test Design• Homepage Impressions• Tasks (6)• Task Completion Survey

• Final Thoughts?• Less than 30 Minutes

Tasks 1. How many credits are in the MS in Accounting program?

2. Find the average GMAT score for a Full-time MBA student

3. What is the cost of the Part-time MBA program?

4. When is the next Information Session being held?

5. Find the graduate admissions application

6. Find the graduate admissions’ office phone number

Completion Survey

After each task, the user was asked:1. Did you complete the task? (Yes/No)2. How satisfied are you, on a scale of 1 to 5*?3. How easy was it, on a scale of 1 to 5*?

*5 being very satisfied

40%60% 60% 60%

80% 80%100%

80% 80%60%

80% 80%

Task completion rate

Task completed from testers' perspectiveTask completed from users' perspective

Site is attractive and has a good balance of graphics/text.

Site was easy to navigate.

Information is easy to read.

I always felt I knew what was possible to do next.

My mistakes were easy to correct.

0 0.5 1 1.5 2 2.5 3 3.5 4

Users’ Final Impressions

MaybeNoYes

Number of user responses

Key Finding:Too Many Navigational Options

What’d We Do With All This?

Recommendations

1. Added an Admissions tab to the top nav (Duh.) 2. Linked directly to each graduate program from the

homepage3. Restructured Admissions nav 4. Created a new, central MBA “landing” page

Recommendations

5. Decrease jargon6. Make functional info clearer + easier to find7. Social media more prominent 8. Reduce image file sizes9. Delete homepage “touts”

All Aboard!

All Aboard!

Dear all,

You may have noticed some changes to the Kogod website recently. They are the result of a usability study of the website we conducted this summer. The focus of our testing was to determine how easy the website is for our primary audience—prospective students—to use. Based on the study's results, we've implemented several changes to the website's layout, navigation and terminology. A few more changes lie ahead.

We expect that this process will make it easier for our prospects to find the information they're looking for, and to take action. A follow-up study will be conducted in 2013 to determine whether further improvements are needed.

If you have any questions, do let me know.

Relationships

Challenging.

The players

8 Faculty Program Directors

Your BFFs <3

1. Data

2. Admissions

3. School Goals

Results @ 6 WeeksAdmissions Homepage: 7,637 pvs (+3%)MS Finance: 3,819 pvs (+50%)MS Accounting: 3,115 pvs (+60%)MS Sustainability Mgmt : 1,325 pvs (+5%)MS Taxation: 1,106 pvs (+84%)MS Real Estate: 1,095 pvs (+63%)Renamed “Tuition & Fees”: 2,147 pvs (+150%)New MBA “landing” page: 2,617 pvs (n/a)New MS Applicants page: 2,515 pvs, 02:19 TOP

…Spring 2013…

Redesigned All Degree Pages• Clearer Calls to Action• Consistency

Redesigned Graduate Programs Index Page

Remember:

Always send a status update to stakeholders.

Don’t forget SEO.

…Summer 2013…No substantial change in visitorsAdmissions homepage: +3.8%

Tuition & Fees: +190.7%

But we were DOWN:• Pageviews -6.7%• Avg visit duration down (slightly) • Pages per visit -7.3%

…Summer 2013…

• Usability testing with same tasks • 2-3 additional tasks

• Tested microsite for direct marketing campaign

• Tested ad landing pages • Tested on desktop + mobile devices

Follow-up Findings

• 3/5 liked the degrees listed on homepage• 2/5 immediately clicked on “Admissions” tab• Users liked the MBA landing page• All 5 appreciated the simplicity, were not

overwhelmed by information

Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7

100% 100%

60%

80%

100% 100%

80%

100% 100%

80% 80%

100% 100% 100%

Task Completion Rate

Completion from user's perspective Actual Completion

The overall site is attactive

Site was easy to navigate.

Information is easy to read.

I always felt I knew what was possible to do next.

My mistakes were easy to correct.

0 1 2 3 4 5 6

1

3

4

5

5

5

5

Users' Final Impressions

YesMaybeNo

Does this mean we’re there?

Not quite.

• Confusion about “part-time” vs. Professional MBA• Difficulty finding GMAT scores• Tuition confusion (PMBA)• Admissions events

Finally,

• Usability testing with same tasks • 2-3 additional tasks• Tested microsite for direct marketing

campaign• Tested ad landing pages • Tested on desktop + mobile devices

Photo by Cade Martin

• Disconnect between what users say and what they do• Scrolling, task completion

• Career outcomes very important• Photo reaction

Finally,

Questions?

@jackiesauter