Database Web Application User Test 2

USABILITY TESTING 2Database Web Application

Test Information

OVERVIEWTechSmith Morae was used on a laptop computer to conduct usability testing of the recently revised WVU Libraries Database

web application, using test questions from the first round of testing that were still relevant to the web application. This round of

usability testing was internal and focused on Undergraduate Student employees.

Test Information

TARGET AUDIENCE73%* of WVU Libraries users are undergraduate students who are 48% female or 52% male, are primarily Caucasian,

African American, Hispanic, or Asian in ethnicity, are United States citizens or are international students, are 21 years in their

average age, and are 49% from the state of West Virginia or 51% from other United States, territories, or countries.

Target audience numbers are populated from both survey polls and server statistics.*

Test Information

DATABASE USE96%* of the WVU Libraries Database web application usage is from non-Downtown Campus Library WVU employee

computer users ‒ 64% of which are returning users, and 61% of which are from ages 18 to 34 ‒ on and off Morgantown

campus, whereas 4% of Database web application usage is from WVU employee at the Downtown Campus Library. The

Geographic Breakdown of these 96% of users is as follows: 99% from the United States; 86% from the state of West

Virginia; 77% from Morgantown, West Virginia; 1% outside the United States.

Statistics are calculated by Google Analytics.*

Test Information

PARTICIPANTS

Ajay Hannah Kyle Robert Samantha Taija

https://testing.lib.wvu.edu/usability5/AJ.wmv

https://testing.lib.wvu.edu/usability5/Hannah.wmv

https://testing.lib.wvu.edu/usability5/Kyle.wmv

https://testing.lib.wvu.edu/usability5/Robert.wmv

https://testing.lib.wvu.edu/usability5/Samantha.wmv

https://testing.lib.wvu.edu/usability5/Taija.wmv

https://testing.lib.wvu.edu/usability5/AJ.wmv

https://testing.lib.wvu.edu/usability5/Hannah.wmv

https://testing.lib.wvu.edu/usability5/Kyle.wmv

https://testing.lib.wvu.edu/usability5/Robert.wmv

https://testing.lib.wvu.edu/usability5/Samantha.wmv

https://testing.lib.wvu.edu/usability5/Taija.wmv

TASK ANALYSISTest 1 & Test 2

Test 1 & Test 2

TASKS1. Go to WVU Libraries’ Databases.

2. Find the database PubMed.

3. What is the earliest coverage in the database SportDiscus?

4. What is a database for the subject Marketing, that is also an

encyclopedia?

5. Your assignment is to listen to an aria from an opera. What

database will you use?

6. How many databases cover the subject of psychology?

7. Find a subject guide for engineering.

8. Go to the tutorial for the database Ad$pender.

9. What are the best databases for finding sources in biology?

10. What is the starting year of coverage for the database Web of

Science?

11. How can you get help with database selection or searching?

12. You need a brief video clip on chemistry lab safety. Where

would you find it?

13. Find 2 databases that contain statistics for market research.

14. What is a database for Theater that is contains Images?

15. How many primary sources can you find for West Virginia

databases?

■ - Omitted Tasks in Test 2

Test 1 Results:All Tasks

AVERAGE TIME* Average time includes delivery of tasks.

Test 1 & 2 Comparison:All Tasks


0.74

1.17

1.07

0.46

0.66

0.99

0.84

1.72

0.95

0.6

0.84

0.61

0.41

1.33

0.87

0.88

0.98

0.64

1.43

1.46

1.39

1.03

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Task 1

Task 2

Task 3

Task 4

Task 5

Task 6

Task 7

Task 8

Task 9

Task 10

Task 11

Average Time (Minutes)

Task

Test 2 Test 1


SUCCESSRATES* Percentages based on a 0 to 3 scoring system.


ERRORRATE* Error rate is a standard deviation based direct path to information as it is known to test administrator, and the rate of the user encountering a functionality or system error.



■ - Test 1 ■ - Test 2

SYSTEM USABILITY SCALETest 1 & Test 2

Test Information

*(SUS) QUESTIONS1. I think that I would like to use this system frequently

2. I found the system unnecessarily complex

3. I thought that the system was easy to use

4. I think that I would need the support of a technical person to

be able to use this system

5. I found the various functions in this system were well

integrated

6. I thought there was too much inconsistency in this system

7. I would imagine that most people would learn to use this

system very quickly

8. I found the system very cumbersome to use

9. I felt very confident using the system

10. I needed to learn a lot of things before I could get going with

this system

* System Usability Scale (SUS)


SUS SURVEY* Results are based on survey results taken after usability testing, and are concerned with the Database web application interface and functionality.

30%

70%

90%

50%

10%

20%

70%

30%

10%

50%

50%

40%

30%

20%

30%

30%

50%

70%

30%

60%

30%

50%

30%

40%

10%

I think that I would like to use this system frequently

I found the system unnecessarily complex

I thought that the system was easy to use

I think that I would need the support of a technical person to be able to use thissystem

I found the various functions in this system were well integrated

I thought there was too much inconsistency in this system

I would imagine that most people would learn to use this system very quickly

I found the system very cumbersome to use

I felt very confident using the system

I needed to learn a lot of things before I could get going with this system

Strongly Disagree Disagree N/A Agree Strongly Agree


SUS SURVEY* Results are based on survey results taken after usability testing, and are concerned with the Database web application interface and functionality.

49.98%

49.98%

49.98%

66.64%

33.32%

49.98%

49.98%

33.32%

33.32%

16.66%

33.32%

66.64%

33.32%

33.32%

16.66%

16.66%

16.66%

33.32%

66.64%

33.32%

66.64%

33.32%

16.66%

33.32%

33.32%

33.32%

I think that I would like to use this system frequently

I found the system unnecessarily complex

I thought that the system was easy to use

I think that I would need the support of a technical person to be able touse this system

I found the various functions in this system were well integrated

I thought there was too much inconsistency in this system

I would imagine that most people would learn to use this system veryquickly

I found the system very cumbersome to use

I felt very confident using the system

I needed to learn a lot of things before I could get going with thissystem

Strongly Disagree Disagree N/A Agree Strongly Agree

SUMMARYNotes & Observations

TASK 1WOuld type lib.wvu.eduWOuld click on Databse tabsWent to websiteWent to the library databases website, clicked on tab, then quick linkWent to google first to search for 'WVU Libraries' clicked on submeta info in googleDoesn't know how to get to databses.didnt click on databses quick lik - didnt scrolld own

TASK 2WOuld go by titlewent to databses by

TASK 3Would use Met Opera on Demand, Or Opera in VideoWould searc h in searchbox for opera, Met Opera on DemandWould say theatre...WOuld just use concert hallWould go to Subject/MusicWould click on viudeo and soundwould add title to search boxWent to Video and soundWent to video and soundWent to theatre in videoUser didn't know music wasn't a daabseTHought search was general search instead of just title

searchTHinks that Video & Sound is a databseStudent thought subjects was a datbaseOpera on demandMet Opera on demandDatabases from search results showed no sugegstions -errorChoose trail database because star

TASK 4Would Search - need to change to Databases by titleWould go click on title PWanting to search for subject in search box - Found 1Not seeing number for results - make largerMusic Dattbase didn't loadLeft justify results , and list at boottom?Isn't using subjects or scrolling dowqnIsn't seeign results immeadetlyIS using search box to search for subject - found 7Didn't notice databases by subject524ish

TASK 5Would click on EWas cobfused and searcheduser doesn't know what subject giuide is, went out to search onlieStudent sees two 'Engineering' databses, si confused

search box should possibly search annotations & titles & subjectsPossibly reword to include just 1 engineering subject, have subject guides as a navigation item, or show up search resultsis searching for subject guideDoesn't know what a subject guide isDeosnt't know what Subject Guide is./ Didnt see first time around

TASK 6We should state 'Database Title Search' that it only searches databse titlesUSer doesn;t know whicjh are best - agricola/academic search compelteStart HEre is sugegstedPossibly reword Start Here to signfy best databasesNot using yellow databases.mising yellow box as imporatnace or suggestionLisated at topI would just search for oneI really dont kno a good journal... normally journals would have imapct scoresDidn't associate yellow databses with best databases.

Live Notes

1/2

TASK 7Would click on Ask A librarianWould just clickon Ask A Librarianscrolle ddown to help with databses - would callI would call to talk to a librarian or talk to someone in personDidnt knwo there was text measssaging optionCOntacting someone at bottom or Ask A LibrarianCLick Aska librarianAsk A real person'Oh, I didn't know there was a text messaging.'

TASK 8Would use seach box as a keyword sechwould use films on demandWould search through individual Databases under chemistryWould go to YouTubeWOuld go to Chemistry/Would go to Subject GUide/Would go to chemistry or video/soundWould c lik on first one to searchWent to video and soundWent to subject guide but nothing about videos was there. Was this a good question?Went to Chemistry,m but is reading through for videosWent to chemistryUser would search in search boxSearch Box results dont suggest databasespossibly change video/sound from subject o mediaNo suggested databses for search results

Need to show video resources or medua resourccesDont see anythign that says videoscan't find chemistry video; searched in search boxCan get toi chemistry, but doesn't seem to know how to get specific media

TASK 9Would use ProQuest Statistical Insight and Richard K MilelrWould go to marketingWent to marketingUser is not using in-page searchingStatista & iLibrary StatisticsStatistaReading through finding research or statisticsRead to find statistics, OECD and StatistaQuestion for eResources-- should there be a Statistics subject area?Keeps trying to search in boxIS rreading through for media typesIs reading through Marketing Subject - Ibis WOrld, and PAssportIndividually reading through definitions of Marketing databasesChose EMIS MAket research and IBS World

TASK 10Would search and go to Theatre in Video

Would just use google for searchingWould just try to click one that says video to see whats in itWould choose encylcopedia of cltohing and fashonWent to threatre, searching for media typesSearching subject in search boxSearched in page of theatere for imagesScroleld to theatreJStor includes high-quqality mages (Student was readinthrough)Is there an advanced search here?IDKFOund some magazine ones... would choose thosecant find medium types

TASK 11Would think newspapers are primary sourcesWent to WV, found 2User is reading through for 'Primary SOurces'THought the titles under subject were the databse titlessearched for west virginia instead of using link; search box should search subjects as well as titles?not going to subjects/ WVI would search west virginaiFOund 2Didn't knwo what rpimary sources meant . said 11 resultsCLicked West VIrignia411 results

Live Notes

2/2

Average Time- 54% of task completion required more time in Test 2 when compared to

Test 1- 36% of task completion required less time in Test 2 when compared to

Test 1- 10% of task completion required the same amount of time in Test 2

when compared to Test 1

Success Rate- Success rates for both 'Completed with ease' and 'Completed' were

largely reduced in Test 2 when compared to Test 1- Success rates for both 'Completed with difficulty' and 'Failed to

complete' largely increased in Test 2 when compared to Test 1

Error Rate/Standard Deviation- The standard deviation of error rates have majorly increased in Test 2

when compared to Test 1

SUSThe following 70% of SUS responses decreased in favorable response:

- I needed to learn a lot of things before I could get going with this system

- I found the system very cumbersome to use- I thought there was too much inconsistency in this system- I found the various functions in this system were well integrated- I thought that the system was easy to use- I think that I would like to use this system frequently

Data Observation

TRENDS

ENHANCEMENTSSuggested

Data from the second round of usability testing indicates a False-Consensus Effect (from client to user) - which can be seen in the Average Time, Success Rate, and Error Rate metric comparison - and is illustrated on page 24 of this report.

The majority of enhancements to content and user interface are maintenance-specific only; however, potential major impact areas reside with the development of the search box, increasing its functionality, and through conducting further user testing.

Both search box development and customer experience, or CX-based user testing should provide insight/dramatically reduce the percentage of failure rates, and decrease the amount of time for task completion.

Content- Change ‘Database Search’ to ‘Database Title-Only Search’ or

‘Database Title Search’- Database descriptions rewritten/reformatted and simplified to be more

readable/scan-able as users scroll- Add ‘Image’ type to ‘Database Subjects’, or revise ‘Database Subjects’

to exclude media types (see *)- Change verbiage from popular databases from ‘Start Here’ to

something more assistive/top-picks orientated

Development- Database search box searches database description and subject- Fix error on search box ‘no results’ to return popular databases.- Integrate chat box into the desktop user interface- Integrate a subject guide list- Modal information for primary resources, subject guides, and types

(see *)

Usability Testing- Shift usability testing to be more formative and away from summative- Conduct future usability testing focusing only on target audience- Conduct future usability testing focusing on qualitative results and user

desirability

User Interface- Prioritize ‘Database by Subject’ section over ‘Trial Databases’ section- Make item or media types stand out in Database description, or add

media keywords/tags to the bottom of Database description *

Other- Determine both the stakeholder’s and the user’s goals/priorities for

project

ENHANCEMENTS

Database Web Application User Test 2

Education

Transcript of Database Web Application User Test 2