PENN STATE SPECIAL COLLECTIONS WEB SITE USER STUDIES

Michelle BeldenSpring 2011

HOMEPAGE SCREENSHOT

GETTING STARTED Read two books available on our shelves:

Web Site Design with the Patron in Mind, by Susanna Davidsen & Everyl Yankee Design talk : understanding the roles of usability

practitioners, web designers, and web developers in user-centered web design, by Brenda Reeb

Reviewed mission statement of PSU Special Collections(“…to provide primary source materials for researchers

from Penn State and around the world”)

GETTING STARTED, CONTINUED

• Brainstormed user groups• Brainstormed user tasks• Got input/approval from colleagues• Met with Usability Team• Became certified for IRB• Secured funding

STUMBLING BLOCKS IRB (Institutional Review Board) process Funding/reimbursement process Recruitment

OTHER NOTESo Not testing finding aids or digital projectso Specific eye towards administrative organization o Having deadline was good to force IT cooperation

STUDY PROTOCOL Introductions

Explanation of the study:

We are testing the usability of the website, NOT your skills as a searcher. Please think aloud while performing the tasks,

about how and why you make choices, and what confuses or frustrates you.

We can clarify the tasks, but cannot help you perform them. The session will be recorded, and notes will be taken, but your involvement will be

anonymous. Recordings will only ever be shown within the department for internal purposes. Your name will never be associated with the recordings or the notes.

After you have done the tasks we will ask you to complete a brief survey. The entire session should last about 1 hour.

Participation in this study is voluntary, and you can choose to end this session any time.

Do you have any questions? Consent forms Set up computer and give participant list of tasks Take notes Questionnaire Ask again if any questions Thank them, provide compensation, get receipt

TASK LIST Warm-up:

Use our website to answer the following questions: What hours is the Special Collections Library open to the public? Where is the Special Collections library located?

PSU students (undergrad and grad): You have been assigned to write a paper using historical documents from the Penn State Special Collections

library. Your professor has told you that there are special procedures for using Special Collections. Using our website, find two ways using a special collections library is different from using a general academic library.

You have found sources within Special Collections and written your paper. Use our website to find the correct format for citing those materials.

PSU staff: Your filing cabinets are overflowing, and your supervisor has asked that you transfer old “stuff” to the

University Archives to make room for current “stuff”. Locate information on what records we collect on our website.

General:  Can you locate the following on our website: (Each subject asked 2)

Digital images of Pennsylvania railroads? Information about special collections materials related to the 8th air force? Information about special collections materials related to Amish settlements in Pennsylvania? Information about special collections materials related to African American history?

Can you use our website to: Figure out who you would ask a question about the United Steelworkers archives? Order a Fred Waring Christmas CD? Find a yearbook photo of David Arthur Valentine, who graduated in 1961? Find out if we have video of the October 1940 football game with Temple?

QUESTIONNAIRE1) How much time do you spend using the Internet for homework or work, per week?

___ Never ___ 1-5 hours ___ 5-10 hours ___More than 10 hours

2) Do you consider yourself an expert computer/Internet user? Yes/No

3) Have you ever been oriented to special collections (generally or at Penn State) through a class, tour, online tutorial, or other outreach activity? Yes/No

4) In the last year, how many times have you used special collections materials?At Penn State: Never, Once or twice, Between 2 and 5 times, More than 5 timesAnywhere Else: Never, Once or twice, Between 2 and 5 times, More than 5 times

5) The Special Collections Library Web Site is easy to use: __ Strongly Agree __Agree __Disagree __Strongly Disagree

6) If you selected Disagree or Strongly Disagree please tell us what causes difficulties:

7) What changes would you make to improve the Special Collections Library’s website?

OBSERVATIONS – TASK CONSTRUCTION Terms matter

When phrase “historical materials” in question, they go to “Historical Collections” unit – same for “archival materials” and University Archives

“Reference” confused for citation Along these lines, subject knowledge matters –

“Amish” in question, “Pennsylvania German” in collection title

Order of questions can influence approach May need to change order from study to study

OBSERVATIONS – SEARCH (GENERAL)

About half didn’t even look at the side bar navigation

Everyone likes to use the site search But it is unclear that this searches all the

University Libraries. They also want to search on format (and can’t)

OBSERVATIONS – SEARCHING SUBJECTS None of those looking for Amish materials

found our largest Amish collection Only one person looking for African-American

materials found our largest A/A collection Most commented they wanted collections

listed by topic

OBSERVATIONS – THE USELESSNESS OF LONG EXPLANATIONS ON WEBSITES

Half of the subjects commented there was too much text on any given page

Several could be on the right page but not see a link right in front of them, if there were too many “ other things” to look at

Yadda yadda yadda yadda yaddaYadda yadda yadda yadda yaddaYadda yadda yadda yadda yaddaYadda yadda yadda yadda yadda

Yadda yadda yadda yadda yaddaYadda yadda yadda yadda yadda

Yadda yadda yadda yadda yaddaYadda yadda yadda yadda yadda

Yadda yadda yadda yadda yaddaYadda yadda yadda yadda yadda

IMMEDIATE CHANGES Add curator contact info to all collection pages Add “(yearbook)” to LaVie link Add citation info to tutorial & online reference pages Add link to digital collections from search collections page Hopefully, organize by subject rather than unit Hopefully, further edit long textual descriptions on top-

level pages

POSSIBLE SUBJECT-BASED DESIGN

NEXT STEPS First round not perfect, but we’ve learned a

lot, not just about the web site but about the user study process.

For the next round, we may use a more ethnographic approach.

Google Analytics. Site search just for Special Collections? Better finding aids & FA search (then test). I will be asking to join the Usability Team.

AREAS FOR FUTURE RESEARCH/COLLABORATION/BRAINSTORMING

Apply Kulthau? Standardized list of questions for

archival website user studies? Explore ideas for “Research Help”:

Diagrams/visual dictionary of common terms Possible to do mouseover definitions in CMS? FAQ since looks simpler/quicker than tutorial? "Search Tips“

Thinking about how to illustrate concepts/collections with just one image

KULTHAU’S INFORMATION SEARCH PROCESS

http://comminfo.rutgers.edu/~kuhlthau/information_search_process.htm

ONLINE TUTORIAL & TOUR http://www.libraries.psu.edu/psul/speccolls/tutorial.html http://www.libraries.psu.edu/psul/speccolls/tour.html http://animoto.com/play/M2PuJCs1K062vokzVTvb0w?utm_content=main_link

MORE PICTURES TO EXPLAIN THINGS What do we mean by “collection”

Records of an organization

Personal papers

TAKING A CUE FROM E-COMMERCE?