Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea
-
Upload
chancellor-swanson -
Category
Documents
-
view
10 -
download
0
description
Transcript of Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea
![Page 1: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/1.jpg)
Mid-Atlantic Chapter of the Medical Library Association2002 Annual Meeting
Collecting User Preferences for Web-Based MEDLINE Training:
A Capitol Idea
Kelly Near
Andrea Horne
Karen Grandage
Claude Moore Health Sciences LibraryClaude Moore Health Sciences Library
![Page 2: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/2.jpg)
Overview
• Proliferation of electronic resources
• Busy health professionals and students
• Need for continual end-user training and support, available any time & any place
![Page 3: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/3.jpg)
Tutorial Study Objectives:
Compare user preferences
Gain experience with new technologies for delivering online training
Evaluate the educational effectiveness of each tutorial format
Evaluate the cost/benefit ratio for each tutorial format
![Page 4: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/4.jpg)
Background
• Advancements in technologyWeb frames
Streaming audio & video
Flash
Java Script Applets
![Page 5: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/5.jpg)
Background –MEDLINE Training
• Still our “bread and butter” resource
• Medical and Nursing School curriculum training sessions
• Monthly classes
• Individual consultations
![Page 6: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/6.jpg)
Planning
• Always allow more time than you think you’ll need!
• We began with the assumption that online tutorials were an important part of educational delivery
![Page 7: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/7.jpg)
Planning
• Tutorial formats were chosen based on - learning style theory
our skill & comfort level with the necessary technology
![Page 8: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/8.jpg)
Tutorial A
• Interactive
• Created in Web frames
• Search instructions in the left frame
• “Live” version of the MEDLINE database active in the right frame
Tutorial A
![Page 9: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/9.jpg)
Tutorial B
• Short animated screen capture illustrations of commonly performed database tasks
• Audio narrative
Tutorial B
![Page 10: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/10.jpg)
Tutorial C
• Classroom format
• Streaming video and audio
• Clips of an instructor interspersed with screen shots of the database
Tutorial C
![Page 11: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/11.jpg)
Study Design• Wanted our study participants to be a
cross-section of our usual MEDLINE class students
• 10 participants were randomly assigned to each tutorial
• We did not determine their MEDLINE searching abilities prior to the testing
• None of the tutorials were meant to be as detailed as an actual class
![Page 12: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/12.jpg)
• Debate: - should they answer specific search questions, such as “what is the symbol used to truncate in Ovid MEDLINE”
or- should they conduct a complete search on a given topic, and have it measured against a “gold standard”?
![Page 13: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/13.jpg)
Search Question
• Use MEDLINE
1996 – present
Perform a comprehensive search on the question:
Is there a connection between head injuries and depression in children?
![Page 14: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/14.jpg)
Six Steps to a Good Search
• Rated each search on a 6 point scale1 point for selecting the MeSH term
“craniocerebral trauma”
1 point for exploding it
1 point for selecting the MeSH term “depression”
1 point for NOT exploding it!
1 point for combining the terms with AND
1 point for applying the age limit
![Page 15: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/15.jpg)
Gold Standard
Explode !
![Page 16: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/16.jpg)
Developing Tutorial A
Key: =development time $=money spent
• Key features: HTML frames Ovid Jumpstarts
• Products used: HTML Assistant Notepad
• Cost = 0• Challenges:
Condensing a full class down to key learning points and brief descriptions
![Page 17: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/17.jpg)
Developing Tutorial B
• Key features: Short video &
audio clips with text
• Products used: SnagIt! Microphone and
headset
• Cost = $39.95 for single
user copy
$Key: =development time $=money spent
![Page 18: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/18.jpg)
Developing Tutorial B
• Challenges:Time required to write a concise script
Time for the multiple takes needed to do a clean audio track …
especially using non-union help!
Take 5!
![Page 19: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/19.jpg)
![Page 20: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/20.jpg)
Starting a Subject Search This example will demonstrate a search for articles about oral cancers in teenage boys who use chewing tobacco.Begin by separating your search topic into individual concepts.Type your first search concept into the box labeled Enter Keyword or Phrase. For our search, we begin by typing in the keywords "oral cancer".Be sure that the checkmark is in the box by Map Term to Subject Heading
Next TopicReturn to Tutorial Home
![Page 21: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/21.jpg)
Developing Tutorial C
• Key features: Used our classroom as
the “stage”
Hired a “professional actress” so fewer retakes were necessary
• Products used: Adobe Premiere
• Cost = $399
$$$$
Key: =development time $=money spent
![Page 22: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/22.jpg)
Developing Tutorial C
• Challenges:
iMovie and Final Cut Pro didn’t allow us to intersperse SnagIt video clips with instructor video
Next tried videotaping the screen in the classroom as the MEDLINE search was being done
Adobe Premiere allowed us to combine the video and the SnagIt clips into one product
![Page 23: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/23.jpg)
![Page 24: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/24.jpg)
Recruitment
• Sent email to medical students in all 4 years inviting them to participate in the study
• Posted recruitment flyers around the Library, Medical School and neighboring Research Buildings
• Word-of-mouth seemed to help
• So did the FREE lunch!
![Page 25: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/25.jpg)
Pre-Testing
• Pre-tested the process with 3 library staff
• Identified problems with inconsistent instructions
• Refined test questions and tutorials based on this input
![Page 26: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/26.jpg)
Testing
• Randomly assigned 10 participants to each tutorial
• The tutorials were taken in a library classroom
• After completing the tutorial, they were each given the same search to complete in Ovid MEDLINE
![Page 27: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/27.jpg)
Interviews
• Participants were asked12 questionsExamples
Have you ever taken a MEDLINE class before?
Did you have any technical problems with the tutorial?
Did you refer back to the tutorial to complete the steps for the test search?
![Page 28: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/28.jpg)
Free Lunch
• Testing over the middle of the day worked very well as the lure of a
FREE LUNCH
was irresistible to all!
![Page 29: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/29.jpg)
Test Scores
0
1
2
3
4
5
6
A (4.5) B (5.1) C (4.9)
A (4.5)B (5.1)C (4.9)
Number of questions
Number correct
![Page 30: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/30.jpg)
ANOVA Test Results
Sum of
Squares df
Mean Square F Sig.
Between Groups
Within Groups
Total
1.867
30.300
32.167
2
27
29
.933
1.122
.832 .446
P value needs to be below 5% - this is 45%
![Page 31: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/31.jpg)
Explode
0
2
4
6
8
10
Tutorial A(6/10)
Tutorial B(10/10)
Tutorial C(7/10)
Tutorial A (6/10)Tutorial B (10/10)Tutorial C (7/10)
Number
of
Participants
Number Exploding
![Page 32: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/32.jpg)
Chi Square Analysis
• Chi Square analysis included 3 separate tests
• Only 1 showed a significant difference
• No significant statistical differences were found overall
![Page 33: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/33.jpg)
Tutorial Preferences
• Overall, interview results showed positive acceptance for all three tutorial types
A B C
![Page 34: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/34.jpg)
Reported Learning Preferences
Online TutorialOnline TutorialClassClass
PrintedPrinted
22
4
3
![Page 35: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/35.jpg)
Lessons Learned …
• Only 8 users had not used MEDLINE before
• We looked at this data to determine if there was a difference in the scores between new and experienced MEDLINE users
• Did not find any difference between the 2 groups
![Page 36: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/36.jpg)
Lessons Learned …
• May need a larger sample size
• AVI files (from SnagIt!) are too large
• Try Snapz Pro (Macintosh only) for video captures
![Page 37: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/37.jpg)
One Caveat …
• Ten days before we were to start testing - we found we needed approval from the University’s Human Investigation!
![Page 38: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/38.jpg)
Conclusions
• Scores = no difference
• Next, we looked at production costs and time and found …
MAJOR DIFFERENCES
![Page 39: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/39.jpg)
…and the Winner is -- Tutorial A!
![Page 40: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/40.jpg)
Future Directions
![Page 41: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/41.jpg)
Tutorial A = the one to trust!
![Page 42: Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea](https://reader030.fdocuments.us/reader030/viewer/2022032607/56813043550346895d95e6e9/html5/thumbnails/42.jpg)
Questions ?
Collecting User Preferences for Web-Based MEDLINE Training:
A Capitol Idea
Kelly Near, Andrea Horne, Karen Grandage