Using screen-capture technology to understand health

information seeking behaviors and assess

e-health literacyDanielle CarlockFaculty Librarian

Scottsdale Community College, AZ, USAd.carlock@scottsdalecc.edu

OVERVIEW

• Institutional Profile

• Methodology

• Results

• Next Steps

SCOTTSDALE COMMUNITY COLLEGE INSTITUTIONAL PROFILE

• Public, two year college in Scottsdale, AZ, USA

• Approximately 10,000 students• Part of the Maricopa Community

College District, serving 250,000 students

• General education requirements for Associate degree include a Natural Sciences course

FON 241LL COURSE

• FON 241LL “Principles of Human Nutrition Laboratory” is a 1 credit course that fulfills the Natural Science requirement

• Taken mainly by non-health science majors

• Students evaluate their risk for cardiovascular disease, hypertension, type II diabetes, and obesity

eHEALTH LITERACY

“The ability to seek, find, understand, and appraise health

information from electronic sources and apply the knowledge gained to

addressing or solving a health problem.”

(Norman and Skinner, 2006)

INSTRUCTIONAL METHODOLOGY

PRETEST(AUGUST)

FLIPPED INSTRUCTION FOR

CONSUMER HEALTH(SEPTEMBER)

FLIPPED INSTRUCTION FOR SCHOLARY HEALTH

(SEPTEMBER)

3 LABS THAT REINFORCED THE

SKILLS(SEPT-NOV)

POSTTEST(DECEMBER)

PRETEST/POSTTEST METHODOLOGY

• Before and after instruction students were given two searching prompts:

1. A personal health information need2. An academic health information need

• Searches screen-captured using Panopto

• Searches scored against rubrics

SCHOLARLY INFORMATION PROMPT & RUBRIC

Prompt: Your nutrition instructor has assigned a research paper on the role of nutrition in type II

diabetes. Locate one scholarly source on the role of nutrition in type II diabetes.

Rubric (5 points): CRITERIA NO YES

ARTICLE IS SCHOLARLY 0 2

ARTICLE IS RESEARCH ARTICLE 0 2

ARTICLE IS RELEVANT 0 0.5

ARTICLE IS CURRENT 0 0.5

OVERALL RESULTS-SCHOLARLY PRETEST

MEASURE RESULTS (n=36)

Mean score 35%

Mean search duration

2.5 minutes

Mean # of sites viewed

0.6 sites

Areas of weakness Not knowing what a scholarly source is or how to find one

SEARCH SOURCES (Prior to instruction)

SEARCH SOURCE # OF STUDENTS

% OF STUDENTS

GOOGLE 19 53%LIBRARY WEB SCALE 7 19%GOOGLE SCHOLAR 3 8%LIBRARY DATABASE 2 5.5%LIBRARY RESEARCH GUIDE 2 5.5%

MULTIPLE SEARCH SOURCES 2 5.5%

CONSUMER HEALTH SITE 1 2.5%

TYPES OF SOURCES SELECTED (Prior to instruction)

TYPE OF SOURCE # OF STUDENTS % OF STUDENTS

CONSUMER SITE 16 44%REVIEW 7 19.5%POSITION PAPER 6 17%RESEARCH 4 11%ONLINE REFERENCE WORK

2 5.5%

NONE 1 3%

SEARCHING TRENDS (Prior to instruction)

• For about half, no distinction in search methods/sources for consumer/personal vs. scholarly searches

• Use of filters/limits, Boolean & synonyms virtually non-existent

• Search revisions did not necessarily lead to better outcomes

PRETEST/POSTTEST COMPARISON-SCHOLARLY

MEASURE PRETEST POSTTESTMean score 35% 72% **

Mean search duration 2 minutes, 30 seconds

2 minutes, 38 seconds

Mean # of sites viewed 0.6 1.5Areas of weakness Knowing what

scholarly sources are and where to find them

Relevancy & source distinctionUse of filters

**Pretest and posttest scores significantly different using a Wilcoxon Signed Rank Test, (V=35, p<0.001)

TYPES OF SEARCH SOURCES(Before and after instruction)

SOURCE % PRIOR TO INSTRUCTION

% AFTER INSTRUCTION

Google/internet search engine 53% 11%Google Scholar 8% 28%PubMed or PubMed Central 0% 20%Science Direct 0% 17%BioMed Central 0% 6%Library Web Scale 19% 3%Academic Search Premier 5.5% 0%Library Research Guide 5.5% 0%Consumer health website 2% 0%Multiple sources 5.5% 17%

SOURCES SELECTED (Before and after instruction)

TYPE OF SOURCE % OF STUDENTS BEFORE INSTRUCTION

% OF STUDENTS AFTER INSTRUCTION

Research article 11% 67%Review article 19.5% 11%Consumer health site 44% 8%Position paper 17% 5.5%Letter to the editor 0% 3%Book chapter 0% 3%Online reference work 5.5% 0%None 3% 3%

SEARCHING TRENDS (After instruction)

•Increase in the use of Boolean (from 8% to 53%)

•Increase in the use of filters (from 11% to 30%)

•Large number of multiple search sources & revisions suggests difficulties with recognizing topically relevant research articles

IMPLICATIONS FOR INSTRUCTION

• Introduce fewer databases and emphasize filters/limits

• More practice exercises on topical relevancy and source type

TAKE AWAYS

• Students must be explicitly taught what scholarly sources are and how to find them

• If there is not enough time for practice and checking for understanding instruction may lead to confusion, over-searching and reversion to Google

• One shots: less is more

NEXT STEPS

• Repeating the study with approximately 100 students this semester

• Interested in finding others that want to replicate the study

WORK CITED

Norman CD, Skinner HA. 2006. eHealth Literacy: Essential Skills for Consumer Health in a Networked World. Journal of Medical Internet Research:8(2):e9.

ACKNOWLEDGEMENTS

Thank you to:• Maricopa Center for Learning and Instruction

which funded the fellowship

• Robert H. Martin, SCC Nutrition faculty, for his collaboration on the project

• SCC Library Division faculty for feedback on the presentation