Dodd - The information literacy impact factor: how to measure value
-
Upload
il-group-cilip-information-literacy-group -
Category
Documents
-
view
517 -
download
1
Transcript of Dodd - The information literacy impact factor: how to measure value
![Page 1: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/1.jpg)
Lorna Dodd
College Liaison LibrarianUCD Library | Dublin 4. email: [email protected] Tel: +353 (0)1 716 7074 www.ucd.ie/library
The Information Literacy Impact factor: How to Measure Value
![Page 2: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/2.jpg)
Outline
Why conduct a survey of module coordinators?
Review of the Literature
Rationale
How the survey was conducted
Survey results
Implications for the future
Conclusions
Lorna Dodd: LILAC 2013
![Page 3: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/3.jpg)
Why?
Lorna Dodd: LILAC 2013
![Page 4: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/4.jpg)
Why?Looking Back
& Looking Forward
Lorna Dodd: LILAC 2013
![Page 5: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/5.jpg)
Looking Back
Large University
No single strategy for the development of Information Literacy
Heavy load of information skills delivery
Driven by individuals
Sometimes evaluated but rarely measured
No clear indication in meeting learning outcomes(Dodd, L. & Kendlin, V., 2010)
Lorna Dodd: LILAC 2013
![Page 6: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/6.jpg)
Looking Forward…
New Library Strategic Plan 2010-2014
“monitor, measure and evaluate the Library’s teaching and learning strategy and
activities.” (UCD Library, 2010)
Lorna Dodd: LILAC 2013
![Page 7: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/7.jpg)
Looking forward
Realignment of staffing structure From approx 17 Liaison Librarians to 6 College
Liaison Librarians
Move from module to programme approach
Move from responsive to consultation
Measure impact on student learning
Lorna Dodd: LILAC 2013
![Page 8: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/8.jpg)
Looking Forward…
Lorna Dodd: LILAC 2013
![Page 9: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/9.jpg)
Literature Review (1)
A lot available discussing why it is important to measure impact
Much of what has been written focuses on Library use rather than instruction.
(Stone 2011; Poll & Payne 2006; Schilling & Applegate 2012).
Limitations of traditional organisational approaches at measuring success and meeting targets.
Most libraries focus on process and output indicators rather than measuring impact.
Measuring impact on student learning requires systematic evaluation of training
(Markless & Streatfield 2006; Schilling & Applegate 2012))
Lorna Dodd: LILAC 2013
![Page 10: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/10.jpg)
Literature Review (2) Several problems arise when measuring impact.
Data protection rules Differing value of services for specific groups Difficulty in measuring long-term impact Time consuming nature of measuring impact of Library
instruction.
Important to use results of any study conducted to measure the impact of Library services Benchmarking activities Improving services Justifying resources used for services Campaigning for increased funding.
Common/Successful ways to measure impact of Library instruction Surveys; Pre and post-tests; Self-assessments; Behavioural observations.
(Poll & Payne 2006)
Lorna Dodd: LILAC 2013
![Page 11: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/11.jpg)
Literature Review (3)
2006 LIR/SCONUL measuring impact initiativeDeveloped impact process
Identified key performance indicators
Used to evaluate the effectiveness of interventions supporting teaching and learning
Research provided information regarding emerging issues throughout the process.
Results included: Improved relationships with academic community A raised profile for Library staff Better understanding of how the Library can support
academic programmes.
(Markless & Streatfield 2006; Blagden, 2005)Lorna Dodd: LILAC 2013
![Page 12: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/12.jpg)
Literature Review (4)
2012 CONUL Information Literacy SurveyGathered feedback on the impact of information literacy instruction
Found library instruction had a positive impact on students’ skill development
Information Literacy often assessed
Targeted respondents
Academics already fully engaged
First year students only
Small sample from each institution
(CONUL Advisory Committee on Information Literacy, 2013)
Lorna Dodd: LILAC 2013
![Page 13: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/13.jpg)
We needed to:
Make academics aware of new approach
Create a strategy to get academics to embrace new approach
Identify what has been successful to date
Collect feedback from those not actively engaged with the Library
Identify what methods would be useful going forward
Problems:
Measuring impact retrospectively
Perceived impact is subjective
External factors influence development of information skills
More likely to opt for options that are familiar
Rationale
Lorna Dodd: LILAC 2013
![Page 14: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/14.jpg)
Survey
Seeking information on modules
Two parts:1. Evaluating information skills delivery to date2. Identifying what would be useful in the future
Survey Monkey 11 questions Mainly quantitative Multiple choice questions Measurement/ranking questions
Lorna Dodd: LILAC 2013
![Page 15: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/15.jpg)
Survey
Introduction:
Level of module – undergraduate/taught postgraduate
If there has been Library instruction
Part 1:
Kind of instruction
Impact of instruction on students’ abilities
Part 2:
Should all students have library instruction
When should library instruction happen?
Which services should be developed
Free Question
Lorna Dodd: LILAC 2013
![Page 16: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/16.jpg)
Promotion
Approx. 700 module coordinators
Only possible to target academic staff
Not all module coordinators are academic staff and not all academic staff are module coordinators
Targeting email – approx 1,900
College Liaison Librarian contacts
Three weeks
Lorna Dodd: LILAC 2013
![Page 17: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/17.jpg)
Survey Results
Type of instruction
Impact of instruction
When and who?
Preferred services for the future
Qualitative feedbackLorna Dodd: LILAC 2013
![Page 18: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/18.jpg)
![Page 19: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/19.jpg)
Type of Library Instruction
Lorna Dodd: LILAC 2013
![Page 20: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/20.jpg)
![Page 21: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/21.jpg)
![Page 22: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/22.jpg)
Impact of Library
Instruction
Lorna Dodd: LILAC 2013
![Page 23: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/23.jpg)
![Page 24: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/24.jpg)
Lorna Dodd: LILAC 2013
![Page 25: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/25.jpg)
Lorna Dodd: LILAC 2013
![Page 26: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/26.jpg)
Lorna Dodd: LILAC 2013
![Page 27: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/27.jpg)
Lorna Dodd: LILAC 2013
![Page 28: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/28.jpg)
Lorna Dodd: LILAC 2013
![Page 29: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/29.jpg)
Lorna Dodd: LILAC 2013
Mostly Taught Postgraduate: EndNote Special Collections PBL One-on-One Consultation
![Page 30: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/30.jpg)
“Students have always received instruction to date
and so difficult to answer the comparison questions below”
Lorna Dodd: LILAC 2013
![Page 31: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/31.jpg)
Lorna Dodd: LILAC 2013
Should ALL students have Library Instruction?
![Page 32: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/32.jpg)
Lorna Dodd: LILAC 2013
Should ALL students have Library Instruction?
![Page 33: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/33.jpg)
Lorna Dodd: LILAC 2013
Should Library Instruction happen in every year?
![Page 34: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/34.jpg)
Lorna Dodd: LILAC 2013
Should Library Instruction happen in every year?
![Page 35: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/35.jpg)
Lorna Dodd: LILAC 2013
Which Stage?
![Page 36: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/36.jpg)
“Instruction in Year one should be built on in subsequent years
to enhance skills by year”
Lorna Dodd: LILAC 2013
![Page 37: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/37.jpg)
Preferred Services for the
FutureLorna Dodd: LILAC 2013
![Page 38: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/38.jpg)
Lorna Dodd: LILAC 2013
![Page 39: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/39.jpg)
Lorna Dodd: LILAC 2013
![Page 40: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/40.jpg)
Lorna Dodd: LILAC 2013
Influence of instruction had on preference?
Instruction had
First Choice Second Choice Last Choice
Tour Subject Specific Workshops
Generic workshopTutorialOnline Video
Tour
Lecture Subject Specific Workshops
Generic WorkshopTutorial
Tour/Online Video
Workshop Subject Specific Workshops
Generic WorkshopTutorial
Tour
No Instruction Subject Specific Workshops
Subject Specific Workshops
Tour
![Page 41: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/41.jpg)
Lorna Dodd: LILAC 2013
![Page 42: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/42.jpg)
Lorna Dodd: LILAC 2013
![Page 43: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/43.jpg)
Lorna Dodd: LILAC 2013
![Page 44: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/44.jpg)
“One of the best approaches would be to create materials that are reusable and for any student, regardless of level.
Just because a student moves up a level doesn't mean they've acquired all of the
learning they should have”
“…really good online video materials, from complete intro to specific and
advanced, and available on demand is the way to go”.
“Tutorials (tutors would be trained by Library staff)”
Lorna Dodd: LILAC 2013
![Page 45: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/45.jpg)
Qualitative Feedback
Lorna Dodd: LILAC 2013
![Page 46: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/46.jpg)
Comments Themes
20% (53) of those who completed the survey made comments
In line with our new strategy
Online options
Constructive comments & suggestions on current and future services
Complimenting services to date
Comments on the survey
Lorna Dodd: LILAC 2013
![Page 47: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/47.jpg)
Comments
5 respondents made comments inline with our new strategy Programme Approach Reusable Learning object and Online tools
6 respondents made comments about proposed future instruction
Lorna Dodd: LILAC 2013
![Page 48: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/48.jpg)
Comments 12 respondents made comments about current
services and changes
Problems with current service:
“…same 'introductory' talks by librarian using lecture slots of various modules for
at least three times (the third time he skipped as he found it ridiculous)
“Students just skip the guest lecture.”
Lorna Dodd: LILAC 2013
![Page 49: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/49.jpg)
Comments
Benefits of current service:
“…requires personal contact with a friendly face early on.
“The current reduction in library services is a real retrograde step especially with the explosion in available information
sources”
Lorna Dodd: LILAC 2013
![Page 50: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/50.jpg)
Comments
Over half (28) complimented current strategy
“we really love our librarians; thank you”
‘Campaigning’ for traditional serviceAcknowledging individualsRecognising value of information literacy to students
Lorna Dodd: LILAC 2013
![Page 51: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/51.jpg)
“…it makes a noticeable difference in the quality of assignment students produce. The Library Tutorial is typically rated as one of the most important lectures they have in that Stage 1 module.”
Lorna Dodd: LILAC 2013
![Page 52: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/52.jpg)
Implications for the Future
Lorna Dodd: LILAC 2013
![Page 53: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/53.jpg)
Impact of Instruction
Definite benefit in Library instruction
Active learning environment is preferred
Stronger evidence in the discovery and use of resources
Remains difficult to identify the ‘real’ impact
Lorna Dodd: LILAC 2013
![Page 54: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/54.jpg)
Timing of Instruction
Most module coordinators feel all students should have Library instruction
Instruction for students at every stage is desirable
Taught Postgraduates Beginning and middle of Semester 1
Undergraduates Stage 1 Stage 3
Lorna Dodd: LILAC 2013
![Page 55: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/55.jpg)
Preferred Services Active learning sessions tailored to subject needs
most preferred
Tours perceived as ineffective
Questions:
Asked to identify how effective instruction was at improving students’ information literacy skills. When they request instruction, do they usually
consider this? Or do they feel students need to know about the
library rather than what they need in order to transition from 2nd level, develop researching skills etc.
Lorna Dodd: LILAC 2013
![Page 56: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/56.jpg)
Questions
Did respondents choose workshop environment over online version because they are more familiar?
If so, then why did the majority of those who had no instruction choose workshop?
If delivering workshops on this scale is unsustainable, what can we do ensure effectiveness?
What work do we need to do in terms of marketing?
Lorna Dodd: LILAC 2013
![Page 57: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/57.jpg)
Conclusion
Shared understanding between Library & Academic in:
Benefit of Instruction Taking a programme approach Ensuring all students get an equal
opportunity (core modules)
Difference in: Most appropriate method of delivery
Lorna Dodd: LILAC 2013
![Page 58: Dodd - The information literacy impact factor: how to measure value](https://reader036.fdocuments.us/reader036/viewer/2022062513/555dd1e4d8b42ab56b8b5340/html5/thumbnails/58.jpg)
References
Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1 impact projects’, Library & Information Research (LIR) vol. 29 (91)
Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to address sustainability in the delivery of information literacy components in UCD’: Librarians' Information Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland.
Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UK university libraries on student learning and research: A facilitated action research approach’, Journal of Information Management vol. 26 pp. 3-15
Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’, Library Hi Tech vol. 24 (4) pp. 547-562
Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a comparison of the efficacy of commonly used measures in library instruction’, Journal of the Medical Library Association, vol. 100 (4) pp. 258-269
Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary report on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22
UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’
http://www.ucd.ie/library/news_publicity/showcase_strategy/
Lorna Dodd: LILAC 2013