1. 1. Measure for Measure Using collaborative assessment to build stronger information literacy skills PRESENTED BY: Susan Gardner Archambault
2. 2. Private Jesuit and Marymount University in Los Angeles, CA 6,064 Undergraduates 2,189 Graduates Library Open 24/5 New Core Curriculum in Fall of 2013 LMU
3. 3. Info. Literacy Learning Outcomes
4. 4. Info lit flag Core Structure
5. 5. Rhetorical Arts Course Criteria Assign at least 10% of the final course grade on the basis of information literacy, with a librarian-led workshop and one or more course-integrated assignments INFORMATION LITERACY OUTCOMES: 1. Conceptualize an effective research strategy and then collect, interpret, evaluate and cite evidence in written and oral communication 2. Distinguish between types of information resources and how these resources meet the needs of different levels of scholarship and different academic disciplines
6. 6. Spring 2014 1272 first-year students had a required library visit Across 72 sections of Rhetorical Arts
7. 7. 3 Info Lit Course Assignments in Common Syllabus
8. 8. Library Visit Agenda 1. Intro to LibGuide 2. Why use the library? 3. Evaluating sources (RADAR game) 4. Finding sources (demo)
9. 9. Library Support Material
10. 10. Rhetorical Arts 1000 LibGuide
11. 11. RADAR Game Students were paired up and given a source to evaluate Using the resources in the course LibGuide the student pairs answered questions about the source they were given Discussion with the librarian after completing the RADAR Game
12. 12. Gamification Incorporating Friendly Competition (points) Motivational Feedback (stars) Measure Progress (showing star count) Reward effort (getting the answer right on the first try earns more points)
13. 13. Research Strategies Tutorial (Recommended Homework)
14. 14. Research Questions How effective was the face-to-face instruction? How effective were the course-integrated assignments? How effective was the library support material?
15. 15. Methodology
16. 16. Direct Measures Overall Averages: Student scores across 100 sampled annotated bibliographies. Scored with a calibrated rubric by a group of volunteer R.A. instructors
17. 17. Indirect Measures Survey: Stratified random sample of 300 students (48% response rate) All instructors teaching a section (57% response rate)
18. 18. Comments Survey comments were analyzed using NVivo and coded
19. 19. Major Findings & Applications
20. 20. High-Scoring Areas: o citing sources correctly o identifying the main purpose of a source Low-Scoring Areas: o identifying intended audience o authority of the author o accuracy of the evidence o bias Annotated Bibliography
21. 21. 3.54 4 0 1 2 3 4 5 Helpfulness/Evaluating Evidence Students Instructors Annotated Bibliography
22. 22. Comment No. of Times Not helpful for writing/research process 7 Too many steps/make simpler 3 Annotated Bibliography/Comments
23. 23. Comment No. of Times Fun/Engaging 21 Helpful 15 Competition 8 Interactive 7 Liked partner work 6 Not fun/engaging 6 Too easy 6 Not useful 5 RADAR Game/Student Comments
24. 24. Students who listed a major in the College of Science & Engineering liked the RADAR game more than the average student Statistically Significant
25. 25. Annotated Bibliography Revised
26. 26. Added Hints Added More Discussion RADAR Game Revised
27. 27. Pre-Search Worksheet
28. 28. 56%22% 22% Instructors (Mean = 3.44 / 5) Agree Disagree Neutral 51%31% 18% Students (Mean = 3.17 / 5) Agree Disagree Neutral Helpfulness in Developing a Topic
29. 29. Comment No. of Times Too easy/not complex enough 17 Did not integrate with or apply to other projects/assignments 5 Add working bibliography of sources 4 Layout needs more space 4 Add reflective questions re: terminology & inclusion/exclusion choices 3 Add citation tracking element 1 Have instructor approve the topic 1 Allow space to change positions 1 ID stakeholder and their position 1 Pre-Search Worksheet Student/Faculty Comments
30. 30. Research Diary
31. 31. 3.29 3.71 3.33 3.5 2.77 2.58 0 1 2 3 4 5 Reflecting on search process Helped collect info Grading rubric/significant aspects of learning Grading rubric/clear criteria Student Instructor Research Diary
32. 32. Comment No. of Times Busy work/not helpful 24 Add section on relevancy of sources/source reflection/how to use/more source interaction 6 Timing/better integration with other assignments (earlier in semester) 2 Research Diary Student/Faculty Comments
33. 33. Pre-Search Worksheet Revised
34. 34. Continued
35. 35. Library Visits/Library Instruction
36. 36. 75% 17% 8% Instructors (Mean = 3.92 / 5) Valuable Somewhat Not Very Valuable 46% 30% 24% Students (Mean = 3.22 / 5) Valuable Somewhat Not Very Valuable How Valuable Are Library Visits?
37. 37. 4.3 4.32 4 3.88 3.57 3.57 3.55 3.67 3.8 3.8 3.41 3.58 3.5 3.53 0 1 2 3 4 5 Scholarly vs. Popular Getting Help from Librarian Finding Info Finding a Topic Evaluating Appropriateness/ Info Citing Sources Using Evidence/Make Arguments Student Instructor How Helpful Are Library Visits?
38. 38. Comment No. of Times LibGuide 30 Services Offered by Librarians 29 Research Databases 24 Research Strategies 16 Evaluating Sources 13 What Did Students Find Most Helpful?
39. 39. Comment No. of Times Fine no suggestions for improvement 50 Research databases/keyword searching 16 Finding scholarly sources 13 Repetition of FYS or other course 10 Too easy 10 Better relevance or integration with real assignment 8 Citing 8 Navigating library web site or LibGuide 8 What Needs Improvement? Student/Faculty Comments
40. 40. Revised: Finding Sources Activity (Team-based Active Learning)
41. 41. LibGuide Student Ranking
42. 42. Comment No. of Times Couldnt find/unaware of 14 Too complicated/confusing/too many links 9 LibGuide: Student/Instructor Comments
43. 43. Research Strategies Tutorial
44. 44. 29% 50% 21% Instructors Who Required Tutorial Yes No Not Sure 55% 45% Students Who Completed Tutorial Yes No Research Strategies Tutorial: Usage
45. 45. Student Ranking of Tutorial 3.71 out of 5
46. 46. Revised: Tutorial as Mandatory Homework Assignment
47. 47. Encourage better integration of library visit and info lit assignments by assigning the same research topic throughout multiple assignments Additional Improvement
48. 48. Thank You: William H. Hannon Library Research Incentive Travel Grant Susan Gardner Archambault susan.archambault@lmu.edu @susanarcham