Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American...
-
Upload
morgan-dean -
Category
Documents
-
view
220 -
download
2
Transcript of Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American...
![Page 1: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/1.jpg)
Magia G. KrausePh.D. Candidate
School of InformationUniversity of Michigan
Society of American Archivists Annual MeetingAugust 14, 2009
Undergraduates in the Archives:
Measuring the Impact of Archival Instruction Through Rubric-based Assessment
![Page 2: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/2.jpg)
Instruction and Assessment in Academic Archives
Krause, Magia G.(2008). Learning in the archives: A report on instructional practices, Journal of Archival Organization, 6(4) 233-268.
![Page 3: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/3.jpg)
Overview of Teaching ExperimentResearch Questions
Do undergraduate students learn from archival instruction?
What do undergraduate students learn from archival instruction?
Goals of studyIntroduction of a reliable assessment tool to
measure student learning Introduction of archival literacy scale that
might compliment a curriculum of undergraduate research education in archives
![Page 4: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/4.jpg)
MethodologyQuasi-experiment in the fieldBackground
US history course in a large state university82 students total
Control and treatment groupsTo compare students that did not receive
archival instruction with students that didPre- and Post- Document Analysis Exercise
To provide a baseline comparison of the groups and look for an improvement in treatment group
![Page 5: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/5.jpg)
Archival InstructionDescription of treatment (aka archival
instruction)Lecture/presentation to entire class by
archivist Hour-long session in the archives with 4
stationsBibliographic InstructionCritical ThinkingPhotograph AnalysisFootnote/Citation Analysis
![Page 6: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/6.jpg)
Photographs from the Instruction Session
![Page 7: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/7.jpg)
Document Analysis ExerciseDivided into 3 sections
Adapted from NARA’s Teaching with DocumentsWritten Document AnalysisPhotograph AnalysisFinding Aid Analysis
Difference between Pre- and Post- TestsAlternative primary sources and finding aid
![Page 8: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/8.jpg)
Assessing Student LearningRubric
Basic definitionScoring guide that includes criteria and levels of
performanceTypes of rubrics
Holistic and analyticSteps in creating a rubric
Criteria : Archival Literacy SkillsObservationInterpretation/Historical ContextEvaluation/Critical ThinkingResearch Skills
![Page 9: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/9.jpg)
Rubric for this study
![Page 10: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/10.jpg)
Using the Rubric3 raters
Two professional archivists with at least five years of experience teaching undergraduates and me
Important for increasing the reliability of the scores
Achieving kappa
Final resultsOur Fleiss Kappa for the pretest was .80218 and .788 for
the post-test, both of which are considered excellent strengths of agreement.
![Page 11: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/11.jpg)
Statistical TestsNon-parametric tests
Useful for analyzing ordinal and nominal dataGreat for small samplesTests used:
Mann Whitney U Test for Independent SamplesResults
The tests confirmed that the null hypothesis could be rejected with 95% confidence
Students in the treatment group did better on the post-test after participating in archival instruction. In fact, there is a statistically significant difference between the two groups’ scores for each of the criteria.
![Page 12: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/12.jpg)
![Page 13: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/13.jpg)
![Page 14: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/14.jpg)
DiscussionThe Research Skills Problem
Disappointing resultsPossible factors
Students not actively engaged during instructionFollow-up assignment involved pre-selected
documents – no chance to practice searching for and identifying documents
Literature on undergraduate information-seeking behavior
![Page 15: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/15.jpg)
Archival Literacy ScaleElements
4 categories of the rubric:ObservationInterpretationEvaluationResearch Skills
Testing for reliabilityResults
![Page 16: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/16.jpg)
ConclusionsLimitations to the generalizability of results
– Need to replicate the study to increase the
reliability of the instrumentsWish for better instructional design for
instruction componentNeed for more communication and
collaboration between archivists to develop and share curricula and standardized tools to assist in instruction and assessment
![Page 17: Magia G. Krause Ph.D. Candidate School of Information University of Michigan Society of American Archivists Annual Meeting August 14, 2009 Undergraduates.](https://reader036.fdocuments.us/reader036/viewer/2022081513/56649cf15503460f949c0c86/html5/thumbnails/17.jpg)
Thank YouFor more information about the study
described in this presentation, please contact: Magia Krause [email protected].