Culture of Assessment NCLB Outcomes-based assessment Accountability.
-
Upload
caiden-doherty -
Category
Documents
-
view
219 -
download
3
Transcript of Culture of Assessment NCLB Outcomes-based assessment Accountability.
Culture of Assessment
NCLB
Outcomes-based assessment
Accountability
Pressure to Assess
At UNO--MyMapp
Programmatic assessment
ACRL training
Model for real-life library assessment
Advantage of developing model before committing resources
Feedback from ACRL instructors
http://www.ala.org/ala/acrl/acrlproftools/assessingstudent.cfm
Ask yourself a few questions Need to Determine:
ScopeMeasurement instrumentPopulationTimeframeDelivery MethodSustainability
What do you want to measure? Determine type of instrument
Pre and Posttest to measure ENGL116 Composition II students’ information literacy skills
Pool of questionsVariety of questions—multiple answer, drag
and drop and fill in the blank
Who can help?
Institutional assistance Faculty with expertise in assessment English faculty to assist in question
development and review Graduate students to assist with data
analysis Other librarians with assessment
experience
Consultants
Use consultants to:
Define your scopeSuggest testing instrument modelsTest your testHelp navigate IRB processReview results and draw conclusions
Delivery
Online vs. Paper Course management software
Blackboard○ Campus IT support○ Familiar○ Convenient○ Accessible 24-7
Time
Planning for assessment—our timeline Timeline for launching Timeline for measuring
Longitudinal?One time measure?
What can you sustain?
Money
Do you need it? What it did for us
IncentiveStipend for consultantsStudent employee wagesData analysis
Staff
How to create buy-in with:Library staffLibrariansEnglish professorsAdjunctsCampus stakeholders
Highlight benefits to all stakeholders
Curriculum
Build on what you have in place Keep it simple to start Be consistent with learning objectives Reinforce key concepts Share decision-making with English or
other faculty stakeholders
Promotion
English Department faculty meetings Library Instruction sessions Library Newsletter—website Continuous need for promotion every
semester until it becomes routine
Benefits of assessment in library Working with faculty consultants
campus-wide opened our eyes to the ways in which different units measure learning
Engaging in larger assessment conversation developing on campus—future committee work opportunities
Theory
Tests
Consistent
Not Consistent
Modify Hypothesis
MeasurementsPre and post questionnairesStudent surveysBlackboard organizationControlled group of librarians, English
instructors, and class sections
QuestionsCollaboration with English departmentPrioritized outcomes with ACRL IL StandardsUsed challenging, performance-based questions
Background and Online Assessment ToolsWhy did we select Blackboard?
Saving time and money with little trainingBlackboard – The Foundation
Sample Size15 sections of English 1160/1164184 students completed both questionnaires
Pre-questionnaire mean = 8.23 out of 15 Post-questionnaire mean = 9.89 out of 15 Statistically significant (p < .001)
Findings from Initial Semester
What did you find most useful in the library instruction?A compleat waste of my time and tuition.Learned what some stuff didAll of the engines that we could use to find things,
and how to use them.The library instructor only showed us a couple,
but after being shown how to use a few, I went through and figured out how to use others without being intimidated by all of the information.
Everything..I woul not have found my sources if it wasnt for the library instruction
Changes implemented after initial semesterAdvice offered from faculty consultants, library
instructors, and English faculty:
147 completed the pre and post questionnaires8 scores rejected as “outliers”139 scores used
Pre-questionnaire mean score = 16.10 Post-questionnaire mean score = 15.61 Results were NOT statistically significant
Results from Second Semester
Second Semester – Survey Questions
Second Semester – Survey Questions
Second Semester – Survey Questions
Correlation between Score Difference and Number of Days Between Tests– Second Semester
Correlations
1 -.081
.344
139 139
-.081 1
.344
139 139
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
difference
Number of Days
differenceNumberof Days
Factors affecting success
Rewards Focused and clarified the purposes of the
information literacy sessions Prioritized and used the most relevant
standards and outcomes that fit our program and institution
Continued to build relationships with the English department faculty
Future assessment ideas
Verbal demonstrations Keyboard analysis Surveys—in depth Longitudinal study of information literacy
skills Citation analysis—MyMapp e-portfolio
Practical Tips
Thank You
Questions? Discussion