Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer...
-
Upload
kane-fielding -
Category
Documents
-
view
217 -
download
0
Transcript of Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer...
![Page 1: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/1.jpg)
Assessment for Impact:
Turning Data into Tangible Results
Laura MillerTim MortonPaul Rittelmeyer
Library Assessment ConferenceSeattle, WA
August 5, 2008
![Page 2: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/2.jpg)
• Balanced Scorecard
• LibQUAL+
• Surveys• Users • Staff
• Focus Groups
• Follow-up Interviews
• Statistics (circ, ILL, gate counts, purchase requests,
etc.)
Where Does Data Come From?
![Page 3: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/3.jpg)
• Improve Library Services
• Inform Collections Decisions
• Influence Satisfaction in the
Workplace
• Support Budget Requests
• Contribute to Staff Development
What Does Data Do?
![Page 4: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/4.jpg)
Metric U.4.b. Turnaround time for new book and ILL requests (external resources).
Target1: Satisfy the turnaround targets 75% of the time.Target2: Satisfy the turnaround targets 50% of the time.
Method: New Books: Fill requests in 7 days. A sample of in-print US titles requested by University-affiliated patrons will be drawn from the request database by Management Information Services. Turnaround time from patron request to availability will be measured.ILL: Fill requests in 7 days. Turnaround time will be measured by ILS from patron request to notification that the book or article is available.
Improve Library ServiceRush Order (aka Purchase Express)
![Page 5: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/5.jpg)
Improve Library Service
Acquisitions(order to receipt)
2-4 days
AUL
OK
![Page 6: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/6.jpg)
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
AUL
OK OK
![Page 7: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/7.jpg)
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
AUL
OK OK
Balanced Scorecard
![Page 8: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/8.jpg)
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
Patron request/selector review/placing order
?
AUL
OK OK
Balanced Scorecard
![Page 9: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/9.jpg)
Improve Library Service
Acquisitions(order to receipt)
2-4 days
Cataloging(receipt to shelf prep to
notifying patron)1 day
Patron request/selector review/placing order
?
AUL
OK OK
Balanced Scorecard• Selector training• Back ups for email/order placement
• Funding for expedited shipping
![Page 10: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/10.jpg)
Inform Collections DecisionsLibQUAL+ and satisfaction with journals
![Page 11: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/11.jpg)
Inform Collections DecisionsLibQUAL+ and satisfaction with journals
LibQUAL+ 2006UVa Faculty and Graduate Student Ratings of Journal Collections
5.00
6.00
7.00
8.00
9.00
ArchitectureFaculty
EducationFaculty
EngineeringFaculty
HumanitiesFaculty
Science/MathFaculty
SocialScienceFaculty
ArchitectureGrads
EducationGrads
EngineeringGrads
HumanitiesGrads
Science/MathGrads
SocialScienceGrads
![Page 12: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/12.jpg)
• Focused on low-scoring areas
• Approached a diverse group of faculty
• Asked for specific needs, wants and
suggestions
• Kept the interviews brief
Targeted InterviewsFollowing up with faculty
Inform Collections Decisions
![Page 13: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/13.jpg)
• Dissatisfaction was not caused by single factor – no ‘smoking
gun’
• Perceived access was a problem – in one area we already
subscribed
to all but one of the requested journals
• Lack of foreign titles
• Missing or incomplete backfiles
What We Learned?
Inform Collections Decisions
![Page 14: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/14.jpg)
• To address access, we instituted a major, ongoing re-design of our
website
• We invested in a electronic resource management system (Serials
Solutions 360)
• We prioritized the purchase of science backfiles (Wiley, Elsevier) and
new publisher packages
• In the arts, we put significant funds into the purchase of quality
images, or, if they could not be bought, we digitized them ourselves
The Results
Inform Collections Decisions
![Page 15: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/15.jpg)
Influence Staff Satisfaction
Library Worklife Survey• Conducted in even-numbered years
• Available to all staff via online form with email reminders
• Response Rates: 2004 – 60%, 2006 – 52.2%
Two-Step Analysis/Response Process
1.Online survey• Concerns identified
2.Focus groups, one-on-one meetings• What can be done to address concerns?
![Page 16: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/16.jpg)
Influence Staff Satisfaction
Problems Identified Solutions Implemented
• Staff don’t know about other departments
• Open Houses for departments
• Staff don’t know what’s going on system-wide
• “Did You Know?”, Job/Position Announcements
• Administration not available to hear concerns
• Brown Bag Lunches, Open Door Policies
Impacts Seen 2006 2004 Change
• Library staff in other departments are familiar with what I do. 28.3% 17.3%
11.0%
• The Library administration is willing and available to listen to 52.3% 37.3%
15.0% my concerns, comments, and recommendations.
• The Library administration effectively communicates to staff. 36.9% 36.6%
0.3%
• Collaboration is encouraged and supported in my job. 69.0% 75.8% -
6.8%
• I get cooperation from other departments when we work together. 71.2% 74.0% -
2.8%
![Page 17: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/17.jpg)
Influence Staff Satisfaction
Problems Identified Solutions Implemented
• Need more staff and better distribution of work
• 24 new positions created
• Inadequate facilities, technology and support resources
• Library IT involved with allocations• Facilities staff increased• Balanced scorecard metric for facilities improvement
Impacts Seen 2006 2004 Change
• I have adequate resources and materials to complete assignments. 59.5% 49.2%
10.3%
• I feel safe in my work environment. 87.9% 81.5% 6.4%
• The Library is concerned about and addresses my ergonomic needs 67.0% 76.3% -
9.3%
• I am physically comfortable in my work environment (temperature, light, noise). 56.4% 44.6% 11.8%
![Page 18: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/18.jpg)
Influence Staff Satisfaction
Problems Identified Solutions Implemented
• Like Administration, HR not always available
• New HR staff hired• HR moves from Admin area into library
• Salaries are inadequate • Extra allocations towards salary adjustments
• No opportunity for growth • Better advertising of existing policies• Mentoring programImpacts Seen 2006 2004 Change
• My salary and benefits are reasonable for the work I do. 43.9% 21.5% 22.4%
• The salary I receive is equitable when compared to co-workers in similar grade levels or ranks. 49.0% 30.9% 18.1%
• My work at the Library helps me achieve my career goals. 55.4% 35.9% 19.5%
• There are opportunities for me to advance at the Library. 36.8% 28.1% 8.7%
• There are adequate staff development opportunities provided by the Library. 64.2% 57.3% 6.9%
![Page 19: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/19.jpg)
Influence Staff Satisfaction
The Overall Impact
2006 2004Change
• My work at the Library helps me achieve my career goals. 55.4% 35.9%
19.5%
• I am satisfied with my job. 63.2% 58.5%
4.7%
• Overall Job Satisfaction Category 65.5% 58.0% 7.5%
![Page 20: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/20.jpg)
Support Budget Requests
• 80% of Music faculty responded to the 2004 Faculty Survey. Of these respondents, 88% chose sound recordings as their highest priority, making it the highest priority item for the department.
• While every Music faculty respondent rated the resource “sound recordings” on a 1-5 scale, the category received one of the lower ratings for satisfaction, a 3.88.
• The rating for sound recordings among faculty as a whole was 3.89.
![Page 21: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/21.jpg)
Support Budget RequestsMusic Library Budget
![Page 22: Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5,](https://reader031.fdocuments.us/reader031/viewer/2022012922/5519bc43550346695e8b458c/html5/thumbnails/22.jpg)
• Raising awareness of traditional practices as assessment tools (ref counts, gate counts, circ data, ILL data)
• Need to review, revise, reassess our BSC metrics and other tools. Assessment is ongoing.
• Managers and all staff need to learn how to better use data and assessment results in daily operations.
• Continue to move assessment out of MIS only. Position MIS as consultant/coordinator or assessment done throughout the organization.
• Assessment is for everyone!
Challenges AheadPromoting a Culture of Assessment