Building a Culture of Library Assessment

BUILDING AND SUSTAINING A CULTURE OF ASSESSMENT IN

YOUR INSTRUCTION PROGRAM

MEREDITH FARKASPORTLAND STATE UNIVERSITY

http://www.flickr.com/photos/londonmatt/4648615038/

Sunday, January 26, 14



•Assessment is the norm•Doing it for the right reasons•Customer-service focused•Culture of learning, curiosity•Decisions based on data•New initiatives tied to performance

measures•Changes based on what is learned

WHAT IS A CULTURE OF ASSESSMENT?


•Be better teachers•Make better decisions•Demonstrate value to ourselves and others•Use data to advocate for the library•Use data to grow the IL program•Hold ourselves to the same standards as

any other academic department•Accreditation

WHY BUILD A CULTURE OF ASSESSMENT?


•Focused on doing what is needed for accreditation•Coercive and top-down•Little support •Coalition of the willing•Data is gathered, but rarely used •Not seen as an integral part of teaching

and learning•Often a lot of cynicism around assessment

ASSESSMENT AT MANY INSTITUTIONS


WHAT DOES IT TAKES TO BUILD A CULTURE OF

ASSESSMENT?


“For a scholarship of assessment to thrive, we must align faculty culture, institutional structures, and leadership for change. The importance of this point cannot be overstated. A meaningful assessment program is more than just a new activity to be undertaken, it is a change in how we think about what we do in higher education.”Don Haviland. (2009). “Leading Assessment: From Faculty Reluctance to Faculty Engagement.” Academic Leadership 9(2).


•Meredith Farkas, Lisa Janicke Hinchliffe (UIUC), & Amy Harris Houk (UNCG)•Survey of libraries at BA/MA/PhD.

institutions in U.S.•n=1604•Contacted the library director•Asked him/her to refer survey to leader

of instructional assessment program•Response rate = 42% (n=672)

A SURVEY OF LIBRARIES, NOT LIBRARIANS


•Questions about institutional characteristics•Inquiry about presence/absence of 16

facilitating factors named in other publications•does the presence/absence facilitate or

hinder culture of assessment•Open-ended questions about what they

think most helps (or hinders) them in building an assessment culture

SURVEY ITSELF


FACILITATING FACTORS1.Campus-‐wide assessment

initiative2.Library involvement in campus-‐

wide assessment initiative3.Clear expectations for

assessment in the library4.Assessment is a priority of

library administration 5.Library has adopted learning

outcomes6.Library has assessment plan7.Library has shared

understanding of the purpose of assessment

8.Library staff/faculty are supported in doing assessment

9.Education and training about assessment are supported

10.Library has assessment expertise in-‐house

11.Library has systems/technologies to support assessment

12.Library leadership offers explicit support to get staff doing assessment

13.Data is available to interested parties

14.Leadership uses assessment data in decision-‐making

15.Librarians use results to improve practice

16.Library culture is user-‐focused


RESPONSES BY CARNEGIE CLASS

Total = 672 (42%)

BA = 234 (36%)MA = 262 (39%)PhD = 155 (54%)

Other = 18


WHO COMPLETED THE SURVEY?

Most "others" were public service librarians and administrators


WOULD YOU DESCRIBE YOUR LIBRARY AS

HAVING A CULTURE OF ASSESSMENT?


WHICH INSTITUTIONAL CHARACTERISTICS CORRELATE WITH A

CULTURE OF ASSESSMENT?


REGIONAL ACCREDITING BODY


HAVING A CAMPUS-WIDE ASSESSMENT INITIATIVE


Correlates:• Clear expectations for assessment (6%)*• Library having an assessment plan (6%)• Shared understanding of the purpose of assessment

(9%)• Library staff feeling supported (10%)• Library having a culture of assessment (10%)

*% of those without a campus-wide assessment initiative who reported the presence of these other factors

WITHOUT A CAMPUS-WIDE ASSESSMENT PUSH


HAVING PEOPLE DEDICATED TO INSTRUCTION OR ASSESSMENT


HAVING COMMITTEES DEDICATED TO INSTRUCTION OR

ASSESSMENT


•Carnegie classification (BA/MA/PhD)•Public/private•Tenure/no tenure •Faculty status/not faculty

WHICH DIDN’T CORRELATE?


WHICH FACILITATING FACTORS CORRELATE

WITH HAVING A CULTURE OF

ASSESSMENT?


ALL OF THEM.


• 86% Library has clear expectations for assessment (p= less than 0.001)

• 85% Library has an assessment plan (p= less than 0.0001)

• 78% Library administration uses assessment data (p=0.0031)

• 77% There is a shared understanding of the purpose of assessment in the library (p= less than 0.0001)

• 77% Library faculty/staff are adequately supported in their assessment work (p= less than 0.0001)

FACILITATING FACTORS MOST STRONGLY ASSOCIATED WITH A

CULTURE OF ASSESSMENT

*% of respondents who report their institution as having a culture of assessment who also report having each of these facilitating factors


• 81% Assessment is not a priority of administration (p= less than 0.0001)

• 80% Library culture is not user-focused (p= less than 0.0001)

• 72% Library is not involved in the campus-wide assessment initiative (p= less than 0.0001)

• 70% Assessment data is not available to interested parties (p= less than 0.0001)

• 68% Library leadership does not use data in decision-making (p=0.0031)

FACTORS MOST STRONGLY ASSOCIATED WITH NOT HAVING A

CULTURE OF ASSESSMENT

*% of respondents who report their institution as not having a culture of assessment who also reported not having these specific facilitating factors.


•Institution-wide Emphasis•Accreditation•Library Administrative Interest or

Support•Library Organizational Culture (user-

focused, curious, etc.)•Professional Development/Training/

Education

OPEN ENDED RESPONSES: WHICH FACTORS MOST FACILITATE?


•Lack of Staffing •Lack of Time•Lack of Library Administrative

Leadership or Support•Lack of Expertise/Lack of Access to

Training•Institution Doesn't Prioritize

Assessment

OPEN ENDED RESPONSES: WHICH FACTORS MOST HINDER?


SO WHAT DO WE NEED TO CREATE A CULTURE

OF ASSESSMENT?


•Value assessment and articulates why we are doing this•Promote assessment•Built into planning•Built into reward structures•Support library faculty/staff engaging in

assessment•Use assessment•Let library faculty/staff drive the bus

ADMINISTRATION WALKS THE TALK


Fewer than half of libraries...•Have clear expectations for assessment

(46%)•Have an assessment plan (41%)

Of libraries reporting clear expectations and an assessment plan, 92% report having a culture of assessment

EXPECTATIONS MUST BE CLEAR


S U P P O RT

http://www.flickr.com/photos/68751915@N05/6355360253/




•Time to learn, develop, conduct and analyze•Building expertise in assessment•Incentives, grants, awards•Professional development•Technologies for collecting, analyzing and

retrieving data

SHOW ME THE MONEY!


•Most librarians don’t have training in assessment of IL instruction•Need to speak a common language•Need to be able to develop assessment

tools and analyze results

EDUCATION


WHAT DO YOU DO WITH YOUR ASSESSMENT RESULTS?

http://www.flickr.com/photos/laughingmonk/44551444/




•Results are easy to get•Results are shared•Results are used to make changes and

decisions•Results are discussed•Results are shared outside of the library

HOW YOU USE THEM MATTERS


•Too busy with other work•Won’t be rewarded for assessment work•Don’t know how to use/analyze data•Lack of centralized support in library•Not valued by their library

Oakleaf and Hinchliffe. (2008). “Assessment Cycle or Circular File: Do Academic Librarians Use Information Literacy Assessment Data?” Proceedings of the 2008 Library Assessment Conference.

WHY DON’T LIBRARIANS USE THEIR ASSESSMENT RESULTS?


COMMON LIBRARIAN CONCERNS ABOUT ASSESSMENT

•Assessment imperative coming from the outside or above

•See it more about accountability than learning•Afraid of negative consequences for bad

assessment results•Time constraints•Loss of control•Uncertainty of how the info will be used


ADDRESSING CONCERNS

•Focus on student learning•Start assessing “where the light is best”•Let librarians develop their own assessment

instruments•Make sure there are no consequences for poor

results•Give people time to reflect on their assessment

results and improve their teaching•Meet often to discuss what everyone is learning

from assessment


IN THE END...

http://www.flickr.com/photos/eschipul/4160817135/Sunday, January 26, 14

http://www.flickr.com/photos/eschipul/4160817135/

http://www.flickr.com/photos/eschipul/4160817135/

Questions?Find me at

meredith.wolfwater.com/wordpressmgfarkas (at) gmail.com

flickr, twitter: librarianmerfacebook: meredithfarkas

http://www.flickr.com/photos/trucolorsfly/2401196653/




Building a Culture of Library Assessment

Education

Transcript of Building a Culture of Library Assessment