ARL New Measures—a View from Cornell

Anne R. KenneyARL Survey Coordinators

and SPEC Liaisons MeetingALA June 2007

Cornell and Assessment

• Participant in ARL Making Library Assessment Work project (August 2006)

• New Scope of Research and Assessment Unit (RAU) Fall 2006– organizational effectiveness, use and

usability, service innovation

Making Library Assessment Work Recommendations

• Examine organizational placement, resources, skill sets, and function of assessment

• Continue LibQUAL and develop customized survey(s)

• Gather more qualitative data from customers• Put ARL statistics into perspective• Develop a usability program• Utilize measures of success, move from

assessment to action• Limit scope of data farm project

Research and Assessment Unit

Team: Xin Li, Rich Entlich, Ellie Buckley, David Banush, Linda Miller

RAU Charge #1: Report on Library Performance

• Prioritize efforts• Assemble and interpret data• Produce final reports

Charge #2: Manage Assessment Data

David Banush, “Options for a CUL Data Mart” (July 2007)

The unhappiness with data collection coupled with a perceived need for even more data collection may perhaps be traced to a very basic problem: an unclear or perhaps non-existent rationale for gathering the data… Several interview subjects expressed significant doubts about data-driven decision models generally. Some maintain that data are not used by the CUL administration in decision making and asked why more information would have any added value.

David Banush

Charge #3: Conduct Hi-Priority Projects

“intelligence unit” – Environmental scans– Patterns and trends

“consultant” – Design assessment

instrument– Administer assessment

projects– Analyze and report findings

with recommendations/ observations

ARL New Measures: The Cornell Perspective

• Reviewed ARL Board approved action items along with Yvonna Lincoln and Bruce Thompson reports – Find the characteristics of the library of the future

insightful– Applaud ARL’s shift from output-focused to outcome-

based measures and collaboration– Look to ARL for clear definitions and communication

plan for the new measures

ARL New Measures: The Cornell Perspective

• Assessed potential impact on CUL– No immediate impact other than adjustments of data

in some sub-categories– If implemented, significant additional investment in

data collection, requiring data outside library’s control– Some data may not be practically attainable– New measures may lead to incomplete picture, e.g.,

expenditure-focused index.– New measures may help CUL embrace culture of

assessment

New Measures: “Cost-Benefit” Estimate

Next Steps• Actively participate in ARL decision making process;

evaluate and help shape definitions.• Discuss/determine in CUL executive committees what

decisions to make for functional areas.• Decide what success indicators are, what data captures

the success levels, how to organize the data to reveal performance.

• Decide to what extent to invest resources for the new data collection (cost-benefit) and what activities to drop so resources can be re-directed.

• Connect the New Measurements initiative with ARL visiting program officers’ findings about CUL

• Be prepared to make tough choices, tying direction and resources to new measures.

Operationalize the to-dos Focus on local so to create an effective way to support ARL

directions• Examine the value of data for CUL decision making and

priorities—internal and external data and measures• Tie library measures more closely to university

measures/goals/requirements• Make cost-benefit choices• Streamline existing data collection effort by 50% or more

within the next 12 months• Identify gaps, estimate additional investment needed for

new measures• Pilot new approach with co-sponsored project

Make the “so what” Leap• Resist the temptation of data collection based on “it

would be interesting, if we know…”• Focus on outcome measures.• Take a hard look at data collection with vague goals.• Recognize value of good enough data.• Run RAU like a business with customer-service

focus.• Have clear charge, authority, leadership, and goals• Build staff skills• Recognize that culture change and politics are hard.• Dare to say, “so what.”• Spend the money when your skills and timeline may

compromise the result, e.g. persona.