1

Conclusion

2

Metrics Revisited

Inputs Outputs Quality and Satisfaction

Interrelated. Rely heavily on subjective responses and self-reporting

OutcomesImpacts on community- Our effects

3

Outcomes Assessment

Measuring change as a direct result of interaction with the library

Most often addressed in academic and school libraries Types of change:

Knowledge Behavior Awareness/ Percpetions Attitudes/ Opinions

4

Assessment Defined

The ongoing process of:

Establishing clear, measurable expected outcomes of student learning

Ensuring that students have sufficient opportunities to achieve those outcomes

Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations

Using the resulting information to understand and improve student learning [at both the course and program] level Suskie (2004, p. 3)

5

NEASC STANDARD SEVEN: Library and Information Resources

1: The institution makes available the library and information resources necessary for the fulfillment of its mission and purposes. These resources support the academic and research program and the intellectual and cultural development of students, faculty, and staff. Library and information resources may include the holdings and necessary services and equipment of libraries, media centers, computer centers, language laboratories, museums, and any other repositories of information or technological systems required for the support of institutional offerings. Clear and disseminated policies govern access, usage, and maintenance of the library, information resources, and services. The institution ensures that students use these resources as an integral part of their education.

2: Through the institution's ownership or guaranteed access, sufficient collections, information technology systems, and services are readily accessible to students wherever programs are located or however they are delivered. These collections, systems, and services are sufficient in quality, level, diversity, quantity, and currency to support and enrich the institution's academic offerings. The institution provides facilities adequate to house the collections and equipment so as to foster an atmosphere conducive to inquiry, study, and learning among students, faculty, and staff. --- Satisfaction?

6

NEASC STANDARD SEVEN (continued) 3: The institution provides sufficient and consistent financial support for the effective

maintenance and improvement of the institution's library, information resources, and services. It makes provision for their proper maintenance, preservation, currency, and security. It allocates resources for scholarly support services compatible with its instructional and research programs and the needs of faculty and students.

4: Professionally qualified and numerically adequate staff administer the institution's library, information resources, and services. The institution provides appropriate orientation and training for use of these resources, as well as instruction in basic information literacy.

5: The institution participates in the exchange of resources and services with other institutions and within networks as necessary to support and supplement its educational programs. It provides appropriate support for distance learning students and faculty, such as on-line reference service and contractual access to relevant off-campus library resources.

6: The institution regularly and systematically evaluates the adequacy and utilization of its library, information resources, and services and uses the results of the data to improve and increase the effectiveness of these services (satisfaction, service quality?)

7

NEASC STANDARD Four:Programs and Instruction

Compliance = outcomes student learning outcomes student outcomes

Undergraduate Degree Programs Graduate Degree Programs Scholarship and Research Instruction Admissions and Retention

http://community.pmc.edu/neasc/standards/Default.htm

8

More on Accreditation

Regional Accreditation- 3 use “information literacy” 3 use “equivalent language” NCC makes the point that today’s students must be prepared to be

“knowledge workers” in the sense that they must not just master certain information, but must be able to comprehend, synthesize and apply that information. More than just technologically literate, these workers will be valued for their “capacity to sift and winnow massive amounts of information in order to discover or create new or better understandings”

Support for collaboration Professional and Discipline-Specific Accreditors also paying

attention to information literacy Important to fields focused on Evidenced-Based Practice

9

Student Outcomes versus Student Learning Outcomes

Student outcomes are aggregate statistics on groups of students (e.g., graduation rates, retention rates, transfer rates, course and program completion rates, and job placement: employment rates for a graduating class)

Such outcomes are institutional outcomes and are used to compare institutional performance

They do not measure changes in students themselves due to their college experience

these outcomes are outputs and reflect what the institution has accomplished; they do not reflect what (or how much) students learned

Student learning outcomes —development of students: demonstrable acquisition of specific knowledge and skills

How well do students Transfer and apply concepts,

principles, ways of knowing, and problem solving across their major program of study?

Integrate their core curriculum, general studies, or liberal studies into their major program or field of study?

Develop understanding, behaviors, attitudes, values, and dispositions that the institution asserts it develops?*

* Maki (2004)

10

Student Learning Outcomes What should students learn?

How does the content of one course relate to another? How well are they learning it?

Across courses What evidence do we gather to answer these

questions? Beyond graded assignments, course evaluations

How do we use that evidence to improve learning Looking a courses as comprising a program

11

Purposes of Assessment

Impacting student learning

Working collaboratively: thinking beyond the set of courses “I” teach

Demonstrating accountability and accreditation—

Meeting institutional mission

Demonstrating institutional effectiveness

12

Institutional Effectiveness Defined as how well an institution achieves its

mission and major institutional goals “Since student learning is the heart of most

institutional missions, the assessment of student learning is a major component of the assessment of institutional effectiveness.”

However, institutional effectiveness does examine other aspects such as scholarship and research, community service, etc.

Suskie, 9-10

13

Student Learning Outcomes

ConceptualLeadership

Critical ThinkingCreative ThinkingProblem Solving

Information LiteracyGlobal Citizen

Values (moral, etc.)Foundations and Skillsfor Lifelong Learning

SkillsOral/written communication

Foreign language communicationTechnological sophisticationQuantitative reasoning ability

Other

Do “satisfaction” and “service quality” impact performance?

14

Direct Methods

Embedded course assessment (performance on assignments, etc.; minute paper)

Portfolio assessment Performance (internships,

practicum, student teaching) Professional jurors or

evaluators Capstone course/experience

Experimental research designs), with pre- and post-testing

Use of standardized tests Think-aloud protocol Directed conversation Videotape/audiotape

evaluation Analysis of

theses/dissertations/ senior papers (content analysis,

interviews, or oral defense)

15

Indirect Methods

Surveys (self-reporting) and self-assessments

Curriculum and syllabus evaluation

Exit interviews Observation

Other

16

A culture of

Assessment

Evidence

Evaluation

17

Have They Learned Anything?

18

So What?

Or Why Evaluation Research?

19

Evaluation Research Addresses Questions Such As What improvements in a program, service, or policy

might result in continuous quality improvement and better accountability?

How well does a program, service, or policy reach its target population and meet the group’s information needs and expectations?

Is the program, service, or policy being implemented in the ways envisioned?

Is it effective?

20

Utility

What are the findings good for? What decision in the real world do the findings

impact? What insights will the study provide? Can the study findings be dismissed with “so

what?”

21

Evaluation (for a Purpose)

Feedback loop (basic systems model)

Decision making

Accountability Summary judgment

(summative evaluation) Continuous [quality]

improvement Benchmarking and Best

Practices

22

Evaluation Research

Is also a political and managerial activity, an input into the complex mosaic from which we can make policy decisions and allocations for the planning, design, implementation, and continuance of programs to better the human

In this sense evaluation research is part of social policy and public administration

23

Evaluation Research--Steps

Reflective inquiry

Procedures

Conduct studyAnalyze data

Prepare/deliver report

Reliability

Validity

Communication skills

Setting up action plan

24

What are important areas for “evaluation research”?

______________ ______________ ______________

etc.

25

Evaluation of Information Services

Systems Analysis Other Research Courses*

*Strongly recommended for anyone planning a career in academic libraries, especially university libraries

Courses in Individual Concentrations

26

Scholarly CommunicationKey Issues Time lag between authorship, peer review,

publication, and dissemination Information overload

27

Future of PublishingOur Core Interests Competitive market Easy distribution and reuse Innovative applications of technology Quality assurance Permanent preservation (“archiving”)

28

Electronic Scholarly Publishing Some Models Digital library model Electronic book model Electronic collection model Retrospective model Preprint model Peer review lite model Commercial publishing model Academic server model

29

Electronic Scholarly Publishing Some Models (continued) Prestigious publishing model University publishing cooperative Public domain model Government server model