Eureka! Strategies for Assessing Co- curricular Programs Across · PDF file ·...

Eureka! Strategies for Assessing Co-

curricular Programs

Across Three Different

InstitutionsHigher Learning

Commission’s Annual Conference, 2016

Today’s Session: • One definition of co-curricular assessment• Brief examination of the HLC criteria related to co-curricular

assessment• Presentations from 3 institutions on how they approach co-

curricular assessment• Discussion of similarities and differences from the panelists• Open questions and discussion• The answer to today’s puzzler!

DIVISION OF STUDENT AFFAIRSASSESSMENT

“That which we call a rose by any other

name would smell as sweet”

Defining Co-curricular Assessment

3.E.1. “Co-curricular programs are suited to the institution’s mission and contribute to the educational experience of its students.”

3.E.1. “Co-curricular programs are suited to the institution’s mission and contribute to the educational experience of its students.”

4.B.2. “The institution assesses achievement of the learning outcomes that it claims for its curricular and co-curricular programs.”

4.B.2. “The institution assesses achievement of the learning outcomes that it claims for its curricular and co-curricular programs.”

Should the operations of the Bursar’s Office be considered a co-curricular program?

Disclaimer: The following statements represent the views of the presenter and may not necessarily represent the views of the HLC


Intentionality: Is the program designed to promote student learning?


Claims: Do you make claims that your program results in learning or contributes to an enriched educational environment?


Outside the classroom: Is the program outside the formal

classroom?


Defining Characteristics• Intentionality

– Is your program designed to promote student learning / development or give students the opportunity to apply their learning in new situations?

• Claims– Do you (or your institution) make claims that your program does

these things (promote learning / development) or contributes to the enriched educational environment?

• Outside the classroom– Is the program outside the formal classroom (i.e., it’s not a class

activity)?• If you meet these, you are running a “co-curricular” program• You should be assessing student learning / development

If my program / department doesn’t meet these criteria, then do I get a “free pass”

on assessment?


5.D. “The institution works systematically to improve its performance.” (Other criteria apply as well)


Co-Curricular Assessment• Assessment of student learning and

development that occurs in co-curricular programs

• Can look very different in different types of institutions!

Eureka! Assessing Co-curricular Programs at Three Different Types

of Institutions

A Community College Perspective

Jonathan Keiser, PhDSenior Associate Vice Chancellor – Educational Quality

City Colleges of Chicago

“Students Juggle Priorities - Most students attend classes and study while working; caring for dependents; and struggling to balance personal, academic, and financial challenges”

Published by the Center for Community College Student Engagement, Characteristics of Community College Students ©2014

Community College Environments

Community College Environments

Published by the Center for Community College Student Engagement, Characteristics of Community College Students ©2014

Co-curricular Survey

8, 8%

64, 64%

28, 28%

WHO

VP/Dean Assessment Committee Dept. ChairsVP/Dean = 3Assessment Committee = 26Depart. Chairs = 11N = 40


This is happening, it’s active and

ongoing.30%

This is happening, but I’m not familiar with the process.

27%

This is definitely not happening.

18%

I’m not sure this is happening.

25%


∗ Surveys∗ As a part of the AQIP process∗ Through conversations with administrators∗ CCSSE∗ Course SLOs∗ Data collection across the college∗ Multidisciplinary collaboration∗ Writing Across the Curriculum (WAC)∗ Assessment Committee

How is co-curricular assessment being accomplished?


∗ Closing the loop∗ Lack of participation in the process∗ Lack of time for data analysis∗ Strategic response planning based on findings∗ It’s difficult to understand∗ Multiple people are involved∗ Funding and approvals∗ Mishandling of data∗ Employee turnover leads to inconsistent results∗ No culture embracing effort∗ Lack of motivation / no understanding of benefit∗ It’s new, therefore unfamiliar∗ Knowing what’s next after collecting data∗ Resistance to change∗ Fear of judgment

Obstacles


∗ “Based on my limited knowledge, I would say, Be prepared to find out results you don't like. And I would say, Fixing a lot of the co-curricular issues requires a significant budgetary commitment.”

∗ “I think we all need to be trained on what co-curricular assessment is and what our roles are in this type of assessment. I have to admit that I do not truly understand what it entails.”

∗ “Reading other institutions' AQIP systems portfolios, provided online by the Higher Learning Commission, is where we have gone to see how it is done.”

∗ “We need alignment between the assessment of student learning and co-curricular assessment or at least some connection between the two processes.”

∗ “Begin by having co-curricular areas identify their mission goals and objectives. We found that this helped craft the learning outcomes.”

Advice

Co-curricular Operational Effectiveness Model

Student Services Department with Missions and Outcomes Academic Support Services Library

Admissions & Registrar Office MSEIP Minorities in Science and Engineering

Academic Advising Student Activities

Athletics Testing Center

Business & Operational Services Transfer Center

Career Planning and Placement Center Educational Talent Search - TRiO

Disability Access Center Veteran Services

Office of Financial Aid Wellness Center

Office of Information Technology

The Institutional Mismatch

∗ Traditional wisdom: Learning outcomes need to be aligned at course, program, and institutional levels

∗ But where are many general education goals, like “tolerance” and “teamwork” and “the ability to function in an increasingly diverse world” taught and assessed

∗ Or is “taught” the right word???

Shifting Perspectives

∗ What happens if we substitute the word “learned” for the word “taught”?

∗ What are the implications of “Where are learning outcomes learned and assessed”?

∗ Emphasis on student demonstration of knowledge and skills, not the topics covered

∗ Ability to do or apply supersedes knowing∗ Responsibility for learning is shared∗ Site of learning becomes less specific, and boundaries

become more fungible

Learning Reconsidered 2

∗ “Learning” is not exclusively classroom-based

∗ Many valued outcomes are the result of processes outside the classroom

∗ “Learning” is a process based on three interdependent student experiences:

∗ Understanding academic content and processes

∗ Student development∗ Identity formation

∗ Meaning is constructed from inquiry, interaction, conversation, and life experience

Learning Reconsidered 2

∗ Responsibility for “learning” exists outside the classroom

∗ Responsibility for “learning” doesn’t always take the same form; some entities on campus produce it, some facilitate it, some support it

∗ Responsibility for assessing learning exists outside the classroom as well

Co-curricular Operational Effectiveness Model

Student Services Department with Missions and Outcomes Academic Support Services Library

Admissions & Registrar Office MSEIP Minorities in Science and Engineering

Academic Advising Student Activities

Athletics Testing Center

Business & Operational Services Transfer Center

Career Planning and Placement Center Educational Talent Search - TRiO

Disability Access Center Veteran Services

Office of Financial Aid Wellness Center

Office of Information Technology

Some Core Areas Where Learning Can Occur in Non-classroom Settings

Developmental Outcomes Requirements

Critical ThinkingWorking with OthersGlobal CompetenceCivic Engagement Ethical BehaviorCommunication

Adaptability

Need to be intentional

Need to be planned Need to be part of

the structure of a student’s experience

Need to be assessed

SMALL SCHOOL EXAMPLE

Presented by Marie Baehr, Vice President for Academic Affairs

What does Coe consider Co-curricular?

Coe provides students “an education that aims at preparing them for life following graduation. Our reason to exist …is to ready students intellectually, professionally, and socially to lead productive and satisfying lives…” [mission]

What is Coe co-curricular?

Students are required to live on campus for all four years with few exceptions, thus setting up expectations for out-of-class learning—for both Coe and its students.

What is Co-curricular? Pretty much everything that is not curricular!

Some examples

• Tutoring

• Residence hall programming

• Athletics

• Clubs

• Orientation

• Inclusion Office

• Student government

Some ExamplesEvidence

• Focus group input

• Satisfaction AND attitude surveys

• Feedback from faculty

• Feedback from community

Staff engagement/development

Structure and resources

• Mostly seamless work between VP for Academics and VP for Student development

• Comparable professional support for staff and for faculty

• Keeping it simple

Implementation for Accreditation

Create a process that gives Coe useful information to know how processes are working and where improvements need to be made, and accreditation does not then need to be the driver.


Co-curricular Assessment at NDSU


• About 12,000 undergraduate students and 2,000 graduate students

• Engineering, Agriculture, Nursing, Business

• Pharmacy, Architecture, Ph.D. programs


Major Activity / Program / Service

Operational Effectiveness

Student Learning /

Development Outcomes

Are we doing things right?

Are we doing the right things?


Operational Effectiveness• Facility usage• Satisfaction• Cost efficiencies• Level of participation

Student Learning and Development• What students are

expected to know, be able to do, or dispositions as a result of the program / activity / service


Operational Effectiveness• Institutional data

(visits, participation, card swipes, etc.)

• Surveys• Secret shopper• Self-report• Observer feedback• Fidelity studies

Student Learning and Development• Rubrics• Minute papers• Performance

assessment• Surveys• Portfolios• Reflection papers• Interviews


Learning Outcomes• Reflect the

knowledge, skills, abilities, or attitudes students should develop in response to the program / event

Service Outcomes• Describe what the

student will know / be able to do as a result of the service provided and

• Clearly communicate an important aspect of the work of the Division


Assessment Structures:• Assessment linked with

annual strategic planning process

• Each strategic plan strategy includes assessment –operational or student learning / development


• Reports due June 1st

• Plans due September 1st

• Reports are reviewed during the summer by the Student Affairs Assessment Council– SAAC is a group of

volunteers who provide direction and guidance to assessment


• All departments have assessment coordinator

• Most discuss assessment at meetings in the spring and have a summer retreat for planning the next year

• Staff development:– Assessment handbook

for new participants– Assessment Academy

(3-semester workshop series)

– Periodic events (e.g., “Survey Ninja”

– Online resources


https://www.ndsu.edu/vpsa/assessment


• Reporting and communicating results to different audiences

• Traditional reports• Monthly Student

Affairs Insight 1-page report

• Brief bullet-point summaries

• Infographics• Twitter and social

media


Resources for assessment• Office of Student Affairs

Assessment–1-person office opened in 2013

• Campus Labs Baseline license• Qualtrics license• Larger departments have portion

of staff member assigned to work on assessment / planning


Does this always work?


Counseling Center• CCAPS periodically administered to clients

(learning outcomes / development)• Mental health consortium project (learning

outcomes and needs assessment)• Tracking no-show rate and adding text-

messaging reminders to reduce no-show rate (service outcomes: operational effectiveness)


Disability Services• Identified students’ self-advocacy as

targeted learning outcome• Plan on asking for self-assessment from

students (at intake), evaluation by the assigned DS coordinator, annual check points

• Client satisfaction and tracking usage rates (operational / service outcomes)


Department of Residence Life• Used rubrics to assess each floor’s

development over the year in three areas: community, academic skills, and wellness (student learning outcomes)

• In response to rubric data and other existing assessment data (including Campus Labs benchmarking survey), completely re-worked the curriculum for 2014-2015

• Track satisfaction with housing and vacancy rates (service outcomes: operational)


Wellness Center• Student employees self-assessed using

rubrics on growth over time on critical learning outcomes (e.g., time management, ability to handle difficult customers, etc.)

• Modified training program and events during the year for student employees

• Track center usage, demographics of users, client satisfaction (operational / service outcome)


Student Health Service

• Survey feedback from users and non-users and analysis of usage data, decided to open more same-day, walk-in appointments (operational)

• Accreditor requires multiple Quality Improvement efforts (operational)

• Smaller student learning outcomes mission, but have looked at tobacco cessation education, individual patient knowledge

Some Conclusions


Some Key Similarities• Distinction between student learning

outcomes assessment and operational measures

• Intentionality about assessment – not trying to assess everything that moves


Some Key Differences• Students’ expectations for co-curricular

experiences• Scale and capacity• Value co-curricular experiences differently

and approach them differently


Contact InformationMarie Baehr: [email protected] Keiser: [email protected] Penn: [email protected]

mailto:[email protected]



Eureka! Strategies for Assessing Co- curricular Programs Across · PDF file ·...

Documents

Transcript of Eureka! Strategies for Assessing Co- curricular Programs Across · PDF file ·...