Don Dodson, Senior Vice Provost

Diane Jonte-Pace, Vice Provost for Undergraduate Studies

Carol Ann Gittens, Director, Office of Assessment

Learning Assessment and Program Review: Finding Meaningful and Manageable Approaches

Acknowledgments and Purpose Today

ACKNOWLEDGING FACULTY CONCERNS: Faculty are not trained for assessment Assessment and program review are time

consuming Effective assessment strategies are difficult

to develop Departments have gathered lots of data but have few answers Faculty need guidance and support from experts

PURPOSE TODAY: Invite dialogue about assessment and program review Identify ways to make them more manageable and meaningful

Defining Terms

Program Review

Refers to periodic evaluation of an academic program based on a self study which examines all aspects of the program, including student learning outcomes.

Assessment

Refers to collection and analysis of quantitative or qualitative evidence about what students actually learn in relation to specified objectives.

Levels of Assessment

Course– Involves designing and grading course assignments– Determines what individual students are learning– Provides feedback to students about their own learning – Informs individual course modifications over time

Program – Asks how well students are achieving objectives of the major or minor– Gathers direct evidence of student learning across program’s curriculum– Reflects on program’s contribution to institution-wide learning goals

Institution – Evaluates overarching institutional learning outcomes (Core Curriculum)

What are Faculty Being Asked to Do for Assessment?

Grade student work in courses

Include program learning objectives on department syllabi where appropriate

Complete Annual Learning Outcomes Assessment Report

Include Core learning objectives on new Core syllabi

Provide selected student work from Core courses– Only 2-3 Core areas per year– Only a few selected courses– Only a small number of students per course (approximately 5)– Invitation to help score student work (not required!)

How Much Flexibility Do We Have Around Assessment?

Faculty determine course-level learning objectives, construct assignments, and grade assignments

Departments create departmental learning goals and objectives

Departments determine meaningful questions for program improvement

Departments design their own assessment plans and data collection strategies (with help, if needed)

Faculty can communicate Learning Objectives for Core through syllabus, website, or other vehicle

What’s the Mandate for Assessment and Program Review?

Federal Regulation

Accrediting Agencies

Board of Trustees

University Administration

Student and Faculty Success

Better Use of Resources

Insight into Strengths / Challenges

Desire to Improve Programs

Why Does This Feel So Burdensome?

Steep learning curve for individuals, institutions, and higher education generally

Many simultaneous initiatives:– Student learning assessment– Program Review– Core Curriculum– WASC self study– Specialized accreditation self studies

Inadequate communication and coordination

Some common correctable mistakes

Common Correctable Mistakes

Assessment Setting more objectives than can reasonably be assessed Setting unclear objectives Assessing the same objective too frequently Using methods that are unnecessarily time-consuming Designing add-on assessments rather than making use of existing course

assignments or program activities

Program Review Not using program review data provided by Institutional Research Office Not integrating self study for University with self study for professional accreditation

Help is available!

Institutional Support

Office of Assessment (Carol Ann Gittens)– Assistance refining learning goals and objectives for meaningful assessment– Assistance streamlining approaches for manageable assessment– Assistance developing effective and efficient strategies and tools for

evaluating student learning– Assistance with data analysis and interpretation (quantitative and qualitative)– Support with all assessment endeavors

Provost’s Office (Don Dodson)– Assistance with program review expectations and design

Dean’s Offices– Feedback and support for both assessment and program review

Additional Institutional Support

Core Curriculum– Honoraria and summer workshops– Stipends for Core scoring parties

Annual Assessment / Program Review– Course releases and funding

Faculty Development Workshops

Responses to Top 10 FSC Recommendations

1. Decrease frequency of program review and course goal assessment

Provost’s Office will propose 8-year program review cycle to Academic Affairs Committee

Frequency of course goal assessment is an individual faculty decision Frequency of program goal assessment is a program-level decision Frequency of institutional goal assessment (Core Curriculum) will require

participation of faculty in individual courses, typically on a volunteer basis

2. Have assessment experts help departments streamline process This service is available through the Office of Assessment and external

consultants

3. Hire staff members to do program review WASC requires full faculty involvement Institutional Research Office since 2005-06 has made relevant data

available on compact disc to programs doing self studies. It will try to improve data currency and user training in future.

Responses to Top 10 FSC Recommendations

– Stop assessments that are not shown to be scientifically valid Unrealistic standard that would probably stop most course grading as well Office of Assessment can assist with planning and data analysis

5. Improve structure and clarity of assessment tools given to departments from above

Other than Curriculum Alignment Matrix, departments are not required to use assessment tools given from above

6. Make program review specific to each department WASC expects some consistency Consistency is desirable because departments may overlook key issues and

because it helps administration determine priorities for program support Current guidelines state: “Programs may request exceptions to the official

guidelines or calendar if such changes would be conducive to a more effective program review.”

Responses to Top 10 FSC Recommendations

7. Use existing data rather than creating new tools To the extent possible, this is always desirable and is consistent with advice given by

the Office of Assessment

8. Improve assessment situation by negotiating with WASC WASC doesn’t negotiate WASC Standards and Criteria for Evaluation are determined through consultation with

all accredited institutions in the region, with an eye to both federal regulations and best practices in higher education

9. Deans read and provide feedback on program review reports Agree

10. Make program review process work for professional accreditation needs Each school currently decides how integrate the two processes in consultation with the

Provost’s Office. Advice: do one self study based on accreditation guidelines, with a supplement to address some specific University questions

SCU Web Resources

Office of Assessment:

http://www.scu.edu/provost/assessment/

Core 2009:

http://www.scu.edu/Core2009

WASC

http://www.scu.edu/WASC (select Resources)

Discussion

What questions do you have?

What suggestions do you have?