Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302
description
Transcript of Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302
![Page 1: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/1.jpg)
Anne Marie Karlberg Director of Institutional Research and Assessment
(360) [email protected]
http://faculty.whatcom.ctc.edu/InstResearch/index.htm
Assessment 101
PNAIRP Conference Thursday October 1, 2009
![Page 2: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/2.jpg)
Outcomes
1. Define assessment.
2. Identify the components of a successful assessment program.
3. Articulate three types of assessment information or data and provide examples of each.
4. Identify three levels of assessment.
5. Identify two phases in creating an effective outcomes process.
![Page 3: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/3.jpg)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
![Page 4: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/4.jpg)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
![Page 5: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/5.jpg)
What is assessment?
“ The systematic collection of information about student learning…to inform decisions about how to improve learning.”
(Walvoord, 2004, p. 2)
![Page 6: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/6.jpg)
Purposes
1. improvement (formative)1. improvement (formative)
2. accountability (summative)
![Page 7: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/7.jpg)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
![Page 8: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/8.jpg)
Educational Evaluation Model: CIPPBy Daniel Stufflebeam
![Page 9: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/9.jpg)
Using CIPP to Create an Effective Assessment Program
![Page 10: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/10.jpg)
Using CIPP to Create an Effective Assessment Program: Context
![Page 11: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/11.jpg)
Context: What do we know about the context of our
colleges? At Whatcom Community College…
8.3% of our employees
18.9% of our degree and certificate seeking students
16.0% percent of Whatcom County residents
…indicate they are of color.
![Page 12: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/12.jpg)
Using CIPP to Create an Effective Assessment Program: Inputs
![Page 13: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/13.jpg)
Resources
• human resources
• financial support
• technical support
• administrator and faculty support
![Page 14: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/14.jpg)
Plans and Strategies
Whatcom will…
•revise its mission statement and familiarize faculty / staff with mission
•implement the strategic plan
•implement the assessment plan
![Page 15: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/15.jpg)
Using CIPP to Create an Effective Assessment Program: Processes
![Page 16: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/16.jpg)
Implementation
Link assessment program to mission statement • redirect resources towards priorities • increase responsiveness to community needs•measure the extent to which we are successful
Provide opportunities for meaningful, collaborative college-wide conversations, by structuring assessment• in an ongoing, simplified, participatory way • relevant and meaningful
![Page 17: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/17.jpg)
Embed assessment throughout college
![Page 18: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/18.jpg)
Learning and teaching practices
Apply meaningful, relevant and contextualized experiences for students, such as…
→ using self-reflection→ applying concepts to a relevant context→ teaching material to peers→ discovering connections between subjects
![Page 19: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/19.jpg)
Using CIPP to Create an Effective Assessment Program: Products
![Page 20: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/20.jpg)
1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations
Assessment data
![Page 21: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/21.jpg)
1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations
2.Indirect indicators (perceptions): e.g., surveys, focus groups, interviews
Assessment data
![Page 22: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/22.jpg)
1.Direct indicators (outcomes): e.g., essays, capstone projects, demonstrations, presentations
2.Indirect indicators (perceptions): e.g., surveys, focus groups, interviews
3.Institutional data: e.g., retention, graduation, enrollment, transfer trends
Assessment data
![Page 23: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/23.jpg)
A.College level
B.Program level
C.Course level
Assessment levels
![Page 24: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/24.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
3. Institutional data (rates and numbers)
Assessment Products: Examples
![Page 25: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/25.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
3. Institutional data (rates and numbers)
Assessment Products: Examples
![Page 26: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/26.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
![Page 27: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/27.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
![Page 28: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/28.jpg)
Using CIPP to Create an Effective Assessment Program
![Page 29: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/29.jpg)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
![Page 30: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/30.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products
![Page 31: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/31.jpg)
Two phases of an outcomes process
![Page 32: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/32.jpg)
(1) Development of an outcomes process
![Page 33: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/33.jpg)
(2) Implementation of an outcomes process
![Page 34: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/34.jpg)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
![Page 35: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/35.jpg)
Using CIPP to Create an Effective Assessment Program
![Page 36: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/36.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
![Page 37: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/37.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
![Page 38: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/38.jpg)
Direct indicators: College outcomes plan
Goal (# of CLAs)
Whatcom will…
Baseline (May 2008) 2008 - 2009
2009 - 2010
2010 - 2011
2011 - 2012
(1) Development of the college outcomes processa. educate faculty / staff / students about assessment No formal
initiativesongoing ongoing ongoing ongoing
b. revise Core Learning Abilities (CLAs) and outcomes Identified 5 CLAs in 1998
Revise CLAs and outcomes
refine review review
c. develop assessment tools (e.g., scoring guides / rubrics) to measure the outcomes
5 review review
d. determine which courses will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)
5 review review
e. include outcomes on syllabi 1 2f. collect the instructional assignments, activities, projects, or experiences, in required courses that will be used to teach outcomes at entry, midway, and exit
1 2
g. collect the activities, experiences, projects, essays, or assignments in required courses that will be used to assess outcomes at entry and exit
1 2
h. attach anchor papers (i.e., examples) for each level of the scoring guide/rubric scale
1 2
(2) Implementation of the college outcomes processa. assess students at (entry and) exit for outcomes 1b. analyze the (entry and) exit assessment data 1c. present analysis to faculty and students and consult on the results 1d. use the data to improve and revise curriculum 1e. document the process; create an assessment report about how the data were used to improve learning
1
![Page 39: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/39.jpg)
Direct indicators: College outcomes plan
Goal (# of CLAs)
Whatcom will…
Baseline (May 2008)
2008 - 2009
2009 - 2010
2010 - 2011
2011 - 2012
(1) Development of the college outcomes processa. educate faculty / staff about assessment No formal
initiativesOngoing
b. revise Core Learning Abilities (CLAs) and outcomes
Identified 5 CLAs in
1998
Revise CLAs and outcomes
Refine Review
c. develop assessment tools (e.g., scoring guides / rubrics, etc.) to measure the outcomes
Review
d. determine which courses will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)
Review
![Page 40: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/40.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
![Page 41: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/41.jpg)
Indirect indicators:College survey schedule
Goal
Survey
Baseline data (May 2008)
2008 – 2009
2009 – 2010
2010 – 2011
2011 – 2012
2012 – 2013
Student Opinion Survey (every 6 years – alternate with the CCSSE)
Conducted January 2008
Community College Survey of Student Engagement (every 6 years – alternate with the Student Opinion Survey)
Faculty Survey of Student Engagement (every 6 years)
Staff satisfaction survey (every 4 years) Conducted in 2007)
Alumni Survey (ACT with Whatcom-specific questions) (every 10 years)
![Page 42: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/42.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
![Page 43: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/43.jpg)
Institutional data: College-level plan
Type of Data
Baseline May 2008
2008 – 2009
2009 – 2010
2010 – 2011
2011 – 2012
Enrollment numbers
Profile of all students
Grade distribution
Graduation rates / numbers
Retention rates
Success after transfer to WWU
![Page 44: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/44.jpg)
Overview
1. What is assessment?
2. Components of an effective assessment program
3. Phases of an outcomes process
4. Assessment plan
5. Website
![Page 45: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/45.jpg)
Website
•WCC Homepage
• “About Whatcom”
• “Assessment / Inst Research”
http://www.faculty.whatcom.ctc.edu/InstResearch/
![Page 46: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/46.jpg)
Comments and questions?
![Page 47: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/47.jpg)
Using CIPP to Create an Effective Assessment Program
![Page 48: Anne Marie Karlberg Director of Institutional Research and Assessment (360) 383-3302](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814f63550346895dbd18f7/html5/thumbnails/48.jpg)
Type of data College Program Course
1. Direct indicators (outcomes assessment)
• Core Learning Abilities and outcomes
• Program outcomes
• Course outcomes
2. Indirect indicators (surveys, interviews)
•Student opinion survey
•CCSSE
•Graduate survey
•Alumni survey
•Student evaluation of courses
3. Institutional data (rates and numbers)
• Graduation • Performanc
e after transfer
•Student enrollment
•Retention •Transfer
•Course completion
•Grade distribution
Assessment Products: Examples
Anne Marie Karlberg (Director of Institutional Research and Assessment)(360) 383-3302
[email protected]://faculty.whatcom.ctc.edu/InstResearch/index.htm