Assessment CycleAssessment Cycle
California Lutheran UniversityCalifornia Lutheran University
Deans’ CouncilDeans’ Council
February 6, 2006February 6, 2006
Assessment Cycle
Assessment Cycle
Strategic
Plan 2014
Strategic
Plan 2014
Assessment SymposiumAssessment Symposium
Program
Review
CaliforniaCalifornia Lutheran UniversityLutheran University Dedicated to educating leaders for a
global society who are strong in character and judgment, confident in their identity and vocation, and committed to service and justice.
Core ValuesCore ValuesExcellence Service Diversity Respect
Responsibility Courage Pursuit of Truth
Assessment SymposiumAssessment Symposium
Assessment Cycle
Assessment Cycle
Strategic
Plan 2014
Strategic
Plan 2014Assessment Symposium
Assessment Symposium
Program
Review
PurposePurpose
Report Report assessment evidenceassessment evidence and ways and ways
in which the evidence in which the evidence was and will be was and will be
usedused to to improve student learningimprove student learning and and
the the quality of our service.quality of our service.
Assessment Symposium
Assessment Symposium
Guiding PrinciplesGuiding Principles
SimpleSimple
MeaningfulMeaningful
Assessment Symposium
Assessment Symposium
BenefitsBenefits
Sharing of research and learnings Sharing of research and learnings
(e.g. Program Review)(e.g. Program Review) Engages various members of the Engages various members of the
University CommunityUniversity Community
Listen, learn, and appreciate othersListen, learn, and appreciate others
Assessment Symposium
Assessment Symposium
BenefitsBenefits
Supportive environment via Supportive environment via
roundtable format roundtable format
Encourages assessment to become Encourages assessment to become
part of our practicepart of our practice
Research in Action!Research in Action!
Assessment Symposium
Assessment Symposium
FeaturesFeatures
5 step model5 step model
5 – 10 minute presentation 5 – 10 minute presentation
One goalOne goal
Standardized PowerPoint formatStandardized PowerPoint format
Review and practice sessionsReview and practice sessions
Assessment Symposium
Assessment Symposium
5 Step Model5 Step Model
1.1. What What goalgoal was set? was set?
2.2. What What research designresearch design / measure was / measure was
used?used?
3.3. What What datadata were collected? were collected?
4.4. What emerged from data (What emerged from data (analysisanalysis)?)?
5.5. What What recommendations recommendations are made?are made? Measurable goals set?Measurable goals set?
Assessment Symposium
Assessment Symposium
ExamplesExamples
AcademicsAcademics Program Review including External Program Review including External
ReviewReview External Review onlyExternal Review only
OperationsOperations
Assessment Symposium
Assessment Symposium
External Review Assessment GoalExternal Review Assessment Goal Review of Foundations Courses and Review of Foundations Courses and
Student Teaching Seminar conducted by Student Teaching Seminar conducted by an external university team an external university team
Purpose: to investigatePurpose: to investigate• Repetition of conceptsRepetition of concepts• If “diversity” met NCATE standard 4If “diversity” met NCATE standard 4• If field experiences met NCATE standard 4If field experiences met NCATE standard 4• If Foundation learning outcomes aligned with If Foundation learning outcomes aligned with
Student TeachingStudent Teaching• If exit survey provided data for improvementIf exit survey provided data for improvement
Research Design and Data Research Design and Data CollectionCollection
Content analysis of syllabi including Content analysis of syllabi including candidate learning outcomes and anchor candidate learning outcomes and anchor activitiesactivities
Analysis of exit surveyAnalysis of exit survey Interviews with Dean and Foundations Interviews with Dean and Foundations
CoordinatorCoordinator
Analysis of DataAnalysis of Data
FindingsFindings Syllabi content analysis shows appropriate Syllabi content analysis shows appropriate
contentcontent ““Diversity” depth is sufficientDiversity” depth is sufficient Field experiences appropriate for Foundations Field experiences appropriate for Foundations
coursescourses Appropriate alignment of Foundations and Appropriate alignment of Foundations and
Student Teaching Seminar learning outcomesStudent Teaching Seminar learning outcomes Exit survey should be strengthenedExit survey should be strengthened
Future GoalsFuture Goals
Goals for upcoming year:Goals for upcoming year: Continue to align online and onsite syllabi and Continue to align online and onsite syllabi and
anchor activitiesanchor activities Distribute specific TPE responsibility between Distribute specific TPE responsibility between
Foundation coursesFoundation courses Specify school demographics for Field Specify school demographics for Field
Activities to ensure experience in diversityActivities to ensure experience in diversity Gather exit portfolio data to help guide Gather exit portfolio data to help guide
program improvement decisionsprogram improvement decisions
Operations Operations Assessment GoalAssessment Goal
Reduce the number of complaints received Reduce the number of complaints received
from credential studentsfrom credential students
Research DesignResearch Design Measures Used to Collect Data Measures Used to Collect Data
Toward Goal ProgressToward Goal Progress Established categories for complaints Established categories for complaints
and recorded data dailyand recorded data daily
Resulting from previous Data Analysis:Resulting from previous Data Analysis: Created position of Student Service Created position of Student Service
CoordinatorCoordinator Provided additional training in customer Provided additional training in customer
service and technical areasservice and technical areas Implemented the development of staff Implemented the development of staff
professional portfoliosprofessional portfolios
Data Collection ResultsData Collection Results
Credential Complaints
0
20
40
60
80
100
120
140
160
180
FY02 FY03 FY04
Series1
Analysis of DataAnalysis of Data Fewer complaints received in areas with closer Fewer complaints received in areas with closer
proximity of a credential supervisor.proximity of a credential supervisor. Mis-advisement has the most room for Mis-advisement has the most room for
improvement; most related to changes or improvement; most related to changes or confusion about state or university regulations or confusion about state or university regulations or policies.policies.
Fewer complaints were received in locations Fewer complaints were received in locations where professional development was regularly where professional development was regularly utilized.utilized.
Greater number of student-centered initiatives Greater number of student-centered initiatives and fewer complaints received from areas where and fewer complaints received from areas where staff had developed professional portfolios.staff had developed professional portfolios.
Future GoalsFuture Goals Develop, implement and measure Develop, implement and measure
additional ways to communicate policies additional ways to communicate policies in order to decrease “mis-advisement” in order to decrease “mis-advisement” complaints.complaints.
Determine the relationship between Determine the relationship between complaints and participation in complaints and participation in professional development and proximity professional development and proximity of credential supervision provided.of credential supervision provided.
Design and implement staff professional Design and implement staff professional portfolios state-wide.portfolios state-wide.
Assessment SymposiumAssessment Symposium May 2006, a.m.May 2006, a.m. FormatFormat
Slide 1: IntroductionSlide 1: Introduction
Slide 2Slide 2 Slide 3Slide 3 Slide 4Slide 4 Slide 5Slide 5 Slide 6Slide 6
2005-06 Goal2005-06 Goal Assessment Assessment DesignDesign
Data Data CollectionCollection
Analysis of Analysis of DataData
2006 - 072006 - 07
GoalGoal
What goal was What goal was set? What set? What aspects of aspects of student student learning / learning / service will be service will be assessed? assessed?
Identify Identify measures measures (instrument / (instrument / tool) used to tool) used to provide data on provide data on progress progress toward goal. toward goal. What process What process was used?was used?
What data What data were were collected? collected?
What themes What themes emerged? emerged? What was What was learned? learned?
Recommendations Recommendations for growth based for growth based on analysis on analysis What measurable What measurable goal is set for goal is set for
2006 - 07? 2006 - 07?
Program ReviewProgram Review
Assessment Cycle
Assessment Cycle
Strategic
Plan 2014
Strategic
Plan 2014Assessment Symposium
Assessment Symposium
Program
Review
Program ReviewProgram Review
In Progress:In Progress: 5 reviews completed Spring 20065 reviews completed Spring 2006 Revised Template, incorporating Assessment Revised Template, incorporating Assessment
Plan into Program ReviewPlan into Program Review Program Review HandbookProgram Review Handbook March Workshops for upcoming ReviewsMarch Workshops for upcoming Reviews
External ReviewersExternal Reviewers Internal ReviewersInternal Reviewers
External ReviewersExternal Reviewers
CredentialsCredentials External perspectiveExternal perspective
Lutheran link
Aspiring link
California
National
Study components of program through a Study components of program through a
series of specific review objectives or answer series of specific review objectives or answer
questions as prepared by program faculty. questions as prepared by program faculty.
Program Review
Program Review
External ReviewersExternal Reviewers
Receive copies of the Program Review Receive copies of the Program Review Report data in advance Report data in advance
May be asked to respond to questions such May be asked to respond to questions such as:as: Do references in syllabi adequately cover current Do references in syllabi adequately cover current
knowledge in the field?knowledge in the field? Do CVs of faculty demonstrate appropriate Do CVs of faculty demonstrate appropriate
qualifications for teaching in this program?qualifications for teaching in this program? Does the curriculum demonstrate an Does the curriculum demonstrate an
undergraduate / graduate level of instruction?undergraduate / graduate level of instruction? Are assessments appropriate?Are assessments appropriate?
Program Review
Program Review
External ReviewersExternal Reviewers
Complete a site visit and conduct Complete a site visit and conduct
interviews with administrators, interviews with administrators,
faculty, students, employers, and faculty, students, employers, and
alumni, as well as review facilities. alumni, as well as review facilities.
Program Review
Program Review
External Reviewers’ ReportExternal Reviewers’ Report Three to five pagesThree to five pages
Prepare preliminary report on-sitePrepare preliminary report on-site
Responds to the objectives or questions Responds to the objectives or questions
Includes three parts: general comments, singles out Includes three parts: general comments, singles out
features of the program that merit commendations, features of the program that merit commendations,
and makes recommendations where there is room and makes recommendations where there is room
for improvement for improvement
Program Review
Program Review
Internal ReviewersInternal Reviewers
California Lutheran University faculty California Lutheran University faculty
members from outside the college or members from outside the college or
school of the program under reviewschool of the program under review
Assessment Committee, Assessment Assessment Committee, Assessment
Director, Associate Provost, Provost, Director, Associate Provost, Provost,
GPEC, …GPEC, …
Program Review
Program Review
Assessment Cycle
Assessment Cycle
Strategic
Plan 2014
Strategic
Plan 2014
Assessment SymposiumAssessment Symposium
Program
Review
CaliforniaCalifornia Lutheran Lutheran UniversityUniversity
Dedicated to educating leaders for a global society who are strong in
character and judgment, confident in their identity and vocation, and
committed to service and justice.
Core Core ValuesValues
Excellence Service Diversity Respect
Responsibility Courage Pursuit of Truth
Assessment CycleAssessment Cycle
California Lutheran UniversityCalifornia Lutheran UniversityDeans’ CouncilDeans’ Council
February 6, 2006February 6, 2006
Halyna KornutaHalyna KornutaDirector of Assessment and Educational EffectivenessDirector of Assessment and Educational Effectiveness