An initiative of the Research & Planning Group
for California Community Colleges
Assessment Results:The Who, the What, the Where and the How
BRIC Las PositasRob Johnstone, Bob Gabriner, Greg Stoup, RP GroupBased on the work of Bob Pacheco & Fred Trapp, RP Group
An initiative of the Research & Planning Group
for California Community Colleges
2
What Sparked these Questions?
The ACCJC Institutional Effectiveness Rubric (Part III Student Learning Outcomes)
Proficiency Comprehensive assessment reports exist and are
completed on a regular basis.
An initiative of the Research & Planning Group
for California Community Colleges
3
Where Did We Look? With Whom Did We Consult?
ACCJC. Institutional Effectiveness RubricACCJC. 2002 StandardsACCJC. ThemesACCJC. Guide to Evaluating InstitutionsProfessional literatureEfforts of national groups/institutesInstitutional web sites, listservs and
colleagues Around the country 4-years, 2-years, for-profit.
An initiative of the Research & Planning Group
for California Community Colleges
4
Our Review Showed
Five Overarching Uses of Assessment Evidence:
1. Help faculty explore the student learning process.
2. Determine the extent to which the curriculum is working.
3. Where can time, energy and/or money be allocated for continuous improvement in learning?
4. Exploit the writing process and dialogue about results to gain broader institutional learning experiences.
5. Help meet the quality assurance pledge to the community.
An initiative of the Research & Planning Group
for California Community Colleges
Area 1: Help faculty explore the student learning process.
An initiative of the Research & Planning Group
for California Community Colleges
6
1. Help faculty explore the student learning process.
Happens at the Course LevelCCBC (Maryland)
Middle States Commission on Higher Education
Community College Futures Assembly, Bellwether Award, 2008 Instructional Programs & Services for High Impact
Course Level AssessmentCHEA award winner, 2006
Institutional Progress in Student Learning Outcomes
National Council on Student Development (NCSD) Exemplary Practice Award Winner
An initiative of the Research & Planning Group
for California Community Colleges
7
Projects are at least three semesters long Individual and high-impact courses (all sections)
included Project proposal by a faculty group
Measurable objectives External review & approval in selecting
methods/instrument & analyzing results. Benchmarking should be included if possible.
Controls and sample size considered. Course improvements based on data analysis Reassessment expected Results/report shared across the college and web
posted
1. Help faculty explore the student learning process.
Happens at the Course LevelCCBC (Maryland)
An initiative of the Research & Planning Group
for California Community Colleges
8
Learning Outcomes Assessment Final Report Template1. Design & proposal for the LOA project2. Implementation of design & data collection3. Redesign of the course to improve student learning4. Implementation of course revisions & reassessment
of student learning5. Final analysis and results
Eavesdropping Two-page executive summaries available http://www.ccbcmd.edu/loa/CrseAssess.html
1. Help faculty explore the student learning process.
Happens at the Course LevelCCBC (Maryland)
An initiative of the Research & Planning Group
for California Community Colleges
9
CHEM 108 An initial “failure” turned to success and collaboration
with a four-year schoolHLTH 101
Addressing an achievement gap with professional development and incrEavesd communication with students
CRJU 101 and 202 Statewide group assessment development effort and
creativity in the interventions used
1. Help faculty explore the student learning process.
Happens at the Course LevelCCBC (Maryland)
An initiative of the Research & Planning Group
for California Community Colleges
An initiative of the Research & Planning Group
for California Community Colleges
11
GEG 1 Assessment part of the program review 2008
GEG 1 appendixGEG 1L appendix
Eaves dropping http://www.rcc.edu/administration/academicAffairs/
effectiveness/assess/resources.cfm
1. Help faculty explore the student learning process.
Happens at the Course LevelCCBC (Maryland)
An initiative of the Research & Planning Group
for California Community Colleges
Area 2: Determine the extent to which the curriculum is working.
An initiative of the Research & Planning Group
for California Community Colleges
2. Determine the extent to which the curriculum is working.
Three Levels of Curriculum Development
Design and
Goals
The Teaching
Is it Being
Learned
Assessment Informs this Improvement
If you do not assess, you lose the opportunity to improve.
An initiative of the Research & Planning Group
for California Community Colleges
This Kind of Analysis Happens at the Program Level
The Study of the Learning Process happens at the course level where faculty’s natural curiosity about student learning takes place.
Curriculum analysis occurs at the program level.
An initiative of the Research & Planning Group
for California Community Colleges
15
What Are the Most Common Things Included?
Assessment focus- course, program, general ed, etc.
What outcomes were assessed?How and when were they assessed?Who was assessed?What were the results?Who reviewed the results, made sense of the
them and what conclusions were reached?What are the implications for practice and/or
policy or future assessment work?How were the results used?
An initiative of the Research & Planning Group
for California Community Colleges
16
Program-level ReportingHocking College
North Central Association of Colleges & Schools, Higher Learning Commission
CHEA award winner, 2008 Institutional Progress in Student Learning
Outcomes
An initiative of the Research & Planning Group
for California Community Colleges
17
Hocking College (OH)Program-level Report
All programs have individual assessmentplansMission statement & central objective for assessment
Institutional success skills (GE)Program exit competenciesCriteria for and means of assessment
Reporting of results
An initiative of the Research & Planning Group
for California Community Colleges
18
Hocking College (OH)
Learning outcomes data collected in a student E-portfolio Directing internal and external evidence (1 to 10
measures) Indirect evidence (1 to 4 measures)
Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric Eight general education outcomes (student success
skills) Discipline-specific exit competencies or outcomes
An initiative of the Research & Planning Group
for California Community Colleges
19
Hocking College (OH)Program-level Report
Eaves dropping Cloud reference, not college URL as links
are broken there Various reports available in each program
profile Curriculum matrix Criteria statements (exit competencies) Instructional Program Outcomes (assessment
plan) Trend Charts for performance criteria
An initiative of the Research & Planning Group
for California Community Colleges
Hocking College (OH)Program-level Report
Example reports and analysis Culinary Arts Technology Forestry Management Technology Nursing Technology
20
An initiative of the Research & Planning Group
for California Community Colleges
Area 3: Where can time, energy and/or money be allocated for continuous improvement in learning?
An initiative of the Research & Planning Group
for California Community Colleges
22
Where can time, energy and/or money be allocated for continuous improvement in
learning?
This tends to happen via general education level assessment & reports, or
Another method is to create “institution-level” reports, where the outcomes of course-level, program-level, and general education level or integrated and summarized.
An initiative of the Research & Planning Group
for California Community Colleges
23
General-Education ReportsMesa College (AZ)
North Central Association of Colleges & Schools, Higher Learning Commission
CHEA Award winner, 2007 Institutional progress in Student Learning
Outcomes
An initiative of the Research & Planning Group
for California Community Colleges
24
Mesa CollegeGeneral Education Report
Multiple outcomes assessed Annually
Annual Report elementsExecutive SummaryMethodologyResults & observations (GE & workplace)Indirect measures findingsAppendices of past results
An initiative of the Research & Planning Group
for California Community Colleges
25
Mesa Community College (AZ)
General education studies completed 2007-08; 2005-06 Numeracy Scientific inquiry Problem solving/critical thinking Information literacy Workplace skills (CTE)
General education studies completed 2006-07; 2004-05 Arts & humanities Cultural diversity Oral communication Written communication
An initiative of the Research & Planning Group
for California Community Colleges
26
Mesa College (AZ)General Education Reports
Eaves dropping
http://www.mesacc.edu/about/orp/assessment/index.htmlAnnual reports and summaries available
Eight years of history and experience
An initiative of the Research & Planning Group
for California Community Colleges
An initiative of the Research & Planning Group
for California Community Colleges
28
General-Education ReportsMesa College (AZ)
New England Association of Schools and Colleges, Commission on Institutions of Higher Education
Cited in the Art and Science of Assessing General Education Outcomes: A Practical Guide (AAC&U, 2005)
An initiative of the Research & Planning Group
for California Community Colleges
29
Capital CC (CT)General Education Reports
Multiple reports One per outcome Each study commonly takes a year
Report elements Introduction Methods Results and findings Conclusions and recommendations Implications for future assessments Appendices of assignment, rubric, notes to
teachers, etc.
An initiative of the Research & Planning Group
for California Community Colleges
30
Capital CC (CT)General Education Reports
General education studies completed Writing, 2001-02 Math, 2002-03 Critical thinking, 2003-04 Global perspective, 2004-05
Eaves dropping Annual reports and summaries available http://www.ccc.commnet.edu/slat/
An initiative of the Research & Planning Group
for California Community Colleges
An initiative of the Research & Planning Group
for California Community Colleges
32
Portland CC (OR)General Education Reports
Northwest Accrediting CommissionEaves Dropping
http://www.pcc.edu/resources/academic/learning-assessment/
One general education theme a year Learning Assessment Focus for 2009-10- Critical
Thinking & Problem Solving Physical Science, Geology and General Science Bioscience Technology Management and Supervisory Development Culinary Assistant Program
An initiative of the Research & Planning Group
for California Community Colleges
33
Truman State University (MO)Various Reports
Southern Association of Schools and Colleges, Commission on Colleges
Eaves Dropping Assessment work began in 1970; National Leader http://assessment.truman.edu/
Assessment Almanac- A compilation of results from each year’s assessment work (versions from 1997 to 2009 are posted)
General Education outcomes are assessed in the context of the major field of study Portfolio Project- required of all seniors to show best work
assessed by faculty for the nature & quality of the liberal arts and sciences learning outcomes (versions from 1997 to 2008 are posted)
An initiative of the Research & Planning Group
for California Community Colleges
34
Who Writes the Reports?
Course-level, program-level & general education Teaching faculty study team with technical assistance
from institutional research or assessment committee
No “lone ranger” authorsInstitutional summary
Academic administrator with assistance from Learning outcomes coordinator or assessment committee
Compilation of work accomplished in one or two academic years across the institution
An initiative of the Research & Planning Group
for California Community Colleges
35
Where Are the Reports?
Location of course, program and general education comprehensive assessment reports Teaching faculty study team members, assessment
committee chair, assessment websiteLocation of institutional summary reports
Academic administrator, assessment committee chair, assessment website
Not in the library basement, actively used to promote a learning organization
An initiative of the Research & Planning Group
for California Community Colleges
Area 4: Exploit the writing process and dialogue about results to gain broader institutional learning experiences.
An initiative of the Research & Planning Group
for California Community Colleges
37
To all affected participantsCampus committees
Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership
Campus fairs, brown-bag lunches, poster sessions for information sharing
Faculty professional development programsAccreditation self-study committee work groupsCollege web site for the public
3. Exploit the writing process and dialogue about results to gain broader institutional learning
experiences.
An initiative of the Research & Planning Group
for California Community Colleges
38
Learning organization Environment that promotes a culture of learning Individual & group learning enriches & enhances the
organization as a whole Systematic problem solving using data for decisions Learning from experiences in assessing organizational
performance Comparing yourself to others (benchmarking) and
borrowing ideas
Adriana Kezar ed. Organizational Learning in Higher Education New Directions for Higher Education. No. 131, Fall 2005. Jossey-Bass.
Exploit the writing process and dialogue about results to gain broader institutional learning
experiences.
An initiative of the Research & Planning Group
for California Community Colleges
Characteristics of Organization Learning
Researchers have found some critical features of learning organizations (Lieberman, 2005, pp. 87-98). In particular, a college as an effective organizational learner:
Maintains a scholarly approach to the questions and problems that the institution faces;
Approaches the campus problems as learners and not as experts;
Develops a culture of evidence that drives decision-making;
Links the organizational learning to the college’s mission;
Makes connections throughout the college and not just as individual units (e. g. , faculty, administration); and
Recognizes and rewards the college’s efforts to become a “learner.”
39
An initiative of the Research & Planning Group
for California Community Colleges
40
Exploit the writing process and dialogue about results to gain broader institutional learning
experiences.
The assessment data sense-making process = a faculty learning experience
Linking results to future interventions = a learning experience for Faculty, assessment committee, academic
administration, planning & resource allocation groupsUsing results to inform an intervention, then reassess =
a learning experience (accomplished one or more terms later) for Faculty, assessment committee, academic
administrationReference for future assessment work and other groups
on campus
An initiative of the Research & Planning Group
for California Community Colleges
41
Exploit the writing process and dialogue about results to gain broader institutional learning
experiences.
Hocking College (OH) Student E-portfolio Annual summary
Improvements in the program in the previous year brought on by study of assessment results
Expenditures of time, money & materials for the assessment program
Requests for assistance in implementing assessment Recommendations for altering the institution’s
assessment process Transition from evaluating individual students to
assessing groups of students & the curriculum experience
An initiative of the Research & Planning Group
for California Community Colleges
42
Community College of Baltimore County (MD) Learning Outcomes Assessment Advisory Board
Links findings in assessment reports to other college-wide initiatives and professional development opportunities
Use of assessment processes and (findings) results Challenged faculty to reexamine prompts used in
assessment Clarity of written prompt & extent it supports program
goals Common assignment options and common rubric
increases faculty understanding and buy-in Builds faculty unity toward common goals Public web page enhances communication and
accessibility to information
Exploit the writing process and dialogue about results to gain broader institutional learning
experiences.
An initiative of the Research & Planning Group
for California Community Colleges
43
CHEA award winner, 2010 Institutional Progress in Student Learning
Outcomes
Exploit the writing process and dialogue about results to gain broader institutional learning
experiences.
An initiative of the Research & Planning Group
for California Community Colleges
44
Reports as Institutional Learning& Resource Allocation
Northern Arizona University- Seals of Assessment Achievement & Excellence
Purpose:To recognize programs that have
demonstrated significant progress with assessing student learning
To promote “best practices” in assessment by sharing practical experiences
To encourage programs to showcase program-level achievements and to adjust curricula when appropriate.
An initiative of the Research & Planning Group
for California Community Colleges
45
Reports as Institutional Learning& Resource Allocation
Feedback & recognitionFeedback rubric for annual assessment
reports Conversations and action Collection and analysis of evidence Implementation of findings
Recognition (achievement & excellence)
An initiative of the Research & Planning Group
for California Community Colleges
46
Reports as Institutional Learning& Resource Allocation
Seal of Assessment AchievementAcademic programs earning this recognition
have demonstrated in their annual report that learning outcomes have been assessed through two or
more methods, and findings have been discussed among
the faculty.
An initiative of the Research & Planning Group
for California Community Colleges
47
Reports as Institutional Learning& Resource Allocation
Seal of Assessment Excellence Academic programs earning this recognition
have demonstrated a thorough implementation of assessment plan(s) the reporting of meaningful assessment data the discussion of findings among faculty and perhaps students the use of findings to showcase student achievements and to make curricular adjustments.
An initiative of the Research & Planning Group
for California Community Colleges
48
Reports as Institutional Learning& Resource Allocation
Mesa College (AZ)Results Outreach Committee
Promotes use of outcomes data in relation to faculty development, pedagogy and academic climate
Groups of faculty offer a proposal for summer or academic year work above the course level
Resulting report placed on the web and used for campus discussion and action
An initiative of the Research & Planning Group
for California Community Colleges
Area 5: Help meet the quality assurance pledge to the community.
An initiative of the Research & Planning Group
for California Community Colleges
50
5. Help meet the quality assurance pledge to the community.
National Institute for Learning Outcomes Assessment (NILOA) Assists institutions & others in discovering & adopting
promising practices in the assessment of college student learning outcomes.
Documenting what students learn, know and can do is of growing interest to colleges and universities, accrediting groups, higher education associations, foundations and others beyond campus, including students, their families, employers, and policy makers.
An initiative of the Research & Planning Group
for California Community Colleges
NILOA 2010 Webscan report Exploring the Landscape: What
Institutional Websites Reveal About Student learning Outcomes Assessment Activities
2010 Connecting State Policies on Assessment with Institutional Assessment Activity
Eaves dropping www.learningoutcomeassessment.org
51
Help meet the quality assurance pledge to the community.
An initiative of the Research & Planning Group
for California Community Colleges
52
Promising Vehicles for Expanding Information to the PublicBrief narrative report from annual
assessment reportsSimple statistical reports on learning
outcomes or surveysBest practices stories supported by
assessmentPeter Ewell. Accreditation & the Provision of Additional Information to the Public about
Institutional and Program Performance, CHEA, May 2004
Help meet the quality assurance pledge to the community.
An initiative of the Research & Planning Group
for California Community Colleges
53
Voluntary System of Accountability APLU & AASCU (520 public institutions, award 70% of
bachelor’s degrees in the US each year) Started 2007 College Profile (includes learning outcomes & links to
campus) Proactive initiative to document learning gains and
average institutional scores (choice of 3 national instruments)
Proactive initiative to illustrate unique campus learning outcomes assessment work
Promoting a learning institution Eaves Dropping
http://www.collegeportraits.org/
Help meet the quality assurance pledge to the community.
An initiative of the Research & Planning Group
for California Community Colleges
54
VSA Example, Cal Poly Pomona
http://www.collegeportraits.org/map Cal Poly Pomona
http://www.csupomona.edu/~academic/programs/ge_assessment/
Help meet the quality assurance pledge to the community.
An initiative of the Research & Planning Group
for California Community Colleges
55
Quality Assurance to the Public
National Association of independent Colleges and Universities Assessment programs on campus tied to institution’s
mission Eaves Dropping
http://www.naicu.edu/special_initiatives/accountability/Student_Assessment/id.514/default.asp
Pepperdine University http://services.pepperdine.edu/oie/learning-outcomes/
learning-outcomes-overview.aspx
An initiative of the Research & Planning Group
for California Community Colleges
56
Where Can I Go?Resource
Filesanywhere.comhttp://www.filesanywhere.com/fs/v.aspx?
v=8a6b678c606573bdac6b
Top Related