Using Portfolios for Program Assessment
description
Transcript of Using Portfolios for Program Assessment
Using Portfolios for Program Assessment
Sakai ConferenceMay 31, 2006
Judy Patton, Candyce Reynolds, and Wende Morgaine
Portland State University
Agenda
E-portfolio for institutional assessment and accreditation
E-portfolio for program assessmentE-portfolio as a pedagogical tool for
learningMoving from web-based portfolio
development to OSP
Assessment Practices
Assessment Serving at least 3 goals:
Institutional---------Programmatic---------Classroom/Pedagogical
The nature of interpretation, meaning of the assessment changes as a function of the audience and the goal of the assessment.
Portland State University
Urban university Doctoral intensive Seven colleges/schools – Arts and Sciences,
Urban and Public Affairs, Business, Fine and Performing Arts, Engineering and Computer Science, Grad School of Social Work, Grad School of Education
24,000 students 80% undergraduate
Key Influences – Research on own students
High percentage of first generation students Broad range of prior academic experiences/preparation High percentage of returning/adult students High proportion of students who began college work
elsewhere Majority of students commute/ no on-campus “place” Majority of students work half-time or more/ off-campus Large number who attend part-time and take more than
6 years to graduate
PSU Assessment Cycle
Using Portfolio Thinking for Accountability and Improvement
Electronic institutional portfolios initiated with Urban Universities Portfolio Project
UUPP had reform of accreditation self-study process as a goal
Institutional portfolios allow institutions to show multiple examples of their work and offer a view of development over time
Web provides a medium for showing the interconnectedness of institutional work
www.portfolio.pdx.edu
Table 1. Basic principles of the biological sciences (see text) as met by individual courses.
Biology Courses * 1 2 3 4 5 6 7 8
101-103 Gen Bio/lab X X X X X X X X
175 Evol Concepts X X X
252,253 Princ Bio X X X X
301, 302, 303 Hum AP X X X X
336 Cell Biology** X X X X
338 Intro Molec Bio X X X X X X
341 Genetics** X
357 Gen Ecology** X X X
387 Vert Zoo X X X X X
412 Animal Behav X X X X X
Student e-Portfolios
Purpose of University Studies - PSU
The purpose of the general education program at PSU is to facilitate the acquisition of knowledge, abilities, and attitudes which will form a foundation for lifelong learning among its students.
PSU University StudiesPSU’s 4 level general education programFour goals:
Inquiry and Critical Thinking Communication Diversity of Human Experience Ethical and Social Responsibility
UNIVERSITY STUDIES REQUIREMENTS
45 credits
credit
15
12
12
6
TT210
FRESHMAN INQUIRY
UPPER DIVISION CLUSTER
SENIOR CAPSTONE
SOPHOMORE INQUIRY
1 2 3
1 2 3
Why Portfolios?
The best evidence of student learning is found in actual student work.
A tool for organizing, assessing, and displaying student work is the student portfolio.
Benefits of the Portfolio
Portfolio-based assessment systems are built upon organized collections of actual student work.
Portfolios provide a means to evaluate student progress and development as well as achievement.
Portfolios provide a means for all students including those with different learning challenges to document and display their achievements.
Portfolios provide an opportunity for students to document their learning both in the classroom and in their participation in outside activities.
Why Portfolios?
Portfolios contribute to student learning.Electronic portfolios are collections of
digitally represented artifacts that: Document practice, Include reflection, Integrate experience, Map to goals and/or standards.
Key Performances:
“A key performance is the work (evidence) a student submits to demonstrate progress toward or achievement of a learning goal.”
For example: a research paper, exam, a creative work, a taped oral presentation, a business plan, or the results of an experiment.
Includes supporting material such as student self-reflection, peer review, and faculty comment.
Rubrics: Rubrics are scoring systems which define the
evidence (see “Key Performance”) needed to demonstrate achievement of particular learning goals set by the major and/ or the institution.
Rubrics are diagnostic (not just the student’s best work) allowing us to pinpoint student progress (or lack thereof) and achievement.
E-Portfolio Benefits: Electronic vs. Hard-Copy Hyperlinks – (encourage metacognition). Multiple Goals (critical thinking, writing, visual
communication, group work). Curricular and Extra-curricular—includes
learning which occurs within, between, and outside the classroom.
Technical Skills.
E-Portfolio Benefits: Electronic vs. Hard-Copy Scholarship as Public enterprise. Public Assessment. Audience Consideration. Students Working with Students – peer tutoring
and interaction.
Main point
Value on student work and voiceEmphasis on using assessment as a
means to give student a sense of their capacity
Most importantly, assessment serving the need of student learning.
Benefits to Students:
Students understand and can articulate what they have learned.
Students can organize and display key performances and other material to demonstrate what they know and can do.
Digital medium allow a variety of kinds of work samples (i.e. art, oral presentation, performances…).
Benefits to Students:
Digital structure gives students the flexibility to organize their presentation portfolios in different ways for different audiences.
Provides students with the opportunity to communicate their accomplishments to graduate programs and employers in ways that give meaning to the transcript and grade point average.
Assessment of Portfolios
Freshman Inquiry Rubrics Locally developed six point rubrics for 5 areas Average
freshman would be expected to score at Levels 2- 3 Portfolio review process
Faculty across campus serve on committee 1 rubric per day Norming/calibration 2 raters for each assignment 3Rd rater if scores do not match
Summative evaluation for program Formative assessment for faculty teaching teams
PSU Student Portfolio Project:
Four level e-portfolio. Organized around “key performances”. Student progress and achievement determined through
the application of scoring systems or rubrics. Developmental: Learning assessed through actual
student work over time. Diagnostic: Areas of adequate/ inadequate progress can
be targeted and specifically dealt with. Curricular, Co-curricular, and Extra-curricular: Includes
student learning which occurs within, between, and outside the classroom.
Closing the Loop
Data from eportfolio review is shared with faculty teams
Teams are asked to discuss the results and determine what, if any, changes will be made in the curriculum
Reports are sent back to the program administration
Each year, program assessment committee studies annual assessment results and creates a new plan based on current questions.
Portfolio Assignment
Centered around the 4 goals of UNSTGeneral Reflection essayReflection on each goalStudents provide evidence of achievement
in the goal area80% of portfolios are now electronic
Assessment Process
Faculty Responsibility to attach the classroom activities to the larger program goals
Random Sample of Portfolios are gathered (n=150)
Faculty in UNST and in other departments rate achievement using rubrics
Team report Program report Part of University assessment process
Rubrics
Writing (part of Communication goal)Quantitative Literacy (part of
Communication goal)Ethics & Social ResponsibilityDiversity of Human ExperienceCritical ThinkingAll Rubrics—6 point scale
Comparison of Rubric scores
0
0.5
1
1.5
2
2.5
3
3.5
4
writing ethics diversity critical QL
2001
2002
2003
Team Differences
Team Differences
Findings and Responses
Consistency of scores Want Diversity scores to be higher
Faculty Development in area of diversity Teams differed
Team Reports allow reflection and ability to address how they will improve their curriculum in the next year
Learning from each other Quantitative Literacy Diversity
Faculty Development is a key component of improving student learning
Examples