2006 Fall Workshop
PLANNING and ASSESSMENT:A TUTORIAL FOR NEW DEPARTMENT CHAIRS –
A REFRESHER COURSE FOR OTHERS
Planning and Assessment Workshop
Session 2
Assessment of Student Learning Outcomes and Departmental Goals
T. Gilmour Reeve
Director of Strategic Planning
Kari Wood
Coordinator, Teaching, Learning, & Technology Center
2:00 – 3:45
November 10, 2006
OUTCOMES ASSESSMENTS OF STUDENTS’ LEARNING
Two boys are walking down the street. The first boy says, “I’ve been really busy this summer. I’ve been teaching my dog to talk.”
His friend responds, “Wow! I can’t wait to have a conversation with your dog.”
The first boy shakes his head. “I said I’ve been teaching him. I didn’t say he learned anything.”
M.J. Allen
Assessing Academic Programs inHigher Education
Planning and Assessment CyclePlanning
(purpose & goals)
Implementation
courses, programs, strategies
Assessment
(measurement & evaluation)
Planning and AssessmentPossible Responses to Assessment:
1. No changes needed; 2. Change Purpose and Goals;3. Change Programs, research, service; 4. Change
Assessments.
Planning
purpose & goals
Implementation
Courses, programs, strategies
Assessment
measurement & evaluation
Levels of Educational Assessment
o InstitutionalSACSCOC – “Institutional Effectiveness”
Addresses all areas and units in delivering all programs and services
o Academic departments and programsDepartmental goals and expected student outcomes for degree programs
o Individual coursesStudent assessment in specific courses & teacher evaluations
Planning and Assessment Workshop – Session 2
There are three basic aspects to these SACSCOC standards:
o The institution DOES planning and assessment;
o The institution USES planning and assessment to improve programs and services;
o The institution DOCUMENTS the planning and assessment activities and the improvements made.
Planning and Assessment Workshop – Session 2
Texas Tech University:
Planning and Assessment
University Policies: OP 10.13 – Strategic Planning and Assessment
“The university, including all academic programs and support operations, is engaged in an ongoing and comprehensive process of planning and assessment. All areas (divisions and colleges) and units (departments, centers, and institutes within areas) must conform to the university policies as specified in this OP.”
Planning and Assessment Workshop – Session 2
Texas Tech University:
Planning and Assessment
University Policies: OP 10.13 (cont.)“Academic programs, including program content, quality,
and assessment, are the primary responsibility of the faculty within the academic disciplines. Learning outcomes assessments of academic programs are included in the departments’ annual assessment reports.”
Planning and Assessment Workshop – Session 2
OP 32.06 – Faculty Responsibility
Specifically, within the first week of the semester, faculty members must provide a course syllabus that includes: a. The course outline;
b. The expected learning outcomes from the course;
c. The methods of assessing those outcomes; and
d. The criteria for grade determination.
Assessment ofStudent Learning
Outcomesin the Courses
Related Terms
(1)TestingAssociated primarily with traditional methods of assessment
(2) EvaluationRequires the interpretation and use of information to make judgments about student learning and/or pedagogy
(3) GradingMay provide insufficient evidence of student learning…Why?
Assessment is an ongoing process of:
1)Establishing clear, measurable outcomes of student learning;
2) Ensuring that students have sufficient opportunities to achieve those outcomes;
3) Systematically gathering, analyzing, and interpreting evidence of student learning; and
4) Using the resulting information to understand and improve the learning process.
Definition
A learning outcome is a specific statement that describes:
(1) knowledge,(2) skills, and(3) attitudes
that students are expected to learn upon successful completion of a course.
Examples
After completing this course, students will be able to:
Art Appreciation Differentiate major works of art
based on the period they represent.
Introduction to Research Methods
Compare and contrast quantitative and qualitative research methodology.
GuidelinesEffective learning outcome
statements:
A = Use concrete, action verbs
C = Are specific to the course
E = Focus on the end, not the means
S = Are student-centered
+A = Are assessable
ACES+A
Action Verbs: Cognitive Domain
Remember
Understand
Apply Analyze Evaluate Create
CountDefineDescribeIdentifyLabelListOutlineQuoteReciteSelect
ComputeConvertDefendDiscussEstimateExplainInferParaphrasePredictSummarize
CalculateClassifyDemonstrateGraphModifyPrepareProduceSolveTranslateUse
AnalyzeArrangeCombineDevelopDiagramDifferentiateDiscriminateIllustrateRelateSeparate
AppraiseAssessCompareContrastCritiqueInterpretJudgeRateSupportTest
ComposeCompileCreateDesignDeviseGenerateIntegratePlanProposeReconstruct
Adapted from Anderson and Krathwohl’s (2001) revised version of Bloom’s taxonomy
Action Verbs: Other Domains
Affective: Argue, convince, debate, express, internalize, justify, support, uphold
Psychomotor: Assemble, build, dissect, dribble, sculpt, repair, sketch, suture
Outcomes are course-specific.
Limit your learning outcomes to 5-10 statements for each course.
These statements will focus on the most important outcomes for the course, not the program or an individual unit.
ACES+A
Focus on the end, not the means.
ACES+A
What should students know or be able to do upon successful completion of the course?
NOT In what activities will students
participate throughout the course?
Outcomes are student-centered.
ACES+A
What should students know or be able to do upon successful completion of the course?
NOT What will instructors do
throughout the course?
Outcomes are assessable.
ACES+A
How can the outcome be measured?
How can we use the information to improve the learning process?
Assessment is an ongoing process of:
1) Establishing clear, measurable outcomes of student learning;
2)Ensuring that students have sufficient opportunities to achieve those outcomes;
3) Systematically gathering, analyzing, and interpreting evidence of student learning; and
4) Using the resulting information to understand and improve the learning process.
Assessment is an ongoing process of:
1) Establishing clear, measurable outcomes of student learning;
2) Ensuring that students have sufficient opportunities to achieve those outcomes;
3)Systematically gathering, analyzing, and interpreting evidence of student learning; and
4) Using the resulting information to understand and improve the learning process.
Best Assessment Practices
1) Assess before, during, and after student learning has occurred.
2) Align assessment methods with the stated learning outcomes.
3) Use a variety of assessment methods.
Framework of Assessment Methods
Traditional Alternative
Selected-Response
Constructed-Response
Product Performance
Process-Focused
Multiple-choiceTrue-FalseMatching
Fill-in-the-blankShort answerLabel a diagram“Show your work”Essay
Research paperLab reportStory/playPoemPortfolioArt exhibitScience projectModelVideo/audiotapePoster presentationPowerpoint
DanceRecital DemonstrationAthletic skillDramatic readingEnactment
DebateTeach a lessonObservationInterviewPresentationDouble-entry journal
http://newton.nap.edu/html/classroom_assessment/ch4_t1.html
Classroom Assessment Technique (CAT)
Short-Writes:
o One Minute Paper: What was the most important thing you learned in class? What question remains unanswered?
o Muddiest Point: What was most confusing about your reading assignment last night?
o Application Cards: Describe one practical application of what we’ve discussed today.
Assessment is an ongoing process of:
1) Establishing clear, measurable outcomes of student learning;
2) Ensuring that students have sufficient opportunities to achieve those outcomes;
3) Systematically gathering, analyzing, and interpreting evidence of student learning; and
4)Using the resulting information to understand and improve the learning process.
Closing the Loop
Are the learning outcomes appropriate?
Is the curriculum aligned with the stated learning outcomes?
Are your instructional methods effective?
Closing the Loop (cont.)
“There is no such thing as effective teaching in the absence of learning. Teaching without learning is just talking.”
-Angelo & Cross (1993)
Planning and Assessment Workshop – Session 2
ASSESSMENT OF STUDENT LEARNING IN THE COURSE
QUESTIONS AND COMMENTS?
Planning and Assessment Workshop – Session 2
ASSESSING STUDENT LEARNING IN THE ACADEMIC PROGRAMS
(PROGRAM-LEVEL ASSESSMENT OF STUDENT LEARNING OUTCOMES)
Planning and Assessment Workshop – Session 2
The term “program-level” refers to learning outcomes and assessment methods related to a degree programProgram-level assessment investigates what the students are learning in their degree programs
Program-level assessment is not the same as the course grades for students.
Planning and Assessment Workshop – Session 2
“Seventy-five percent of students will earn a grade of X [“C” or “B”] or better in Course 1234”
Concern for using course grades as measures in program level assessment:
o A grade is an aggregate of the student’s performances on class assignments (not directly tied to a specific expected outcome)
o Different students will earn the same course grades but will perform differently on course assignments
o Focusing on grades for assessing programs may encourage grade inflation
o Using a minimal course grade of “C” only states what is required for graduation
Planning and Assessment Workshop – Session 2
Program-level learning outcomes and assessment are reflected in the Strategic Plan – on the goals related to undergraduate and graduate education
But the expected outcomes for each degree program do not have to be stated in the STRATEGIC PLANS but are reported in the ANNUAL ASSESSMENT REPORTS
Planning and Assessment Workshop – Session 2
o The first step in writing and assessing program-level learning outcomes is determining the purpose of the degree program
o It is important to engage faculty in this (and all subsequent) steps in the process
Planning and Assessment Workshop – Session 2
Questions to ask to help determine the course purpose include:
o What role does this program play in the department or college?
o What is unique or special about this academic area/degree program?
o What will the students “do” with their degrees?
Planning and Assessment Workshop – Session 2
Once the program’s purpose has been established, specific learning outcomes can be developed
A program-level expected learning outcome is a formal statement of what the program’s faculty expects students to learn in a particular degree program
Planning and Assessment Workshop – Session 2
Expected learning outcomes define:
o What faculty expect students to know at the end of the degree program
ANDo What faculty expect students to
be able to do at the end of the degree program
Planning and Assessment Workshop – Session 2
Once the learning outcomes for a degree have been established, a variety of methods are used to assess if the students’ learning matches those expected outcomes.
Planning and Assessment Workshop – Session 2
Once the learning outcomes for adegree have been established, a
varietyof methods are used to assess if
thestudents’ learning matches those
expected outcomes.
Planning and Assessment Workshop – Session 2
Departments should develop assessment plans that outline
how and when they will be investigating how well both the current students and students
who have graduated have learned/mastered the program-
level learning outcomes
Planning and Assessment Workshop – Session 2
Methods to Assess Current Students’ Learning
o Capstone courseso Review of senior projectso Focus groupso Students’ performance on
comprehensive exams
Planning and Assessment Workshop – Session 2
Methods to Assess Graduated Students’ Learning
o Graduate follow-up surveyso Performance on licensure examso Job placement datao Exit interview/surveys
Planning and Assessment Workshop – Session 2
Strategies for using outcomes assessments of student learning:
o Engage the facultyo Begin with discussions at first faculty
meetings of the yearo Use available resources (TLTC and Office
of Strategic Planning – see websites)o End the year with a faculty review –
make improvements based on assessments .
CLOSE THE LOOP!
Planning and Assessment Workshop – Session 2
See examples in handouts for incorporating outcomes assessments in the Annual Assessment Reports
Also additional information on outcomes assessment is provided in the handouts
Questions and comments?