Module 2

Norming Expectations and Assessments for Student Learning

2.15.05

Cycle ofAssessment

andImprovement

ChooseImprovement

TakeStock

NORMSTUDENTLEARNINGOUTCOMES

NormAssessments

MakeImprovements

andEvaluate

BuildInstitutional

Capacity

IdentifyImprovement

Strategies

Cycle ofAssessmentImprovement

ChooseImprovement

TakeStock

NormStudentLearningOutcomes

NORMASSESSMENTS

MakeImprovements

andEvaluate

BuildInstitutional

Capacity

IdentifyImprovement

Strategies

NB —— note well

Separating -student learning outcomes

from

-student learning outcomes assessment

-wastes time, -confuses the coherence of programs, and-looses opportunities for

collaboration and collective norming

Step by Step:

ImprovingStudent Learning

Step 1: Dialogue• Dialogue about a department’s

philosophy and assumptions of teaching and learning should guide the choice of student learning outcomes:– A developmental philosophy would set

increments of increasingly complex SLOs

– A problem-based learning philosophy would implement complex, “messy” learning outcomes in every course

ACTIVITY: DIALOGUE1

1. What are the philosophies or principles about teaching and learning on which we’ve built our curriculum, instructional design, pedagogy, or use of educational tools?

2. What methods of teaching or educational experiences develop the knowledge, understanding, habits of the mind, ways of knowing, and problem solving that this discipline, profession, trade, program or institution values?

3. How do we impart those ways of thinking, knowing and problem solving to students?

4. Which students benefit from which teaching strategies, educational processes, or educational experiences?

1Maki, P.L. (2004). Assessing for Learning. American Association for Higher Education, Sterling, VA: Stylus.

On poster paper

• Write at least one response to each bulleted question.

Step 2: Norm Expectations• Faculty discuss and debate what

students should know and be able to demonstrate upon completion of one course or the program.

• Expectations are collaboratively

authored and collectively accepted. Full- and adjunct/ part-time faculty who teach a course come to consensus.

1Maki, P.L. (2004). Assessing for Learning. American Association for Higher Education, Sterling, VA: Stylus.

ACTIVITY: DIALOGUE ABOUT EXPECTATIONS

• What are the critical minimum learning/ performance outcomes for students in this course?

• What do students need to progress to the next course in this sequence? (Do students usually take more courses in this sequence?)

• What outcomes do students need to meet their own goals (vocational certificate, AAS, transfer?)

• What outcomes has institution defined?

On poster paper

• Write at least one response to each bulleted question.

Collectively Set Expectations for a Course

EXPECTATIONS• use active verbs that identify student

performance: create, apply, construct, formulate

• align with desired outcomes for program or degree

• align with disciplinary philosophy of learning• are collaboratively authored and collectively

accepted• incorporate professional, licensure, or

standardized outcomes• can be assessed qualitatively or

quantitatively

Step 3: Norm and Implement Assessment(s)

• Align assessment procedures with programmatic philosophy/ assumptions about learning- What are the values of program?

• Collectively identify evidence/ criteria/ performance indicators/ primary traits

• Collectively set rubric for scoring

Assessment should fit the nature of a program’s goals using methods which reflect the type of learning that is valued, rather than methods which are most easily constructed or scored. For example, if courses emphasize open-ended problems, the assessment procedure should emphasize the same objective. Facing students with a multiple-choice test to measure open-ended problem solving would be inappropriate.

<www.maa.org/past/ql/ql_part4.html>

Using Criteria/ Standards/ Traits

• Share with students• Consistently grade each student’s work

according to criteria and rubric• Guide students in self-use of criteria• Collect and analyze outcomes• Use outcomes to improve teaching• Share learning activities, outcomes,

assessments, criteria, rubrics with other faculty who teach this lesson

1Adapted from Scroggins, B. (2004). Targeting Student Learning. Modesto,CA: Modesto Junior College.

Criteria/ Standard/

Primary Trait

Excellent Superior Satis-factory

Poor Unsat

STARTHERE

BUILDING A RUBRIC1

1Adapted from Scroggins, B. (2004). Targeting Student Learning. Modesto ,CA: Modesto Junior College

ACTIVITY: MORE DIALOGUE1

What do faculty do now?• How do you determine an A from a B from a C;

acceptable from not acceptable? (“I know it when you see it” versus written criteria for grading every assignment?

• Are your criteria easy enough to use that you use them every time? (consistent use)

• Do you share criteria and rubrics with students before assignments? Do you encourage students to self-assess their work using the criteria and rubrics?

• How do you use the results of assessment, of criteria to improve your assessment methods, criteria, and/or rubrics?

1Adapted from Scroggins, B. (2004). Targeting Student Learning. Modesto,CA: Modesto Junior College

REVIEW: Thinking through SLOs

• What outcomes do you want for your students?

• From whom do you want these outcomes? (targeted ~ all)

• When do you want these outcomes? (entry — middle — exit)

• What evidence will be credible and agreed upon by faculty? (assessment method)

• What criteria is acceptable? (criteria)

ACTIVITY: NORM SLOs

•DRAFT SLOs components (columns), using one of the formats offered or making one to meet your local needs.

Step 4: Compile and share results

• Gather results/ data across sections of a course and/or across courses within a program and/or across courses in a sequence

• Share and discuss results; identify gaps in student learning; discuss effectiveness of current teaching and assessment methods

• Use accessible terminology and formats for reporting data

Analyzing Results

• Plotting students’ achievement along agreed-upon criteria/ standards/ traits focuses attention on patterns of strengths and weaknesses

• Visually presented using comparative tables focuses attention on why some groups do well while others do not

• Focus attention on WHY?

SLO Statements

Assessment Methods/Criteria

Results of Assessments

ACTIVITY: COMPILING RESULTS

IdentifyingImprovement

Strategies

Cycle ofAssessment

andImprovement

ChooseImprovement

TakeStock

NormStudentLearningOutcomes

NormAssessments

MakeImprovements

andEvaluate

BuildInstitutional

Capacity

IDENTIFYIMPROVEMENT

STRATEGIES

Collecting data

≠Improvement in student learning

outcomes

Step 5: Collectively set improvement plan

• Collectively identify patterns of weakness in student learning

• Allow time to reach consensus about “Why” and about changes/ improvements to address gaps

• Collectively distribute specific tasks and locations for piloting improvements

Potential improvement areas

• Improvements in instruction• Improvements in curriculum/

content• Improvements in student support• Changes in motivation and

engagement of students???

Instructional Improvement

• More active forms of learning, including projects, applications, simulations

• Decreased use of lecturing• Alignment between values/ desired

goals of program and pedagogy/ assessment procedures

• • may also improve motivation and engagement of students

ACTIVITY: IMPROVING INSTRUCTION

• How does your institution focus on instructional skills of faculty now?

• Are there opportunities for faculty to observe one another, team teach, video and review their own teaching, or “practice” effective pedagogy in other ways?

• What is your institutional culture surrounding teaching? (I never hear chalk anymore; I hear lots of chairs moving; I don’t see instructors in front of the room, etc)

• What role does pedagogy play in hiring and personnel evaluation at your campus?

On poster paper

• Write at least one response for each question.

Curricular Improvement

• Interdisciplinary connections (increases relevance)

• Learning communities of linked courses (increases “community”)

• Infusion of applications into liberal arts or infusion of math/communication skills into career-technical courses

• • interdisciplinary connections may improve student motivation and engagement

Appropriate improvements

• Students don’t connect what they learn in one course with content/ concepts of another course: link courses

• Developmental students don’t connect learning activities with their own goals: contextual courses

• Even students who are placed into math or English courses can’t do the required work: re-evaluate placement instrument

• Technical students don’t have necessary basic skills: contextual courses; infusion of academic into technical content

ACTIVITY: CURRICULAR IMPROVEMENT

• Describe previous curriculum reform endeavors on your campus. What gaps in student learning were these changes to address? What data did you collect and analyze? How effective was/ were those curriculum reform(s)?

• Which curriculum improvements seem promising to meet the learning gaps you’ve identified?

On poster paper

• Write at least one response for each question.

• Student services decentralized: connected to disciplines, departments

• Student services centralized: one stop shop

• Student services faculty engaged in teaching

• How does student services contribute to student learning outcomes?

Student services improvement

ACTIVITY: IMPROVING LEARNING VIA STUDENT SERVICES

• Describe previous reforms in student services on your campus.

• What gaps in student learning were these changes to address? What data did you collect and analyze? How effective was/ were those student services reform(s)?

• Which student services revisions seem promising?

On poster paper

• Write at least one response for each question.

Improvement in Student Motivation and

Engagement

• Difficult to address directly• Usually results from improvements in

instruction, curriculum or student services

• • Many efforts to “fix” students contradict the CCC mission

Moving from data to improvement

• Where will your campus target its initial efforts?– target population– content area– program

• Which strategy(ies) are aligned with those improvement goals?

• Which strategy(ies) appear to offer the best cost/benefit?

Reporting Out

• Prioritize your efforts. • Where will you put your first

efforts to improve student learning: improvements in instruction, curriculum, student support, or??

• Why?

EvaluatingImprovement

Strategies

Cycle ofAssessment

andImprovement

ChooseImprovement

TakeStock

NormStudentLearningOutcomes

NormAssessments

MAKEIMPROVEMENTS

ANDEVALUATE

BuildInstitutional

Capacity

IdentifyImprovement

Strategies

Making and Evaluating Improvement Strategies

• Improvement strategies are aligned with student learning gaps

• Baseline data provides comparison• Collaboratively author and

collectively accept new outcomes, learning activities, curriculum/ student services reform

• Collect data on effectiveness of improvement strategies

Step 6: Continue SLOA Cycle

• Reassess how well students improve based on implemented changes

• Periodically update assessment plans that build in a cycle of inquiry

Mapping Backwards from Outcomes to Learning Activities

• Establish desired outcome• Then credible assessment• Then criteria/ primary traits• Then rubric for scoring • Then content/ learning

activities that align with outcome and assessment

Confusion: Where to Start??

• Some colleges start with institutional outcomes: Gen Ed

• Some colleges start with Perkins/ VTEA funded occupational programs

• WHERE is not nearly as important as STARTING because this is continuous professional learning to improve student learning

• Ready, Fire, Aim as you gain experience

Homework Report

• How does Program Review compare to SLOACs?

• What benefits/ disadvantages would there be for revising Program Review to incorporate some/all SLOAC data-driven decision making?

Norena Norton Badway, Ph.D.Principal

Phone 209-951-7477 home office 209-946-2168 University office 209-601-7121

Email badway@aol.comnbadway@pacific.edu