Open Pathways:Testing the Degree Profile

Activity 1Marshall University

January – March 2012

Activity 1(in a nutshell)

First: Select 3 to 5 pre-capstone/culminating experience courses in which course outcomes can be clearly mapped to your degree program outcomes.

Second: Map the course outcomes to the degree program outcomes to the Degree Profile outcomes that are appropriate.

Third:Evaluate course/program outcomes based on mapping and alignment with the Degree Profile and make changes as needed.

At the conclusion of this workshop, participants will– Evaluate/revise measurable student learning outcomes at the

program level.– Determine whether or not these outcomes can be assessed in

students’ capstone or culminating experiences.– Map these outcomes to three to five courses in the program

where they are practiced and measured prior to final assessment in the capstone or culminating experience.

– Map course to program to Degree Profile outcomes for the three to five pre-capstone/culminating experience courses chosen.

– Evaluate course and program outcomes based on mapping.– Make changes as needed.

Activity 1What to consider when reviewing student learning outcomes (Hatfield, 2009).

1. Keep the number manageable – 6 to 8 – think essential learning outcomes upon graduation – higher levels of cognition.

2. Faculty should agree on meaning of outcomes and the criteria to be used for their assessment.

3. Outcomes should be written so that, for example:

Students will (action verb) (something) ―Students will design a creative work representing at

least two disciplines.―Students will develop a position on a significant public

policy issue.

Activity 1What to consider when reviewing student learning outcomes (Suskie, 2009).

Can you measure your student learning outcomes?˗ AVOID: learn, know, understand, acquire, think…..

Do your learning outcomes:– emphasize higher order cognitive skills?– emphasize product over process?– can you clearly explain why each is important/essential?

Activity 1What to consider when reviewing student learning outcomes (Suskie, 2009).

Think about the end rather than the means of getting there. For example:

– When students complete a research paper, what should they be able to do?

• Evaluate research in the discipline;• Formulate a research question;• Conduct research in the discipline;• Write clearly in the discipline.

Activity 1Example of degree program outcomes.

Upon completion of the program, students will

Why are these outcomes important?

Evaluate research in the discipline To be competent clinicians, speech-language pathologists must evaluate the validity and reliability of evidence from scientific studies regarding the efficacy of various treatment methods. They must identify where research is lacking, i.e. questions that still need to be answered.

Write clearly in the discipline To share findings of research and/or client diagnostic/treatment information with other professionals, it is important that future speech-language pathologists write clearly and concisely to their targeted audience.

Activity 1Can you measure learning outcomes in the capstone/culminating

experience?

– The capstone/culminating experience should result in one or more products students create.

– You should be able to assess what students have created with rubrics you have developed for each program learning outcome.

Activity 1All about mapping.

• Select 3 to 5 courses that are essential to your program and MAP (connect) the course learning outcomes to your program learning outcomes.

Activity 1All about mapping.

Keep these 3 things in mind:

1. Students should have a chance to practice the outcomes before the final assessment in the capstone/culminating experience.

2. Each outcome should be assessed at least one time before the capstone/culminating experience using significant forms of assessment.

3. Although you may be accustomed to using exams as a form of assessment, we would like you to use other more appropriate and meaningful methods.

Activity 1Suggestions on how to approach mapping.

• The following slide offers a template that you can adapt for program outcome/course mapping.

Activity 1Suggestions on how to approach mapping.

Example (not exhaustive) [ see Maki (2004) for other examples]

Course/Program Outcome

Course 1 Course 2 Course 3 Course 4 Course 5 Capstone/Culminating Experience

PO 1 X X

PO2 X X

PO3 X X

PO4 X X X

PO5 X X X X

PO6 X X

Activity 1Suggestions on how to approach mapping.

• The following slide offers a template you can use for each one of the 3-5 courses you select.

Activity 1Suggestions on how to approach mapping.

Template for Syllabus Chart

Course Outcomes

How Practiced in this Course

How Evaluated in this Course

Program Outcomes

Degree Profile Areas/Outcomes

Students will . . .

[list relevant teaching and learning activities here]

[list assessments—papers, projects, presentations—that evaluate students’ ability to demonstrate this particular outcome]

[list number(s) that correspond to program outcome list above]

[list one or more of the Degree Profile Areas/Outcomes here]

This table provides a means of mapping, for each individual course you select, the course outcomes, to program outcomes, to Degree Profile outcomes.

Activity 1How useful is the Degree Profile?

Consider how your course and program outcomes align with the Degree Profile Areas and Outcomes.

• Do your course and program outcomes address the same level of cognitive ability?

• Do your course and program outcomes align with the Degree Profile outcomes at the appropriate level?

• If course and program outcomes do not align with Degree Profile outcomes, please explain why.

• Can you improve your learning outcomes by better aligning them with the Degree Profile outcomes?

• Can you identify essential program learning outcomes that are absent from the Degree Profile and that you believe should be included?

Activity 1Example: PSC 207 for the BA in Political Science Program

Shawn Schulenberg and Marybeth Beller contributed to this slide

A: Course Learning Outcomes

B: Program Learning Outcomes

C: DP Areas of Learning and Learning Outcomes

D: Changes: Course Outcomes

Changes: Program Outcomes

Students will Students will

Analyze political arguments.

Evaluate political problems and situations.

Identify major concepts in Political Science. Evaluate major political science methods, concepts and theories.

Specialized Knowledge: Evaluates a complex challenge using perspectives and scholarship from political science and at least one other field.

Broad, Integrative Knowledge: Judges the likelihood that the combination of disciplinary perspectives (referenced above) would contribute to the resolution of the challenge.

IS: Analytic Inquiry: Evaluates theories and approaches to the challenge.

IS: Communication fluency: Constructs a sustained, coherent argument.

Change course outcome to “evaluate” political arguments.

Consider eliminating “Identify major concepts in Political Science” as a program-level outcome.

Consider adding a program level outcome regarding communication fluency.

Activity 1Example: PSC 207 for the BA in Political Science Program

Shawn Schulenberg and Marybeth Beller contributed to this slideA: Course Learning Outcomes

B: Program Learning Outcomes

C: DP Areas of Learning and Learning Outcomes

D: Changes: Course Outcomes

Changes: Program Outcomes

Students will Students will

Analyze the relationships between cultural norms and democratic principles in various countries around the world.

Evaluate political problems and situations.

Compare and contrast the rights and responsibilities of citizenship from a global perspective.

Draw reasonable inferences and observations.

Select appropriate information and technological resources to find and/or disseminate information central to Political Science.

Specialized Knowledge: Clarifies a complex issue using perspectives and scholarship from PSC and at least one other field.

IS: Analytic Inquiry: Compares and contrasts approaches to complex issues.

IS: Use of Information Resources: Incorporates multiple information resources in different media into presentations, etc

IS: Engaging Diverse Perspectives: Analyzes how cultural perspectives could affect interpretation of problems in politics.

IS: Communication Fluency: Delivers a presentation using more than one medium, using collaboration to work through an issue.

No change needed to course level outcome.

Misalignment between course outcome and program outcome, “Evaluate political problems and situations.” Consider not using this course outcome to assess this particular program outcome.

Consider eliminating “draw reasonable inferences and observations” and make this a criterion to achieve the “evaluate political problems and situations” outcome, which will be evaluated in other courses.

Consider adding a program level outcome regarding communication fluency.

Activity 1Example: CD 623 for MS in CD Program

A: Course Learning Outcomes

B: Program Learning Outcomes

C: DP Areas of Learning and Learning Outcomes

D: Changes: Course Outcomes

Changes: Program Outcomes

Students will evaluate research on the prevention and early intervention outcomes for children with disorders of fluency.

Students will evaluate child growth and development relative to normal processes of language and speech communication.

Students will critique scientific theories related to linguistic communication and its disorders.

Students will communicate effectively in the discourse of the discipline.

Specialized Knowledge: Elucidates the major theories, research methods and approaches to inquiry, and/or schools of practice in the field; articulates relevant sources; and illustrates their relationship to allied fields.

Broad, Integrative Knowledge: Articulates and defends the significance and implications of his specialized work in terms of challenges, trends and developments in a social context. Intellectual Skills: Analytic Inquiry; Use of Information Resources; Communication Fluency.

Need an outcome for communication.

Communication Fluency: Add metacognitive reflection to project.

No Change Needed

Communication Fluency: Metacognitive reflection an important part of assessment.

Activity 1Example: CD 623 for MS in CD Program

A: Course Learning Outcomes

B: Program Learning Outcomes

C: DP Areas of Learning and Learning Outcomes

D: Changes: Course Outcomes

Changes: Program Outcomes

Students will describe the characteristics of developmental stuttering, cluttering, neurogenic disfluency, and psychogenic disfluency.

Students will identify characteristics of disorders of language, speech, voice, and hearing.

Specialized Knowledge: Defines and uses principal terms in the field. – Bachelor’s Level

We feel that this outcome is appropriate to this particular course because it is necessary to support higher level outcomes.

We note a mismatch between the course outcome “describe” and the program outcome “identify.” As “identify” represents a low level of cognitive functioning, and since it will not be possible for students to demonstrate higher order skills without this competency, we do not think it needs to be stated explicitly as a program outcome and will eliminate it as such.

Next Steps: Activity 2March 26 – May 16

On the basis of work done in activity 1, update assessment plan as follows:

1. Make necessary revisions to program’s capstone.

2. Make necessary changes to program and/or essential pre-capstone/culminating experience course outcomes.

3. Identify assessment measures for each program outcome.

4. Develop rubrics for each program outcome. *

5. Submit updated assessment plan. *

Resources: Assessment Website

• www.marshall.edu/assessment– Entrance to Assessment website

• http://www.marshall.edu/assessment/Resources/AssessmentResources.aspx– Assessment Definitions– Bloom’s Taxonomy of Measurable Verbs– AAC&U Value Rubrics

Resources: Open Pathways Website

• www.marshall.edu/hlcopenpathways– Will be available Friday, January 20

References

• Adelman, C., Ewell, P., Gaston, P., & Schneider, C. G. (2011). The degree qualifications profile, Indianapolis, IN: Lumina Foundation for Education, Inc.

• Hatfield, S. (2009). Assessing your program level assessment plan. IDEA Paper # 45, retrieved from http://www.theideacenter.org/sites/default/files/IDEA_Paper_45.pdf : Manhattan, KS: the IDEA Center.

• Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing, LLC.

• Rhodes, Terrel, ed. 2010. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington, DC: Association of American Colleges and Universities.

• Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed). San Francisco, CA: Jossey-Bass.