Conducting Assessment to Support Teaching and Scholarship...CONDUCTING LEARNING ASSESSMENT TO...

1

Assessment and Learning Resource Repository

ACSC Crawford Hall 230 & 231

2550 Campus Rd, Honolulu, HI 96822 manoa.hawaii.edu/assessment

Conducting Assessment to Support Teaching and Scholarship

Hill, Y. (2020, September 30). Conducting assessment to support teaching and scholarship [Online workshop]. Assessment and Curriculum Support Center and Center for Teaching Excellence. University of Hawai‘i at Mānoa, Honolulu, HI, United States.

https://manoa.hawaii.edu/assessment

Conducting Assessment to

Support Teaching and Scholarship

A workshop for all instructional faculty/staff

Yao Z. Hill, Associate Specialist

PresenterPresentation NotesCONDUCTING LEARNING ASSESSMENT TO SUPPORT TEACHING AND SCHOLARSHIPWe invite all teaching faculty and staff, especially recently hired faculty, lecturers, & graduate teaching assistants to this interactive workshop. Come and explore powerful ways for using program learning assessment tools to enhance teaching, collaborate with colleagues on curriculum matters that impact all students, and contribute to the program quality and coherence.��Participants will:Reflect on ways to use student learning outcomes, the program curriculum map, signature assignment, and rubrics to enhance teaching.Learn to reflect on their assessment efforts and effective ways to describe it in their contract renewal and tenure and promotion documents.Explore scholarship in teaching, learning, and assessment through the national Assignment Library and Learning Improvement projects.��This workshop is co-sponsored by the Center for Teaching Excellence and Assessment and Curriculum Support Center.

DATEWednesday, September 30, 2020TIME2:00 PM - 3:30 PMLOCATIONONLINE (Zoom)PRESENTED BY:Yao Zhang Hill, Assessment and Curriculum Support Center

Housekeeping

• Access your individualized handout here: http://go.hawaii.edu/3c3

PresenterPresentation Noteshttps://drive.google.com/drive/u/0/folders/1JlHQsXrppVzh1jW39ZWKY6GhOmcVH58A

Session outcome

Plan 1 strategy to use program learning assessment tools & processes to support teaching and scholarship

Student learning

outcomes

Curriculum map

Signature assignment Rubric

Assessment is an integral part of teaching

PresenterPresentation NotesThroughout the workshop, I want to convey that assessment is an integral part of teaching.

Activity 1:

• Skim the sample narrative.• Highlight program learning

assessment tools/processes• Describe your impression

4 min individual + 6 min breakoutgo.hawaii.edu/3c3

PresenterPresentation Notes10 min activity: First, read the sample narrative designed to be included in one’s tenure promotion dossier. Identify the assessment tools and processes that this faculty used in his/her reflective teaching practice, without me telling you what these tools/processes mean. You can underline keywords or use markers to highlight.Participants contribute and call out answers.Can you describe your holistic impression of this colleague? How does this narrative make you feel about this colleague?

Program Learning Outcomes (PLO) Assessment

An on-going process designed to monitor and improve student learning. Faculty: a) develop explicit statements of what students should learn (i.e., student learning outcomes); b) verify that the program is designed to foster this learning (alignment); c) collect data/evidence that indicate student attainment (assessment results); d) use these data to improve student learning (close the loop). (Allen, M., 2008)

PresenterPresentation NotesSeeing how a colleague can use program learning assessment to support his/her reflective teaching, let me start with defining program learning outcomes. In this definition of program learning outcome assessment, we can see that the ultimate purpose is to improve student learning. The action toward the improvement is informed by making our expectations clear through student learning outcome statements, aligning learning opportunities with the outcomes, collecting the evaluating learning evidence, and using the data.

StudentLearning

Outcomes

LearningOpportunities

Collect & Evaluate Evidence

Interpret Results

Use Results

Assessment cycle

PresenterPresentation NotesThis diagram reflects the typical steps that a program would go through in program learning outcome assessment. The process provides the logic and the language to talk about how a program is accountable for its educational quality. The cycle starts with the program faculty articulating expectations of student learning outcomes in terms of knowledge, skills, and dispositions that they expect of a graduating senior; examining whether there are sufficient learning opportunities for students to meet faculty expectations; collecting & evaluating evidence of student learning, and interpreting and using the results.

StudentLearning

Outcomes



Interpret Results

Use Results

Assessment cycle

PresenterPresentation NotesStudent learning outcomes is the first powerful reflection tool that I want to introduce.

SLO Definition: Action-oriented statement of…

knowledge skills values

PresenterPresentation NotesAction-oriented statement of the knowledge, skills, and/or dispositions students are expected to know, to be able to do or to value upon successful program completion.

Domains of Learning

Cognitive Affective Psychomotor

PresenterPresentation NotesThere are three major learning domains: cognitive domain, addressing intellectual knowledge and skills; affective domain addressing values and attitudes, about how students feel; and psychomotor domain, about student skills using their body and hand movement. Let’s take a look at some examples.

Example SLOs

Cognitive

• Students can explain major theories in the field of study.

• Students can critically evaluate and synthesize multiple sources of evidence.

• Students can design and conduct undergraduate researchprojects.

PresenterPresentation NotesCognitively, students can explain theories, evaluate and synthesize evidence, and design and conduct research.

Example SLOs

• Students can skillfully explore self-initiated projects on XXX.

• Students seek out indigenousperspectives when addressing community challenges.

• Students be able to advocatesustainable living principles to the community.

Affective

PresenterPresentation NotesAffectively, students can explore self-initiated projects, seek out perspectives, and advocate sustainable living principles.

Example SLOs

Students can• Embody structural dance

movements• Play music instruments• Operate equipment

Psychomotor

PresenterPresentation NotesIn terms of psychomotor skills using their body and especially their hands, students can embody dance movements, play instruments like a piano, and operate equipment.

Using program SLOs to support teaching—An example

PresenterPresentation NotesFor many instructional faculty, knowing their program learning outcomes is very illuminative and allows them to rethink their own course(s). Profound changes can happen just by aligning one’s own course with the program outcomes. I once taught Language Testing in Second Language Studies and I can demonstrate how the alignment process can help instructors like me to refine courses.

Make the alignment clearer

Course SLO:conduct item analysis

Program SLO 1Critical thinking

Evaluate item quality & usefulness through content & descriptive statistical analysis

PresenterPresentation NotesIn my course, I had a course SLO that students will be able to conduct item analysis. My task for students is to calculate item statistics such as item difficulty in Excel and report the results in Word. It involves the skill of using Excel but it doesn’t necessarily address critical thinking. To make my course more aligned with critical thinking, I could rephrase my course outcome this way: Evaluate… through…analysis. Now I make it clear that students have to make a judgement of the quality and usefulness of test items. To me, that is clearly aligned with critical thinking.

Address program SLOs not emphasized before

Program SLO 5: Develop and apply sound framework in assessment

New Course SLODiagnose test items & improve a test based on item

analysis results.

PresenterPresentation NotesI also found that Program SLO 5 speaks to applying framework in assessment, and my course is the only course that deals with assessment. This makes me feel a great responsibility to make sure that application happens in my course. After this consideration, I further modified my Course SLO to “Diagnose test items & improve a test based on item analysis results.” This means my students not only have to analyze the test items and make judgement on their quality, but they also have to improve the items that show poor quality.

Assignment redesign to align with new SLO

Item Analysis Report

• Excel calculations• Item diagnosis• Recommendation for

revision

PresenterPresentation NotesIn seeing whether students have achieved the learning outcomes, I realized my previous assignment in which students do calculations in Excel was not good enough. So, I had to modify the assignment to make it better aligned with the new Course SLO.

Scaffold learning experience

• Revise instructions• Tips sheet• Annotated sample

reports• Rubrics

Photo by Bernard Hermant on Unsplash

PresenterPresentation NotesThe first time I gave out the assignment without detailed instructions, the assignments turned back to me varied in quality. Each time that I taught the course, I identified where students struggled and designed scaffolding activities/materials to help students turn in high quality assignments.

https://unsplash.com/@bernardhermant?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyTexthttps://unsplash.com/s/photos/scaffold?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText

Use programs SLOs to support teaching• Align course SLOs with program expectations• address important knowledge/skill not emphasized before• refine assignments/tasks to clearly align with the SLOs• provide necessary scaffolding learning experience• help students connect their learning by communicating the

program SLOs that you address.

Activity 21. Locate your program SLOs2. Plan a strategy to use program SLOs to support your teaching (handout)

PresenterPresentation Notes5 min

Where to find program SLOs

• Departmental Website• Assessment Office Website:

https://manoa.hawaii.edu/assessment/update2/view.php

Learning Outcomes



Interpret Results

Use Results

Assessment cycle

Curriculum map:learning opportunities program SLOs

Courses SLO 1Knowledge

SLO 2Critical

thinking

SLO 3Research

SLO 4Written Comm.

SLO 5 Ethical

Reasoning

Course 101 X X X X

Course 102 X X X

Course 201 X X

Course 202 X X X

Mandatory advising X X

Capstone X X X X X

PresenterPresentation NotesIn program assessment, the curriculum map is another powerful tool that can engage thoughtful reflection on program coherence. The map makes the learning opportunities very clear under each program SLO.

Imagine that I am the student in this course. Which outcome would I likely fall short of in achievement when I graduate?



SLO 2Critical

thinking

SLO 3Research

SLO 4Written Comm.

SLO 5 Ethical

Reasoning

Course 101 X X X X

Course 102 X X X

Course 201 X X

Course 202 X X X


Capstone X X X X X

?

PresenterPresentation NotesEthical Reasoning! It only appeared in the capstone course, and capstone is a place for students to organize, synthesize, and reflect to achieve a higher level of learning. It is not a good place to introduce a new SLO. Clearly there is a gap in the curriculum. As individual instructors, we can think about how to fill that gap through assignments and activities. We can also think about how to collaborate with colleagues to collaboratively design learning activities to help fulfill the gap.



SLO 2Critical

thinking

SLO 3Research

SLO 4Written Comm.

SLO 5 Ethical

Reasoning

Course 101 X X X X

Course 102 X X X

Course 201 X X

Course 202 X X X


Capstone X X X X X

PresenterPresentation NotesNow, let’s take a look at SLO 2: Critical thinking. Every course claims to address this SLO. How can we make sure that we are not overlapping with each other and we don’t create a big jump in student learning experience from one course to another, so that there can be an implicit gap in the curriculum?

Your course

Prior course

Subsequent course

Major theoriesResearch writingCritical thinking

PresenterPresentation NotesLook at the curriculum from another angle. We are expecting students to gain a high level of complex skills upon program completion.

Let’s say that you are teaching a course right in the middle of the curriculum, it can be a 200- or 300-level course. In your course, you require students to have gained certain foundational knowledge or skills to be successful in your course. In almost every discipline, if we teach middle or upper level courses, we have assumptions of students’ prior knowledge or skills in the discipline.

Did you talk to instructors teaching the lower level courses and ask them what you should expect students to know or be able to do before entering your class? Have you made your assumption of prior knowledge clear to your students? Did you investigate what prior knowledge or skills the students brought in? Did you try to provide support for students to make up the knowledge/skills that were needed before entering the class? For example, asking them to review certain topics before coming into the class or having a review class?

Use curriculum map to support teaching• identify the curriculum gap and collaborate with

colleagues to fill it• collaboratively design learning tasks/activities to

scaffold learning throughout the curriculum• collegially communicate one’s expectations of

students’ prior knowledge/skills• find out colleagues’ expectations of student

knowledge/skills existing in one’s course

Activity 3

Locate your program’s curriculum map

Select a strategy to use the curriculum map to support your teaching

Share in small groups.

PresenterPresentation Notes3 minutes. Share for 6 minutes

Breakout Room Task: Think of one strategy to implement and share with a colleague (6 min)

Curriculum map

PresenterPresentation NotesCurriculum map reviews the vertical alignment of the curriculum.

Signature Assignment

PresenterPresentation NotesSignature assignment helps with horizontal alignment of the curriculum


A generic task, problem, case, or project that can be tailored or contextualized in different disciplines or course contexts. (Driscoll, 2016, p. 11).

PresenterPresentation NotesDriscoll, A. (2016, February). Aligning direct and indirect assessment strategies with outcomes. In Assessment 201: Advanced topics in assessment Resource Binder, p.10-28. WASC Senior College and University Commission Educational Programs. Chaminade University, Honolulu, HI.

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Eng 100

Sections of the same course

PresenterPresentation NotesThis semester, we have 47 sections of English 100. How can we ensure comparable learning experiences across all sections?

Sample Signature Assignment for Eng 100• Write 3-5 pages• Argue for a position• Include 3 sources• Specify audience• Genres to consider:

• Academic essay for general public• A letter to GSO• A white paper to legislature • A transcript of a talk or media product (pod cast)

Quan Survey Qual

Similar types of courses

PresenterPresentation NotesSay you are teaching graduate level courses and in your program there is only one research method course being required. Should students have comparable opportunity to practice using those methods? If one course is giving out a multiple-choice exam, and another asks students to conduct research and write up the research report, the students in the latter class may gain more solid learning experience and retain the skill better.

Sample Research Proposal signature assignment• Students write a 5-8 page research proposal that

include• Research topic• Justification of methods selection• Description of methods• Proposed analysis procedure

ChineseOC

Oral Communication (OC) designated course

Political Science

OC

EngineeringOC

PresenterPresentation NotesWe have a lot of General Education requirements. How are we making sure that our Gen Ed requirements are helping students achieve what we want them to gain? How can we promote cross discipline conversations about oral communication? What are the essential skills that we all want our students to be able to demonstrate? How do we development signature assignments to allow students to demonstrate those skills?

UH Mānoa Oral Presentation Signature AssignmentTask: Students will prepare and deliver a formal oral presentation. Length: 5-20 minutesPurpose: persuade or informAudience: Specify intended audienceSupporting material: Utilize appropriate supporting materials

Oral Communication Teaching and Assessment resource for facultyhttps://manoa.hawaii.edu/assessment/reports/gened/oral-communication/oc-resources/

PresenterPresentation NotesUH Mānoa has a signature assignment on oral presentation.

Signature assignment development

SLOs


EvaluationCriteria

PresenterPresentation NotesSignature assignments need to be developed with the learning outcomes in mind. If your outcome includes critical thinking, your signature assignment will ask students to argue, to evaluate, to analyze, rather than to define, explain, and describe. Once you have the signature assignment, the next step is to develop evaluation criteria. The criteria, in turn, need to align and address the target learning outcomes.

Use signature assignment to support teaching• Collaborate with colleagues for shared learning expectations

• Refine assignments to align with the shared expectations

• Collaborate with colleagues to design a signature assignment

• Collaboratively brainstorm ways to enhance assignment and teaching

PresenterPresentation NotesCollaborate with colleagues teaching the same or similar courses to form shared learning expectationsRefine the assignment to align with the shared expectationsCollaborate with colleagues to design a signature assignmentDiscuss student performance with fellow colleagues and collaboratively brainstorm ways to enhance assignment and teaching

Activity 4

Select a strategy to use a signature assignment to support your teaching.

Plan one activity to implement.

PresenterPresentation Notes3 minutes

Learning Outcomes



Interpret Results

Use Results

Assessment cycle


PresenterPresentation NotesOnce a signature assignment is developed, it can serve both as learning opportunities and as learning evidence. Using signature assignments can make assessment efficient and promote faculty conversation and collaboration.

Learning Outcomes



Interpret Results

Use Results

Assessment cycle

PresenterPresentation NotesOnce you have collaborated with your colleagues either in your department or faculty teaching the same Gen Ed designated courses to design a signature assignment, you will need to think about how to evaluate students’ work. Very often, we ask students to do complex performance tasks that involve students integrating knowledge and skills at multiple dimensions. We need an evaluation tool that can capture that complexity. A rubric is a commonly used evaluation tool.

Rubric

A rubric is a scoring guide that describes the criteria that faculty use to evaluate student performance, understanding, or

behavior.

PresenterPresentation NotesFirst thing first, let’s take a look at the definition of the rubric.

VALUE Oral Presentation Rubric

Organization

Language

Delivery

Supporting Material

Central message https://drive.google.com/file/d/1MDJRFjFrPXOw8TvFxq8RHF7tMB02hOOo/view

Capstone4

Milestone 3

Milestone 2

Benchmark 1

PresenterPresentation NotesIt is often shaped like a matrix, which describes levels of achievement in a specific area of performance, understanding, or behavior.The words in red boxes represent evaluation dimensions or evaluation criteria. The words in the green boxes represent different levels of performance. We call them performance scales.The words in each square are called performance descriptors.

https://drive.google.com/file/d/1MDJRFjFrPXOw8TvFxq8RHF7tMB02hOOo/view

How can this rubric be revised?

Rubric Categories1. Grammar2. Word choice3. Typos4. Content organization

Program Outcome:Apply critical thinking and rhetorical skills to produce coherent written works

Need to also evaluate critical thinking

PresenterPresentation NotesWhat you see are rubric criteria for a writing task in a foreign language course. What is missing?

Collaboratively decide on the criteria, performance description, & definition of satisfactory performance

PresenterPresentation NotesThe best way to develop a rubric is to ask your colleagues what they think is important. Ask them to give examples and provide descriptions.

Adapt an existing rubric

https://www.aacu.org/value-rubrics

https://www.aacu.org/value-rubricshttps://www.aacu.org/value-rubrics


http://manoa.hawaii.edu/assessment/resources/rubricbank.htm

http://manoa.hawaii.edu/assessment/resources/rubricbank.htm

Data dashboard template: http://go.hawaii.edu/AZW

PresenterPresentation NotesHow to analyze rubric results? Present the percentage of the students who met or exceeded your expectations. In this chart, the dark green bar represents the percentage of the students who met the standards on the final presentation and the light green bar showed the percentage of students who met the expectations on the presentation rehearsal. We can observe that students made a big improvement from the rehearsal to the final presentation, but still only 60% of the students met our expectations on delivery.

http://go.hawaii.edu/AZW

Use rubrics to support teaching

• align the rubric with program SLOs• collaboratively decide what is important to evaluate

with colleagues• collaboratively define satisfactory performance• collaboratively evaluate and refine the rubric

Activity 5

Select a strategy to use a rubric to support your teaching.

Plan one activity to implement.


Learning Outcomes



Interpret Results

Use Results

Assessment cycle

PresenterPresentation NotesResults from student performance on a well-designed assignment and rubric are likely to generate meaningful discussion about program curriculum quality and promote the use of the results.

Breakout Room Task:Pick one signature assignment/rubric activity that you plan to try. Share and give feedback to a colleague.


Scholarship Opportunities

PresenterPresentation NotesExample of results and samples of use of results. We see more and more professional organizations value scholarship of teaching and learning. They are supporting this more and more. You can look into your professional organization conferences to publish on teaching and learning in addition to your subject area.

Publish your assignment—locally

Assessment and Learning Resource Repository

http://go.hawaii.edu/ATb

PresenterPresentation NotesIf you want to start small, share your work with the campus first. Our office is building an Assessment and Learning Resource Repository. We welcome assignment examples, rubrics, and any other resources that you think can be useful for your colleagues. Your work remains your intellectual property. However, it will be assigned a creative commons license. This means that when other people use your work, they have to cite you, but they can adapt for their own use without your permission. It is one step toward open educational resources while still acknowledging your intellectual contribution.

https://drive.google.com/drive/folders/16f1dxiQgU7GEfDUold9lM0dnQR04rUODhttp://go.hawaii.edu/ATb

Publish your assignment—nationally

https://www.learningoutcomesassessment.org/ourwork/assignment-library/Email: [email protected]

NILOA

PresenterPresentation NotesAt the next level, consider submitting to the national Assignment Library, for the opportunity to receive peer feedback from experts in your field beyond UH and make your work known globally. This is a project organized by the National Institute for Learning Outcomes Assessment, or NILOA. National leaders of learning assessment have long recognized the important function that excellent assignments play in quality assessment.

There is more work involved when publishing your assignment on this forum. You will need to clearly state your learning outcomes, have a well-developed set of evaluation criteria, describe how you’ve scaffolded student learning, the assessment results, and your reflections on how the assignment went. But once you have done that, and this workshop is designed to prepare you for doing that, do consider contributing to this forum. Make our work known. Elevate UH’s reputation in quality education at the national level. I have seen many assignments from CUNY: City University of New York. It would be nice when I offer this workshop in the future, I can use examples from you.

NILOA logo image source: https://www.learningoutcomesassessment.org/wp-content/uploads/2019/05/NILOA2019Logonowords-120x120.png

https://www.learningoutcomesassessment.org/ourwork/assignment-library/https://www.learningoutcomesassessment.org/ourwork/assignment-library/mailto:[email protected]

https://www.learning-improvement.org/

Tell your story of learning improvement

Assess Change Re-Assess

Fulcher et al. (2004)

PresenterPresentation NotesAnd if you want to even further pursue the scholarship in program level teaching, learning, and assessment, the Learning Improvement Community is a faculty-friendly space for you to share how you collaborated with colleagues to assess student learning, using your assignment, made adjustment to pedagogy collectively with your colleagues, and re-assessed student learning and showed improvement. Our office is here to support you in making your story happen and getting it published.Scale image: Photo by Theme Inn on UnsplashPig image: Photo by Christopher Carson on Unsplash

Fulcher et. al. (2014). A simple model for learning improvement: Weigh pig, feed pig, weigh pig: https://www.learningoutcomesassessment.org/documents/Occasional_Paper_23.pdf

https://www.learning-improvement.org/https://www.learningoutcomesassessment.org/documents/Occasional_Paper_23.pdf

Disciplinary Conferences

Assessment & SoTL Conferences

LocalHawaii Educational Research Association Annual Conference (HERA) (Jan/Feb) Pacific Association for Institutional Research (PacAIR)Hawai’i-Pacific Evaluation Association (HPEA)NationalAALHE ConferenceIUPUI Assessment InstituteAnnual Scholarship on Teacher and Learning Conference List

http://www.hawaii.edu/hera/conferenceinfo.htmhttp://www.pacairweb.org/https://h-pea.org/https://www.aalhe.org/http://assessmentinstitute.iupui.edu/http://www.washington.edu/teaching/sotl-annual-conferences/

Thank you!

Yao [email protected]

Q & AEvaluate the session here: https://forms.gle/8wCzw4SEGXj5Ssvi7

mailto:[email protected]://forms.gle/8wCzw4SEGXj5Ssvi7

Assessment & Curriculum Support Center University of Hawai‘i at Mānoa Crawford Hall 230 & 231 2550 Campus Road, Honolulu, HI 96822

[email protected] (808) 956-4283 (808) 956-6669

manoa.hawaii.edu/assessment

1

Conducting Learning Assessment to Support Teaching and Scholarship

Dr. Yao Z. Hill ([email protected])

Activity 1: Articulate Integrating Assessment in Teaching Practice Activity Instruction: The following narrative represents sample language that is intended to be included in the tenure and promotion dossier.

1. Skim the narrative and highlight learning assessment tools/processes mentioned.

2. Write down your impression of this colleague.

My approach to teaching the course Research Ethics transformed after I learned that this course is the only course in the BS program that substantially addresses the ethics-related program student learning outcome (SLO), which is: “Students will function professionally and ethically when conducting research.” Since this is a senior level course, students’ performance in this course no longer just represents their achievement in my course but also as a graduating senior from this degree program. With this realization, I asked myself several questions:

1. Are my expectations of the students the same as the program expectations? What does “function ethically” entail to other faculty and to the program as a whole?

2. If another faculty teaches this course, would students learn the same or differently? What should be the essential learning experience and what kind of evidence can show that students have achieved the program expectations?

3. Is this one course enough? Where else can this be addressed in our program curriculum? Who else might be interested in addressing ethical deliberation in their courses? How can I collaborate with them?

In the process of seeking the answers to these questions, I joined the Curriculum Committee, collected input from faculty colleagues, and revised my course SLOs to explicitly reflect program expectations of “function ethically.” For example, SLO1: “Students will be able to follow the IRB process when conducting their research;” SLO2: “Students will be able to perform ethical deliberation in every step of the research process that reflects ethical standards of the field;” and so on. Working with colleagues, I led and coordinated the development of rubrics to assess

mailto:[email protected]


[email protected] (808) 956-4283 (808) 956-6669


2

the outcomes achievement. I provided students both the SLOs and the rubrics right at the beginning of the course so that students clearly know the program expectations and the path to progress toward mastery. In addition, with the committee’s feedback, I revised my course final assignment to make it a signature assignment, so that instructors other than me can use it as the culminating evidence of student ability both for the course and for program assessment purposes. To help students achieve higher levels of mastery, I designed many scaffolding activities throughout the semester. For example, … Furthermore, I approached two faculty members teaching lower level courses and used my expertise in ethics to help them develop classroom activities and assignments related to ethical deliberation.

Working collaboratively with faculty beyond the boundary of the classroom helped me to make transformative improvements to the course. Students came more prepared after going through the learning experience in the lower level classes. The activities and assignments made more sense for them because the expectations are clear and aligned with program expectations. As a result, 80% of the students performed satisfactorily on the final culminating assignment after the course transformation, compared to 60% before the changes. More importantly, I believe that I significantly contributed to the curriculum coherence and quality in building students’ ethical reasoning ability and ethical conduct in research. My teaching and assessment work positively impacted my class, the Ethical Research course in general, and the program.

(Disclaimer: This is constructed sample language designed for inclusion in a faculty’s dossier. Individual faculty members should check with their own department personnel evaluation criteria to see whether a narrative like this would be pertinent/appropriate.)

mailto:[email protected]


[email protected] (808) 956-4283 (808) 956-6669


3

Student learning outcomes (SLOs) Definition: Action-oriented statement of the knowledge, skills, and/or dispositions students are expected to know, to be able to do, and to value upon successful course/project/program completion Resource: Find program SLOs here: https://manoa.hawaii.edu/assessment/update2/view.php?view=programs

Activity 2: Locate the program SLOs that your course aligns with Which program learning outcomes do you emphasize in your course?

Select a strategy to use program SLOs to support your teaching

❏ Clearly align the course with program SLOs [Align = substantially and explicitly address that program SLO in my course]

❏ Address important knowledge/skill not emphasized before ❏ Refine assignments/tasks to clearly align with the SLOs ❏ Provide students with necessary scaffolding learning experience ❏ Help students connect their learning by communicating the program SLOs that you address ❏ Other:

Plan one activity to implement here:

mailto:[email protected]://manoa.hawaii.edu/assessment/resources/how-to/develop-program-student-learning-outcomes-slos/https://manoa.hawaii.edu/assessment/update2/view.php?view=programs


[email protected] (808) 956-4283 (808) 956-6669


4

Curriculum Map Definition: A matrix that shows the alignment between learning opportunities and program SLOs

Activity 3: Locate your program’s curriculum map here. Select a strategy to use the curriculum map to support your teaching

❏ Identify a curriculum gap and collaborate with colleagues to fill it ❏ Collaboratively design learning tasks/activities to scaffold learning throughout the curriculum ❏ Collegially communicate my expectations of students’ prior knowledge/skills to instructors teaching a lower

level course ❏ Find out colleagues’ expectations of student knowledge/skills existing from my course ❏ Explain to students my assumptions of their prior knowledge/skills ❏ Design activities to help students connect their prior learning to my course ❏ Refine my course so that my students are better prepared for subsequent courses (alternatives: future

professional tasks, community/civic engagement, graduate school)


mailto:[email protected]://manoa.hawaii.edu/assessment/resources/how-to/curriculum-mapping-curriculum-matrix/https://manoa.hawaii.edu/assessment/update2/view.php?view=programs


[email protected] (808) 956-4283 (808) 956-6669


5

Signature Assignment Definition: A generic task, problem, case, or project that can be tailored or contextualized in different disciplines or course contexts (Driscoll, 2016, p. 11)

Sample Signature Assignments English 100

3-5 pages in length Argue for a position Include 3 sources Specify audience Genres to consider:

● Academic essay for general public

● A letter to GSO ● A white paper to legislature ● A transcript of a talk or media

product (podcast)

Research Methods Course -- Research Proposal 5-8 pages in length Includes:

● Research topic ● Justification of methods

selection ● Description of methods ● Proposed analysis

procedure

UH Mānoa Oral Presentation Signature Assignment Task: Students will prepare and deliver a formal oral presentation. Length: 5-20 minutes Purpose: persuade or inform Audience: Specify intended audience Supporting material: Utilize appropriate supporting materials OC teaching and assessment resources are here.

Activity 4: Select a strategy to use a signature assignment to support your teaching

❏ Collaborate with colleagues teaching the same or similar courses to form shared learning expectations ❏ Refine the assignment to align with the shared expectations ❏ Collaborate with colleagues to design a signature assignment ❏ Discuss student performance with fellow colleagues and collaboratively brainstorm ways to enhance

assignment and teaching


mailto:[email protected]://manoa.hawaii.edu/assessment/reports/gened/oral-communication/oc-resources/


[email protected] (808) 956-4283 (808) 956-6669


6

Rubrics Definition: A scoring guide that describes the criteria that faculty use to evaluate student performance, understanding, or behavior. It is often shaped like a matrix, which describes levels of achievement in a specific area of performance, understanding, or behavior. Resources

● VALUE rubrics: https://www.aacu.org/value/rubrics ● AO Rubric Bank: http://manoa.hawaii.edu/assessment/resources/rubricbank.htm ● ACSC Assessment and Learning Resource Repository: http://go.hawaii.edu/ATb

Activity 5: Select a strategy to use a rubric to support your teaching

❏ Align the rubric with program SLOs ❏ Collaboratively decide what is important to evaluate with colleagues ❏ Collaboratively define satisfactory performance ❏ Collaboratively evaluate and refine the rubric ❏ Adapt the program rubrics together with your students ❏ Share the rubric with students ❏ Train students to self-evaluate using the rubrics ❏ Track student progress by analyzing rubric data ❏ Use rubric results to identify SLOs to enhance


Upcoming Rubric Use and Development Online Workshop can be registered for here.

Activity 6: Write an email to yourself Remind yourself next time that you are going to describe your teaching practice and how you will describe your efforts integrating assessment to support teaching and scholarship. Use bullet points to note down key ideas.

mailto:[email protected]://www.aacu.org/value/rubricshttp://manoa.hawaii.edu/assessment/resources/rubricbank.htmhttp://go.hawaii.edu/ATbhttps://hawaii.zoom.us/meeting/register/tJEsc-GoqzMvHtfcp2c-UOoH5Mtn_UJKQPjx

Conduct Assessment to Supporting Teaching and Scholarship HandoutConduct Assessment to Support Teaching and Scholarship Cover Page.pdfConduct Assessment to Supporting Teaching and ScholarshipConducting Assessment to Support Teaching and ScholarshipHousekeeping Session outcomeAssessment is an integral part of teachingActivity 1:Program Learning Outcomes (PLO) AssessmentAssessment cycleAssessment cycleSLO Definition: Action-oriented statement of…Domains of LearningExample SLOsExample SLOsExample SLOsUsing program SLOs to support teaching—An exampleMake the alignment clearerAddress program SLOs not emphasized beforeAssignment redesign to align with new SLOScaffold learning experienceUse programs SLOs to support teachingActivity 2�1. Locate your program SLOs�2. Plan a strategy to use program SLOs to support your teaching (handout)Where to find program SLOsAssessment cycleCurriculum map:�learning opportunities program SLOsCurriculum map:�learning opportunities program SLOsCurriculum map:�learning opportunities program SLOsSlide Number 26Use curriculum map to support teachingActivity 3��Locate your program’s curriculum map��Select a strategy to use the curriculum map to support your teaching��Share in small groups.Breakout Room Task: Think of one strategy to implement and share with a colleague (6 min)Slide Number 30Slide Number 31Signature AssignmentSections of the same courseSample Signature Assignment for Eng 100Similar types of coursesSample Research Proposal signature assignmentOral Communication (OC) designated courseUH Mānoa Oral Presentation Signature AssignmentSignature assignment developmentUse signature assignment to support teachingActivity 4��Select a strategy to use a signature assignment to support your teaching.��Plan one activity to implement.Assessment cycleAssessment cycleRubricVALUE Oral Presentation RubricHow can this rubric be revised?Slide Number 47Adapt an existing rubricAdapt an existing rubricAdapt an existing rubricSlide Number 51Use rubrics to support teachingActivity 5��Select a strategy to use a rubric to support your teaching.��Plan one activity to implement.Assessment cycleBreakout Room Task:�Pick one signature assignment/rubric activity that you plan to try. Share and give feedback to a colleague. Scholarship OpportunitiesPublish your assignment—locallyPublish your assignment—nationallySlide Number 59Disciplinary ConferencesAssessment & SoTL Conferences�Thank you!��Yao Hill�[email protected]

_Master Handout.pdfActivity 1: Articulate Integrating Assessment in Teaching PracticeStudent learning outcomes (SLOs)Curriculum MapSignature AssignmentRubricsUpcoming Rubric Use and Development Online Workshop can be registered for here.Activity 6: Write an email to yourself

Conducting Assessment to Support Teaching and Scholarship...CONDUCTING LEARNING ASSESSMENT TO...

Documents

Transcript of Conducting Assessment to Support Teaching and Scholarship...CONDUCTING LEARNING ASSESSMENT TO...