What Works - Course and Program Assessment in STEM
description
Transcript of What Works - Course and Program Assessment in STEM
![Page 1: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/1.jpg)
What Works - Course and Program Assessment in STEM
PKAL Network – 7 October 2013Dr. Dianne Raubenheimer,
Director of Research, Planning and Assessment, Meredith College
![Page 2: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/2.jpg)
By the end of this session, participants will be able to: Identify some purposes of assessmentExplain selected assessment terminologyArticulate ways to align outcomes,
teaching, learning and assessment Consider strategies for linking course and
program assessment
![Page 3: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/3.jpg)
Ice Breaker: Write down some thoughts
A show of hands:Gathering information Measuring goals and outcomesUsing diverse measures and sourcesAnalyzing and interpreting informationReporting resultsEngaging stakeholdersMaking improvements
![Page 4: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/4.jpg)
Assessment is the systematic collection, review, and use
of information about educational programs undertaken for the purpose
of improving student learning and development
(Palomba & Banta,1999).
![Page 5: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/5.jpg)
Task: Individually - Write down different types of
assessment tasks you use in your courses. Use one small post-it for each type of assessment.
As a group - Sort the post-its into two columns – direct and indirect assessment methods Direct assessments provide for direct examination or
observation of student knowledge of skills against measureable learning outcomes.
Indirect assessments of student learning ascertain the perceived extent of value of learning experiences. They assess opinions or thoughts about student knowledge or skills.
![Page 6: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/6.jpg)
Task: Looking at the different assessment
strategies. In your groups, discuss some thoughts about the purposes of assessment
You may be able to cluster some of the methods with particular purposes
![Page 7: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/7.jpg)
Assessment of student learning (knowledge, skills and attitudes)
Improvement of student learning Improvement of teaching Program development, monitoring and
improvement SoTL, SoA, SoP
Accreditation Public accountability / Performance
scorecard
Internal / Intrinsic
External / Extrinsic
![Page 8: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/8.jpg)
Outcomes
Reciprocal relationship
![Page 9: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/9.jpg)
An example from NC State : Using an observation rubric, observed 15
instructors in different disciplines teaching All indicated they were using innovative
practices with technology Instructor noted the learning outcomes for the
lesson being taught Observer used the SOLO taxonomy to score the
SOLO level of instruction Observer used the SOLO taxonomy to score the
SOLO level at students were engaged in the class
Then gathered student work from the faculty that tested the outcome
Examined the assessment task and student work using the SOLO taxonomy (Raubenheimer, et al, 2007)
![Page 10: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/10.jpg)
SOLO category Representation Type of outcome (Biggs, 2004)
Unanticipated Extension(Extended Abstract)
Create SynthesiseHypothesise ValidatePredict DebateTheorise
Logically Related (Relational)
Apply OutlineDistinguish AnalyseClassify ContrastSummarise Categorise
Multiple Points(Multistructural)
Explain DefineList SolveDescribe Interpret
Single Point (Unistructural)
State RecogniseRecall QuoteNote Name
![Page 11: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/11.jpg)
Instructor SOLO
2
5
10
1
2
3
8
12
6
2
6
21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Single point (e.g. tell,state, recall)
Multiple points (e.g.rework, clarify, explain)
Logically related (e.g.inquire, apply, distinguish)
Unanticipated extension(e.g. imagine, elaborate)
Always
Often
Sometimes
Rarely
Never
![Page 12: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/12.jpg)
Student Solo
5
7
9
123
4
3
1
4
3
1
2
1
22
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Single point (e.g. tell, state, recall) Multiple points (e.g. clarify,
explain, define, describe)
Logically related (e.g. inquire,
apply, analyze)
Unanticipated extension (e.g.
imagine, elaborate, synthesize)
Always
Often
Sometimes
Rarely
Never
![Page 13: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/13.jpg)
Scored Using SOLO Taxonomy
Level of outcome taught
Level of related student assessment task
Single 3 7
Multiple point 2 3
Logically related 8 2
Unanticipated extension
0 1
![Page 14: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/14.jpg)
In your groups: Discuss ways in which you align teaching
and assessment of student learning outcomes?
Select 3 individuals to each act as chair, scribe and reporter for the group
![Page 15: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/15.jpg)
http://www.arapahoe.edu/about-acc/presidents-office/assessment
![Page 16: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/16.jpg)
On your own:Use the “Program Level-Assessment of Student
Learning: Self-assessment worksheet” to assess your program at your institution
In your groups: Discuss ways in which you align teaching
and assessment of student learning outcomes in your courses (columns 1-3)?
Select 3 individuals to each act as chair, scribe and reporter for the group
![Page 17: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/17.jpg)
Ensure the program outcomes reflect what you want students to have learned/attained/ achieved by the time they graduateGenerating appropriate program outcomesDevelop program outcomes - course
assessment matricesSelecting appropriate assessment
tasks/methodsDevelop a system for keeping track of
program dataGather, review data and use data for ongoing,
systematic program review and improvement
![Page 18: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/18.jpg)
Develop an implementation matrix associated with program outcomesH= high, M=medium; Low = L
Course1
Course2
Course3
Course4
Course5
Etc
Outcome 1
H M H
Outcome 2
M H M L
Outcome 3
H L M
Outcome 4
H L L H
Etc
![Page 19: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/19.jpg)
Develop an implementation and assessment matrix associated with program outcomes I = implement; A = assessCourse
1Course2
Course3
Course4
Course5
Etc
Outcome 1
I & A I I & A
Outcome 2
I I & A I I
Outcome 3
I & A I I & A
Outcome 4
I & A I I I & A
Etc
![Page 20: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/20.jpg)
Expected Outcomes
BIO183
BME 201
BME 203
BME 204
BME210
BME 252
BME 301
BME 302
Co-curricula/ other sources
Outcome (a) Graduating senior survey
Sub-Outcome a1 A A A
Sub-Outcome a2 A ASub-Outcome a3 A A ASub-Outcome a4 A A FE examOutcome (b) Graduating
senior surveySub-Outcome b1 x UG Research
Sub-Outcome b2 A A A UG Research
Sub-Outcome b3 A A
Sub-Outcome b4 A A
Outcome (e) Graduating senior survey
Sub-Outcome e1 A A A A Design day
Sub-Outcome e2 A A
Sub-Outcome e3 A
Sub-Outcome e4 ASub-Outcome e5 A
Articulate sub-outcomes on the matrix
![Page 21: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/21.jpg)
GraduatingSenior Survey
FacultySurvey
WPS 332Test & homework
scores
WPS 360Math-
intensive test questions
WPS 475Math skills
test
Goal Actual Goal Actual Goal Actual Goal Actual Goal ActualE:
P: 30%+C:
NI:<10%
E: 16%P: 78%C: 7%NI: 0%
E: P: 30%+
C: NI:<10
%
E: 11%P: 67%C: 9%NI: 3%
Avg. scoresB or
higher
Avg. Score
B+
Avg. scoresB or
higher
Avg. Score
B+
Avg. scoresB or
higher
Avg. Score
B+
Determine assessment methods and expected performance criteria(level of performance )
![Page 22: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/22.jpg)
Develop a system for recording and tracking results
![Page 23: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/23.jpg)
Best practices indicate that assessment
is systematic and systemic. It occurs at multiple levels in an
institution, vertically and horizontally,
involving processes that are connected and integrated
![Page 24: What Works - Course and Program Assessment in STEM](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813e61550346895da86978/html5/thumbnails/24.jpg)
Assessment: Arapahoe Community College http://www.arapahoe.edu/about-acc/presidents-office/assessment
Biggs, J. (2004). Teaching for quality learning at university. Berkshire, UK: Open University Press. Heer, R. (2009). A Model of Learning Objectives. Center for Excellence in Learning and Teaching,
Iowa State University. Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution.
Sterling, VA: Stylus. Mertler, C.A. (2003). Classroom assessment: A practical guide for educators. Los Angeles: Pyrczak. Ozturk, H. & Raubenheimer, C.D. (2011). PAT: An online program A\assessment tool. Paper
presented at 2011 American Society for Engineering Education International Conference, Vancouver, Canada.
Palomba, C.A & Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in higher education. San Francisco, CA: Jossey-Bass.
Raubenheimer, C. D., Spurlin, J., Martin, S., & Mehlenbacher, B. Faculty in Technology-Rich Contexts: Connecting Teaching, Learning, and Assessment in the Classroom. 2007 UNC TLT Conference Proceedings.
Rogers, G. (2004). Self-Assessment: Quality Assurance of Program Level-Assessment of Student Learning. Rose Hulman Institute of Technology.
Spurlin, J.E., Rajala, S.A., & Lavelle, J.P. (2008). Designing better engineering education through assessment: A practical resource for faculty and department chairs on using assessment and ABET criteria to improve student learning. Sterling, VA, Stylus Publishing.