Peer-Assessment in Higher Education: A Review of Recent...
Transcript of Peer-Assessment in Higher Education: A Review of Recent...
![Page 1: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/1.jpg)
Peer-Assessment in Higher Education: AReview of Recent Studies
Michael Mogessie Ashenafi
Department of Information Science and EngineeringUniversity of Trento
06 November, 2015
![Page 2: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/2.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
2/52
![Page 3: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/3.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Introduction
Assessment in education - varies with goals
Continuous vs one-offGoal - measuring performance and/or improving studentlearningTerminologies - Summative and Formative
3/52
![Page 4: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/4.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Introduction
Assessment in education - varies with goalsContinuous vs one-off
Goal - measuring performance and/or improving studentlearningTerminologies - Summative and Formative
3/52
![Page 5: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/5.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Introduction
Assessment in education - varies with goalsContinuous vs one-offGoal - measuring performance and/or improving studentlearning
Terminologies - Summative and Formative
3/52
![Page 6: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/6.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Introduction
Assessment in education - varies with goalsContinuous vs one-offGoal - measuring performance and/or improving studentlearningTerminologies - Summative and Formative
3/52
![Page 7: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/7.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Summative Assessment
intended to measure degree of achievement
either one-off or carried out at intervals - Mid-terms, finalsCriterion-referenced (absolute grading) or normative(relative to other students)
4/52
![Page 8: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/8.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Summative Assessment
intended to measure degree of achievementeither one-off or carried out at intervals - Mid-terms, finals
Criterion-referenced (absolute grading) or normative(relative to other students)
4/52
![Page 9: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/9.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Summative Assessment
intended to measure degree of achievementeither one-off or carried out at intervals - Mid-terms, finalsCriterion-referenced (absolute grading) or normative(relative to other students)
4/52
![Page 10: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/10.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Formative Assessment
student-centered
Goal - to provide support and feedback to studentsHelps students monitor their own progressAlso helps the teacher to adjust their instructionaccordinglyShould not contribute towards final grades
5/52
![Page 11: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/11.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Formative Assessment
student-centeredGoal - to provide support and feedback to students
Helps students monitor their own progressAlso helps the teacher to adjust their instructionaccordinglyShould not contribute towards final grades
5/52
![Page 12: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/12.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Formative Assessment
student-centeredGoal - to provide support and feedback to studentsHelps students monitor their own progress
Also helps the teacher to adjust their instructionaccordinglyShould not contribute towards final grades
5/52
![Page 13: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/13.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Formative Assessment
student-centeredGoal - to provide support and feedback to studentsHelps students monitor their own progressAlso helps the teacher to adjust their instructionaccordingly
Should not contribute towards final grades
5/52
![Page 14: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/14.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Formative Assessment
student-centeredGoal - to provide support and feedback to studentsHelps students monitor their own progressAlso helps the teacher to adjust their instructionaccordinglyShould not contribute towards final grades
5/52
![Page 15: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/15.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Non-Traditional forms of Assessment
The teacher is not the sole assessor
significant involvement of studentspurely formative, or a blendE.g. Self-assessment, peer-assessment
6/52
![Page 16: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/16.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Non-Traditional forms of Assessment
The teacher is not the sole assessorsignificant involvement of students
purely formative, or a blendE.g. Self-assessment, peer-assessment
6/52
![Page 17: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/17.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Non-Traditional forms of Assessment
The teacher is not the sole assessorsignificant involvement of studentspurely formative, or a blendE.g. Self-assessment, peer-assessment
6/52
![Page 18: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/18.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Peer-Assessment
“... an arrangement in which individuals consider theamount, level, value, worth, quality, or success of theproducts or outcomes of learning of peers of similarstatus.”
Topping(1998)
7/52
![Page 19: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/19.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In this study ...
Over two-decades of research in peer-assessment
The million dollar question is - Does it really work?This review examines recent literature:
to find out if there’s a clear-cut answerto identify challenges and opportunitiesto recommend ways to tackle challenges in the practice
8/52
![Page 20: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/20.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In this study ...
Over two-decades of research in peer-assessmentThe million dollar question is - Does it really work?
This review examines recent literature:
to find out if there’s a clear-cut answerto identify challenges and opportunitiesto recommend ways to tackle challenges in the practice
8/52
![Page 21: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/21.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In this study ...
Over two-decades of research in peer-assessmentThe million dollar question is - Does it really work?This review examines recent literature:
to find out if there’s a clear-cut answerto identify challenges and opportunitiesto recommend ways to tackle challenges in the practice
8/52
![Page 22: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/22.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In this study ...
Over two-decades of research in peer-assessmentThe million dollar question is - Does it really work?This review examines recent literature:
to find out if there’s a clear-cut answer
to identify challenges and opportunitiesto recommend ways to tackle challenges in the practice
8/52
![Page 23: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/23.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In this study ...
Over two-decades of research in peer-assessmentThe million dollar question is - Does it really work?This review examines recent literature:
to find out if there’s a clear-cut answerto identify challenges and opportunities
to recommend ways to tackle challenges in the practice
8/52
![Page 24: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/24.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In this study ...
Over two-decades of research in peer-assessmentThe million dollar question is - Does it really work?This review examines recent literature:
to find out if there’s a clear-cut answerto identify challenges and opportunitiesto recommend ways to tackle challenges in the practice
8/52
![Page 25: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/25.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
9/52
![Page 26: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/26.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Topping (1998) - A qualitative study
Reviewed 109 studies to find out if PA works
Identified many variables among the studieswhat subject?nature of the PA task assessed: educational vs.professionalformative or summative?what is being assessed?do peer-assigned scores agree with those of the teacher’s?
His conclusion:too many variablesno concrete evidence regarding the soundness orpracticality of PA in higher education
10/52
![Page 27: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/27.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Topping (1998) - A qualitative study
Reviewed 109 studies to find out if PA worksIdentified many variables among the studies
what subject?nature of the PA task assessed: educational vs.professionalformative or summative?what is being assessed?do peer-assigned scores agree with those of the teacher’s?
His conclusion:too many variablesno concrete evidence regarding the soundness orpracticality of PA in higher education
10/52
![Page 28: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/28.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Topping (1998) - A qualitative study
Reviewed 109 studies to find out if PA worksIdentified many variables among the studies
what subject?nature of the PA task assessed: educational vs.professionalformative or summative?what is being assessed?do peer-assigned scores agree with those of the teacher’s?
His conclusion:too many variablesno concrete evidence regarding the soundness orpracticality of PA in higher education
10/52
![Page 29: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/29.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - A meta-analytic study
conducted a meta-analytic review of 56 studies comparingpeer and teacher marks
Variables identifiedpopulation characteristicswork being assessedcourse levelnature of assessment criterianumber of teachers and students involved per assessmenttask
Their conclusion: On average, student marks agreed withteacher marks:
mean r=0.69 - the higher the bettermean effect size d=0.24 - the lower the better
11/52
![Page 30: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/30.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - A meta-analytic study
conducted a meta-analytic review of 56 studies comparingpeer and teacher marksVariables identified
population characteristicswork being assessedcourse levelnature of assessment criterianumber of teachers and students involved per assessmenttask
Their conclusion: On average, student marks agreed withteacher marks:
mean r=0.69 - the higher the bettermean effect size d=0.24 - the lower the better
11/52
![Page 31: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/31.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - A meta-analytic study
conducted a meta-analytic review of 56 studies comparingpeer and teacher marksVariables identified
population characteristicswork being assessedcourse levelnature of assessment criterianumber of teachers and students involved per assessmenttask
Their conclusion: On average, student marks agreed withteacher marks:
mean r=0.69 - the higher the bettermean effect size d=0.24 - the lower the better
11/52
![Page 32: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/32.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - Six Influential Factors
assessing individual dimensions vs overall judgementsusing well-specified criteria
The nature of the assessment task - educational product orprocess vs. professional practiceBetter experimental designs (e.g. sample sizes)→ betteragreementNumber of students involved per assessment taskThe subject area - less agreements in medical educationInvolving students in the development of assessmentcriteria→ better agreement
12/52
![Page 33: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/33.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - Six Influential Factors
assessing individual dimensions vs overall judgementsusing well-specified criteriaThe nature of the assessment task - educational product orprocess vs. professional practice
Better experimental designs (e.g. sample sizes)→ betteragreementNumber of students involved per assessment taskThe subject area - less agreements in medical educationInvolving students in the development of assessmentcriteria→ better agreement
12/52
![Page 34: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/34.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - Six Influential Factors
assessing individual dimensions vs overall judgementsusing well-specified criteriaThe nature of the assessment task - educational product orprocess vs. professional practiceBetter experimental designs (e.g. sample sizes)→ betteragreement
Number of students involved per assessment taskThe subject area - less agreements in medical educationInvolving students in the development of assessmentcriteria→ better agreement
12/52
![Page 35: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/35.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - Six Influential Factors
assessing individual dimensions vs overall judgementsusing well-specified criteriaThe nature of the assessment task - educational product orprocess vs. professional practiceBetter experimental designs (e.g. sample sizes)→ betteragreementNumber of students involved per assessment task
The subject area - less agreements in medical educationInvolving students in the development of assessmentcriteria→ better agreement
12/52
![Page 36: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/36.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - Six Influential Factors
assessing individual dimensions vs overall judgementsusing well-specified criteriaThe nature of the assessment task - educational product orprocess vs. professional practiceBetter experimental designs (e.g. sample sizes)→ betteragreementNumber of students involved per assessment taskThe subject area - less agreements in medical education
Involving students in the development of assessmentcriteria→ better agreement
12/52
![Page 37: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/37.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Falchikov and Goldfinch (2000) - Six Influential Factors
assessing individual dimensions vs overall judgementsusing well-specified criteriaThe nature of the assessment task - educational product orprocess vs. professional practiceBetter experimental designs (e.g. sample sizes)→ betteragreementNumber of students involved per assessment taskThe subject area - less agreements in medical educationInvolving students in the development of assessmentcriteria→ better agreement
12/52
![Page 38: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/38.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
13/52
![Page 39: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/39.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Inclusion Factors
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
14/52
![Page 40: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/40.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Inclusion Factors
The Selection Process
Keywords - peer assessment, peer grading, peerevaluation, peer review, peer feedback, peer interactionGoogle ScholarJournal articles and conference proceedings publishedsince 2000Not computer-based or web-based (Luxton-Reilly (2009)provides a comprehensive review)Final list included 64 studies
15/52
![Page 41: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/41.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
16/52
![Page 42: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/42.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Two Main Categories
Literature Reviews
Student involvementVariables of peer-assessmentQuality of peer-assessment
Case studies, action research and peer assessmentinstruments
The value of peer-feedbackPeer-assessment design strategiesPerceptions of students and teachersPsychological and social factors in peer-assessmentValidity and reliability of peer-assessment
17/52
![Page 43: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/43.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Two Main Categories
Literature ReviewsStudent involvementVariables of peer-assessmentQuality of peer-assessment
Case studies, action research and peer assessmentinstruments
The value of peer-feedbackPeer-assessment design strategiesPerceptions of students and teachersPsychological and social factors in peer-assessmentValidity and reliability of peer-assessment
17/52
![Page 44: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/44.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Two Main Categories
Literature ReviewsStudent involvementVariables of peer-assessmentQuality of peer-assessment
Case studies, action research and peer assessmentinstruments
The value of peer-feedbackPeer-assessment design strategiesPerceptions of students and teachersPsychological and social factors in peer-assessmentValidity and reliability of peer-assessment
17/52
![Page 45: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/45.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
18/52
![Page 46: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/46.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Student Involvement
Several studies recommend that students be activelyinvolved at various stages of PA
Falchikov (2003), Leenknecht et al. (2011), Bloxham &West (2004), Sluijimans et al. (2004)
PA must actively involve students to be effectivePA experiments should allow replicationclear instructions for students regarding processes involved
19/52
![Page 47: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/47.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Student Involvement
Several studies recommend that students be activelyinvolved at various stages of PAFalchikov (2003), Leenknecht et al. (2011), Bloxham &West (2004), Sluijimans et al. (2004)
PA must actively involve students to be effectivePA experiments should allow replicationclear instructions for students regarding processes involved
19/52
![Page 48: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/48.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van Zundert et al. (2010) reviewed 26 articles between1990 and 2007
Identified four variable categories
Psychometric qualitiesDomain-specific skillsPeer-assessment skillsstudents’ attitudes towards PA
20/52
![Page 49: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/49.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van Zundert et al. (2010) reviewed 26 articles between1990 and 2007Identified four variable categories
Psychometric qualitiesDomain-specific skillsPeer-assessment skillsstudents’ attitudes towards PA
20/52
![Page 50: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/50.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Topping (2010) - reveals many uncertainties in PA andidentifies 17 variables
Do peer-peer relationships affect the practice?Should peer-feedback be iterative or one-off?Is assigning multiple students to the same assessment taskeffective?inconsistencies, contradictory results, flaws or limitations ofstudies are revealed
21/52
![Page 51: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/51.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Topping (2010) - reveals many uncertainties in PA andidentifies 17 variables
Do peer-peer relationships affect the practice?Should peer-feedback be iterative or one-off?Is assigning multiple students to the same assessment taskeffective?inconsistencies, contradictory results, flaws or limitations ofstudies are revealed
21/52
![Page 52: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/52.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van den berg et al. (2006a) select 10 of Topping’s 17variablesImportant for optimal peer-assessment design
What is being assessed? Written work? Oralpresentation?, ...Is PA as substitute for teacher’s assessment?Is it mutual, anonymous?Is contact face-to-face?in-class, take-home?Are there any incentives?
22/52
![Page 53: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/53.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van den berg et al. (2006a) select 10 of Topping’s 17variablesImportant for optimal peer-assessment design
What is being assessed? Written work? Oralpresentation?, ...Is PA as substitute for teacher’s assessment?Is it mutual, anonymous?Is contact face-to-face?in-class, take-home?Are there any incentives?
22/52
![Page 54: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/54.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van den berg et al. (2006b) build upon previous researchImpact of variables on oral and written feedback
Peer-feedback is optimal when:
PA conducted in small groups, formative or summativeWritten feedback should be orally explained and discussedwith the assessedBut what about large classes?
23/52
![Page 55: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/55.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van den berg et al. (2006b) build upon previous researchImpact of variables on oral and written feedbackPeer-feedback is optimal when:
PA conducted in small groups, formative or summativeWritten feedback should be orally explained and discussedwith the assessedBut what about large classes?
23/52
![Page 56: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/56.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van Gennip et al. (2009) - interpersonal variables ingroup-based PA
Psychological safetyValue diversityInterdependence - responsible involvement (not specific togroup-based PA)Trust
24/52
![Page 57: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/57.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Variables of peer-assessment
Van Gennip et al. (2009) - interpersonal variables ingroup-based PA
Psychological safetyValue diversityInterdependence - responsible involvement (not specific togroup-based PA)Trust
24/52
![Page 58: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/58.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Quality of Peer-Assessment
Tillema et al. (2011) - How to measure quality of PApractices3 quality criteria should be met at all stages of theassessment process
Authenticity - process needs to actively engage students -representativeness, meaningfulness, cognitive complexity,content coverageTransparency - tasks should be clear, understandable, anddoableGeneralisability - can outcome be generalised to those oftasks measurin the same achievement? - comparability,reproducibility, educational consequences
25/52
![Page 59: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/59.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Quality of Peer-Assessment
Tillema et al. (2011) - How to measure quality of PApractices3 quality criteria should be met at all stages of theassessment process
Authenticity - process needs to actively engage students -representativeness, meaningfulness, cognitive complexity,content coverage
Transparency - tasks should be clear, understandable, anddoableGeneralisability - can outcome be generalised to those oftasks measurin the same achievement? - comparability,reproducibility, educational consequences
25/52
![Page 60: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/60.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Quality of Peer-Assessment
Tillema et al. (2011) - How to measure quality of PApractices3 quality criteria should be met at all stages of theassessment process
Authenticity - process needs to actively engage students -representativeness, meaningfulness, cognitive complexity,content coverageTransparency - tasks should be clear, understandable, anddoable
Generalisability - can outcome be generalised to those oftasks measurin the same achievement? - comparability,reproducibility, educational consequences
25/52
![Page 61: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/61.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Quality of Peer-Assessment
Tillema et al. (2011) - How to measure quality of PApractices3 quality criteria should be met at all stages of theassessment process
Authenticity - process needs to actively engage students -representativeness, meaningfulness, cognitive complexity,content coverageTransparency - tasks should be clear, understandable, anddoableGeneralisability - can outcome be generalised to those oftasks measurin the same achievement? - comparability,reproducibility, educational consequences
25/52
![Page 62: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/62.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Quality of Peer-Assessment
Gielen et al. (2011) - A contrasting viewThe quality being sought is determined by the goal of thePA taskPerhaps more practical; PA is implemented in manycontextsA single set of quality criteria may not be fitting to all
26/52
![Page 63: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/63.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
27/52
![Page 64: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/64.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Miller (2003) - Quality of peer-feedback determined byspecificity of criteria
The more specific the criteria, the more discriminative PA is- risks lowering feedback quality
Strijbos et al. (2010) - Is elaborate feedback good?
The majority of 89 grad students didn’t think soAdequate but had a negative impactDegree of specificity and brevity have varying impacts onstudents with different competence levels
Lin et al. (2001) - In general, specific feedback morehelpful than holistic feedback in improving performance
28/52
![Page 65: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/65.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Miller (2003) - Quality of peer-feedback determined byspecificity of criteria
The more specific the criteria, the more discriminative PA is- risks lowering feedback quality
Strijbos et al. (2010) - Is elaborate feedback good?
The majority of 89 grad students didn’t think soAdequate but had a negative impactDegree of specificity and brevity have varying impacts onstudents with different competence levels
Lin et al. (2001) - In general, specific feedback morehelpful than holistic feedback in improving performance
28/52
![Page 66: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/66.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Miller (2003) - Quality of peer-feedback determined byspecificity of criteria
The more specific the criteria, the more discriminative PA is- risks lowering feedback quality
Strijbos et al. (2010) - Is elaborate feedback good?
The majority of 89 grad students didn’t think soAdequate but had a negative impactDegree of specificity and brevity have varying impacts onstudents with different competence levels
Lin et al. (2001) - In general, specific feedback morehelpful than holistic feedback in improving performance
28/52
![Page 67: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/67.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Miller (2003) - Quality of peer-feedback determined byspecificity of criteria
The more specific the criteria, the more discriminative PA is- risks lowering feedback quality
Strijbos et al. (2010) - Is elaborate feedback good?The majority of 89 grad students didn’t think soAdequate but had a negative impactDegree of specificity and brevity have varying impacts onstudents with different competence levels
Lin et al. (2001) - In general, specific feedback morehelpful than holistic feedback in improving performance
28/52
![Page 68: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/68.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Miller (2003) - Quality of peer-feedback determined byspecificity of criteria
The more specific the criteria, the more discriminative PA is- risks lowering feedback quality
Strijbos et al. (2010) - Is elaborate feedback good?The majority of 89 grad students didn’t think soAdequate but had a negative impactDegree of specificity and brevity have varying impacts onstudents with different competence levels
Lin et al. (2001) - In general, specific feedback morehelpful than holistic feedback in improving performance
28/52
![Page 69: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/69.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Althauser & Darnall (2001), Tsai et al. (2002) - Studentswho provide high-quality feedback tend to incorporatefeedback from peers in their revisions.
Li et al. (2010) - Strong positive relationship between astudent’s quality of feedback and the quality of their ownfinal project.Cho and McArthur (2010) - Feedback from multiple peersis more helpful than that from just one.Hu (2005), Min (2006), Sluijsmans and Prins (2006), Saito(2008) - Training students in providing feedback and in PAskills, in general, improved quality of feedback and workbeing assessed.Chen & Tsai (2009) - Subsequent feedback tends toproduce marginal improvement in the quality of work beingassessed
29/52
![Page 70: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/70.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Althauser & Darnall (2001), Tsai et al. (2002) - Studentswho provide high-quality feedback tend to incorporatefeedback from peers in their revisions.Li et al. (2010) - Strong positive relationship between astudent’s quality of feedback and the quality of their ownfinal project.
Cho and McArthur (2010) - Feedback from multiple peersis more helpful than that from just one.Hu (2005), Min (2006), Sluijsmans and Prins (2006), Saito(2008) - Training students in providing feedback and in PAskills, in general, improved quality of feedback and workbeing assessed.Chen & Tsai (2009) - Subsequent feedback tends toproduce marginal improvement in the quality of work beingassessed
29/52
![Page 71: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/71.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Althauser & Darnall (2001), Tsai et al. (2002) - Studentswho provide high-quality feedback tend to incorporatefeedback from peers in their revisions.Li et al. (2010) - Strong positive relationship between astudent’s quality of feedback and the quality of their ownfinal project.Cho and McArthur (2010) - Feedback from multiple peersis more helpful than that from just one.
Hu (2005), Min (2006), Sluijsmans and Prins (2006), Saito(2008) - Training students in providing feedback and in PAskills, in general, improved quality of feedback and workbeing assessed.Chen & Tsai (2009) - Subsequent feedback tends toproduce marginal improvement in the quality of work beingassessed
29/52
![Page 72: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/72.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Althauser & Darnall (2001), Tsai et al. (2002) - Studentswho provide high-quality feedback tend to incorporatefeedback from peers in their revisions.Li et al. (2010) - Strong positive relationship between astudent’s quality of feedback and the quality of their ownfinal project.Cho and McArthur (2010) - Feedback from multiple peersis more helpful than that from just one.Hu (2005), Min (2006), Sluijsmans and Prins (2006), Saito(2008) - Training students in providing feedback and in PAskills, in general, improved quality of feedback and workbeing assessed.
Chen & Tsai (2009) - Subsequent feedback tends toproduce marginal improvement in the quality of work beingassessed
29/52
![Page 73: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/73.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
The Value of Peer-Feedback
Althauser & Darnall (2001), Tsai et al. (2002) - Studentswho provide high-quality feedback tend to incorporatefeedback from peers in their revisions.Li et al. (2010) - Strong positive relationship between astudent’s quality of feedback and the quality of their ownfinal project.Cho and McArthur (2010) - Feedback from multiple peersis more helpful than that from just one.Hu (2005), Min (2006), Sluijsmans and Prins (2006), Saito(2008) - Training students in providing feedback and in PAskills, in general, improved quality of feedback and workbeing assessed.Chen & Tsai (2009) - Subsequent feedback tends toproduce marginal improvement in the quality of work beingassessed
29/52
![Page 74: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/74.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Topping et al. (2000)
PA conducted in a class of 12 grad studentsFormativeProduct assessed - end-of-second-term academic reportMandatory participation, PA results did not contribute tofinal marksOut-of-class, anonymous, reciprocal14 specific criteria providedStudy sought to investigate peer and teacher scoreagreementsConclusions:
Adequate reliability and validity of the approachMay, however, not generalise to other settings
30/52
![Page 75: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/75.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Topping et al. (2000)PA conducted in a class of 12 grad studentsFormativeProduct assessed - end-of-second-term academic reportMandatory participation, PA results did not contribute tofinal marksOut-of-class, anonymous, reciprocal14 specific criteria providedStudy sought to investigate peer and teacher scoreagreements
Conclusions:
Adequate reliability and validity of the approachMay, however, not generalise to other settings
30/52
![Page 76: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/76.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Topping et al. (2000)PA conducted in a class of 12 grad studentsFormativeProduct assessed - end-of-second-term academic reportMandatory participation, PA results did not contribute tofinal marksOut-of-class, anonymous, reciprocal14 specific criteria providedStudy sought to investigate peer and teacher scoreagreementsConclusions:
Adequate reliability and validity of the approachMay, however, not generalise to other settings
30/52
![Page 77: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/77.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Ballantyne et al. (2002) - One of the largest PA studies
A three-phase study spanning a two-year period1654 students and 30 staff from three departmentsPA procedures outlined and revised together with studentsShortcomings - assessment was manual, anonymity wasnot preserved in some departmentsIncrease in student load - required to meet outside class toexchange assignments and agree on final grades, risk ofbiasOtherwise a thoroughly designed high quality study
31/52
![Page 78: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/78.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Ballantyne et al. (2002) - One of the largest PA studiesA three-phase study spanning a two-year period1654 students and 30 staff from three departmentsPA procedures outlined and revised together with studentsShortcomings - assessment was manual, anonymity wasnot preserved in some departmentsIncrease in student load - required to meet outside class toexchange assignments and agree on final grades, risk ofbiasOtherwise a thoroughly designed high quality study
31/52
![Page 79: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/79.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Automating peer-assessment tasks has severaladvantages
teachers can enjoy PA advantages less the negativeimpacts discussedanonymity, efficient assignment distribution, discussion,and submission of grades easily guaranteedautomation could also help calibrate grades assigned bymultiple peers (Hamer et al. 2005)
32/52
![Page 80: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/80.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Automating peer-assessment tasks has severaladvantagesteachers can enjoy PA advantages less the negativeimpacts discussedanonymity, efficient assignment distribution, discussion,and submission of grades easily guaranteedautomation could also help calibrate grades assigned bymultiple peers (Hamer et al. 2005)
32/52
![Page 81: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/81.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Some variations
the teacher assessing the quality of feedback instead ofanalysing peer-assigned marks (Davies 2006)PA without explicit assessment criteria (Jones & Alcock2014)
33/52
![Page 82: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/82.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Peer-Assessment Design Strategies
Some variationsthe teacher assessing the quality of feedback instead ofanalysing peer-assigned marks (Davies 2006)PA without explicit assessment criteria (Jones & Alcock2014)
33/52
![Page 83: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/83.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Overall positive perceptions of students reported by:McLaughlin & Simpson (2004), Saito & Fujita (2004), Wen& Tsai (2006), Wen et al. (2008), McGarr & Clifford (2013)Chang (2006), Kwok (2008), Wood & Kruzel (2008), XIao &Lucking (2008)
PA is productive and gives me a clearer view of howteachers assess students (Hanrahan & Isaacs 2001)Increased responsibility for others and improved learning(Papinczak et al. 2007)Time-intensive, intellectually challenging, creates a sociallyuncomfortable environment (Topping et al. 2000, Hanrahan& Isaacs 2001, Arnold et al. 2005, Praver et al. 2011)
34/52
![Page 84: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/84.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Overall positive perceptions of students reported by:McLaughlin & Simpson (2004), Saito & Fujita (2004), Wen& Tsai (2006), Wen et al. (2008), McGarr & Clifford (2013)Chang (2006), Kwok (2008), Wood & Kruzel (2008), XIao &Lucking (2008)
PA is productive and gives me a clearer view of howteachers assess students (Hanrahan & Isaacs 2001)
Increased responsibility for others and improved learning(Papinczak et al. 2007)Time-intensive, intellectually challenging, creates a sociallyuncomfortable environment (Topping et al. 2000, Hanrahan& Isaacs 2001, Arnold et al. 2005, Praver et al. 2011)
34/52
![Page 85: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/85.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Overall positive perceptions of students reported by:McLaughlin & Simpson (2004), Saito & Fujita (2004), Wen& Tsai (2006), Wen et al. (2008), McGarr & Clifford (2013)Chang (2006), Kwok (2008), Wood & Kruzel (2008), XIao &Lucking (2008)
PA is productive and gives me a clearer view of howteachers assess students (Hanrahan & Isaacs 2001)Increased responsibility for others and improved learning(Papinczak et al. 2007)
Time-intensive, intellectually challenging, creates a sociallyuncomfortable environment (Topping et al. 2000, Hanrahan& Isaacs 2001, Arnold et al. 2005, Praver et al. 2011)
34/52
![Page 86: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/86.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Overall positive perceptions of students reported by:McLaughlin & Simpson (2004), Saito & Fujita (2004), Wen& Tsai (2006), Wen et al. (2008), McGarr & Clifford (2013)Chang (2006), Kwok (2008), Wood & Kruzel (2008), XIao &Lucking (2008)
PA is productive and gives me a clearer view of howteachers assess students (Hanrahan & Isaacs 2001)Increased responsibility for others and improved learning(Papinczak et al. 2007)Time-intensive, intellectually challenging, creates a sociallyuncomfortable environment (Topping et al. 2000, Hanrahan& Isaacs 2001, Arnold et al. 2005, Praver et al. 2011)
34/52
![Page 87: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/87.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Summative PA undermines learning, especially whenthere’s no feedback (Sluijsmans et al. 2001, Papinczak etal. 2007)
A large survey (1740 students and 460 faculty) found thatsummative PA is considered ineffective and studentsquestion the reliability and expertise of their peers (Liu &Carless 2006)Responses seem favourable of PA as students progressthrough subsequent PA tasks and editions (Sluijsmans etal. 2003)
35/52
![Page 88: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/88.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Summative PA undermines learning, especially whenthere’s no feedback (Sluijsmans et al. 2001, Papinczak etal. 2007)A large survey (1740 students and 460 faculty) found thatsummative PA is considered ineffective and studentsquestion the reliability and expertise of their peers (Liu &Carless 2006)
Responses seem favourable of PA as students progressthrough subsequent PA tasks and editions (Sluijsmans etal. 2003)
35/52
![Page 89: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/89.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Perceptions of Students and Teachers
Summative PA undermines learning, especially whenthere’s no feedback (Sluijsmans et al. 2001, Papinczak etal. 2007)A large survey (1740 students and 460 faculty) found thatsummative PA is considered ineffective and studentsquestion the reliability and expertise of their peers (Liu &Carless 2006)Responses seem favourable of PA as students progressthrough subsequent PA tasks and editions (Sluijsmans etal. 2003)
35/52
![Page 90: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/90.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Psychological and Social Factors in Peer-Assessment
Gender effects are the least studies factors in PA in highereducation (Falchikov & Goldfinch 2000, Falchikov 2003,Topping 2010)
Bias may not be an issue when PA is anonymousThe most affected are those which involve visual contactbetween peersA study involving 41 undergrads (20 females) found thatmales rated males slightly higher than female presenters(Langan et al. 2005)This was not the case for females - (Langan et al. 2005,Langan et al. 2008)A study of 40 students involved in a PA task (20 females)reported that female students found it a stressful task(Pope 2005).
36/52
![Page 91: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/91.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Psychological and Social Factors in Peer-Assessment
Gender effects are the least studies factors in PA in highereducation (Falchikov & Goldfinch 2000, Falchikov 2003,Topping 2010)Bias may not be an issue when PA is anonymous
The most affected are those which involve visual contactbetween peersA study involving 41 undergrads (20 females) found thatmales rated males slightly higher than female presenters(Langan et al. 2005)This was not the case for females - (Langan et al. 2005,Langan et al. 2008)A study of 40 students involved in a PA task (20 females)reported that female students found it a stressful task(Pope 2005).
36/52
![Page 92: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/92.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Psychological and Social Factors in Peer-Assessment
Gender effects are the least studies factors in PA in highereducation (Falchikov & Goldfinch 2000, Falchikov 2003,Topping 2010)Bias may not be an issue when PA is anonymousThe most affected are those which involve visual contactbetween peers
A study involving 41 undergrads (20 females) found thatmales rated males slightly higher than female presenters(Langan et al. 2005)This was not the case for females - (Langan et al. 2005,Langan et al. 2008)A study of 40 students involved in a PA task (20 females)reported that female students found it a stressful task(Pope 2005).
36/52
![Page 93: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/93.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Psychological and Social Factors in Peer-Assessment
Gender effects are the least studies factors in PA in highereducation (Falchikov & Goldfinch 2000, Falchikov 2003,Topping 2010)Bias may not be an issue when PA is anonymousThe most affected are those which involve visual contactbetween peersA study involving 41 undergrads (20 females) found thatmales rated males slightly higher than female presenters(Langan et al. 2005)
This was not the case for females - (Langan et al. 2005,Langan et al. 2008)A study of 40 students involved in a PA task (20 females)reported that female students found it a stressful task(Pope 2005).
36/52
![Page 94: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/94.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Psychological and Social Factors in Peer-Assessment
Gender effects are the least studies factors in PA in highereducation (Falchikov & Goldfinch 2000, Falchikov 2003,Topping 2010)Bias may not be an issue when PA is anonymousThe most affected are those which involve visual contactbetween peersA study involving 41 undergrads (20 females) found thatmales rated males slightly higher than female presenters(Langan et al. 2005)This was not the case for females - (Langan et al. 2005,Langan et al. 2008)
A study of 40 students involved in a PA task (20 females)reported that female students found it a stressful task(Pope 2005).
36/52
![Page 95: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/95.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Psychological and Social Factors in Peer-Assessment
Gender effects are the least studies factors in PA in highereducation (Falchikov & Goldfinch 2000, Falchikov 2003,Topping 2010)Bias may not be an issue when PA is anonymousThe most affected are those which involve visual contactbetween peersA study involving 41 undergrads (20 females) found thatmales rated males slightly higher than female presenters(Langan et al. 2005)This was not the case for females - (Langan et al. 2005,Langan et al. 2008)A study of 40 students involved in a PA task (20 females)reported that female students found it a stressful task(Pope 2005).
36/52
![Page 96: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/96.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
These are the most common studies
Validity - how similar are teacher and peer marks?Reliability - How close are scores assigned by peers(teachers) to the same work? AKA - Inter-rater reliability15 studies were examined8 reported correlation coefficients4 reported mean and standard deviation - effect sizes (d)were computed
d = 2∗[mean(eg)−mean(cg)]sd(eg)+sd(cg)
eg = experimental group, cg = control group
37/52
![Page 97: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/97.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
These are the most common studiesValidity - how similar are teacher and peer marks?
Reliability - How close are scores assigned by peers(teachers) to the same work? AKA - Inter-rater reliability15 studies were examined8 reported correlation coefficients4 reported mean and standard deviation - effect sizes (d)were computed
d = 2∗[mean(eg)−mean(cg)]sd(eg)+sd(cg)
eg = experimental group, cg = control group
37/52
![Page 98: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/98.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
These are the most common studiesValidity - how similar are teacher and peer marks?Reliability - How close are scores assigned by peers(teachers) to the same work? AKA - Inter-rater reliability
15 studies were examined8 reported correlation coefficients4 reported mean and standard deviation - effect sizes (d)were computed
d = 2∗[mean(eg)−mean(cg)]sd(eg)+sd(cg)
eg = experimental group, cg = control group
37/52
![Page 99: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/99.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
These are the most common studiesValidity - how similar are teacher and peer marks?Reliability - How close are scores assigned by peers(teachers) to the same work? AKA - Inter-rater reliability15 studies were examined8 reported correlation coefficients4 reported mean and standard deviation - effect sizes (d)were computed
d = 2∗[mean(eg)−mean(cg)]sd(eg)+sd(cg)
eg = experimental group, cg = control group
37/52
![Page 100: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/100.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
These are the most common studiesValidity - how similar are teacher and peer marks?Reliability - How close are scores assigned by peers(teachers) to the same work? AKA - Inter-rater reliability15 studies were examined8 reported correlation coefficients4 reported mean and standard deviation - effect sizes (d)were computed
d = 2∗[mean(eg)−mean(cg)]sd(eg)+sd(cg)
eg = experimental group, cg = control group
37/52
![Page 101: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/101.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
7 design quality criteria (Falchikov & Goldfinch 2000) +anonymity
Population characteristics reported?Subject area reported?What is assessed, at what level (intro, intermediate,advanced)What instrument and criteria were used, if any?What statistics were reported?# of teachers and students involved per assessment taskDid assessment contribute to final grades?Was assessment anonymous?
Studies missing at least 4 criteria→ poor design (N=3)The rest (N=12)→ high quality (at least 2 missing in many)
38/52
![Page 102: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/102.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
7 design quality criteria (Falchikov & Goldfinch 2000) +anonymity
Population characteristics reported?Subject area reported?What is assessed, at what level (intro, intermediate,advanced)What instrument and criteria were used, if any?What statistics were reported?# of teachers and students involved per assessment taskDid assessment contribute to final grades?Was assessment anonymous?
Studies missing at least 4 criteria→ poor design (N=3)The rest (N=12)→ high quality (at least 2 missing in many)
38/52
![Page 103: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/103.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl
7 design quality criteria (Falchikov & Goldfinch 2000) +anonymity
Population characteristics reported?Subject area reported?What is assessed, at what level (intro, intermediate,advanced)What instrument and criteria were used, if any?What statistics were reported?# of teachers and students involved per assessment taskDid assessment contribute to final grades?Was assessment anonymous?
Studies missing at least 4 criteria→ poor design (N=3)The rest (N=12)→ high quality (at least 2 missing in many)
38/52
![Page 104: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/104.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - DesignPitfalls
Not reporting any statistics (Lindblom-ylanne et al. 2006)
Reporting incomplete or imprecise statistics - e.g. barcharts providing only approximate information (Cho et al.2006)Violation of the very definition of peer-assessment
asking students to assess class participation or effort (Ryanet al. 2007)using students who do not participate in creating theproduct being assessed (De Grez et al. 2012)
Partial anonymity, varied treatment of EGs and CGs (Xiaoand Lucking 2008)Other missing information - age, gender, anonymity,contribution towards final grade, course level
39/52
![Page 105: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/105.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - DesignPitfalls
Not reporting any statistics (Lindblom-ylanne et al. 2006)Reporting incomplete or imprecise statistics - e.g. barcharts providing only approximate information (Cho et al.2006)
Violation of the very definition of peer-assessment
asking students to assess class participation or effort (Ryanet al. 2007)using students who do not participate in creating theproduct being assessed (De Grez et al. 2012)
Partial anonymity, varied treatment of EGs and CGs (Xiaoand Lucking 2008)Other missing information - age, gender, anonymity,contribution towards final grade, course level
39/52
![Page 106: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/106.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - DesignPitfalls
Not reporting any statistics (Lindblom-ylanne et al. 2006)Reporting incomplete or imprecise statistics - e.g. barcharts providing only approximate information (Cho et al.2006)Violation of the very definition of peer-assessment
asking students to assess class participation or effort (Ryanet al. 2007)using students who do not participate in creating theproduct being assessed (De Grez et al. 2012)
Partial anonymity, varied treatment of EGs and CGs (Xiaoand Lucking 2008)Other missing information - age, gender, anonymity,contribution towards final grade, course level
39/52
![Page 107: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/107.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - DesignPitfalls
Not reporting any statistics (Lindblom-ylanne et al. 2006)Reporting incomplete or imprecise statistics - e.g. barcharts providing only approximate information (Cho et al.2006)Violation of the very definition of peer-assessment
asking students to assess class participation or effort (Ryanet al. 2007)using students who do not participate in creating theproduct being assessed (De Grez et al. 2012)
Partial anonymity, varied treatment of EGs and CGs (Xiaoand Lucking 2008)
Other missing information - age, gender, anonymity,contribution towards final grade, course level
39/52
![Page 108: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/108.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - DesignPitfalls
Not reporting any statistics (Lindblom-ylanne et al. 2006)Reporting incomplete or imprecise statistics - e.g. barcharts providing only approximate information (Cho et al.2006)Violation of the very definition of peer-assessment
asking students to assess class participation or effort (Ryanet al. 2007)using students who do not participate in creating theproduct being assessed (De Grez et al. 2012)
Partial anonymity, varied treatment of EGs and CGs (Xiaoand Lucking 2008)Other missing information - age, gender, anonymity,contribution towards final grade, course level
39/52
![Page 109: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/109.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - Results
Mean correlation coefficient(r) of 0.80 and mean effectsizes(d) of 0.27
Corroborates findings by Falchikov & Goldfinch (2000),although with much smaller studiesMost studies varied in the design of assessment tasks
Products assessed - written work, oral presentationDisciplines - education, business, law, medical education,computer science and engineeringStats reported - correlation coefficients, one-way & multipleANOVA, Cronbach’s alpha, t-tests, intraclass correlation,mean and SD
40/52
![Page 110: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/110.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - Results
Mean correlation coefficient(r) of 0.80 and mean effectsizes(d) of 0.27Corroborates findings by Falchikov & Goldfinch (2000),although with much smaller studies
Most studies varied in the design of assessment tasks
Products assessed - written work, oral presentationDisciplines - education, business, law, medical education,computer science and engineeringStats reported - correlation coefficients, one-way & multipleANOVA, Cronbach’s alpha, t-tests, intraclass correlation,mean and SD
40/52
![Page 111: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/111.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Themes of Interest
Validity and Reliability of Peer-Assessmentl - Results
Mean correlation coefficient(r) of 0.80 and mean effectsizes(d) of 0.27Corroborates findings by Falchikov & Goldfinch (2000),although with much smaller studiesMost studies varied in the design of assessment tasks
Products assessed - written work, oral presentationDisciplines - education, business, law, medical education,computer science and engineeringStats reported - correlation coefficients, one-way & multipleANOVA, Cronbach’s alpha, t-tests, intraclass correlation,mean and SD
40/52
![Page 112: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/112.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
41/52
![Page 113: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/113.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Focus of this study was on PA in higher education
Variables of interest have led to a multitude of designstrategiesCommendable studies providing insight into the intricaciesof PA practice
Cho et al. (2006)Ozogul & Sullivan (2009)Smith et al. (2002)Xiao & Lucking (2008)Sahin (2008)
Maintaining anonymity in manual PA becomes a luxury asthe number of students involved increasesLack of common standards - most studies are not readilycomparable
42/52
![Page 114: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/114.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Focus of this study was on PA in higher educationVariables of interest have led to a multitude of designstrategies
Commendable studies providing insight into the intricaciesof PA practice
Cho et al. (2006)Ozogul & Sullivan (2009)Smith et al. (2002)Xiao & Lucking (2008)Sahin (2008)
Maintaining anonymity in manual PA becomes a luxury asthe number of students involved increasesLack of common standards - most studies are not readilycomparable
42/52
![Page 115: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/115.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Focus of this study was on PA in higher educationVariables of interest have led to a multitude of designstrategiesCommendable studies providing insight into the intricaciesof PA practice
Cho et al. (2006)Ozogul & Sullivan (2009)Smith et al. (2002)Xiao & Lucking (2008)Sahin (2008)
Maintaining anonymity in manual PA becomes a luxury asthe number of students involved increasesLack of common standards - most studies are not readilycomparable
42/52
![Page 116: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/116.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Focus of this study was on PA in higher educationVariables of interest have led to a multitude of designstrategiesCommendable studies providing insight into the intricaciesof PA practice
Cho et al. (2006)Ozogul & Sullivan (2009)Smith et al. (2002)Xiao & Lucking (2008)Sahin (2008)
Maintaining anonymity in manual PA becomes a luxury asthe number of students involved increases
Lack of common standards - most studies are not readilycomparable
42/52
![Page 117: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/117.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Focus of this study was on PA in higher educationVariables of interest have led to a multitude of designstrategiesCommendable studies providing insight into the intricaciesof PA practice
Cho et al. (2006)Ozogul & Sullivan (2009)Smith et al. (2002)Xiao & Lucking (2008)Sahin (2008)
Maintaining anonymity in manual PA becomes a luxury asthe number of students involved increasesLack of common standards - most studies are not readilycomparable
42/52
![Page 118: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/118.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Most studies mix experiments and attempt to measureseveral variables - mixed results?
No attempts to take advantage of advances in relateddisciplinesThe vast majority are standalone practices in conventionalclassroomsAdvances in computer science are being applied in almostall social systemsPA has yet to take advantage of these - So far, web-basedPA only
43/52
![Page 119: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/119.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Most studies mix experiments and attempt to measureseveral variables - mixed results?No attempts to take advantage of advances in relateddisciplines
The vast majority are standalone practices in conventionalclassroomsAdvances in computer science are being applied in almostall social systemsPA has yet to take advantage of these - So far, web-basedPA only
43/52
![Page 120: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/120.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Most studies mix experiments and attempt to measureseveral variables - mixed results?No attempts to take advantage of advances in relateddisciplinesThe vast majority are standalone practices in conventionalclassrooms
Advances in computer science are being applied in almostall social systemsPA has yet to take advantage of these - So far, web-basedPA only
43/52
![Page 121: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/121.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Most studies mix experiments and attempt to measureseveral variables - mixed results?No attempts to take advantage of advances in relateddisciplinesThe vast majority are standalone practices in conventionalclassroomsAdvances in computer science are being applied in almostall social systems
PA has yet to take advantage of these - So far, web-basedPA only
43/52
![Page 122: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/122.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Most studies mix experiments and attempt to measureseveral variables - mixed results?No attempts to take advantage of advances in relateddisciplinesThe vast majority are standalone practices in conventionalclassroomsAdvances in computer science are being applied in almostall social systemsPA has yet to take advantage of these - So far, web-basedPA only
43/52
![Page 123: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/123.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Majority PA practices are one-off experiments - how do wetest if it helps long-term learning?
Having PA practice as part of a curriculum is a riskybusiness - who are the stakeholders?Most studies are disconnected and only few build uponprevious studiesLack of studies regarding impacts of gender, race,anonymity, academic dishonestyHow about impact of formative peer-assessment onstudents’ performance on end-of-course exams?Manual peer-assessment lays more burden on bothstudents and teachers
44/52
![Page 124: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/124.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Majority PA practices are one-off experiments - how do wetest if it helps long-term learning?Having PA practice as part of a curriculum is a riskybusiness - who are the stakeholders?
Most studies are disconnected and only few build uponprevious studiesLack of studies regarding impacts of gender, race,anonymity, academic dishonestyHow about impact of formative peer-assessment onstudents’ performance on end-of-course exams?Manual peer-assessment lays more burden on bothstudents and teachers
44/52
![Page 125: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/125.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Majority PA practices are one-off experiments - how do wetest if it helps long-term learning?Having PA practice as part of a curriculum is a riskybusiness - who are the stakeholders?Most studies are disconnected and only few build uponprevious studies
Lack of studies regarding impacts of gender, race,anonymity, academic dishonestyHow about impact of formative peer-assessment onstudents’ performance on end-of-course exams?Manual peer-assessment lays more burden on bothstudents and teachers
44/52
![Page 126: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/126.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Majority PA practices are one-off experiments - how do wetest if it helps long-term learning?Having PA practice as part of a curriculum is a riskybusiness - who are the stakeholders?Most studies are disconnected and only few build uponprevious studiesLack of studies regarding impacts of gender, race,anonymity, academic dishonesty
How about impact of formative peer-assessment onstudents’ performance on end-of-course exams?Manual peer-assessment lays more burden on bothstudents and teachers
44/52
![Page 127: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/127.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Majority PA practices are one-off experiments - how do wetest if it helps long-term learning?Having PA practice as part of a curriculum is a riskybusiness - who are the stakeholders?Most studies are disconnected and only few build uponprevious studiesLack of studies regarding impacts of gender, race,anonymity, academic dishonestyHow about impact of formative peer-assessment onstudents’ performance on end-of-course exams?
Manual peer-assessment lays more burden on bothstudents and teachers
44/52
![Page 128: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/128.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
In Summary
Majority PA practices are one-off experiments - how do wetest if it helps long-term learning?Having PA practice as part of a curriculum is a riskybusiness - who are the stakeholders?Most studies are disconnected and only few build uponprevious studiesLack of studies regarding impacts of gender, race,anonymity, academic dishonestyHow about impact of formative peer-assessment onstudents’ performance on end-of-course exams?Manual peer-assessment lays more burden on bothstudents and teachers
44/52
![Page 129: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/129.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
45/52
![Page 130: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/130.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
Exploring the applicability of educational gamesSome positive results of introducing educational games inthe physical sciencesAlthough most studies focus on K-12 educationThorough reviews of educational games - Randel et al.(1992), Wu et al. (2012)CS advances may help with efficient integration ofeducational games into peer-assessment practicesa way of eliciting participation through collaborative andcompetitive games
46/52
![Page 131: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/131.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
Automation of peer-assessment tasks could help teachersintroduce healthy competition into PA processesAutomation also improves efficiency of PA processes -randomised distribution, collection, marking of PA tasksAutomation helps conduct iterative PA experiments, andmultiple rounds of feedback and reviewAutomation can easily guarantee anonymityAutomation opens the door to ubiquitous learningenvironments (Jones & Jo 2004, Sun & Shen 2014)Automation reduces teacher workload (Bouzidi & Jaillet2009)
47/52
![Page 132: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/132.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
Automation - Advanced opportunitiesapplication tools that detect academic dishonesty,automated essay scoring, social network analysis,automated calibration of peer-assigned scores (Hamer etal. 2005, Giovannella & Scaccia 2014)student performance prediction models based onpeer-assessment data (Ahenafi, Riccardi & Ronchetti,2015)
Is students’ bias regarding their peers’ abilities logical? -Anonymity may provide the answerTeacher plays a student in an automated peer-assessmentenvironment
48/52
![Page 133: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/133.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
Automation - Advanced opportunitiesapplication tools that detect academic dishonesty,automated essay scoring, social network analysis,automated calibration of peer-assigned scores (Hamer etal. 2005, Giovannella & Scaccia 2014)student performance prediction models based onpeer-assessment data (Ahenafi, Riccardi & Ronchetti,2015)
Is students’ bias regarding their peers’ abilities logical? -Anonymity may provide the answer
Teacher plays a student in an automated peer-assessmentenvironment
48/52
![Page 134: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/134.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
Automation - Advanced opportunitiesapplication tools that detect academic dishonesty,automated essay scoring, social network analysis,automated calibration of peer-assigned scores (Hamer etal. 2005, Giovannella & Scaccia 2014)student performance prediction models based onpeer-assessment data (Ahenafi, Riccardi & Ronchetti,2015)
Is students’ bias regarding their peers’ abilities logical? -Anonymity may provide the answerTeacher plays a student in an automated peer-assessmentenvironment
48/52
![Page 135: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/135.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
All in allwe still need robust design quality and measurementstandards - still waiting for the first symposium on PA
An opportune time for scholars in education and computerscience to forge collaborationsNot a practice within education anymore - 21st centuryPA is interdisciplinary
49/52
![Page 136: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/136.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
All in allwe still need robust design quality and measurementstandards - still waiting for the first symposium on PAAn opportune time for scholars in education and computerscience to forge collaborations
Not a practice within education anymore - 21st centuryPA is interdisciplinary
49/52
![Page 137: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/137.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
The Way Forward
All in allwe still need robust design quality and measurementstandards - still waiting for the first symposium on PAAn opportune time for scholars in education and computerscience to forge collaborationsNot a practice within education anymore - 21st centuryPA is interdisciplinary
49/52
![Page 138: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/138.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Outline
1 Introduction
2 20th Century Peer-Assessment
3 21st Century Peer-AssessmentInclusion FactorsThemes of Interest
Literature ReviewsCase studies, action research and peer assessment instruments
4 Discussion
5 Recommendations
6 End of Talk
50/52
![Page 139: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/139.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
References
All references can be retrieved from the article discussed in thistalk
Michael Mogessie Ashenafi (2015): Peer-assessment inhigher education twenty-first century practices, challengesand the way forward, Assessment & Evaluation in HigherEducation, DOI: 10.1080/02602938.2015.1100711
51/52
![Page 140: Peer-Assessment in Higher Education: A Review of Recent Studieslatemar.science.unitn.it/segue_userFiles/EMA-CSL/michael_mogessie... · Peer-Assessment in Higher Education: A Review](https://reader036.fdocuments.us/reader036/viewer/2022070903/5f5df27fa2c6210a1a606acd/html5/thumbnails/140.jpg)
Outline Introduction 20th Century Peer-Assessment 21st Century Peer-Assessment Discussion Recommendations End of Talk
Thank you all!
Questions?
52/52