Provision of personalized feedback at scale using learning analytics
-
Upload
abelardo-pardo -
Category
Education
-
view
6.803 -
download
0
Transcript of Provision of personalized feedback at scale using learning analytics
![Page 1: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/1.jpg)
Provision of personalized feedback at scale using learning analytics
Centre for the Enhancement of Teaching and Learning Faculty of Education Hong Kong University 19/May/2017
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
Antonietta flickr.com
![Page 2: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/2.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 2
Maureen Barlin flickr.com
NewDesignMindset
The role of data
and feedback
Personalised feedback at scale
![Page 3: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/3.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 3
Maureen Barlin flickr.com
New Design Mindset
![Page 4: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/4.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 4
Simple information transfer is not working
Mazur, E. (2009). Farewell, lecture. Science, 323(5910), 50-51.
Krugazor flickr.com
![Page 5: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/5.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 5
![Page 6: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/6.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 6
Active Learning Works
Engage students in the learning process
Bonwell, C. C., & Eison, J. A. (1991). Active Learning: Creating Excitement in the Classroom ASHEERIC Higher Education Report No. 1. Washington, DC, USA: George Washington University.
![Page 7: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/7.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 7
“… robust correlations between student involvement in a subset of ‘educationally
purposive activities’, and positive outcomes of student success and development, including
satisfaction, persistence, academic achievement and social engagement”
Trowler, V. (2010). Student engagement literature review. York, UK: The Higher Education Academy.
Joshua Ganderson flickr.com
![Page 8: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/8.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 8
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823
Christian W
eidinger flickr.com
We, as learners may
• Not know how to promote comprehension, retention, transfer.
• Not assess properly our own learning
• Be biased when judging our learning
• Rely too much on social beliefs
![Page 9: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/9.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 9
Janson Hew
s flickr.com
Frontier between physical and virtual spaces is blurring
![Page 10: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/10.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 10
Beware of technology creating the illusion of rational thinking
Jen R flickr.com
![Page 11: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/11.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 11
“… teaching in higher education will necessarily shift the balance of its efforts towards a greater investment in design
as a way of coping with otherwise intolerable pressures on staff and
resources.”
Goodyear, P. (2015). Teaching as Design. HERDSA Review of Higher Education, 2, 27-50.
![Page 12: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/12.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 12
"There is no such thing as a neutral design"
Jeremy Brooks flickr.com
Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press.
“People make good choices in contexts in which they have experience, good information, and
prompt feedback"
![Page 13: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/13.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 13
“38 meta-analyses investigating 105 correlates of achievement, based on 3,330 effect sizes from almost 2 million students”
Schneider, M., & Preckel, F. (2017). Variables Associated With Achievement in Higher Education: A Systematic Review of Meta-Analyses. Psychological Bulletin. doi:10.1037/bul0000098
• The effectivity of courses is strongly related to what teachers do.
• The effectivity of teaching methods depends on how are implemented
• Teachers can improve the instructional quality of their courses by making a number of small changes
- providing detailed task-focused and improvement-oriented feedback
• The combination of teacher-cantered and student-cantered instructional elements is more effective than either form of instruction alone
Variables associated with achievement
![Page 14: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/14.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 14
Maureen Barlin flickr.com
NewDesignMindset
The roleof data
and feedback
![Page 15: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/15.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 15
Students are less likely to engage in pre-class activities if they are not interactive, do not provide formative feedback, and
not coherently linked with the face-to-face activities
O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85-95. doi:10.1016/j.iheduc.2015.02.002
Dan Klim
ke flickr.com
![Page 16: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/16.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics
If you could choose one…
• Over 800 meta-analyses of student achievements
• 100 factors with potential influence
• Feedback in top five
• (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
16Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses related to achievement. New York: Routledge.
![Page 17: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/17.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 17
Boud, D., & Molloy, E. (Eds.). (2013). Feedback in Higher and Professional Education: Understanding it and doing it well. London and New York: Routledge.
Faruk Ateş flickr.com
“Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work”
![Page 18: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/18.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 18
Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. doi:10.1080/0969595980050102
Gareth C
hristopher flickr.com
Innovations designed to strengthen the frequent feedback that students receive about their learning yield substantial learning gains
![Page 19: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/19.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 19
Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.
Perceived as an administrative chore instead of a pedagogical necessity
Marcin W
ichary flickr.com
![Page 20: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/20.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 20
Ian Ransley flickr.com
How to scale sustainable practices?
Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. doi:10.1080/03075071003642449
• Multi-stage assignments
• Dialogic feedback
• Technology supported
• Self-evaluation
![Page 21: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/21.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 21
Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. doi:10.1007/s40593-016-0105-0
From artificial intelligence to intelligence amplification
Instructors are informed about student engagement but it is up to them to decide if/when/how to act
Tecnalia flickr.com
![Page 22: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/22.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 22
Dm
itry Grigoriev flickr.com
Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs
1st International Conference on Learning Analytics and Knowledge [Online]. https://tekri.athabascau.ca/analytics. [Accessed 2017].
![Page 23: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/23.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 23
• Collect data about how students engage in a learning experience
• Interpret the observations in the context of the instructional design
• Translate knowledge into personalised student support actions
![Page 24: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/24.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 24
Bartimote-Aufflick, K., Reimann P., Pardo, A. The perspective theory brings to learning analytics in the classroom: A realist approach Manuscript in preparation.
Learning Analytics Model
![Page 25: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/25.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 25
Maureen Barlin flickr.com
NewDesignMindset
The role of data
and feedback
Personalised feedbackat scale
![Page 26: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/26.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 26
Example of Highly Instrumentalized Learning Design
![Page 27: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/27.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 27
Weekly real time feedback
![Page 28: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/28.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 28
No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17
![Page 29: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/29.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 29
You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?
Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?
Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.
Thorough work with the task about machine language encoding. Give it a quick review before the midterm.
Q1 Q2 Q3 Q4
Instructor
![Page 30: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/30.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 30
Algorithm
![Page 31: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/31.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 31
AutomaticEmail
![Page 32: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/32.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 32
Helpful feedback
Effect size (Cohen’s d) = 0.49. Medium positive effect
Midterm Scores
Effect size (Cohen’s d) = 0.21. Small positive effect
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D. & Mirriahi, N. (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback. Manuscript in preparation
![Page 33: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/33.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 33
ontasklearning.org
![Page 34: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/34.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 34
• Support instructors to create personalised feedback
• Simple rule-base knowledge encoding
• Provide appropriate view of data sources
• Scale to large and highly diverse cohorts
• Will be released as open-source project Q3/4 2017
• First pilots in Q1/2 2017
• Tutorial in LAK 2017
• Contact us if interested
ontasklearning.org
![Page 35: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/35.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 35
MTC
0316 flickr.com
Conclusions• New L&T design mindset required • Feedback is effective to promote
student engagement • Learning analytics has the
potential to support students at scale
• Use data to provide personalised student support
![Page 36: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/36.jpg)
Abelardo Pardo Provision of Personalised Feedback at Scale Using Learning Analytics 36
![Page 37: Provision of personalized feedback at scale using learning analytics](https://reader034.fdocuments.us/reader034/viewer/2022050613/5a6d545d7f8b9ae5418b5307/html5/thumbnails/37.jpg)
Provision of personalized feedback at scale using learning analytics
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
Antonietta flickr.com
Centre for the Enhancement of Teaching and Learning Faculty of Education Hong Kong University 19/May/2017