PAZAPA: identifying and supporting at-risk students
Transcript of PAZAPA: identifying and supporting at-risk students
![Page 1: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/1.jpg)
PAZAPA: identifying and supporting at-risk students
Serge VerlindeJordi Heeren
Nathalie NouwenKU Leuven, Leuven Language Institute
![Page 2: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/2.jpg)
2
French proficiency in Flanders is on the decline
French is definitely needed on the labourmarket
uncomfortable position of French language teachers at higher education institutions
![Page 3: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/3.jpg)
Reseach question
3
How toidentify& support at-risk students (for French)
byproviding relevant feedback at the right moment?
Rather than providing students with the message ‘we missed you last week; please catch up the next days’ […], intervention can now focus on concrete learning behaviors: working on the tendency to procrastinate, searching for alternative learning materials that may be of more challenging nature. In otherwords: the learning feedback will change from product feedback to feedback provided at the process level, the most effective type of feedback.
Tempelaar, Rienties and Nguyen (2017). Towards Actionable Learning AnalyticsUsing Dispositions
![Page 4: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/4.jpg)
Subject of the study
4
• 367 students1st year of fac. of Law (2015-2016)university of Leuven (KU Leuven)
• selected from dataset of 3909 students (2015-2017)
![Page 5: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/5.jpg)
Teaching approach: blended learning
5
language for specific purposes course
basic grammar (and vocabulary) exercises
student data
initial testresult
learningmanagementsystem
end mark -continuous assessment
![Page 6: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/6.jpg)
Student data
6
• previous studies• end mark for previous studies• number of hours for French and Mathematics• mother tongue
• self-evaluation (0-10)
![Page 7: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/7.jpg)
Initial test
7
• C-test
o 100 itemso often used as placement test
![Page 8: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/8.jpg)
Initial test
8
• Did the students take the test seriously? YES
![Page 9: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/9.jpg)
Initial test
9
• Predictive value of test scores?o Correlation test score – exam score:
• R = .55 (p<0,001)o Regression analysis
• R² = 0.50%
variables β pEnd resultsecundary school
.292 <0.001
Test score .265 <0.001Secundary school type
.252 <0.001
…
YES
![Page 10: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/10.jpg)
Initial test
10
• Division into groups (n = 367)
exam < 10 exam >= 10 % successtest <= 40 32 4 11.1%test <= 50 44 22 33.3%test <= 60 45 67 59.8%test <= 70 16 80 83.3%test > 70 5 52 91.2%
I: 102
II: 112
III: 153
![Page 11: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/11.jpg)
LMS: exercises
11
• 20 grammatical topics, free order• common “fill in the blanks” exercises
• 1 exercise = 2 attemptso first attempt => correction + fact sheeto second attempt => focus on mistakes
correction + fact sheet
• goal: 7/10 for 17/20 topics > reward > final mark
![Page 12: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/12.jpg)
Exercises
12
• feedback_1: correction + fact sheet (after both attempts)
![Page 13: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/13.jpg)
Exercises
13
• feedback_2: score compared to score of their peers
![Page 14: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/14.jpg)
Exercises
14
• feedback_3: dashboard
![Page 15: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/15.jpg)
Exercises
15
• Do weaker students start their exercises later?
start ac. year
wake-up call
I: 42.4% III: 65%
YES
(II: 61.9%)
![Page 16: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/16.jpg)
Exercises
16
• Do exercises help students get better marks?
>= 17/20 ex. >= 7
< 17/20 ex.>= 7
pass 198 27fail 102 40
YES, probably…
Chi2 = 15.251; df = 1; p = 0.0001
![Page 17: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/17.jpg)
Exercises
17
• Are the exercises too difficult?
% first attempt(s)
% last attempt
averageattempts/topics
I 35.2% 75.3% 2.6
II 38.8% 76.4% 2.7
III 43.9% 76.2% 2.1
NO
![Page 18: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/18.jpg)
Exercises
18
• Are some exercises more difficult than others? YES
![Page 19: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/19.jpg)
Exam mark
19
• exam mark * self-evaluation
![Page 20: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/20.jpg)
Research question
20
How toidentify& support at-risk students (French course)
byproviding relevant feedback at the right moment?
![Page 21: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/21.jpg)
Feedback BEFORE analyses
21
• Mixed product and process feedback
for every ex.: correction + fact sheet => second attempt
for every ex.: score comparison (peers)
wake-up call 1 (mid-Nov.): mail with variable message
wake-up call 2 (mid-March): mail with variable message
dashboard
![Page 22: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/22.jpg)
Feedback AFTER analyses
22
• Process feedback
for every ex.: correction + fact sheet => second attempt
for every ex.: score comparison (peers)
wake-up call 1 (mid-Nov.): email with variable message
wake-up call 2 (mid-March): email with variable message
dashboard
asap: personalized email
![Page 23: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/23.jpg)
Feedback AFTER analyses
23
• email to students witho test score <= 50 (if test taken seriously)
o self-evaluation < 5
• content:o start asap with the exerciseso making exercises increases your chances of successo start with exercises on [items]o be aware that you will need at least 3 attempts => 7/10o later on: do not forget to focus on [items]
![Page 24: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/24.jpg)
Feedback AFTER analyses
24
• email to students witho test score <= 50 (if test taken seriously)
o self-evaluation > 6• content:
o you may have overestimated yourselfo start asap with the exerciseso making exercises increases your chances of successo start with exercises on [items]o be aware that you will need at least 3 attempts => 7/10o later on: do not forget to focus on [items]
![Page 25: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/25.jpg)
Conclusion
25
descriptive learning analytics
predictive learning analytics
better identification of at-risk studentsdispositional learning analytics
process feedback
![Page 26: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/26.jpg)
Future work
26
• More motivated group division?• Findings supported by evidence from other groups?• Impact of early (process) feedback?
• Analysis of answers?
![Page 27: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/27.jpg)
Impact of early (feedback) process?
27
2015-2016
2016-2017
I: 42.4% III: 65%(II: 61.9%)
I: 60,3% III: 62,6%(II: 61.8%)
![Page 28: PAZAPA: identifying and supporting at-risk students](https://reader030.fdocuments.us/reader030/viewer/2022012413/616cae794f37dd32e16d6065/html5/thumbnails/28.jpg)
Impact of early (feedback) process?
28
YESBUT… (data from the day before the wake-up call)
2015-16 numberof
2016-17
20 5.6% 0 1.4% 5216 60.2% < 5 74.2% 27059 16.4% 5-9 11.8% 4342 11.7% 10-16 8.2% 3022 6.1% >= 17 4.4% 16
need for constant monitoring…