10 07-17 inform conf slides
-
Upload
ffranceschina -
Category
Education
-
view
494 -
download
2
description
Transcript of 10 07-17 inform conf slides
![Page 1: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/1.jpg)
Assessing IFP students’ needs assessment
Florencia Franceschina
Manchester Metropolitan University
Inform Conference,
University of Reading, 17 July 2010
![Page 2: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/2.jpg)
1. Introduction
• Increased weight of the ‘student voice’ in HE (e.g., NSS, International Student Barometer, etc.)
• The ‘student voice’ is valued in different ways:– Students as a customers– Students as individuals capable of self-directed
learning
• Learning needs self-assessment relevant to either view
![Page 3: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/3.jpg)
Defining self-assessment
“Self-assessment refers to the involvement of learners in making judgements about their own learning […]”.
Boud and Falchikov (1989: 529)
![Page 4: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/4.jpg)
Why assess learning needs?
Because it can help us to identify:
• learners’ strengths and weaknesses
• learners’ goals and expectations
![Page 5: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/5.jpg)
Issue #1: How useful is
learning needs self-assessment?
• Useful for policy makers
• Useful for curriculum developers
• Useful for teachers
• Useful for students
![Page 6: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/6.jpg)
“Whether it be a junior high school student wondering whether to crack open his social studies textbook one more time before the test or a medical student deciding whether to practice her intubating technique before her next shift, students make more effective decisions about where to apply their learning efforts when they can actually discern their strengths and weaknesses.”
(Dunning et al., 2004: 85)
![Page 7: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/7.jpg)
Issue #2: How valid is
learning needs (self-)assessment?
Do students know what they need?
Do even experts know which needs we should explore and aim to address? For example, the relevance of some needs may not be apparent at the time of assessment (Grant, 2002: 157).
![Page 8: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/8.jpg)
Issue #3: How reliable is
learning needs self-assessment?
• Are all learners equally able to make accurate judgements about their learning needs?
• How good are IFP students at it?
![Page 9: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/9.jpg)
2. Brief review of the literature
Self-assessment …
• … is difficult.
![Page 10: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/10.jpg)
Meta-studies reveal typically low correlation between self-views and actual performance:
.21 (Hansford and Hattie, 1982)
.39 (Falchikov and Boud, 1989)
Similar findings by Dunning et al., 2003; Dunning et al., 2004; Powers, 2002; Ross, 1998.
![Page 11: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/11.jpg)
Self-assessment …• … is difficult.• … is not always valued by learners
(e.g., compared to summative assessment)
![Page 12: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/12.jpg)
Self-assessment …• … is difficult.• … is not always valued by learners
(e.g., compared to summative assessment)
• … is modulated by culture.
![Page 13: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/13.jpg)
Self-regulation is subject to the influence of educational/cultural background factors
(e.g., Purdie and Hattie, 1996; Olaussen and Bråten, 1999)
![Page 14: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/14.jpg)
Self-assessment …• … is difficult.• … is not always valued by learners
(e.g., compared to summative assessment)
• … is modulated by culture.• … is modulated by level of study.
![Page 15: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/15.jpg)
Students in advanced courses tend to be more accurate self-assessors than students on introductory courses (Falchikov and Boud, 1989).
![Page 16: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/16.jpg)
Self-assessment …• … is difficult.• … is not always valued by learners (e.g.,
compared to summative assessment)• … is modulated by culture.• … is modulated by level of study.• … is modulated by academic ability level.
![Page 17: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/17.jpg)
“High-performing students were accurate, with accuracy improving over multiple exams. Low-performing students showed moderate prediction accuracy but good postdiction accuracy. Lowest performing students showed gross overconfidence in predictions and postdictions. Judgments of performance were influenced by prior judgments and not prior performance. Performance and judgments of performance had little influence on subsequent test preparation behavior.”
Hacker et al. (2000: 160)
![Page 18: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/18.jpg)
Self-assessment …• … is difficult.• … is not always valued by learners (e.g.,
compared to summative assessment)• … is modulated by culture.• … is modulated by level of study.• … is modulated by academic ability level.• … is differently accurate depending on area
of study.
![Page 19: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/19.jpg)
Students in science tend to be more accurate self-assessors than students in other areas (Falchikov and Boud, 1989).
![Page 20: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/20.jpg)
Self-assessment …• … is difficult.• … is not always valued by learners (e.g.,
compared to summative assessment)• … is modulated by culture.• … is modulated by level of study.• … is modulated by academic ability level.• … is differently accurate depending on area of
study.• … can be developed as a skill.
![Page 21: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/21.jpg)
Although student self-assessment remains inaccurate after training, it does seem to be subject to some improvement. For example:
McDonald (2010) has shown that involving students in meta-assessment training can help mathematics students to improve their test performance.
Chen (2008) found that through feedback and practice students’ self-assessments of English speaking skills become more aligned with tutor assessment.
![Page 22: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/22.jpg)
Wide range of data collection methods available
• Formal / informal
• Planned / opportunistic
• Individual / group
• By self / peer / tutor / outside expert
![Page 23: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/23.jpg)
Some commonly used tools
• Questionnaires• Focus groups• Online discussion fora• Staff-student liaison committees• Diaries• Objective tests• Observation• Action research
![Page 24: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/24.jpg)
3. Methodology
• Cohort of 51 IFP students
• Min. language entry requirement: IELTS 5.5 or equivalent
• Followed up for the duration of IFP (2009-2010)
![Page 25: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/25.jpg)
Mixed nationalities
• Saudi Arabia (22%)• Bahrain (14%)• Nigeria (14%)• Pakistan (8%)• Turkey (8%)• UAE (8%)• China (4%)• India (4%)
• Iran (4%)• Sri Lanka (4%)• DR Congo (2%)• Hong Kong (2%)• Kenya (2%)• Turkish Cyprus (2%)• USA (2%)• Zimbabwe (2%)
![Page 26: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/26.jpg)
Mixed linked degrees
Science / Engineering
53%Business31%
HLSS14%
Other2%
![Page 27: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/27.jpg)
Data collected
• Learning needs questionnaire– Student version (individual self-assessment)– Staff version (group assessment)
• Diagnostic language test• Attendance records• Coursework and exam results (all IFP
modules)• Staff interviews• End-of-year module feedback (EAP module)
![Page 28: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/28.jpg)
Analyses
• Perceived needs vs. observed academic performance
• “Objective” diagnostic measures vs. observed academic performance
![Page 29: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/29.jpg)
4. Findings
Overview
1. “Objective” measures
2. “Subjective” measures
3. Improvements in judgements?
![Page 30: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/30.jpg)
Correlations between ‘objective’ measures
Lang test at entry
EAP unit mark
IFP mark
EAP attend.
IFP attend.
Lang test at entry
- .22 .36
EAP unit mark
- - .81 .67
IFP
mark
- - - .65
EAP attend.
- - - -
IFP attend.
- - - - -
![Page 31: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/31.jpg)
Correlation between language test at entry and EAP unit marks
0
10
20
30
40
50
60
70
80
0 20 40 60 80 100 120 140 160 180
OPT scores at entry
EAP unit marks
![Page 32: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/32.jpg)
Correlation between language test score at entry and overall IFP mark
0102030405060708090
100
0 20 40 60 80 100 120 140 160 180
OPT score at etnry
Overall FY mark
![Page 33: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/33.jpg)
Correlations between ‘objective’ measures
Lang test at entry
EAP unit mark
IFP mark
EAP attend.
IFP attend.
Lang test at entry
- .22 .36
EAP unit mark
- - .81 .67
IFP
mark
- - - .65
EAP attend.
- - - -
IFP attend.
- - - - -
![Page 34: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/34.jpg)
Correlation between EAP unit mark and IFP overall mark
0
10
20
30
40
50
60
70
80
0 10 20 30 40 50 60 70 80 90
EAP unit mark
IFP overal mark
![Page 35: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/35.jpg)
Correlations between ‘objective’ measures
Lang test at entry
EAP unit mark
IFP mark
EAP attend.
IFP attend.
Lang test at entry
- .22 .36
EAP unit mark
- - .81 .67
IFP
mark
- - - .65
EAP attend.
- - - -
IFP attend.
- - - - -
![Page 36: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/36.jpg)
Correlation between EAP attendance and EAP mark
0102030405060708090
100
0 10 20 30 40 50 60 70 80
EAP mark
EAP attendance
![Page 37: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/37.jpg)
Correlation between IFP attendance and overall IFP mark
0102030405060708090
100
0 10 20 30 40 50 60 70 80 90
IFP overall mark
Attendance %
![Page 38: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/38.jpg)
How did “objective” measures work?
• Language test at entry was poor predictor or later performance. This could be due to a number of reasons. (Similarly surprising patterns reported for UCAS tariff points/FY performance elsewhere.)
• Attendance was reasonably good predictor. This confirms findings in previous studies (e.g., Hughes, 2009).
• EAP unit was best predictor of IFP performance.
![Page 39: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/39.jpg)
Staff perceptions about students’ needs at entry
Tutors most frequently identified general language weakness as an issue to be addressed.
“when marking essays, it is obvious that many students have a very weak grasp of syntax sometimes, and this will hold them back in the future.”
![Page 40: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/40.jpg)
“Sometimes there have been some difficulties in their ability to express themselves effectively in written work but this is not always the case. The main problem I have found with international students who do have language difficulties is their lack of commitment in addressing the issue or them denying the fact there is a problem despite overwhelming evidence and continuous advice to attend support classes. Some students believe their English skills are better than they truly are.”
![Page 41: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/41.jpg)
“Required much repetition on basic tasks appear not to understand instructions.”
“there are others who have a very poor command of the English language and struggle with their research and assignments.”
![Page 42: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/42.jpg)
Tutors and support staff thought the IFP students’ numeracy skills and subject background knowledge was on the whole the same or better than in UK students.
![Page 43: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/43.jpg)
Relative difficulty of IFP modules (MMU ‘units’)
Mean unit marks per unit type
0
10
20
30
40
50
60
70
L-type units X-type units N-type units
Mean unit marks
![Page 44: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/44.jpg)
Spread of unit marks per unit type
0102030405060708090
100
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113
Individual unit marks
L-type unit marks X-type unit marks N-type unit marks
![Page 45: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/45.jpg)
Students’ perceptions about their needs at entry
Indirect method of calculating match between students’ perceived needs and actual performance:
1. Self-assessment questionnaire answers coded as 1 if student ticked ‘I need to study more of this’/’I need help with this’, and 0 if student ticked ‘I don’t want much of this’/’I don’t need help with this’
2. EAP unit summative assessment marks were coded as 0 if mark was above mean and 1 if below mean.
3. Match between 1 and 2 above was checked.
![Page 46: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/46.jpg)
Maximum possible score for successful matching of needs and actual performance: 5
(W, R, S, L, IT)
IFP students’ average score: 2.5
![Page 47: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/47.jpg)
Perceived vs observed order of difficulty of subskills
• Predicted by students:W < IT < L < S/R
• Observed:W (38%) < R (43%) < S (50%) < L (52%) < IT (80%)
![Page 48: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/48.jpg)
Correlation between correct prediction index for subskills and actual IFP overall marks
0
1
2
3
4
5
0 10 20 30 40 50 60 70 80 90
IFP overall mark
Correct prediction index
![Page 49: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/49.jpg)
Students’ perceptions about their needs at exit
Needs questionnaire (at exit) answers match performance in subskills at an average score of 2.3 (no improvement)
![Page 50: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/50.jpg)
End-of-year student feedback on EAP unit
0
1
2
3
4
5
6
7
8
9
10
Q1a. vocabulary quizzes
Q1b. glossaries and essay planning notebook
Q1c. vocabulary notebook
Q1d. textbook
Q1e. IT (orange) bookletQ1f. materials on WebCT
Q1g. speaking practice and assessmentQ1h. listening practice and assessmentQ1i. reading practice and assessmentQ1j. writing practice and assessment
Q1k. IT practice and assessment
Question
Average score
How useful?
How enjoyable?
![Page 51: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/51.jpg)
Learning need as judged by students at the end of the IFP
Higher Lower
Speaking Listening
Reading
Writing
IT
![Page 52: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/52.jpg)
EAP unit performance
Assessment area
Mean scores
SD Median
IT 80 27 87
Listening 52 14 53
Speaking 50 15 52
Reading 43 16 47
Writing 38 21 40
![Page 53: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/53.jpg)
5. Summary and Conclusion
• Not all objective measures that seem relevant make useful predictions.
• Tutors and support staff’s global, subjective assessment of students’ learning needs seems to be accurate.
• These IFP students do not seem to be very good at assessing their learning needs in detail, but they are relatively better when their judgements are considered at a more general, group level.
![Page 54: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/54.jpg)
• Given how unreliable IFP students’ judgements seem to be, we should be careful when using these judgements to inform our practice.
• The highest value of learning needs self-assessment is probably for the students themselves.
![Page 55: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/55.jpg)
6. Practical suggestions
• Bear in mind that learning needs self-assessment is difficult, especially for IFP students.
• When interpreting IFP students’ needs self-assessment, remember that it can be influenced by:– cultural factors – academic ability level (good vs. weak students)– area of study (science vs. other)– assessment context (formative/summative; high/low
stakes)
![Page 56: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/56.jpg)
• Students become more accurate with training and practice, so provide them with opportunities to develop this skill.
• Close the feedback loop with staff and students, for example through:– As part of regular assessment feedback– Websites, VLEs, notice boards– Programme committee meetings– Informal reporting by staff in class
![Page 57: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/57.jpg)
• Whenever possible, combine self-assessment with other sources of information about your students’ learning needs.
• Take advantage of the wide range of data collection tools available.
• Methodological tip: ask students to make self-assessment judgements using same scales as ‘objective’ performance measures you will use to compare. (This has its own problems, though.)
![Page 58: 10 07-17 inform conf slides](https://reader033.fdocuments.us/reader033/viewer/2022060116/5583b20ed8b42ae2238b515a/html5/thumbnails/58.jpg)
ReferencesBoud, David, and Falchikov, Nancy. 1989. Quantitative studies of student self-assessment in higher
education: a critical analysis of findings. Higher Education 18:529-549. Chen, Yuh-Mei. 2008. Learning to self-assess oral performance in English: a longitudinal case study.
Language Teaching Research 12:235-262. Dunning, David, Heath, Chip, and Suls, Jerry M. 2004. Flawed self-assessment. Implications for health,
education and the workplace. Psychological Science in the Public Interest 5:31-54. Dunning, David, Johnson, Kerri, Ehrlinger, Joyce, and Kruger, Justin. 2003. Why people fail to recognize
their own incompetence. Current Directions in Psychological Science 12:83-87.Falchikov, Nancy, and Boud, David. 1989. Student self-assessment in higher education: a meta-analysis.
Review of Educational Research 59:395-430.Grant, Janet. 2002. Learning needs assessment: assessing the need. British Medical Journal 324:156-
159. Hacker, Douglas J, Bol, Linda, Horgan, Dianne D, and Rakow, Ernest A. 2000. Test prediction and
performance in a classroom context. Journal of Educational Psychology 92:160-170.Hughes, Nicola T. 2009. Attendance as a measure of student motivation and engagement. InForm 3:7-8.McDonald, Betty. 2010. Improving learning through meta assessment. Active Learning in Higher
Education 11:119-129. Olaussen, Bodil S, and Br 蚯 en, Ivar. 1999. Students' use of strategies for self-regulated learning: cross-
cultural perspectives. Scandinavian Journal of Educational Research 43:409-432.Powers, Donald E. 2002. Self-assessment of reasoning skills. ETS Research Report RR-02-22.
Princeton, NJ: Educational Testing Service (ETS). Purdie, Nola, and Hattie, John. 1996. Cultural differences in the use of strategies for self-regulated
learning. American Educational Research Journal 33:845-871. Ross, Stephen John. 1998. Self-assessment in second language testing: a meta-analysis and analysis of experiential factors. Language Testing 15:1-20.