The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology...
-
Upload
alison-wilkins -
Category
Documents
-
view
219 -
download
0
Transcript of The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology...
![Page 1: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/1.jpg)
The Impact of Evaluation on Thai ELT
Richard Watson ToddKing Mongkut’s University of Technology Thonburi
![Page 2: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/2.jpg)
Impacts of evaluation
Most countries: most influential evaluation is University Entrance Exams– The Chinese experience in 2007
• Parents giving children Ritalin
• Parents trying to close airport
![Page 3: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/3.jpg)
The Thai experience
Survey of secondary school teachers’ problems
156 respondents Most serious problem = the influence of the
university entrance exam
![Page 4: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/4.jpg)
A brief history of Thai university entrance 1967 - 1997: multiple-choice exams as sole
criterion 1998: 10% of marks from school scores
(GPA and percentile ranking) By 2006, school scores increased to 30%
(GPA only)
![Page 5: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/5.jpg)
A brief history of Thai university entrance 2006: name change to O-NET and A-NET
– Inclusion of essay question– Link to school curriculum– Norm-referencing through T-scores– Marking fiasco
2007: multiple-choice only
![Page 6: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/6.jpg)
Future plans for Thai university entrance 2008: Add 3 more core subjects (health,
technology, arts) 2010: A-NET replaced by aptitude test 2010: GPA system simplified 2010: Scores for ethics
![Page 7: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/7.jpg)
Stated reasons for changes
Change away from exams only– validity problems– student stress– growth of tutorial schools– lack of attention to secondary schooling– move to learner-centredness
Keeping school scores to 30%– reliability problems
![Page 8: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/8.jpg)
Stated reasons for changes National Education Act of 1999
– Section 26– "Educational institutions shall assess learners' performance
through observation of their development; personal conduct; learning behaviour; participation in activities and results of the tests accompanying the teaching-learning process commensurate with the different levels and types of education. Educational institutions shall use a variety of methods for providing opportunities for further education and shall also take into consideration results of the assessment of the learners' performance."
![Page 9: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/9.jpg)
Reasons for changes
Conflict between educational ideals and assessment inertia
Conflict between Ministry of Education and Council of University Rectors– MoE: motivate students at secondary schools– CUR: emphasise reliability
![Page 10: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/10.jpg)
Current situation
Constant flip-flopping Heavy emphasis on multiple-choice exams
– 70% from multiple-choice entrance exams– 30% from school GPA, but 60% of school
scores from exams with heavy multiple-choice bias
Emphasis on multiple-choice exams unlikely to change in future
![Page 11: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/11.jpg)
Impact of university entrance
Impact on teaching and learning– teach vocab and grammar– no higher-level thinking– rote learning– ephemeral memory of facts– receptive skills emphasised
![Page 12: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/12.jpg)
Impact of university entrance
Impact on other exams– School exams use entrance exams as model– The case in 2006
• April 2006: O-NET including essay
• June 2006: new semester
• July 2006: mid-term school exams include essay
• August 2006: CUR announces ‘no more essays’
• September 2006: final school exams have no essay
![Page 13: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/13.jpg)
Impact of university entrance
Impacts on students and society– students focus on grades, not learning– more cheating and selfishness at school– reliance on tutorial schools– inequalities in access to higher education
![Page 14: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/14.jpg)
Validity and reliability
Multiple-choice is practical– but 2006 marking fiasco not caused by essays
Multiple-choice is reliable– only if questions are well-designed
![Page 15: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/15.jpg)
Validity and reliability
Poorly designed questions from secondary schools– ______ sports bag do you like?
• a. What
• b. Who
• c. Which
• d. Whose
![Page 16: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/16.jpg)
Validity and reliability
Poorly designed questions from secondary schools– People swimming here do so his own risk
• a. Everybody swim here
• b. People swim here because he like risk
• c. People who swim here will have accident
• d. People did not swim here
![Page 17: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/17.jpg)
Validity and reliability
Content validity: Test what you teach– O-NET based on school curriculum– Use of school GPA scores– Reliance on multiple-choice means prominent
objectives ignored– Multiple-choice may be more a measure of test-
taking strategies than of language ability
![Page 18: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/18.jpg)
Validity and reliability
Predictive validity– Research into predictive validity
• GPA better predictor than exams
• Tutorial schools influence scores
– University entrance exams are not particularly valid for predicting candidates’ future performance
![Page 19: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/19.jpg)
How good is university entrance?
Detrimental effects on teaching and learning Detrimental effects on school evaluations Detrimental effects on students and society Good practicality Reasonable reliability Poor content validity Weak predictive validity
![Page 20: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/20.jpg)
Directions for change
Previously– inconsistent– directionless– knee-jerk reactions– driven by personal agendas
How to decide on and implement changes
![Page 21: The Impact of Evaluation on Thai ELT Richard Watson Todd King Mongkut’s University of Technology Thonburi.](https://reader033.fdocuments.us/reader033/viewer/2022051215/5697bfb61a28abf838c9e3a3/html5/thumbnails/21.jpg)
Implementing changes
Gatekeeping is only one among many considerations
Use National Education Act as a guide Emphasise validity over reliability
Change university entrance from a problem to an agent for beneficial development