BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell,...
-
Upload
yahir-wiggins -
Category
Documents
-
view
215 -
download
0
Transcript of BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell,...
![Page 1: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/1.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Bruce Howell, University of Reading
How should, and how can, ‘low level EAP’
be assessed?
![Page 2: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/2.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Why assess? (e.g. Reading)
Oct-Dec
• EACH COURSE:
• beginning: placement
• end: progress (proficiency)
Jan-Mar
Apr-Jun
Jul-Sep
![Page 3: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/3.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
‘Placement’ – getting on the right track
1. "is this candidate at least the minimum level to join the pre-sessional EAP programme?"
2. "which (e.g. writing) group within the EAP pre-sessional programme should this student be in?"
3. “what specific area needs to be addressed within the EAP in-sessional programme?” (Read 2008)
![Page 4: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/4.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
‘Placement’ – getting on the right track
1. programme screening• yes/no
2. class placement• multi-level
3. diagnostic• descriptions not numbers?
![Page 5: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/5.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
“It must be practical” (1)• time
restraints• e.g.
British Council (O’Sullivan 2010)
![Page 6: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/6.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
“It must be practical” (2)• 10 minutes Dictation (gap-fill sentences)• 10 minutes Kanji ( spelling)• 30 minutes Writing• 5 minutes Speaking (interview)• 3 minutes self-assessment questionnaire
(Toyoda & Hashimoto 2001)
** The more data the better **
BUT a minimum of data may be enough for your situation
![Page 7: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/7.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
But what about EAP?• TEEP: listening and reading into writing; task-
based speaking; academic perspective on topics
• process approach can be replicated in (placement) tests (Lee 2005; Plakans 2009)
• …but is this necessary for low levels?
• experience of TEEP: below ~4.5 can “blow it completely”
• so perhaps the approach should be:– general language competence– university context
![Page 8: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/8.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
How should ‘low level’ EAP be assessed?• Reading Writing Listening Speaking
– yes but … perhaps not possible
• what about just a simple grammar test?– NO!– “use of a single test format is inadequate” (Green and
Weir 2004)
• why not just use international tests or school results?– not always available– expensive– bland, generalised– not always trustworthy, despite claims
![Page 9: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/9.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Popular suggestions
• grammar test• vocabulary test• independent writing test• independent reading test• interview
![Page 10: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/10.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
More suggestions…• linked reading speaking task
– ‘two-in-one’– going beyond formulaic language
• dictation– “high correlation with grammatical knowledge AND
oral/aural proficiency” (Ford-Niwa 1998, cited in Toyoda & Hashimoto 2001)
• a staged/modular procedure:– not all sections are needed (Fulcher 1997)– a pre-test screening section?
• self-assessment
![Page 11: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/11.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
DELNA (Read 2008)
![Page 12: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/12.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Online OPT
![Page 13: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/13.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
British Council (O’Sullivan 2010)
![Page 14: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/14.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Self-assessment• does it work?• research gives differing results (Dandenault
1997)for: against:
• practicality – saves time/money
• involves the learner in the decision-making process
• increases self-awareness
• advantages have a place in classrooms, not tests
• unconvincing for teachers• inviting exaggeration
('cheating'?)• influence of personality
factors• tendency to err towards the
'average' – not to rank high or low
![Page 15: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/15.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
…need to ask meaningful questions
![Page 16: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/16.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
…don’t be too simplistic!(Gere et al 2010)
![Page 17: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/17.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
How can ‘lower level’ EAP be assessed (practical solution)
• depends on your context!
![Page 18: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/18.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Computer-based?• … if you have the resources!
for: against:
• simple, from the administration point of view
• data collection automatic
• scoring can be instant
• certificates can be automatic
• attractive for some
• restrictive of item type• impersonal (unless
complemented with interview etc.)
• off-putting for some
![Page 19: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/19.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
How to check your test is working
• trial/pre-test it first – if possible– imperative (Fulcher 1997)
• get teachers’ views on suitabilty• ask students about their experience• analyse score data – eliminate problem
items• review cut level(s) – are they living up to
expectations?• how does it compare to scores on IELTS
etc?– (concurrent validity)
![Page 20: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/20.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
… but remember
• we cannot assume the test ‘is not working’ just because students do not increase their level in a linear way
• depends on:– motivation/completion of
work/attendance– aptitude for languages– external factors
• e.g. distractions outside class
– etc.
![Page 21: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/21.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
Most important
• assessments must have a system– essential to have stakeholders on board
(Read 2008)– must be easily understood by all– follow-up plans (e.g. split profile)– who does the decision-making?
• advise potential students so that they can make the right decision
• programme screening test:– short test may be OK, but it’s high-stakes …– so it has to have proven reliability and good
security
![Page 22: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/22.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
BibliographyBrantmeier, C. (2006). Advanced L2 learners and reading placement: Self-assessment, CBT, and subsequent performance. System 34 :15–35.
Dandenault, E. J. (1997). Self-Assessment of Communicative Ability: Investigation of a Novel Tool for ESL Learners. MA Dissertation. Montreal: Concordia University. Available at http://spectrum.library.concordia.ca/1227/1/MQ40158.pdf
Fulcher, G. (1997). An English language placement test: issues in reliability and validity. Language Testing 14-113.
Read, J. (2008). Identifying academic language needs through diagnostic assessment. JEAP 7:180- 190
Gere, A. R., Aull L., Green. T., and Porter, A. (2010). Assessing the validity of directed self-placement at a large university. Assessing Writing 15:154–176.
Green, A. B. and Weir. C. J. (2004). Can placement tests inform instructional decisions? Language Testing 21-467.
Lee, Y. J. (2005). A Summary of Construct Validation of an English for Academic Purposes Placement Test. Spaan Fellow Working Papers in Second of Foreign Language Assessment Vol. 3.
O’Sullivan , B. (2010). Developing a Placement Test: A case-study of the British Council ILA project. IATEFL TEA SIG Newsletter Spring 2010.
Plakans, L. (2009). Discourse synthesis in integrated second language writing assessment. Language Testing 26-561.
Toyoda, E. & Hashimoto, Y. (2001). Analysis of a new Japanese Language Placement Test Battery Using G-Theory and Rasch Model Programs. Melbourne Papers on Language Testing 10-1.
![Page 23: BH How should, and how can, low level EAP be assessed? BALEAP SIG Bristol Feb 2011 Bruce Howell, University of Reading How should, and how can, low level.](https://reader038.fdocuments.us/reader038/viewer/2022103111/5518c69855034638098b4ba8/html5/thumbnails/23.jpg)
BH How should, and how can, ‘low level EAP’ be assessed?BALEAP SIG Bristol Feb 2011
What do YOU think?