The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden...
-
Upload
daniel-booker -
Category
Documents
-
view
217 -
download
0
Transcript of The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden...
![Page 1: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/1.jpg)
The New SHAPE Listening & Reading Tests
OR
HOW DID THEY DO THAT?!
BILC Seminar – Stockholm, SwedenOctober 2013
Michael Adubato SHAPE Language Testing Centre
Belgium
![Page 2: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/2.jpg)
GERMANY 5 – 3 SWEDENFriends Arena, Stockholm
16 October 2013
![Page 3: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/3.jpg)
Designing tests for SHAPE – a true learning process
University of Westminster
![Page 4: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/4.jpg)
Aim
To explain the challenges we faced designing
the Listening and Reading tests and how we
overcame them.
![Page 5: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/5.jpg)
The Format – Listening and Reading
SLP 2: 20 x passages20 x MCQ
SLP 3: ? x passages20 x MCQ
SLP 4: ? x passages 20 x MCQ/ SA
SLP 2 – 3Listening: 60 minsReading: 90 mins
SLP 3 – 4Listening: 75 minsReading: 120 mins
![Page 6: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/6.jpg)
Listening:Speed per levelRubricOrientation messageTime to complete MCQ
ReadingReading speed per levelTypes of reading: skim/ scam/ read for detail
Factors affecting Passage Length
Required test length
Required test length
![Page 7: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/7.jpg)
Validity
Contexts English Genres
![Page 8: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/8.jpg)
Paper question
Task Focus Category Paper A Topic Word Length
English Answer Author
1 Answer who, what, when questions
Pers Serial 5: Invitation to party Scottish Dancing 109 British a KW
2 Answer how and why questions
Work Serial 31: Travel order Brussels Airport 127 USA b RW
3 Answer how and why questions
Gen Newpaper article Sleep patterns 119 British a RW
4 Answer how and why questions
Work Equipment request forms New system for booking equipment
124 USA a MR
5 Identify explicity stated main ideas
Work Serial 4: Routine office email exchange
Revised HR policy requiring checking for legality
112 Canadian c RW
6 Distinguish fact from opinion
Work Serial 4: Routine office email exchange
Brown and Laing Account 106 Australian b CH
7 Answer who, what, when questions and how,why questions
Work Serial 40: Request for translator/interpreter services. Include details of requirement
Interpreter required for Ukrainians 166 USA c RW
8 Interpret complex sentences
Work Serial 11: How to use the airconditioning
Fire safety 210 British c JAG
Sample from Examination Schematic
![Page 9: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/9.jpg)
Creating the texts………
![Page 10: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/10.jpg)
Which factors affect the complexity of a text?
• Task focus
• Criteria eg STANAG or CEFR
• Grammar• Lexis • Sentence length• Density of text/passage
• Speed (listening)• Interference (listening)• Level of inference
![Page 11: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/11.jpg)
Answer who, what, when questions
Answer how and why questions
Distinguish fact from rumour or hearsay
Recognise author's intentions
Recognise attitudes and emotions of the writer
Outline the development of an argument
Task Foci
![Page 12: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/12.jpg)
Writing and Setting Process
Undertake TNA
Allocate test items to writers
Train writers Scrutinise in-office
Writer edits
Internal Setting Board
In-house edits
Pilot papers
Produce paper
Produce Schematic
Allocate items to writers
Scrutinise in-office
Writer edits
Order tasks
Initial draft
Internal Setting
Pilot and adjust
Select writers
Select final writers
![Page 13: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/13.jpg)
Recording
![Page 14: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/14.jpg)
Why do people offer to sit a pilot examination?
For a reward
Joy
To please others
They have no reason to do so
![Page 15: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/15.jpg)
Piloting Exam Paper in the UK
![Page 16: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/16.jpg)
Piloting the paper overseas
SLP 2/3/4??
![Page 17: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/17.jpg)
![Page 18: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/18.jpg)
What did SPSS do for us?
(Remember, items increase in difficulty throughout each level of the test.)
Enabled us to see how many people answered a question with a, b, c or d.
![Page 19: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/19.jpg)
Pre-test questionnaires
– Gender
– Nationality
– Language level (variation required)
– Educational level
![Page 20: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/20.jpg)
![Page 21: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/21.jpg)
NOT SHOWN!IF YOU’RE INTERESTED IN THIS, I CAN PROVIDE YOU
WITH A COPY…
Demonstration of Paper C Spreadsheet
![Page 22: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/22.jpg)
![Page 23: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/23.jpg)
![Page 24: The New SHAPE Listening & Reading Tests OR HOW DID THEY DO THAT?! BILC Seminar – Stockholm, Sweden October 2013 Michael Adubato SHAPE Language Testing.](https://reader036.fdocuments.us/reader036/viewer/2022081515/56649e435503460f94b36d51/html5/thumbnails/24.jpg)
Questions?