Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for...
-
Upload
vivian-powers -
Category
Documents
-
view
219 -
download
1
Transcript of Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for...
![Page 1: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/1.jpg)
Conflicting demands in Conflicting demands in integrated reading / writing integrated reading / writing taskstasks
Tom Lumley Australian Council for Educational Research
Annie Brown University of Melbourne
![Page 2: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/2.jpg)
2
Typical Tests of English for Typical Tests of English for Academic PurposesAcademic Purposes
eg• TOEFL - Test of Written English (USA)• IELTS (UK/Australia)
Writing tested as discrete skill
Essay format
Decontextualized, short prompts
No specific content
Students’ background knowledge / opinion
![Page 3: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/3.jpg)
3
New TOEFLNew TOEFL
• Prototype LanguEdge Writing tasks
Integrated Reading / Writing
Reading Input text (c. 750 words)
followed by related Writing Task
plus
Independent (stand-alone) Writing task
(LanguEdgeTM Courseware Handbook for Scoring Speaking and Writing.
© ETS, 2002.)
![Page 4: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/4.jpg)
4
Task instructionsTask instructions
• You have 25 minutes to answer the question below by writing a response based on information from the passage. Typically an effective response will be between 175 and 250 words.
• Your response should present the key ideas and supporting points from the passage necessary to answer the question fully in your own words as much as possible.
• Your response will be scored on the quality of your writing and how well and how completely you convey the relevant information from the passage. Typically an effective response will be between 175 and 250 words.
(LanguEdge Courseware Handbook for Scoring Speaking and Writing. ETS, 2002.)
![Page 5: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/5.jpg)
5
TasksTasks
Task 1: “Early Cinema”Explain how projectors changed the economics of showing films and the experience of watching films.
Task 2: “Nineteenth-century politics in the United States”
Explain the different viewpoints held by nineteenth-century Whigs and Democrats on the economy and the role of government and explain why different people supported each of the two parties.
(LanguEdge Courseware Handbook for Scoring Speaking and Writing. ETS, 2002.)
![Page 6: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/6.jpg)
6
Assessment criteriaAssessment criteria
holistic scale:
• relevance, accuracy• clarity and completeness of ideas• organisation and linking of ideas
• linguistic accuracy• paraphrasing and appropriate reference to
input text
![Page 7: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/7.jpg)
7
Test TakersTest Takers
• written texts - 60 learners of English
n Language Institution place
30 Mandarin University Beijing
20 Korean Lang. centres Melbourne
10 Cantonese University / univ. prep.
Hong Kong/
Melbourne
![Page 8: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/8.jpg)
8
Verbal report dataVerbal report data
• Test takers:Interviews immediately following writing
Collected in L1, translated into English
• 6 ratersVerbal reports as they assessed the texts
produced by the test takers
![Page 9: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/9.jpg)
9
Test taker interpretations of the task Test taker interpretations of the task requirementsrequirements
What does the task require ?
•Do test takers consider them to be principally reading or
writing tasks?
•Do they recognize that they are not required to summarise
the whole text?
•How do they think their responses are assessed?
•Are they more concerned about quality of the content or of
the language they produce?
•How do their views affect the way they approach the task?
![Page 10: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/10.jpg)
10
Findings: task requirementFindings: task requirement
MC09
I should only be summarising and paraphrasing. … I was quite
selective, I tried to look for the relevant parts in the text which
supported what I wanted to write.
BE05It expects me mainly to summarize the central ideas of the
original text and then to re-write it for short. A re-writing of the main content.
• Rater 6 BE05I think this student has done a similar thing to the previous one
in that they don’t seem to be addressing the question. On first go a lot of it’s irrelevant.
![Page 11: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/11.jpg)
11
Findings: task requirementFindings: task requirement
• MC08I planned the structure of my essay in my head . I did not think it
was necessary to draft as the question is quite clear, and I knew what to write.
• Rater 3 MC08principal ideas are inconsistently presented. There’s some
irrelevant stuff in there and it’s not explicit enough about the
key terms of the question.
![Page 12: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/12.jpg)
12
Findings: Findings: text structuretext structure
• MK23My text is a typical organization of an essay, which includes introduction, body and conclusion.
• MC11(in the introduction) I paraphrased the question. … I think it is better to include an introduction. … The purpose of the introduction is … to give the reader some idea of what I am about to write.
• B14
Q: Did you have the introduction part?A: No. I didn’t even think about it.
![Page 13: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/13.jpg)
13
Findings: Findings: text structuretext structure
• HK01Q:Ok, so how did you go about organizing this text?A: No need to organize because there are 3 parts in the
question, so I just wrote one paragraph for each part.
• HK07How can we complete the answer in 250 words only?
• BE07 My conclusion part was simply a repeat of the introduction, not
a conclusion based on the different paragraphs in my writing.
![Page 14: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/14.jpg)
14
Findings: Findings: text structuretext structure
• Rater 5 BE04I would like an introduction. I don’t really know what they’re talking about, … so I am a bit confused when I start reading it. …. if it had a context it would receive a higher mark from me
• Rater 2 MC06This one I’m afraid really has to be a 1 because it spends almost a third of its time saying what it’s going to do and then has very little …
• Rater 3 B10
If I look at organisation. It’s been organised so that the first
sentence … rephrases the question, which is quite a
reasonable way to go about it.
![Page 15: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/15.jpg)
15
Assessment criteriaAssessment criteria
• Your response will be scored on the quality of your writing
and how well and how completely you convey the relevant
information from the passage. Typically an effective response
will be between 175 and 250 words.
(LanguEdge Courseware Handbook for Scoring Speaking and Writing. ETS, 2002.)
![Page 16: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/16.jpg)
16
Assessment criteria: content Assessment criteria: content focusfocus
• MC11I feel that the content is more important, because the task is trying to find out if I understand the passage.
• MC13I focused on getting the ideas right, because if the ideas were incorrect, even if I wrote beautifully, it would still be useless.
• MC06Q: Which is more important to you in answering this task, do you think – content or language?
A: Getting the ideas right, I didn’t have any time to think of my language.
![Page 17: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/17.jpg)
17
Assessment criteria: content Assessment criteria: content focusfocus
• MC08I think getting the ideas right is more important. If you are asked to answer some questions based on a text, you are expected to provide the correct answers, you can’t just write anything. Even if your English is good, it doesn’t count.
• MC06I think the marker will focus on content, he or she will use a checklist and check if I have listed all the points required to answer the question.
![Page 18: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/18.jpg)
18
Assessment criteria: Assessment criteria: real worldreal world
• B09At my study level, accurate language is not a big issue. The key is to have correct ideas.
• MC05Because I don’t know the criteria, so I was sort of guided by the assessment criteria for an academic paper.
![Page 19: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/19.jpg)
19
Assessment criteria: Assessment criteria: language focuslanguage focus
• B10
Q: Which one was more important to you, correct ideas or accurate language?
A: Accurate language. After all, it was a test of writing ability, that is, a test of language.
• B15
Q: Which is more important to you in answering this task, do you think – correct ideas or accurate language?
A: Language, because I feel little difficulty with getting correct ideas.
![Page 20: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/20.jpg)
20
Task completion: copyingTask completion: copying
• BE08
I have copied some sentences, and some phrases, and also
the ideas. I don’t think this is a writing task, but more like a
paraphrasing task, or summarizing task.
• HK01Q : Are you happy about the way you have (copied)?
A : Of course not, but I had no time.
![Page 21: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/21.jpg)
21
Reasons for copyingReasons for copying
BE01
I felt if I changed the words, the meaning of my writing would change. Therefore, I prefer to use the expressions in the original text.
B07
I felt those words fit in very well, and I could find no better words than them.
HK01
“the proper sphere of government action” … “moral beliefs” … I don’t understand those expressions, so I copied the whole sentence to my draft.
![Page 22: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/22.jpg)
22
Difficulties of paraphrasingDifficulties of paraphrasing
• MC05
I was inclined to use my own words than direct paraphrasing. I
find that it is actually easier to use your own words than
paraphrasing, because in paraphrasing you have to find the
right words to express the writer’s ideas, I think that is even
more demanding.
![Page 23: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/23.jpg)
23
RatersRaters: : copied textcopied text
• Rater 5 BE01 So, I don’t get a sense of this person at all because there is too much
of it lifted from the original text…
• Rater 3 B17
the first sentence “projectors largely changed the experience of
watching films” - I’m not sure whether that’s been lifted or
not, but it reads outside the writer’s own voice a bit.
![Page 24: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/24.jpg)
24
RatersRaters: : copied text?copied text?
• Rater 3 MC09
I’m just wondering about how much is plagiarised. Things
like “Also, the economics of showing films increased
because the image could be projected to a large screen for
mass audience ” That seems like it’s come from the text
and I can’t find it. But it seems like it should have, I’m sure
I’ve read that.
![Page 25: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/25.jpg)
25
RatersRaters: : what are we testing?what are we testing?
• Rater 5 B093
and I guess that’s the tension where we’ve got to make up our
mind, what are we testing? Are we testing understanding, or
are we testing language? And so if we’re testing understanding
and the student is able to lift the sentence and then put a
sentence in their own text that explains and elaborates the
previous sentence then can I reward the student for having a
good understanding, and being able to select appropriately from
the text and then elaborate in their own words? It’s a bit of a
tension I face with this one I reckon. So overall though, now
goodness me I have to score it, it’s a problem.
![Page 26: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/26.jpg)
26
Raters: Raters: in praise of paraphrasein praise of paraphrase
• Rater 5 B053 the student, I think, has got an understanding of the structure
of the text … I mean she has attempted to use her own words
a little bit, I think, rather than just lift passages from the text as
I’ve seen before. So, it hasn’t been successful, but she has
tried to paraphrase it herself. So, again, big decisions. I think
there is enough understanding of the content even though
some of the sentences don’t quite make sense.
![Page 27: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/27.jpg)
27
Advice to test takersAdvice to test takers
• a full summary is not required or appropriate
• an introduction is required
• conclusion is not required
![Page 28: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/28.jpg)
28
The task: The task: a trade-offa trade-off
• completing the task by relying on the
language of the input text
OR
• using their own words and hence reducing the quality of the language.
![Page 29: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/29.jpg)
29
Advice to test takers?Advice to test takers?
• focus on content:– too much of a temptation to copy– conflicts with the need to use their own words and
demonstrate their own competence in English
• focus on paraphrase and their own language:– time constraint– difficulty of fulfilling task successfully– memory load
• edit and proofread their work– time constraint
![Page 30: Conflicting demands in integrated reading / writing tasks Tom Lumley Australian Council for Educational Research lumley@acer.edu.au Annie Brown University.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649e1a5503460f94b071db/html5/thumbnails/30.jpg)
30
ConclusionConclusion
• need to redesign the assessment task
Task Input
or
Task focus & instructions
(for information on current TOEFL tasks see www.toefl.com )