Post on 03-Dec-2014
description
ASSESSMENT AS LEARNING: ENGAGING STUDENTS IN ACADEMIC LITERACY IN THEIR FIRST SEMESTERTimothy Jowan Curnow & Anthony J. Liddicoat
Research Centre for Languages and Cultures
University of South Australia
REDESIGNING ASSESSMENT
Redesign applied linguistics courses, including the assessment
Integrate academic literacy of native English speakers and others into teaching, learning and assessment
Focus on academic literacy doesn’t entail a lack of focus on disciplinary content; the two can be integrated
A MODEL OF THE ASSESSMENT PROCESS
Conceptualising Judging ValidatingEliciting
What to assess
How to judge
How to justify
How to elicit
The “ability” or “knowing” of
interest
Criteria for judging
performance, including
1. aspects of competence
2. performance analysis
ValidationTasks/ procedures
that operationalise the construct
Source, Scarino, 2006)
ASSESSMENT IN PRACTICE
Most thought is given to elicitation, very little to anything else. In terms of this model, many assessment codes of
practice deal only with elicitation (and only with the amount of elicitation)
Note that a lack of clarity about the conceptualisation of what is being assessed has a direct impact on all other assessment processes
RETHINKING ASSESSMENT IN THE APPLIED LINGUISTICS MAJOR
Design of an overarching assessment approach for all core courses in applied linguistics
Process: Conceptualising what learnings we wanted to achieve
across the program Conceptualising how these interrelated Conceptualising what tasks would:
allow those learnings to be put into practice give evidence of nature of students’ learning develop students knowledge of the discipline as well as
demonstrating their ability to use this knowledge
Designing tasks (eliciting) Designing assessment criteria (judging)
WHAT LEARNING DID WE WANT TO SEE?
Not simply “body of knowledge” but creation/use of knowledge in different ways. E.g. acquiring knowledge from text linking argument and evidence evaluating information according to purpose ……..
Focus on transformation of knowledge not of reproduction of knowledge. E.g. using knowledge for new purposes, not simply recall making connections between different texts,
experiences, etc. ……
WHAT LEARNING DID WE WANT TO SEE?
Key dimensions determined by team as being1. Critical reading of research2. Analysis of research writing3. Synthesis of research from multiple sources4. Constructing an argument using the research of others5. Analysing language data6. Constructing an argument from language examples7. Understanding the process of research development8. Designing and implementing research projects9. Communicating research findings
Divided between courses in the major 1-4: LANG 1056 5-6: LANG 1055 7-9: Later courses
HOW DID THIS AFFECT OUR APPROACH TO ASSESSMENT?
Realisation that some assessment tasks did not have a clear link either to knowledge or its use, for example: Reading tasks
Purpose: “get the buggers to read” Reading had no purpose other than having been completed Questions mainly showed comprehension of the text (i.e. they
confirmed the text had been read) The tasks had no connection to each other or to larger themes
Oral presentations on texts Purpose: ??? (something oral might be good? Task “inherited”)
Presentations were boring, for tutors and for the students – simple rehearsals of information
Students had nothing to say about the texts because they had no reason to say anything
Students had nothing to say about the texts because they had no framework in which to develop any ideas about it
Very hard to mark (criteria when you don’t have a purpose?)
HOW DID THIS AFFECT OUR APPROACH TO ASSESSMENT?
New assessment plans have a reduced diversity of task types, at least for first year
courses greater coherence between individual assessment tasks
cumulative development across tasks development towards a clear end point, with each task
contributing to the end point
clear links between teaching/learning needs in class and assessment tasks
ASSESSMENT IN LANG 1056 ‘LANGUAGE AND CULTURE’
Developmental focus on essay writing and reading/using research writing
Academic literacy focus combined with content focus
Discrete literacy focus for each assessment task Cumulative learning developed through assessment
ASSESSMENT IN LANG 1056
Critical reading task guided reading task
based on a single set research article
emphasis on identifying and analysing arguments and evidence
emphasis on abstracting information across components of the research article
ASSESSMENT IN LANG 1056
Analytic reading task focused essay-like
response based on a single reading – using a research article to address a new question
essay topic related to reading but not about the reading – knowledge transformation
emphasis on locating and using information for new purposes
requires critical reading as starting point
ANALYTIC READING
Text:Wierzbicka , A. (1991) Cross-cultural pragmatics. The
Hague: Mouton de Gruyter. Ch 2: “Different cultures, different languages, different speech acts”
Task: Your task is to write an analytic response to
Wierzbicka’s paper which addresses the following topic:
The meaning of an utterance in a language is not simply the result of the meanings of the words in the utterance, but is fundamentally influenced by the cultural context in which it is uttered.
ASSESSMENT IN LANG 1056
Analytic reading task focused essay-like
response based on a single reading – using a research article to address a new question
essay topic related to reading but not about the reading – knowledge transformation
emphasis on locating and using information for new purposes
requires critical reading as starting point
ASSESSMENT IN LANG 1056
Synthetic reading task focused essay-like
response based on a set of readings
essay topic related to readings – knowledge transformation
emphasis on using multiple sources to address a question
emphasis on synthesising arguments and evidence from multiple sources
requires critical reading and analytic use of reading as starting point
SYNTHETIC TASK
Texts:Nelson, et al. (2002). Cross-Cultural Pragmatics: Strategy Use in
Egyptian Arabic and American English Refusals. Applied Linguistics 23(2):163-189.
Hickey, L. (2005). Politeness in Spain: Thanks but no “thanks”. In L. Hickey and M. Stewart (eds). Politeness in Europe. Clevedon: Multilingual Matters.
Haugh, M. (2003). Revisiting the conceptualisation of politeness in English and Japanese. Multilingua 23(1-2): 85-110.
Task: Using only these readings, answer the following question in a brief
essay format:
While all cultures value politeness, what it means to be polite varies from culture to culture.
ASSESSMENT IN LANG 1056
Synthetic reading task focused essay-like
response based on a set of readings
essay topic related to readings – knowledge transformation
emphasis on using multiple sources to address a question
emphasis on synthesising arguments and evidence from multiple sources
requires critical reading and analytic use of reading as starting point
ASSESSMENT IN LANG 1056
Essay conventional essay task requires critical reading
and analytic and synthetic use of readings as starting point
involves locating own sources
ASSESSMENT IN LANG 1055
Focus of course is on micro-analysis of language data
Academic literacy focus: analysing language data constructing an argument based on language examples
Previous assessment items included a ‘conventional’ essay, requiring finding own references
Research literature important and much exposure in course, but didn’t need assessment item since focus of LANG 1056
ASSESSMENT IN LANG 1055
Language analysis tasks One task per course module
semantics ‘grammar’ (morphology and syntax) phonetics and phonology
Modelled in tutorials spot patterns in the data develop rules to account for the patterns (choosing between competing analyses)
Assessment criteria include treating the data as the unit for analysis analysis justified on the basis of the data
ASSESSMENT IN LANG 1055
‘Data analysis and justification’ essay replaced previous ‘conventional’ essay half-way through course structure modelled on previous six or seven readings students choose one of eight languages (they’re nearly
all studying a language also, strong engagement) given 30-40 example sentences initial analysis, then an explicit choice, each path with
advantages and disadvantages students explicitly told there is no single right answer need to give evidence for their particular answer, they
are assessed on their justification, and how they argue that their analysis is superior to alternatives
CONCLUSIONS
Important to begin from what you want students to learn
‘Whole-of-major’ approach Students engaged with professional research
literature Academic literacy through discipline-specific
content Effect on students somewhat unclear, since most of
these students are only now finishing their first year, but some tentative outcomes
OUTCOMES?
Course evaluation for LANG 1056 Did the course make you feel more confident in dealing with
reading applied linguistics research? Resounding ‘yes’. Did the course develop your understanding of the types of
writing required in linguistics? Resounding ‘yes’. “I came into the course with little knowledge on reading
linguistics research and now I feel as if I can read research papers confidently”
“I went back over the first reading of the course and I discovered that I could actually understand what the article was trying to say, whereas before, I had much trouble understanding”
Students’ results LANG 1056 essays brought total marks up Some LANG 1055 essays publishable in terms of structure
and arguments