Dynamic assessment of academic writing: macro-Themes and

hyper-Themes

Prithvi ShresthaOpenELT, Department of Languages

The Open University, UK

What I want to talk aboutWhy this study?

Theoretical frameworks: DA and Systemic Functional Linguistics (SFL)

Research methodology

Data collection

Analysis and findings

Conclusion

Questions and suggestions

Why this study?

Lack of DA research in academic writing assessment (see Poehner 2008 for a review) and call for more targeted and ‘dialogic’ tutor support (i.e., feedback) to students with their assignments (e.g., York 2003; Lillis 2006)

An academic writing tutor’s search for sustainable writing development support

This study explored:

Do Dynamic Assessment (DA) procedures enhance students’ writing development?

Defining academic writing and academic writing assessment

Academic writing: academic texts expected to be written by students in a particular discipline in higher education (HE), which entails having the epistemological, linguistic and socio-cultural knowledge required in order to become a part of that disciplinary community.

Academic writing assessment: the range of procedures used to “describe the promise and limitations of a writer” working in a particular rhetorical, linguistic and socio-cultural context (emphasis added, Huot 2003, p. 107)

Theoretical frameworks (1): DADynamic Assessment (DA) driven by Vygotskian

(1978) sociocultural theory of learning

Definition of DA: an “approach to understanding individual differences and their implications for instruction … [that] embeds intervention within the assessment procedure” (Lidz and Gindis, 2003 p.99).

Goal of DA: identify learners’ Zone of Proximal Development (ZPD) and actual ability and help them to move to the next level of zpd; focus on process and not on product; concerned with future rather than the past

Two key concepts in DA: (1) zone of proximal development (ZPD) and (2) mediation

Theoretical frameworks (1): DA (contd.)

ZPD: “the distance between the actual development level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers” (Vygotsky 1978, p. 86).

Mediation: making sense of the world around us via human artefacts, symbolic tools and other humans (Kozulin, 2003).

Dynamic Assessment (DA) in the literature

General education (see review in Haywood and Lidz 2007)

Language education (e.g., Poehner 2005; Anton 2009; Oskoz 2005) – all in the context of foreign/ second language in a face-to-face situation

Theoretical frameworks (2): genre theory based on Systemic Functional

Linguistics

Genre theory based on Systemic Functional Linguistics (SFL) as developed by Halliday and his colleagues (e.g., Halliday and Martin 1993)

Definition of genre: ‘staged, goal-oriented social processes’ (Martin 1997)

Texts/ genres have different functions in different social contexts (e.g., narration, argumentation). Such functions/ purposes are realised through register variables - field (subject matter), tenor (reader-writer relationship) and mode (medium of communication).

SFL genre theory and aca- demic writing assessment

Heavy focus on textual analysis (i.e., textual features)

Diagnostic tool (e.g., Bonnano & Jones 2007)

Criteria for assessment (e.g., Rose, Rose, Farrington, & Page, 2008)

To date no strong focus on SFL-oriented systematic procedures in formative or summative assessment in higher education

Research contextAcademic writing course in open and distance

learning

Majority of students native speakers of English (ca. 400 students)

Tutor-student interaction via online forum and emails

Researcher tutoring one group (ca. 20 students) – two students volunteered to participate

Data collection methodsDynamic assessment ‘sessions’ over six months

(i.e., tutor-student interaction, various drafts of students’ assessment texts) – the assignment tasks required students to apply a business study framework to a business situation/ problem (i.e., case study analysis)

Tutor mediation in these DA sessions followed the basic principles proposed by Haywood and Lidz (2007, p. 42).

Focus of the present presentation: macro-Themes and hyper-Themes of DA1 and DA2 two drafts each

Data analysisAn analysis of macro-Themes and hyper-Themes

in students’ assessment texts following SFL genre theory (e.g., Martin & Rose 2007)

macro-Theme: refers to the introductory paragraph in an essay or analysis which predicts what the text will be about (Martin 1993).

Hyper-Theme: is defined as ‘a clause (or combination of clauses) predicting a pattern of clause Themes constituting a text’s method of development’ (Martin 1993, p. 245).

Findings: macro-Themes (1)Michelle’s (pseudonym) macro-Themes

DA1 draft 1: lengthy background information; lacks sufficient orientation for the reader

DA1 final draft: same except a few minor changes

DA2 first and final drafts: concise and focused; clearly indicates how the analysis will develop following the STEP framework (classification criteria)

Findings: macro-Themes (2)Natasha’s (pseudonym) macro-Themes

DA1 draft 1: some orientation but no clear classification criteria (i.e., STEP factors)

DA1 final draft: clearly states what the analysis will do; clear classification criteria

DA2 first and final drafts: similar to DA1 final; clearly sets out how the analysis will be developed; classification criteria explicit; final version with more business concepts and sentences better connected

Findings: hyper-Themes (1)Michelle’s hyper-Themes (examples)

DA1 draft 1: A major development by Heineken introduced a better tasting non alcoholic beer onto the market. (P4)

DA1 final: Technological factors affect Heineken's marketing strategy in one main way. (P4) 

DA2 draft 1:Although safety has been one of the main focuses for the safer syringe market, there have been some Technological factors which have influenced the market. (P3)

DA2 final: Technological factors influencing the Safer Syringe market include the production of Safer Syringes in large quantities and the type of products available. (P3)

Findings: hyper-Themes (2)Natasha’s hyper-Themes (examples)

DA1 draft 1: Technological factors: It was important to create a tasty product for the regular beer consumers. (P3)

DA1 final: Several technological factors influenced Buckler 's development. (P3)

DA2 draft 1: Technology plays a central role in adoption of the safer syringes. (P3)

DA2 final: Technology also plays a central role in the adoption of safer syringes.

Findings: learner perception“… New method is more relaxed whilst the

traditional one is more stressful, more personal… The method was very encouraging – prompted to think about the problem. Gave an opportunity to try again with some prompts. If required explicit help was given. Very ‘dynamic’ approach…” (Michelle).

“… The new method of assessment helped a lot because there were more specific details that guided in the process of writing. When I see more details, I remember next time while writing… I would like to have a similar experience in other courses… I wish I could do the project longer… Can we start the research again?...” (Natasha).

Conclusion The findings suggest that dynamic assessment

procedures do enhance learners’ academic writing development, albeit at varying levels.

It was also found that both the students showed progress regarding their ability to manage macro-Themes and hyper-Themes in Taxonomic Explanations.

Overall, it appears that DA procedures do help students improve their academic writing if implemented appropriately by considering individual students’ potential to develop. Although these findings cannot be generalised given the specific context in which the DA procedures were implemented, this study has pedagogical implications for writing assessment in a specific discipline in higher education.

Questions and suggestions?Any questions?

Any suggestions?

References Antón, M. (2009). Dynamic Assessment of Advanced Second Language Learners. Foreign Language Annals,

42(3), 576-598.

Bonanno, H., & Jones, J. (2007). Measuring the Academic Skills of University Students. Sydney: Learning Centre, the University of Sydney.

Halliday, M. A. K. & Martin, J. R. (eds.) (1993) Writing science: Literacy and discursive Power, London, The Falmer Press.

Haywood, H. C., & Lidz, C. S. (2007). Dynamic assessment in practice : clinical and educational applications. Cambridge: Cambridge University Press.

Huot, B. (2002). (Re)Articulating Writing Assessment: Assessment for Teaching and Learning. Logan, Utah: Utah State University Press.

Kozulin, A. (2003) “Psychological tool and mediated learning”, in Kozulin, A., Gindis, B., Ageyev, V.S. and Miller, S.M. (eds.) Vygotsky’s educational theory in cultural context, Cambridge, Cambridge University Press.

Lidz, C.S. and Gindis, B. (2003) “Dynamic assessment of the evolving cognitive functions in children” in Kozulin, A., Gindis, B., Ageyev, V.S. and Miller, S.M. (eds.) Vygotsky’s educational theory in cultural context, Cambridge, Cambridge University Press.

Lillis, T. M. (2006). Moving towards an Academic Literacies Pedagogy: Dialogues of Participation. In L. Ganobcsik-Williams (Ed.), Teaching Academic Writing in UK Higher Education. Basingstoke: Palgrave Macmillan.

References (contd.)Martin, J. R. (1993) “Life as a Noun: Arresting the Universe in Science and Humanities”, in Halliday, M. A. K. &

Martin, J. R. (eds.) Writing science: Literacy and discursive Power, London, The Falmer Press.

Martin, J. R., & Rose, D. (2007). Working with Discourse: Meaning beyond the Clause. London and New York: Continuum.

Martin, J. R. (1997). Analysing genre: functional parameters. In J. R. Martin & F. Christie (Eds.), Genre and Institutions: Social Processes in the Workplace and School (Vol. first). London and Washington: Cassell.

Oskoz, A. (2005). Students' dynamic assessment via onlline chat. CALICO Journal, 22(3), 513-536.

Poehner, M. E. (2005). Dynamic assessment of oral proficiency among advanced L2 learners of French. Unpublished PhD Dissertation/Thesis,, Pennsylvania State University, University Park.

Poehner, M. E. (2008). Dynamic assessment : a Vygotskian approach to understanding and promoting L2 development. [New York?]: Springer.

Rose, D., Rose, M., Farrington, S., & Page, S. (2008). Scaffolding academic literacy with indigenous health sciences students: An evaluative study. Journal of English for Academic Purposes, 7(3), 165-179.

Vygotsky, L.S. (1978) Mind in Society: The Development of Higher Psychological Processes, Cambridge, MA, Harvard University Press.

Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45(4), 477-501.