This may be the author’s version of a work that was submitted/acceptedfor publication in the following source:

Willis, Jill, Adie, Lenore, & Klenowski, Val(2013)Conceptualising teachers’ assessment literacies in an era of curriculumand assessment reform.Australian Educational Researcher, 40(2), pp. 241-256.

This file was downloaded from: https://eprints.qut.edu.au/58342/

c© Consult author(s) regarding copyright matters

This work is covered by copyright. Unless the document is being made available under aCreative Commons Licence, you must assume that re-use is limited to personal use andthat permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then referto the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe thatthis work infringes copyright please provide details by email to qut.copyright@qut.edu.au

Notice: Please note that this document may not be the Version of Record(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) canbe identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.

https://doi.org/10.1007/s13384-013-0089-9

1

Conceptualising teacher assessment literacies in an era of

curriculum and assessment reform.

published (2013) Australian Educational Researcher, pp. 1-16.

Jill Willis

Queensland University of Technology, Brisbane.

phone 3138 3798 jill.willis@qut.edu.au

Lenore Adie

Queensland University of Technology, Brisbane.

phone 3138 8552 lenore.adie@qut.edu.au

Val Klenowski

Queensland University of Technology, Brisbane.

phone 3138 3415 val.klenowski@qut.edu.au

Teacher assessment literacy is a phrase that is often used but rarely defined. Yet understanding

teacher assessment literacy is important in an international curriculum and assessment reform

context that continues to challenge teachers’ assessment practices. In this article situated examples

of classroom assessment literacies are analysed using Bernstein’s (1996, 1999) theoretical tools of

vertical and horizontal discourses, classification and framing. Drawing on a sociocultural view of

learning, the authors define teacher assessment literacies as dynamic social practices which are

context dependent and which involve teachers in articulating and negotiating classroom and

cultural knowledges with one another and with learners, in the initiation, development and practice

of assessment to achieve the learning goals of students. This conceptualisation of assessment

literacy aims to make explicit some underpinning theoretical constructs of assessment literacy to

inform dialogue and decision making for policy and practice to benefit student learning and

achievement.

Keywords: assessment literacies; assessment reform; sociocultural theory

2

Introduction

Australian teachers are experiencing considerable change to their

curriculum and assessment practices as the first stages of the Australian

Curriculum are implemented in schools in 2012 and beyond (ACARA 2010). This

climate of curriculum and assessment reform is a shared international experience

(Darling-Hammond 2011) and in Australia is providing opportunities for national

conversations about the future of learning in schools (Yates, Collins, and

O'Connor 2011). It is timely to revisit the meaning of assessment literacy in terms

of the message systems of curriculum, pedagogy and assessment, and the

relevance for contemporary educational contexts. This paper argues that teacher

assessment literacy needs to be positioned within a discussion about learning

theory, and understood as an ethical practice that is social, dynamic and layered.

Assessment literacy is a term that is often used yet rarely defined by those

who use it, reflecting an assumption that the term is self-explanatory (Criswell

2006). In this paper, we apply a sociocultural lens to an explanation of assessment

literacy to highlight it as a social practice, within which negotiation and cultural

knowledge are seen as significant. The following definition is offered:

Assessment literacy is a dynamic context dependent social practice that

involves teachers articulating and negotiating classroom and cultural

knowledges with one another and with learners, in the initiation,

development and practice of assessment to achieve the learning goals of

students.

Schools, through the three interrelated message systems of curriculum, pedagogy

and assessment (Bernstein 1971) regulate access to increasingly diverse social

3

discourses, and the learning opportunities students can access (Lingard 2007;

Zammit 2011). To provide a common language to discuss how students can

become full participants in the multiplicity of schooling discourses in our diverse

and increasingly globalized society, the New London Group (1996) developed the

theory of multiliteracies. Teachers and students are multi-literate when they have

“the capacity to speak up, to negotiate, and to engage critically” (p. 67) in the

various modes of meaning that are “dynamic representational resources,

constantly being remade by their users as they work to achieve their various

cultural purposes” (p. 64). Different literacies are needed for different

communities of practice as they are distinct social groups, each with their own

repertoires (Hipwell and Klenowski 2011;Wenger 1996). Yet the complexities of

the capabilities and knowledges teachers draw on in their assessment communities

of practice are hidden by the phrase ‘assessment literacy’. The verb ‘assess’ has

been nominalised to become a concept, and the adjective ‘literate’ has been

nominalised to create a noun, so teachers can be described as possessing

assessment literacy. Nominalisation is a linguistic feature used to “compact

information – whole conversations – that we assume people (or at least ‘experts’)

are up on” (New London Group 1996 p. 79). To open up the conversation, and

emphasise the complexity of teacher work within multiple assessment discourses,

the phrase ‘assessment literacies’ will be used in the remainder of the article.

The concept of assessment literacies explored in this paper assumes that

discourses within various assessment practices, such as peer assessment,

classroom performances, state tasks or national tests, are representational

resources through which teachers and learners can shape their identity through

participation and through interactions in their various assessment communities of

practice (Gipps 2008). As Gee (2003) notes, all learning is learning to ‘read’,

4

which includes reading words and social and cultural signs within specific

semiotic domains. In this paper, assessment is understood as a specific semiotic

domain. Bernstein’s (1996, 1999) concepts of horizontal and vertical discourse are

drawn from to explicate the multiple dimensions of various assessment literacies

within the semiotics of contemporary assessment practices.

A Bernsteinian analysis for the conceptualisation of assessment

literacies

Bernstein (1999) described vertical and horizontal structures of

educational discourse, providing a theoretical language to distinguish between

ways that knowledge is acquired and transmitted, inducting the user into the

language of the community of practice. Vertical discourses he referred to as

‘official’ or ‘schooled’ knowledge, and horizontal discourses as ‘local’ or

‘common sense’ knowledge (1999, p. 158). Vertical discourse has a “coherent,

explicit and systematically principled structure, hierarchically organised…and

takes the form of specialised languages” (p. 161). It is disciplinary knowledge

“constituted in scientific research communities, literary and artistic organisations”

and encoded in highly symbolic forms that “must be decoded or translated

(pedagogised) in order to be accessible to those outside the specialist domain”

(Singh 2002, p. 575). Horizontal discourse is likely to be “oral, local, context

dependent and specific, tacit, multi-layered” (Bernstein 1999, p. 159) and learning

it “probably requires oral transmission; the experience of a social interactional

relationship” with an expert in the practice (p. 165). Competence in a horizontal

discourse will often not transfer to other contexts, as “the competence is tied to

the social context in which it is enacted” (Moss, 2000, p. 52).

5

Before teachers help students to recognise (read), realise (speak) and move

between the multiple discourses of schooling, they need to be literate in these

discourses themselves, as well as be able to “recontextualise” the knowledges to

induct their students into the discourses (Bernstein 1996, p. 47). Bernstein’s

(1996) concepts of classification, that is the strength of boundaries between

contexts, and framing, the internal logic within a discourse, are proposed as

helpful conceptual tools to analyse, explain and define assessment literacies. The

power relations that maintain the distance between discourses are hidden,

appearing natural, real and “the source of integrity” (Bernstein 1996, p. 21). A

strong classification (C+) indicates that a discourse has a distinct voice,

specialised rules and unique identity, in contrast to weak classification (C-) with

less specialised voices and discourses. While horizontal discourses tend to have

weak classification, they can have weak or strong framing (Bernstein 1999, p. 64).

Framing refers to the kind of talk and spaces that are constructed to regulate the

selection, sequencing, pacing, criteria and rules of social order that control

communication within local pedagogic practice, that is “who controls what”

(Bernstein 1996, p. 27). When framing is strong (F+), the teacher has more

control over communication and the social base, whereas where framing is weak

(F-), the student or “acquirer has more apparent control” (p. 27. emphasis in the

original). Assessment discourses that exhibit strong classification, (for example

economic assessment practices have organisation and specialised language quite

different to assessment practices in visual arts), and strong framing (with well

known symbols and an acknowledged canonical hierarchy of concepts) would be

notated as C+ F+ to indicate the strength of the internal grammars (Bernstein

1999, p. 164). Acquisition of a horizontal discourse with weak framing (F-) is

more problematic, as it is highly situated and local, and expertise is gained

6

through “a tacit acquisition of a particular view of cultural realities” so that what

is counted as knowledge depends on whose perspective is dominant in the social

context (Bernstein 1999, p. 165). Teachers not only recontextualise knowledge

and structure the classroom pedagogic discourse through curriculum and

assessment practices, moving between horizontal and vertical discourses,

additionally power relations are legitimated, contested and negotiated (Singh

2002, p. 578). These fundamental Bernsteinian concepts of vertical and horizontal

discourse and classification and framing are used in the following section to

conceptualise assessment literacy.

Vertical and horizontal discourses of assessment and views of learning

Assessment literacies conceptualised as vertical or horizontal discourses

within different cultural and policy contexts, are underpinned by different

assumptions about learning. Within traditional measurement conceptions of

assessment that emerged from early twentieth century paradigms of scientific

measurement, social efficiency and behaviourist learning theories, assessment

remains separate from instruction. In direct contrast the social-constructivist

paradigms emerging later in the twentieth century emphasise the interrelatedness

of assessment, curriculum and learning theory (Shepard 2000. p 5). Traditional

measurement views of assessment value scientific measurement by objective tests,

and commitment to precise standards (p. 6) and reflect the characteristics of a

vertical discourse with strong classification and framing (C+F+). This view of

learning underpinned early definitions proposed for assessment literacy such as an

ability to “know the difference between sound and unsound assessments” with

sound assessments being described as meeting standards of appropriate purposes,

targets, methods, and achievements with control for bias (Stiggins 1995. p. 238).

7

This definition of assessment literacy may describe the teacher’s role within high

stakes assessment cultures in the USA (Popham 2009), Confucian heritage

cultures (Carless 2011), and within Australian National Assessment Program –

Literacy and Numeracy [NAPLAN] (Klenowski, 2011). With clear skills,

specialist languages and processes (Maton, 2000) they are examples of vertical

assessment discourses. Assessment case studies in other policy contexts such as

Canadian teacher education (DeLuca and Klinger 2010), standards referenced

classroom assessment in Queensland (Adie 2010) and formative or assessment for

learning contexts (Willis 2011; Cowie, 2005) reflect characteristics of horizontal

assessment discourses in that they are local, with the integration of assessment

practices, learning and pedagogy. Assessment literacy in a horizontal discourse is

more like “critical inquiry [into] the practices and processes of assessing – social

and cultural acts of doing assessment in actual contexts” (Wyatt-Smith and Gunn

2009 p. 87), reflecting sociocultural views of learning. Assessment literacy will be

understood differently depending on the view of learning embedded in the cultural

and policy context and so is closely associated with theories of learning. In this

paper, a sociocultural view of learning informs the proposed definition of

assessment literacy.

Assessment literacy from sociocultural perspectives

From sociocultural perspectives (Gee 2004; Rogoff 1990; Wenger 1998)

assessment is viewed as a cultural activity where individuals negotiate meaning

and identity through participating in the activity with the social, cultural and

historical experiences they bring (Elwood 2006; Gipps 1999; Pryor and

Crossouard 2008). Assessment “must be reconceived in more active and relational

terms rather than in terms of static matching and fixed descriptive frameworks”

8

(Bredo 1994, p. 24). Assessment is linked to learning interactions and pedagogical

relationships and is conceived not as “something that is being done to students

[but rather as] something that is being done with and for the students” (Klenowski

2009, p. 89). Drawing from the works of Dewey (1919|1966) and Vygotsky (in

Cole et al. 1978), it importantly shifts the location of learning from inside an

individual’s brain, to be “an aspect of person-environment interaction itself”

(Bredo 1994, p. 24). It expands the definition of assessment literacies beyond a

traditional view of “skills, knowledges and cognitions that reside within the

individual” (Gee 2003) to culturally responsive practices through which teachers

open up opportunities for students to negotiate their identity and practice within

the learning community (Adie 2010; Klenowski 2009; Willis 2010). While these

are worthy goals, they are not simple to achieve as teachers work within multiple

horizontal and vertical assessment discourses simultaneously.

Teacher assessment literacies are situated, complex and challenging.

Assessment literacies within vertical discourses might emphasise the knowledges

that teachers need when identifying misconceptions and gaps in individual

students’ acquired concepts, and the competencies required to adequately

diagnose students’ needs and position them on developmental continua (Griffin,

Woods and Nguyen 2005). The challenge is that for teachers “this is a task of

immense difficulty, one that is far more complex than that implied by the notion

of a ‘gap’” (Black and Wiliam 2006, p. 90). Elwood (2008. p. 93) notes that a

focus on closing the gaps is a constructivist view of learning where “learning gets

located back within the individual” which can lead to teachers attributing lack of

learning to the innate qualities of the learner rather than the teaching (Murphy and

Whitelegg 2006). Teachers who resist this discursive position and see themselves

as agents for change, responsible for inviting students to connect their peer-based

9

and community-based identities with the specialist discourses connected to school

assessment, enable students to achieve learning gains and autonomy through

assessment (Marshall & Drummond 2006). Developing assessment literacies

therefore includes knowing how to create opportunities for students to connect to

the official school discourse, a principle that is highlighted in assessment literature

as “an ethical prerequisite for fair assessment” (Gee 2008, p.76; Elwood & Lundy

2010). However this assumes that teachers have knowledge of students’ local

community identities and can “recontextualise” this knowledge through their

classroom instructional discourse and the regulative discourse that provides the

rules for social order (Bernstein 1996 p. 47). Additionally, developing teacher

assessment literacies requires continuous learning about new curriculum and

assessment policies and contributions to the ensuing discussions and practices

through which new assessment discourses emerge. Policy change requires

teachers to develop new repertoires which significantly impacts on the work of

teachers and the assessment, curriculum and pedagogic discourses of the

classroom (Comber 2012). Rather than a singular assessment literacy, the multiple

discourses indicate that teachers and students need multiple assessment literacies.

A sociocultural view of assessment literacies thus acknowledges that assessment

literacy is not a singular or fixed set of capabilities, but a capability that is situated

and needs to be understood within the assessment culture and policy context of

the community. Through Bernstein’s (1999) concepts of horizontal and vertical

discourses, assessment literacies can be explored so that the embedded literacies

within various assessment discourses emerging and merging in the current era of

curriculum reform can be identified, articulated and further understood.

10

Assessment literacies and ethical considerations in times of educational

reforms: Examples from the Australian assessment context.

Australian teachers are experiencing an era of educational reform as they

implement a national curriculum in English, Maths, Science and History, with

additional discipline areas soon to follow. The Australian curriculum specifies

content to be taught and the achievement standard expected for each year level

and discipline (ACARA 2011). Assessment is discussed in terms of a range of

purposes: to diagnose, to inform learning, to provide consistency in reporting and

to determine levels of achievement in aspects of literacy and numeracy and other

learning areas (ACARA 2010a). Alongside the Australian curriculum is a

National Assessment Program in Literacy and Numeracy (NAPLAN) that

includes annual testing of Years 3, 5, 7 and 9 students’ levels of achievement.

The introduction of standards for learning and assessment has impacted the

curriculum in many countries in two main ways (Centre for Educational Research

and Innovation 2005; Clarke et al.2000; Zepke et al. 2005). First, there is a

horizontal discourse within standards-referenced assessment that aims to align

curriculum and assessment by placing student learning in the centre of the policy

process, and relying on teacher social moderation to reach local agreement about

standards (Learning and Teaching Scotland 2010: Queensland Studies Authority

2009). Second, standards are also used for accountability as seen in national

assessment programs such as NAPLAN. This vertical assessment discourse is

often viewed in competition with a student-centred discourse of assessment yet

the horizontal and vertical assessment discourses overlap and inform teacher

practice. The following varied assessment contexts that exemplify the complexity

of teachers’ work are only a few of the multiple assessment discourses that

Australian teachers are required to be fluent within.

11

The introduction of standards-referenced curriculum into Australian

schools brought with it processes, such as social moderation, to ensure the

reliability and consistency of judgement decisions. The practice of social

moderation is located within the assessment policies in many Australian

educational sites, yet for a number of teachers it is a new assessment practice.

Understanding assessment through the negotiated practice of social moderation

involves teachers in both the vertical discourses of specialised, hierarchically

organised discipline knowledge, and the horizontal discourses of local place-

specific interpretations of standards and criteria. This is a conceptually complex

assessment context involving teachers negotiating their judgement decisions

across systems of knowledge generation which include:

1. Thelocallygeneratedschool‐basedunderstandingsofassessmentpolicy,processesandpractices,ahorizontaldiscourse

2. Theteacher’slearnedknowledgeofadiscipline(forexample,thedifferentdisciplineknowledgesofmathematics,scienceorEnglishthatareverticaldiscourses),and

3. Theteacher’spersonalbeliefsaboutlearningandassessment,andtheroleofstudentsintheseprocesses,ahorizontaldiscoursedevelopedthroughexperience.

Through participation in moderation meetings, teachers are expected to negotiate

a shared understanding of the qualities of a standard by explicitly identifying and

justifying how the evidence denotes a standard. It is also anticipated that teachers’

involvement in these processes of negotiation and explication will enhance their

teaching practice and enable them to support their students to realise the

discipline-specific assessment literacies.

The adoption of standards-based assessment and the understanding that

achievement standards are fuzzy and lacking in precise boundaries of distinction

(Sadler 1987), could be seen to weaken the classification (C-) of a discipline,

particularly those discipline areas with a history of specialised and hierarchically

12

organised knowledge systems. For example, the introduction of open-ended

assessment tasks into a discipline such as mathematics can challenge historic

notions of correctness based on a single answer. When the students’ justification

and reasoning are to be assessed, then a previously strong classification (C+) of

mathematics that involved specialised rules of what is understood as correct

responses will be weakened (C-) as teachers negotiate their local understandings

of what denotes quality in terms of justification and reasoning.

The inclusion of social moderation into a standards-referenced assessment

system is intended to strengthen the framing (F+) so that the social orders that

control the [assessment] communication become embedded in local assessment

practices. However, when teachers are involved in a range of new assessment

practices, for example the introduction of standards-referenced assessment and

social moderation, then we may expect that the framing would be considerably

weakened (F-). Adie (2010) describes such a context in a project that investigated

the introduction of online social moderation processes to middle school teachers

in Queensland, Australia. The teachers were involved in the judgement and online

moderation of newly introduced common assessment tasks in English,

mathematics and science, which were based on stated achievement standards and

judged using task-specific assessment criteria. For the teachers, the practices of

standards-referenced assessment, social moderation, online communication, and

the use of common assessment tasks were all new practices.

To successfully negotiate the assessment context of online moderation, the

teachers needed to be able to move between their disciplinary knowledge and their

local contextual knowledge to justify their judgement decisions, and to form

common understandings of the standards with the other teachers involved in the

meeting. This new assessment discourse presented as weakly framed (F-) and

13

problematic as there was little shared prior knowledge or history within the group

to draw from. While experience in such a context would strengthen the framing of

this discourse, it was apparent in Adie’s (2010) investigation that teachers drew

from other knowledges and attributes such as an ability and willingness to

problem-solve to overcome some of the weaknesses in this discourse. The

teachers’ use of the online tools that enabled annotation of the evidence of a

standard within student work also strengthened the framing of this assessment

practice. Teachers’ willingness to work through and gain control over difficulties

that were discipline-related, technological or part of the negotiation process in the

online moderation meetings, enhanced their participation in the assessment

practice. The introduction of new ways of ‘knowing’ and ‘doing’ assessment

required that the teachers articulate their current understandings and negotiate

how or if these were accommodated in the new practice. Teachers also needed to

draw on other skills and understandings to fully engage with these new

assessment practices.

Through participation in a new social assessment practice, the tools and

shared histories that began to strengthen the framing of the horizontal discourse

were being negotiated. The social and conceptual tools are examples of the

“dynamic representational resources, constantly being remade by their users”

(New London Group 1996, p. 64). Social participation is an important part of

developing literacy within a discourse, as Bernstein (1996) asserts that the

regulative or the social discourse is the dominant discourse. In the example of

social moderation, the social resources enabled the development of an

increasingly strengthened framing of the horizontal assessment discourse.

Bourne (2004) illustrated how a teacher used the deliberate strengthening of the

framing within a horizontal literacy discourse, in partnership with positive social

14

cues, to invite the students to become insiders into the community of meaning.

However, strong framing does not always lead to inclusion. The following

Assessment for Learning contexts illustrate other variations of classification and

framing within horizontal assessment discourses that were used to support

learners.

Assessment for Learning (AfL) practices such as sharing learning goals,

feedback, peer and self-assessment have become part of international educational

policy discourse since Black and Wiliam (1998) highlighted associated positive

achievement gains. In some policy contexts such as Northern Ireland (Gardner

2011) AfL is a systemic priority with the clear skills, specialist languages and

processes indicative of vertical discourse. In the assessment policy context of

Queensland, Australia, AfL has been positioned as a horizontal discourse,

integrated within classroom learning episodes (QSA 2009). Through AfL

practices, teachers expect students to shift from being passive recipients to

becoming active learners who take responsibility for and manage their own

learning (Black and Wiliam 2006). However as Willis (2011) shows teachers can

frame assessment practices in a variety of ways, all with the intention of including

students in the discourse. In a Year 9 science class the teacher strongly framed

(F+) the AfL practices creating a shared repertoire of daily learning goals, a self-

assessment revision quiz, and peer feedback. The strong framing was shaped by

the vertical discourse of science (C+F+) and further enabled by the teacher’s use

of social stories based on his own and his student’s life experiences to

recontextualise the concepts. In a Year 7 class the teacher provided explicit

direction and vocabulary, and used encouraging verbal and nonverbal

communication, to establish cooperative learning habits among student peers

across a number of their subject areas prior to peer assessment (C-F+). This

15

changed over time as students developed skills in collaboration to become a

weaker framing, where the teacher’s questions positioned the students as decision

makers (C-F-). A further effective combination occurred in Year 8 when the

teacher deliberately used weak framing of peer and teacher feedback in an

integrated social science, technology and personal development assessment task to

challenge students to experiment with technology (C-F-). He created positive

relationships with the students although this was also combined with occasional

strong framing of social expectations (F+ of the regulative discourse).

The teachers’ framing of AfL practices occurred within both the

instructional and regulative discourses (Bernstein 1996) and the framing strength

shifted depending on the teachers’ own learning preferences as well as what type

of framing the teacher thought would best help students learn to accomplish the

desired assessment practice at that time. Teacher assessment literacy therefore

included the ability to notice and ‘read’ and respond to the assessment and

learning practices of the students (Morgan and Wyatt-Smith 2000) and to

encourage them “to act in literate ways that make their learning visible and

audible” to themselves and others (Johnston and Costello 2005, p. 262). Positive

social relationships between teachers and students and shared repertoires of AfL

practices made expectations of quality and the practices of experts visible and

accessible to students as explicit and tacit knowledge, and this was achieved

through both strong and weak framing. Not all students are able to read the tacit

social cues about how to participate appropriately in AfL practices (Pryor and

Crossouard 2008), so it becomes the responsibility of “teachers, and more

experienced colleagues, peers and mentors... to open up practice to learners by

engaging them in the joint enterprise… and extending their access to the

repertoire of the tools of the community” (Murphy et al. 2008 p.113). Knowing

16

how to work within the discourses to open up opportunities for students to

succeed has the potential to enable teachers to “interrupt the reproduction of

educational inequality” (Gamble and Hoadley 2004. p. 157). The importance of

teachers ‘reading’ the degree of student participation was a new understanding of

assessment literacy for the teachers linked to ethical assessment practice. It

challenged their traditional beliefs that assessment was purely about measuring

cognitive outcomes, particularly when there was a concurrent national assessment

policy emphasis on national testing through NAPLAN and increased

accountability as school results are published on the MySchool website (ACARA

2010b).

Australia’s increased use of assessment within systems of accountability is

part of a global economisation of education policy (Lingard 2010). Within this

context, ethical assessment and equity principles have gained increased

significance as ethical dilemmas emerge from an over-reliance on high-stakes

testing (Nichols and Berliner 2007; Berliner 2008). International assessment

theorists (Stobart 2008; Berliner 2008) have identified the social science principle,

known as Campbell’s law as relevant in this context of so-called transparency and

more performance-related accountability. This principle states that the more any

quantitative social indicator is used for social decision-making, the more subject it

will be to corruption pressures, and the more likely it will be to distort the social

processes it is intended to monitor. National assessments (social indicators) can

be used to maximise outcomes for accountability and the monitoring of standards

(social decision-making) with unintended consequences for schools, teachers and

students (Elwood and Lundy 2010; Stobart 2008).

Students, particularly those from different ethnic or cultural groups, have

been found to have differential access to curriculum and to level of qualifications,

17

which is seen as a discriminatory practice (Gillbourn and Youdell 2000).

Students can also feel marginalised, can be excluded because of their behaviour or

can be separated into special education provision because educational difficulties

are “pathologised as difficulties inherent within students. This is true not only of

students with disabilities and those defined as ‘having special educational needs’,

but also of those whose socioeconomic status, race, language and gender renders

them problematic to particular teachers in particular schools” (Ainscow 2009, p

6). Such discriminatory practices and collateral effects can emerge in test-based

accountability contexts (Nichols and Berliner 2007). In England and the United

States, for example, the government exerted pressures on schools to improve their

test results. A negative feature of punitive accountability climates is the

emergence of pre-test coaching to improve particular students’ results to help the

school achieve its identified target. In this situation resources are allocated to

those students who have been identified as borderline, and who with help, are

likely to achieve a pass grade. By giving attention to those students who “might

just make it”, other students, such as those who will pass without support and

those who are unlikely to reach the standard to pass, are neglected (Stobart 2008,

127). An important ethical and legal dimension of teacher assessment literacy is

therefore being able to critically consider how assessment can be culture-inclusive

to challenge discriminatory consequences of assessment policy implementation

and promote greater respect for and valuing of difference (Assessment Reform

Group 1999; Tognolini and Stanley 2007; Young and Kim 2010).

In a study of culture-inclusive assessment for Indigenous Australian

students in regional and remote Queensland, Klenowski, Connolly & Funnell

(2012) found that teachers’ pedagogy and assessment practices required change to

open up opportunities for Aboriginal and Torres Strait Islander students to

18

demonstrate their understanding and achievement in mathematics. The research

problem of patterns of under-achievement by Indigenous students as reflected in

standardised tests provided the context for the study. Data across all levels -

school, state, national, and international – reveal that Australian schools are not

addressing equity issues effectively (DEST 2007; Sullivan, Tobias and

McDonough 2006) with Indigenous children scoring significantly lower than non-

Indigenous (De Bortoli and Thomson 2009) and with a decline in the performance

of Indigenous students in numeracy relative to that of the rest of the school

population as the period of time spent at school increases (DEST 2007). Drawing

on Bernstein’s concepts of knowledge codes and structures it is possible to

analyse how the vertical discourse of the subject mathematics and the strong

framing of the didactic approach to the teaching and assessment of this subject

hindered learning for some Indigenous students.

The vertical discourse and the strong framing and classification of the

pedagogic and assessment practices were impenetrable for some students in that

they found the literacy demands challenging and unfamiliar. Circulation of

knowledge in this context of strong classification and framing requires explicit

recontextualisation. It was in examining the assessment opportunities for cultural

connectedness for student learning that Klenowski et al (2012) found that if the

teachers worked more closely with Indigenous Education Workers, teachers could

appreciate fully the students’ cultural identities and contexts. Teachers could in

this way identify cultural influences, funds of knowledge and values supportive

for them to enhance their pedagogic and assessment practices to embrace and

extend the cultural spaces for learning and teaching of their Indigenous students.

In horizontal discourse the circulation of knowledge occurs through social

relations, practices and their contexts. The Indigenous Education Workers were

19

in a position to inform teachers of how mathematical assessment tasks could be

reframed to appear more authentic and engaging for Indigenous students by

responding to their local and cultural circumstances.

The opportunity for Indigenous students to participate in learning and

assessment, and to demonstrate their achievement, is fundamental to equity. Yet

such opportunities are sometimes hindered as Indigenous students find the vertical

discourse and the strong framing of the pedagogy do not recognise, respect or

value their experiences and their methods of demonstration of learning. A

culture-inclusive approach requires teachers to distinguish the ‘funds of

knowledge’ (González et al. 2008) that Indigenous students draw on in their

learning. Effortful teaching that is culture-responsive and allows for different

ways of knowing is also practised to facilitate increased agency for these students.

Berlack (2001), cited by Luke et al. (2002, p. 11), argued “students from a non-

dominant culture experience testing as a form of cultural intimidation.” These

authors state “that students from particular ethnic and racial groups may develop

attitudes and practices of resistance to the surveillance, judgement and

categorisation practices that are affiliated with large-scale testing.” Culture-

inclusive assessment does not attempt to favour the culturally different group,

rather it is recognised that cultural differences can impact on performance, such as

on standardised tests. The variables that may influence test performance include:

the cultural-specificity of how the task or activity in question is framed; the

cultural-specificity of the normative models of child and adolescent development

reflected in the domain specification and constructs of the test; the linguistic codes

and conventions of the test and task and the cultural-specificity of content

knowledge (Luke et.al. 2002).

20

These examples from Australian assessment contexts exemplify the

understanding that teacher assessment literacies include knowledge of the

grammars of both horizontal and vertical discourses in situated assessment

practices, and knowing how and when to frame horizontal discourses to create

both epistemic and social relations to enable full participation of students as

knowers (Maton 2000). It is through participation, and responsive framing that

interlocutors develop fluency in the situated assessment discourses.

Conclusions

We have stated that assessment literacy, as an ethical, social, dynamic and

layered practice needs to be positioned within a discussion about learning theory.

If assessment literacies are understood as a negotiated rather than static or

received understandings then a shared language is needed to enable teachers to

engage in critical inquiry of their assessment practices. Through this shared

discourse, teachers can begin to articulate and question their beliefs and

understandings of assessment. This paper has drawn on Bernstein’s concepts of

horizontal and vertical discourse and classification and framing, as a theoretical

language to open up conversations about assessment. Illustrative examples from

empirical assessment research have been analysed using the terms to propose a

sociocultural understanding of assessment literacies. The definition of assessment

literacy as a dynamic context dependent social practice that involves teachers

articulating and negotiating classroom and cultural knowledges with one another

and with learners, in the initiation, development and practice of assessment to

achieve the learning goals of students, is offered as an invitation for further

dialogue about how to support teachers and learners.

21

When teacher assessment literacies can be understood in terms of

discourses, teachers can be supported in developing the required skills through

participation in supportive communities of practice. The three assessment contexts

highlight this social and situated dimension of assessment literacy. Dialogue, and

learning by doing, give participants the opportunity to read and critically

problematise and recontextualise norms of assessment communities and

understand assessment demands. Acknowledging these varied experiences opens

up opportunities for shared understandings of assessment to be developed within a

local community of practice, as well as a wider global community. Pedagogic and

assessment practices that strive to achieve fairer and more equitable outcomes for

all students and that protect all young people’s rights to opportunities to learn,

progress and succeed, have become a global priority. Participation in assessment

as with participation in education must be ‘inclusive, avoid …discrimination …

and encourage opportunities for marginalized children to be involved … it needs

to provide equality of opportunity for all, without discrimination on any grounds’

(Elwood and Lundy 2010 p. 345). Valid, equitable assessment is a child’s right

(Elwood and Lundy 2010). There are important system-level implications here for

those who inform and support teachers in addressing these new demands.

Teacher assessment literacies are a fundamental issue that needs to be

made more explicit and discussed and debated. Through discussions about what is

meant by assessment literacies, we can open up our practices to develop shared

understandings rather than assuming that we share common needs and knowledge.

When teachers have the opportunity to discuss the implications for student

learning through assessment practices in climates of curriculum and assessment

reforms, teacher professionalism is honoured and assessment literacies are

articulated and developed as teachers negotiate their understanding. The extent of

22

how well teachers, policy officers, teacher educators and researchers and

professional bodies support the development of teacher assessment literacies will

be measured by the development of student assessment literacies. Developing

teacher assessment literacy is part of the ethical and moral responsibility to

provide opportunity for all students to learn.

23

References

Adie, L. (2010). Developing shared understandings of standards-based assessment: online

moderation practices across geographically diverse contexts. PhD thesis, Queensland University of

Technology.

Alexander, R. ed. (2010). Children, their world, their education: final report and

recommendations of the Cambridge Primary Review. Abingdon: Routledge.

Assessment Reform Group. (1999). Assessment for learning: Beyond the black box. Cambridge:

University of Cambridge School of Education.

Australian Curriculum and Reporting Authority (ACARA). (2010a). The Shape of the Australian

Curriculum. http://www.acara.edu.au/curriculum/curriculum.html - 3 Accessed 21st February,

2012.

Australian Curriculum and Reporting Authority (ACARA). (2010b.) My School ™.

http://www.myschool.edu.au/SchoolSearch.aspx Accessed 15th January, 2013.

Australian Curriculum and Reporting Authority (ACARA). (2011). The Australian Curriculum

version 1.2. http://www.australiancurriculum.edu.au/Home Accessed 19th November, 2011.

Ainscow, M. (2009). Local solutions for local contexts: the development of more inclusive

education systems. In Den inkluderende skole I et ledelsesperspektiv (Leadership perspectives on

the inclusive school,ed. Rasmus Alenkaer, Frydenlund: Copenhagen, DK.

Berlack, H. (2001). Race and the achievement gap. Rethinking Schools Online 15, no. 4.

http://www.rethinkingschools.org/restrict.asp?path=archive/15_04/Race154.shtml. Accessed 31

October, 2008

Berliner, D. (2008). Why Rising Test Scores May Not Equal Increased Student Learning.

Dissenthttp://www.dissentmagazine.org/online_articles/why-rising-test-scores-may-not-equal-

increased-student-learning. Accessed 12 January 2012.

Bernstein, B. (1971). On the classification and framing of educational knowledge. In M. F.D.

Young (Ed.), Knowledge and control: New directions for the sociology of education (pp 47 – 69).

London: Collier Macmillan.

Bernstein, B. (1996). Pedagogy, symbolic control and identity: Theory, research and critique.

London: Taylor and Francis.

24

Bernstein, B. (1999). Vertical and Horizontal Discourse: an essay. British Journal of Sociology of

Education. 20 (2) 157 - 173

Black, P. & Wiliam, D. (1998). Assessment and Classroom Learning. Education: Principles,

Policy and Practice. 5, no.1: 7-74.

Black, P. & Wiliam, D. (2006). Developing a theory of formative assessment in J. Gardner (Ed.),

Assessment and Learning. (pp. 81 – 100). London: SAGE publications.

Bourne, J. (2004). Framing talk: Towards a ‘radical visible pedagogy’ in Muller, J. Davies, B. &

Morais, A. Reading Bernstein, Researching Bernstein (pp. 61 – 74). London: RoutledgeFalmer

Bredo, E. (1994). Reconstructing educational psychology: Situated cognition and Deweyian

pragmatism. Educational Psychologist. 29(1): 23-35.

Carless, D. (2011). From testing to productive student learning. Implementing formative

assessment in confucian-heritage settings. New York: Routledge.

Centre for Educational Research and Innovation. (2005). Formative assessment: Improving

learning in secondary classrooms. Paris: Organisation for Economic Co-operation and

Development.

Clarke, M., Madaus, G., Horn, C. & Ramos. M. (2000). Retrospective on educational testing and

assessment in the 20th century. Journal of Curriculum Studies, 32(2): 159-181.

Criswell, J. (2006). Developing assessment literacy: A guide for elementary and middle school

teachers. Norwood, MA: Christopher Gordon publishers.

Cole, M., John-Steiner, V., Scribner, S. & Souberman, E. eds. (1978). L.S. Vygotsky. Mind in

Society. The development of higher psychological processes. Cambridge, MA: Harvard University

Press.

Comber, B. (2012). Mandated literacy assessment and the reorganisation of teachers’ work: federal

policy, local effects. Critical Studies in Education, 53(2), 119–136.

Cowie, B. (2005). Pupil commentary on assessment for learning. Curriculum Journal, 16(2), 137 -

151.

Darling-Hammond, L. Restoring our schools. The Nation.

http://www.srnleads.org/press/news/nation_restoring_our_schools.html Accessed 16 June, 2010.

Darling-Hammond, L. (2011). Teacher knowledge and student learning: making the connection.

Paper presented at the 2011 Vision to Reality: Queensland's new education landscape.

25

http://www.qsa.qld.edu.au/downloads/events/linda_darling_hammond.pdf Accessed on 18th

December, 2011.

De Bortoli, L. & Thomson. S. (2009). The achievement of Australia's Indigenous students in PISA

2000-2006. Melbourne: ACER Press.

DeLuca, C. & Klinger. D. (2010). Assessment literacy development: identifying gaps in teacher

candidates' learning. Assessment in Education: Principles, Policy & Practice, 17(4): 419 - 438.

Department of Education, Science and Training (DEST). (2007). What Works. The Work

Program: Core Issues 4: Numeracy. http://www.whatworks.edu.au Accessed on January 30,

2011.

Dewey, J. (1966). Democracy and education. An introduction to the philosophy of education.

New York: Free Press.

Elwood, J. (2006). Formative assessment: possibilities, boundaries and limitations. Assessment in

Education (2) 215 - 232.

Elwood, J. (2008). Gender issues in testing and assessment. In Murphy, P and Hall, K. eds.,

Learning and practice: agency and identities. London: Sage.

Elwood, J. & Lundy, L. (2010). Revisioning assessment through a children’s rights approach:

implications for policy, process and practice. Research Papers in Education, 25, (3): 335-353

Gamble, J. & Hoadley, U. (2004). Positioning the regulative order in Muller, J. Davies, B. &

Morais, A. Reading Bernstein, Researching Bernstein (pp. 61 – 74). London: RoutledgeFalmer

Gardner, J. (2009). AfL a practical guide. Belfast, Northern Ireland: Council for Curriculum

Examinations and Assessment.

Gee, J. (2003). Opportunity to learn: A language-based perspective on assessment. Assessment in

Education: Principles, Policy & Practice.10, (1) 27 - 46.

Gee, J. (2004). Situated language and learning. A critique of traditional schooling. New York:

Routledge.

Gee, J. (2008). A sociocultural perspective on opportunity to learn. In Moss, P., Pullin, D., Gee,

J.,Haertel, E.,& Jones Young, L. eds. Assessment, equity and opportunity to learn ,76 - 108.

Cambridge: Cambridge University Press.

Gilborn, D., & Youdell, D. (2000). Rationing education: policy, practice, reform and equity.

Milton Keynes: Open University Press,

26

Gipps, C. (1999). Socio-Cultural aspects of assessment. Review of Research in Education, 24: 355

- 392.

Gipps, C. (2002). Beyond Testing: Towards a theory of educational assessment. London:

RoutledgeFalmer

Gipps, C. (2008). Socio-cultural aspects of assessment. In H. Wynne (Ed.) Student assessment and

testing: Vol. 1 (252-291). Thousand Oaks, CA: Sage.

González, N., Moll, L., Floyd Tenery, M.,Rivera, A., Rendón, P., Gonzales, R.,&Amanti. C.

(2008). Funds of Knowledge for Teaching Latino Households, in Hall, K., P. Murphy and J. Soler,

Pedagogy and Practice Culture and Identities. Milton Keynes, UK: Open University.

Griffin, P., Woods, K. & Nguyen. C. (2005). An environmental scan of tools and strategies that

measure progress in school reform. Melbourne: Department of Education and Training.

Hipwell, P. & Klenowski. V. (2011). A case for addressing the literacy demands of assessment.

Journal of language and literacy. 34 (2) 127 – 146.

James, M. (2006). Assessment, teaching and theories of learning. In J. Gardner (Ed.) Assessment

and Learning. (47 – 60). London: SAGE publications.

Johnston, P. & Costello. P. (2005). Principles for literacy assessment. Reading Research Quarterly

40(2) 256 -267.

Kelly, A. (2003) Research as design. Educational Researcher 31(1) 3-4.

Klenowski, V. (2009). Australian Indigenous students: addressing equity issues in assessment.

Teaching Education, 20 (1) 77 - 93.

Klenowski, V & Adie, L. (2009). Moderation as judgment practice: Reconciling system level

accountability and local level practice. Curriculum Perspectives 29(1)10-28.

Klenowski, V. & Wyatt-Smith, C. (2011) The impact of high stakes testing: the Australian story.

Assessment in Education: Principles, Policy and Practice. 19 (1) 65 – 79.

Klenowski, V., Connolly, S. & Funnell, R. (in press, accepted December 2012) Examining the

assessment opportunities for cultural connectedness for student learning: A sociocultural analysis.

In Boyle, C. (Ed) Student Learning: Improving practice, New York: Nova Science Publishers Inc.

Learning and Teaching Scotland. (2010). Assessment for Curriculum for Excellence.

http://www.ltscotland.org.uk/Images/AssessmentforCfE_tcm4-565505.pdf Accessed on 30th July,

2011.

27

Lingard, B. (2007). Pedagogies of indifference. International Journal of Inclusive Education,

11(3) 245 – 266

Lingard, B. (2011). Policy as numbers: ac/counting for educational research. Australian

Educational Research. 38. 355 – 382.

Luke, A., Woods, A., Bahrand, M. & McFarland, M. (2004). Accountability: Inclusive Assessment,

Monitoring and Reporting. Research Report prepared for the Queensland Indigenous Education

Consultative Body. Queensland, Australia.

McTaggart, R. & Curro. G. (2009). Book Language as a Foreign Language: ESL strategies for

Indigenous Learners. Toowong: QCOT

Marshall, B., Drummond, M. (2006). How teachers engage with Assessment for Learning: lessons

from the classroom. Research Papers in Education, 21 (2), 133 – 149.

Maton, K. (2000). Recovering pedagogic discourse: A Bernsteinian approach to the sociology of

educational knowledge. Linguistics and Education. 11 (1) 79 – 98.

Moss, G. (2000). Informal literacies and pedagogic discourse. Linguistics and Education. 11 (1)

47 - 64

Moss, P. (2008). Sociocultural implications for assessment 1: Classroom assessment. In Moss, P.,

Pullin, D., Gee, J., Haertel, E & Jones Young, L. eds. Assessment, equity and opportunity to learn

222 - 294. Cambridge: Cambridge University Press.

Morgan, W. and C. M. Wyatt-Smith, (2000). Im/proper accountability: Towards a theory of

critical literacy and assessment. Assessment in Education: Principles, Policies and Practice, 7, 123

- 142.

Murphy, P. (2008). Gender and subject cultures in practice. In Murphy, P., and K. Hall, eds.

Learning and practice: agency and identities 161-172. London: Sage.

Murphy, P. & Whitelegg. E. (2006). Girls in the physics classroom: a review of the research on

the participation of girls in physics. London: Institute of Physics.

New London Group. (1996). A Pedagogy of multiliteracies: designing social futures. Harvard

Educational Review 66 (1) 60 - 92.

Nichols, S.L. & Berliner. D. (2007). Collateral damage: How high stakes testing corrupts

America’s schools. Cambridge, MA: Harvard Educational Press.

Popham, J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory into

Practice 48 (1) 4-11.

28

Pryor, J. & Crossouard. P. (2008). A socio-cultural theorisation of formative assessment. Oxford

Review of Education 34, no.1: 1 - 20.

Queensland Studies Authority. (2009). P - 12 Assessment Policy.

http://www.qsa.qld.edu.au/downloads/approach/qsa_assessment_policy.pdf Accessed on 20th

September 2011.

Rogoff, B. (1990). Apprenticeship in thinking. Cognitive development in social context. New

York: Oxford University Press.

Sadler, D. R. (1987). Specifying and promulgating achievement standards. Oxford Review of

Education 13 (2) 191-209.

Shepard, L. (2000). The role of assessment in a learning culture. Educational Researcher, 29 (7), 4

– 14.

Singh, P. (2002). Pedagogising Knowledge: Bernstein’s theory of the pedagogic device. British

Journal of Sociology of Education, 23 (4) 571 - 582

Stiggins, R. (1991). Assessment literacy. Phi Delta Kappan 72, no. 7: 534 - 539.

Stiggins, R. (1995). Assessment literacy for the 21st century. Phi Delta Kappan 77 (3) 238 - 246.

Stobart, G. (2008). Testing times: The uses and abuses of assessment. London: Routledge.

Sullivan, P., Tobias, S. & McDonough, A. (2006). Perhaps the decision of some students not to

engage in learning mathematics in school is deliberate. Education Studies in Mathematics. 62 (1)

81-99.

Tognolini, J. &Stanley, G. (2007). Standards-basedassessment: A tool and means to the

development of human capital and capacity building in education. Australian Journal of

Education, 51(2): 129-145.

Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge:

Cambridge University Press

Willis, J. (2010). Assessment for learning as a participative pedagogy. Assessment Matters 2: 65 -

84.

Willis, J. (2011). Affiliation, autonomy and Assessment for Learning. Assessment in Education:

principles, policy and practice. 18 (4) 399 – 415

Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991) To use their minds well: Investigating new

forms of student assessment. Review of Research in Education. 17, 31 - 74

29

Wyatt-Smith, C. & S. Gunn. (2009). Towards theorising assessment as critical inquiry. In Wyatt-

Smith, C. & J. Cumming eds. Educational Assessment in the 21st Century 83 - 102. London:

Springer.

Young, V.M. & Kim. D. H. (2010). Using assessments for instructional improvement: a literature

review. Education Policy Analysis Archives, 18,(19).

Yates, L., Collins. C. & O'Connor, K. eds. (2011). Australia's curriculum dilemmas. State cultures

and the big issues. Melbourne: Melbourne University Press.

Zammit, K. (2011). Connecting multiliteracies and engagement of students from low socio-

economic backgrounds: Using Bernstein’s pedagogic discourse as a bridge. Language and

Education, 25:3, p. 203 – 220.

Zepke, N., Leach, L., Brandon, J., Chapman, J., Neutze, G., Rawlins, P., et al. (2005). Standards-

based assessment in the senior secondary school: A research synthesis. Wellington, New Zealand:

New Zealand Qualifications Authority.