Classroom based assessment:

The teacher's perspective

Tim McNamaratfmcna@unimelb.edu.au

What are the most pressing issues in assessment that you face?

Agree on two in order of priority

Outline

What is assessment? Standards-based assessment and the

ESL teacher Example: ESL standards in Victoria Study of implementation of standards Study of classroom based assessment Impact of external tests (NAPLAN; NCLB)

What is assessment?(1) Assessment - internal

Assessment is a process of observing evidence from

performance and interpreting it so that we can reach

conclusions about what individuals know or can do and

in order to make decisions based on these conclusions

Assessment provides evidence only, and relies on interpretation

Assessment only gives us evidence, not certainty

We interpret the evidence - we draw inferences - we reach conclusions and make claims about students

e.g. ‘this student can follow a normal classroom lesson in maths’

We make decisions about students based on these inferences

e.g. ‘this student no longer needs ESL support’ These claims and decisions can be wrong - just

like with detectives or doctors

Validation: Checking on our inferences

When the claims and decisions we make about test takers matter a lot, we need to check that they are defensible

We need to provide reasoning that links the evidence (the score) to the claim

We need to think about competing explanations for the score - patterns in the score data, student accounts, evidence from judges can be used to investigate these issues

This investigation is known as validation Has parallel in the investigation of inferences in

legal trials

What is assessment? (2) Assessment - external

Standards are key methods for managing education systems – funding, student pathways, accountability

Format of standards

Standards are typically formulated as an ordered series of statements about levels of achievement or stages of development.

(There may be multiple sets of ordered statements for different aspects of language development)

Possible functions of standards Planning: to act as a series of objectives of goals for

teaching and learning; involve clear and specific statements of teaching aims

Professional understanding: to inform teachers about the typical progress of learning; more complex statements include contextual and interpretative information in order to help the teacher understand more fully the nature of the emergent ability in the learner

Accountability: to act as statements of learning outcomes for administrative purposes - tends to be dominant function

Structure of ESL standards, Victorian Essential Learning Standards (VELS)

STAGE BANDS (YEARS OF SCHOOLING)

A1 Lower primary (Years P to 2)

A2

B1 Upper primary (Years 3 to 6)

B2

B3

S1 Secondary (Years 7 to 10)

S2

S3

S4

BL Little literacy, middle/upper primary

SL Little literacy, secondary

ESL Companion to VELS

Relationship to the English standards

Using the English Standards will not be an accurate reflection of an ESL student’s learning, and can indicate that there are problems in the student’s learning progress, when in fact the student is actually making adequate progress learning English as their second or subsequent language.

As ESL students learn more English, their learning begins to correspond to the English Standards. If an assessment of an ESL student on the English Standards is within acceptable year level parameters, then it is time to start using the English Standards for assessing and reporting progress. If not, then the ESL Standards should continue to be used.

ESL Developmental Continuum

STAGE BL - Reading

BL beginning

(B0.1)

Students beginning to work towards the standard at BL are new to English and new to literacy. They may have little or no experience of formal learning and do not have literacy strategies already acquired through developing literacy in a first language. They will be able to draw on general learning skills and strategies that they have used to function effectively in their own communities, for example observing, memorising, classifying. They may be reluctant to participate in reading activities. They may recognise their own language in writing, if it has a written form, and may recognise that English print is different from their own language. 

BL progressing towards

(B0.2)

Students progressing towards the standard at BL show interest in print and recognise some environmental print including their name. They can recognise and name some letters. They watch and listen as texts are read aloud to them but may not join in. They rely on peer or teacher support to complete structured activities. They show an interest in books and focus on illustrations. They demonstrate reading-like behaviour such as holding a book, sitting and looking at a book, turning pages and looking at pictures. They are starting to build a bank of English words they recognise, such as environmental print. They read some short, familiar texts that have been well-introduced in class.

BL Standard

(B0.3)

At Stage BL, students read a wide range of familiar, short, simple, repetitive, fictional and everyday texts, and complete simple, structured activities based on them. They retell a simple familiar story, and sequence a simple familiar process with sentences and pictures. They show early understanding that texts are written for a variety of purposes. They show beginning understanding of the sound/symbol relationships of English. They read some familiar words and phrases in context, and recognise, can name, and know the sounds some common letters and letter groups usually make. They recognise that meaning is carried by intonation, and they listen for key words and for repetition of words and phrases in texts read aloud. They focus on illustrations and other non-print features when reading. They use word lists and personal dictionaries to assist them to read new words.

STAGE BL – Speaking and Listening

BL beginning

(B0.1)

Students beginning to work towards the standard at BL have very little or no oral English. They do not respond meaningfully to English. They will join in activities, watching and copying what other students do in the classroom but may not speak. They may spontaneously repeat words or phrases without understanding their meaning. They may not speak in the classroom except to same language peers. They may initially attempt to communicate with the teacher using their own language. They are likely to listen to extended texts in English with visual support.

BL progressing towards

(B0.2)

Students progressing towards the standard at BL are settling into situations where English is the dominant language. They begin to understand that communication with teachers and peers needs to be conducted in English. They begin to learn the very basic oral English needed to manage learning in an English-speaking classroom, where the teacher adapts spoken texts to assist the students. Through their first language experiences, they understand that different forms of language and levels of politeness are used in different situations and contexts. They begin to adapt their limited, emerging English language resources to respond to new communicative and functional demands. They recognise the importance of non-verbal communication. They begin to become familiar with patterns in the sounds, intonation, rhythm, grammar and meaning of English.

BL Standard

(B0.3)

At Stage BL, students communicate simply but effectively in familiar, basic social and classroom contexts, using simple formulaic and creative structures. They learn through English, well supported by context. They contribute relatively complex ideas through simple English, and use simple English to respond to the ideas of others. Students’ English is characterised by varying grammatical accuracy, a short ‘telegraphic’ structure, simple subject/verb/object construction and overgeneralisation of rules. They use common adjectives to describe or add emphasis. They use repetitive grammar patterns copied from stories, songs, rhymes or the media. Students’ pronunciation, stress and intonation are comprehensible, but carry elements of first language pronunciation. They use some basic communication strategies, asking for repetition, and questioning to check understanding, clarify or confirm. They use some basic strategies to initiate and sustain simple conversations in English, restating, repeating or re-pronouncing as appropriate. 

Indicators of progress – 4 aspects:

Texts and responses to texts focuses on producing and responding to oral English texts used for social interaction and in the school context across the curriculum.

Cultural conventions of language use focuses on understanding and using spoken English in a variety of contexts and identifying how different contexts affect the way spoken English is used and interpreted.

Linguistic structures and features focuses on control over the structures and features of spoken English.

Maintaining and negotiating communication focuses on the strategies students at this stage typically use to speak in and learn English.

Teaching strategies – Speaking and Listening Listening cloze Reaching a consensus Barrier games Dictogloss Running dictation Pair dictation Language experience Role play Picture stories Surveys Tell me

Teaching strategies – Writing

Scaffolding writing using the curriculum cycle

Language experience Dictogloss Pair dictation Running dictation Deconstruction Joint construction

How will evidence be collected to know

what standard a student is at? Using teachers as the agents: Formal assessment

Teacher develops own instruments (less reliable) Teacher draws on task bank developed by language testers

(more reliable) Informal assessment

Teacher uses naturally occurring assessment opportunities to form picture of child’s standard

Teacher reports standard of each studentsUsing external assessment: tests developed by language testers (more reliable)

The assessment pyramidLEVELS

(NUMBERED)

LEVEL SUMMARIES

DIMENSION DESCRIPTIONS

WITHIN EACH DIMENSION, EXAMPLES PROVIDED

[ADVICE TO TEACHERS] DETAILED EXAMPLES

TEACHER CHOOSES ACTIVITY & CRITERIA

Competing demands in standards-based assessment

Validity demands Managerialist demands

Teacher/

learner demands Intellectual defensibility of construct Evidence of Reliability Other validity evidence Concern for consequences

Reporting Accountability

Meaningfulness in instructional process Facilitation of learning Enhanced quality of teaching Minimization of administrative burden on teachers

Study of use of ESL assessment frameworks by teachers in Victoria

Macqueen, S., L. Harding & C. Elder (2011) Investigating the validity of the ESL VELS student data. Report to DEECD.

Interview study, 21 representatives from 9 schools All participants were teachers in Victorian primary or secondary

schools or language centres with responsibility for assessing English as a Second Language (ESL) students.

The teachers ranged from fairly new graduates to very experienced teachers, including teachers with support, co-ordination and management roles.

Schools from different geographical regions were selected

Hill (2010) study of assessment cultures

Compared assessment cultures of 2 language classrooms in Melbourne:

Yr 6 (last year of primary) Yr 7 (first year of secondary)

as part of ‘transition’ study

Year-long ethnographic study of two experienced teachers of Indonesian

Classroom based assessment - definition

Any reflection by teachers (and/or learners) on the qualities of a learner’s (or group of learners’) work, and the use of that information by teachers (and/or learners) for teaching, learning (feedback), reporting, management or socialization purposes.

‘Assessment opportunity’

Any action, interaction or artifact (planned or unplanned, deliberate or unconscious, explicit or embedded) which has the potential to provide information on the qualities of a learner’s (or group of learners’) performance.

1. WHAT DO LANGUAGE TEACHERS DO?

1.1 Planning Assessment

Is there planning for assessment? How detailed is planning? What is its intended relationship to instruction? How does it relate to external standards and frameworks?

1.2 Framing Assessment

Is assessment made explicit to learners? How is this done?

1.3 Conducting Assessment

What opportunities does the classroom provide for assessment? Does assessment tend to focus on the class, group/pairs of students or individuals?

1.4 Using Assessment Data

How is assessment-related information used?

Teaching

Learning (feedback)

Person-referenced

Task-referenced Confirmatory

Explanatory

CorrectiveReporting

Management

Socialization

2. WHAT DO TEACHERS LOOK FOR?

What information about valued enterprises, qualities and standards is available

2.1 In Advance in written/ verbal instructions and/or assessment rubrics?

2.2 In Feedback in written and/or verbal feedback?

2.3 In Reporting in reporting deliberations and/or in written reports?

3. WHAT THEORY OR 'STANDARDS' DO THEY USE?

3. Teacher Theories & Beliefs What does the data reveal about teachers’ beliefs about

the subject or content areasecond language learning and teachingthe nature of assessment?

4. DO LEARNERS SHARE THE SAME UNDERSTANDINGS?

4. Learner Theories & Beliefs What does the data reveal about learners’ beliefs about

the subject or content areasecond language learning and teachingthe nature of assessment?

NAPLAN – ACTA submission We submit that NAPLAN assessments currently do not

permit the reporting of quality information about EAL/D students’ progress or school performance to parents, principals, school communities and systems because:

1. the test data are not disaggregated to reveal EAL/D learners’ performance

2. the tests are inaccurate, inconsistent and lack validity for EAL/D learners

3. the focus on English literacy fails to provide a basis for innovation and quality teaching of EAL/D learners, and is narrowing and distorting the teaching EAL/D learners receive.

We recognise that accurate, quality data is essential to implementing National Partnership agreements and the ongoing development of effective policies and programs.

1. Without data on how EAL/D learners are performing on NAPLAN tests, it is difficult to sustain claims for provision that is appropriate to their educational needs and, at the national level, impossible to monitor the effectiveness of any such provision.

2. Failure to identify these learners within the NAPLAN reporting regime is having adverse flow-on effects for their educational experiences and outcomes.

Identifying EAL/D learning needs requires reputable, specific-purpose EAL/D assessment tools and frameworks that are appropriate for use in Australian schools. EAL/D assessment frameworks now exist in all states, territories and systems, and can be used with confidence as a basis for the immediate disaggregation of EAL/D learners’ NAPLAN results. In fact, Australia has led the world in developing assessment methodologies for EAL/D learners in schools, and some have been used as models internationally.

US: No Child Left Behind

Strict, controversial measures of managing educational system through tests

EAL/D achievement measured separately Accommodations allowed in content area

assessments translation dictionaries longer time

New EAL/D frameworks for NCLB (Bailey & Huang 2011; Llosa 2011)

WIDA Consortium: separate strands for English in relation to

content areas: language arts mathematics science social studies