Assessment and accountability: From teachable moments to tests Peter Afflerbach University of...
-
Upload
trinity-schroeder -
Category
Documents
-
view
216 -
download
2
Transcript of Assessment and accountability: From teachable moments to tests Peter Afflerbach University of...
Assessment and accountability: Assessment and accountability:
From teachable moments to From teachable moments to teststests
Peter AfflerbachPeter Afflerbach
University of MarylandUniversity of Maryland
[email protected]@umd.edu
Reading Research ConferenceReading Research Conference
International Reading AssociationInternational Reading Association
Atlanta, GAAtlanta, GA
May 3, 2008May 3, 2008
Some past history…and assessment
Public School 33 Queens
Junior High School 109 Queens
Jamaica High School, Jamaica, Queens
1974 Betty Crocker “Leader of Tomorrow” Award
(Afflerbach, 2007; Black & Wiliam, 1998; Calfee & Hiebert, 1991; Crooks, 1988; Davis, 1998; Pellegrino, Chudowsky & Glaser, 2001)
The state of research on assessment
Assessment as the dependent variable in much reading research
Assessment itself is much less likely to be the focus of research
Psychometric inquiry related to validity and reliability
Questions about most useful, most effective,
most efficient assessment need our attention
Assessment in the nexus
Politics
Economics
Agendas
READING ASSESSMENT
Student growth and achievement
Teacher professional development
High quality teaching
Zone of proximal development
Curriculum materials
Framing questions and an observation
Where is our assessment focus?
Do we attend to one type of assessmentat the expense of others?
Is our assessment focus one that may actually
impede student achievement and limit teacher effectiveness?
Is our assessment one that supports students and teachers in their critical, daily work?
How might a balance of assessment, focused on both teachable moments and tests,
help us reach our goals?
Accountability is measured by a single test
on a single day, butaccountability is accomplishedwith daily, useful assessments
that inform our best instruction.
Accountability for teaching and learning Accountability for teaching and learning will be realized when there is balance will be realized when there is balance
in literacy assessment, in literacy assessment, including balance in:including balance in:
• • meeting the needs of different meeting the needs of different audiences and purposes of assessmentaudiences and purposes of assessment
• • formative assessment and summative formative assessment and summative assessmentassessment
• • the assessment that is donethe assessment that is done to to or or forfor students and assessment that is done students and assessment that is done withwith and and byby students students
• • the assessment of what students learn the assessment of what students learn and how students use this knowledgeand how students use this knowledge
• • the assessment of cognitive and the assessment of cognitive and affective factors related to literacyaffective factors related to literacy
……the need for balance in the need for balance in literacy assessment, literacy assessment, including balance in:including balance in:
• • the demands for teacher and school the demands for teacher and school accountability and professional development accountability and professional development opportunities that help teachers develop opportunities that help teachers develop expertise in assessmentexpertise in assessment
The The contextscontexts in which we conduct in which we conduct reading assessment are influenced by (and reading assessment are influenced by (and reflect) political, economic and social reflect) political, economic and social
factorsfactors
• The prevalence of high stakes testing, NCLB or no NCLB
In 2008…In 2008…
• Traditions and habits of assessment
• The tests that our children take today look much like those that we took when we were in school
• How does our current understanding of literacy influence our assessments?
• How does our current understanding of assessment influence our assessments?
‘‘The Report Card’The Report Card’
What do you think Johnny did next?What do you think Johnny did next? 1. hid his report card1. hid his report card 2. sneaked away2. sneaked away 3. scolded his mother3. scolded his mother 4. showed his report card to his parents4. showed his report card to his parents
(Nelson Denny Reading Test, 1966)(Nelson Denny Reading Test, 1966)
‘‘Saved by a Fly’Saved by a Fly’
What is the main idea of the story?What is the main idea of the story? 1. A moose drinks all the water from a river1. A moose drinks all the water from a river 2. A group of animals is afraid of a big moose2. A group of animals is afraid of a big moose 3. A fly gets rid of a troublesome moose3. A fly gets rid of a troublesome moose 4. A group of animals is always quarreling4. A group of animals is always quarreling
(Maryland State Assessment, 2006)(Maryland State Assessment, 2006)
Necessary Necessary alignmentsalignments for for useful reading useful reading assessments:assessments:
The construct of readingThe construct of reading
Reading standards and Reading standards and
benchmarksbenchmarks
Reading curriculum and instructionReading curriculum and instruction
Reading assessmentReading assessment
1. Balance in meeting the needs of different 1. Balance in meeting the needs of different
audiences and purposes of assessmentaudiences and purposes of assessmentAudience Purpose
Students To report on learning and communicate progress To motivate and encourage
To teach children about assessment and how to assess their own work and
progress To build student independence
Teachers To determine nature of student learning To inform instruction
To evaluate students and construct grades
To diagnose student strengths and weaknesses
Afflerbach, 2007
Taxpayers To demonstrate that tax dollars are well spent
Afflerbach, 2007
Politicians To establish accountability of schools
To inform the public of school progress
School To determine instructional program administrators effectiveness
To prove school and teacher accountability
Audience Purpose
Parents To inform of their children’s achievement
To help connect home and school efforts to support student
1. Balance in meeting the needs of different 1. Balance in meeting the needs of different
audiences and purposes of assessment (cont.)audiences and purposes of assessment (cont.)
2. Balance in formative assessment and 2. Balance in formative assessment and summative assessmentsummative assessment
Formative assessment provides information for:
• Understanding individual students and their immediate needs
• The teachable moment
• Creating scaffolded approaches to instruction
(Black & Wiliam, 1998; Crooks, 1988)
Summative assessment provides information for:
• Comparing students’ literacy products
• Judging students’ achievement in relation to benchmarks, standards and each other
(National Reading Panel, 2000; Pellegrino et al, 2001)
In a thoughtful assessment system, formative and summative assessment work
together…
Formative
-ball control
-kicking
-passing
-vision
-resilience
-position
-speed
-team play
-creativity
Summative
-score at end of game
-team standing at end of season
-personalimprovementfrom season to season
Zone of proximal developmentZone of proximal development, , assessmentassessment and and teachable momentsteachable moments
Student's next level of competency and achievement
Assessment and teaching
Zone of proximal development
Teaching and assessment
Student's current level of competency and achievement
(Vygotsky, 1978; Palincsar & Brown, 1984; Rosenshine & Meister, 1994; Palincsar &
Herrenkohl, 2002)
Student's next level of competency and achievement
Where our best teaching happens
Assessment and teaching
Zone of proximal development
Teaching and assessment
Where accountability is created
Student's current level of competency and achievement
A bit more on this…
Formative assessment is used for:
Understanding individual studentsand their immediate needs
Informing the teachable moment
Creating scaffolded approaches to reading instruction
Formative assessment done well leads to good news on summative assessments
Is the daily assessment that identifies individual student needs and allows us to
address them high stakes assessment ?
Important balance questions:
Who is best served by the current array of assessments?
What percentage of assessment across the school year
is formative? Summative?
Is this a good thing?
A test score tells us little or nothing of the means by which it was achieved.
3. Balance the assessment that is done3. Balance the assessment that is done toto or or forfor students students
and assessment that is done and assessment that is done withwith and and byby students students What does assessment mean to our
students?
How do students conceptualize assessment?
If a universal goal of reading instruction is to help
students become independent and successful, are we doing enough
to foster independence?
How can assessment help foster this independence?
“The black box”
My work in my work out = The grade
The need to make transparent and tangible
what assessment “is”…
Independent and successful readers continually monitor and evaluate their learning and
performance
Metacognition and comprehension monitoring are learned…
Who teaches them? How are they taught?
(Black & Wiliam, 1998; Flavell, 1978; Markman, 1979;
Pressley & Afflerbach, 1995)
Do classroom assessments help students become
independent in assessment?
Are students “insiders” or “outsiders” to the culture of assessment?
Is assessment done to or for students?Is assessment done with and by
students?
Without the ability to self-assess reading
(or any other learning), how will students be independent?
It is legitimate to further characterize the broadpoint appearance as a major archeological horizon marker for the eastern seaboard. In the terms of Willey and Phillips, a horizon is “a primarily spatial continuity represented by cultural traits and assemblages whose nature and mode of occurrence permit the assumption of a broad and rapid spread.” That a quick expansion of the broadpoint-using peoples took place is indicated by the narrow range of available radiocarbon dates, along with a correspondingly wide areal distribution of components. Once established, the broadpoint horizon developed as a “whole cultural pattern or tradition” in its own right by persisting and evolving over an expansive region for 500 to 1000 years.
Broadpoint is a type of arrowhead
How are we metacognitive?
Self-assessment happens when students have metacognitive strategies to use, the mindset that they should use them,
and motivation to do so.
Self-assessment is strategic, so our successes with strategy instruction can inform our teaching of self-assessment strategies.
Explain, model, think-aloud and practice.
With reading…
(Pressley, 2005; Duffy, 1993)
4. Balance the assessment of 4. Balance the assessment of cognitivecognitive and and
affectiveaffective factors related to learning factors related to learning Representative State Level Intended Learning Outcomes
Beginning in kindergarten and by the end of second grade students will be able to:
1. Demonstrate a positive learning attitude.
2. Develop social skills and ethical responsibility.
3. Demonstrate responsible emotional and cognitive behaviors.
4. Understand and use basic concepts and skills.
The assessment of cognitive and affective factors
that contribute to and influence achievement
Most state and local standards address affective goals and outcomes in their
curriculum goals and standards and learning benchmark descriptions
Self-concept as a reader
Are these assessed? To what end?
(Chapman & Tunmer, 1995; Guthrie & Wigfield, 1997; CORI Projects; McKenna & Kear, 1990)
Self-esteem as a reader
Motivation to read
When we think of our teaching successes, do we think of students who
scored well on tests?
Or do we think of students who went from reluctant readers to enthusiastic
readers?
Do we think of students who evolved from easily discouraged readers to readers whose motivation helped them
persevere through challenges?
Do we remember students who avoided reading at all costs evolving into
students who learned to love reading?
5. Balance the assessment of 5. Balance the assessment of what students learn from readingwhat students learn from reading and and how students use this knowledgehow students use this knowledge
The 2009 Framework of the National Assessment of Educational Progress derives from expert consensus and conceptualizes reading as:
Reading is an active and complex process that involves• Understanding written text• Developing and interpreting meaning • Using meaning as appropriate to type of text, purpose and situation
(National Center for Education Statistics, 2007)
“Using meaning as appropriate to type of
text, purpose and situation…”
In the 21st century, can our students
•Identify claims and evidence in texts they read?
•Conduct critical evaluations of texts?
•Make judgments of texts and authors?
•Apply what they learn from reading to identify and solve problems?
•Synthesize information from text and non-text sources?
(Coiro & Dobler, 2007; Gee, 2000)
• Meaning construction andcritical evaluation is assessed with …
- Multiple choice items
- Brief constructed responses
- Extended constructed responses
- Performances and judgments
Increasing
complex
performance
and
assessment
(Pellegrino, Chudowsky & Glaser, 2001)
The application of what is learned from reading
Dolgeville, NY
This Factory's Bats Are Going, Going, Gone; As Home of McGwire's 'Big Stick,' Struggling Upstate Town Gets a Lift
And that is the best economic news in a long, long time for Dolgeville, tucked in the hills north of the Mohawk River and 23 miles east of Utica. The century-old clapboard houses and the flower beds here are well tended, but behind the trim exterior, the village is fading, its population and job base declining steadily, like so many places upstate that have lost the manufacturing that once fed them.
Jamestown lost its furniture makers and the Crescent wrench works, Troy lost the Arrow shirt collar factory, and Schenectady lost the American Locomotive Company and most of General Electric. Dolgeville's soon-to-be relic is the Daniel Green shoe factory, a great stone castle along East Canada Creek that once employed nearly 1,000 people, but has seen a series of layoffs. Most of the jobs have moved overseas, and the factory will cease production in June.
RICHARD PEREZ-PENA NY Times April 25, 1999
6. Balance the demands for accountability 6. Balance the demands for accountability with with
professional development opportunities professional development opportunities
that help teachers develop that help teachers develop expertise in assessmentexpertise in assessment
How do teachers become expert at assessment?
How can teachers attain reliability in their
classroom-based assessments?
Opportunity costs of different approaches
to reading assessment
(Darling-Hammond, 1997; Stiggins & Conklin, 1992)
How are teachers supported
in the extremely challenging work of becoming expert
at classroom assessment?
Professional development in classroom assessment is central to school success. For example, a teacher’s checklist for asking appropriate questions can be used to sharpen daily assessment routines:
_____ I ask questions that are appropriately phrased and understood by students._____ I ask questions that are at an appropriate level for the materials being covered. _____ I ask questions that require students to think at various intellectual levels._____ My questions follow a logical sequence._____ Student responses are used to guide my next questions._____ My questions are consistent with the intended goals or objectives of the lesson._____ I ask questions that assess student understanding. _____ I ask processing questions if a student’s answer is incomplete or superficial. _____ I encourage students to answer difficult questions by providing cues or rephrasing._____ I avoid closed-ended questions that restrict students’ demonstration of learning.
Slack, J. Questioning strategies to improve student thinking and comprehension. Southeast Educational Development Laboratory. Reprinted with permission.
Towards expertise in classroom assessment
Vertical and horizontal reading assessment
In for the long term
Avoiding faddism
Opportunity costs
Connect teachable momentswith tests
(Afflerbach, 2007; Johnston, 1991)
ConclusionsConclusions
Effective reading assessment is conceptualized and practiced with clear understanding of
how teachable moments connect with tests
The extreme focus on high stakes tests, AYP and means to meet it skews school assessment towards summary statements, and away from
valuable formative assessment
ConclusionsConclusions
Such formative assessment is that which informs our teachable moments—
Providing information that describesstudents’ zones of proximal development--and where and what and how our instruction
should address students’ needs
Reading assessment is influenced by powers great and small--and this often creates imbalances for teachers and students
ConclusionsConclusions
Our addressing imbalances should be informed by our goals
for (and with) reading assessment
To help us understand student learning and
to help us guide student learning
ConclusionsConclusions
This duality of reading assessment allows us to promote:
-Strategic, skillful reading
-Lifelong, enthusiastic readers
-The Matthew effect
Cognitive and affective and social development
Independence and self-assessment
Lastly…
Achieving balance in our reading assessment programs, materials and procedures promotes accountability
from teachable moment to test