1

Developing Expertise Developing Expertise with Classroom with Classroom

Assessment in K-12 Assessment in K-12 ScienceScience

Maryl Gearhart, Sam Nagashima, Jennifer Pfotenhauer, Cheryl Schwab, Craig Strang

Center for the Assessment and Evaluation of Student Learning

(CAESL)University of California, Berkeley

University of California, Los AngelesLawrence Hall of Science

2

Developing Expertise with Developing Expertise with Classroom Assessment in K-12 Classroom Assessment in K-12

ScienceScience

WestEdDiane Carnahan

Karen Cerwin

Kathy DiRanna

Jo Topps

U.C. BerkeleyMaryl Gearhart

Jennifer Pfotenhauer

Cheryl Schwab

UCLAJoan Herman

Sam Nagashima

Ellen Osmundson

Lawrence Hall of ScienceLynn Barakos

Craig Strang

3

Overview of SessionOverview of SessionPresentations

• Project purpose and contextProject purpose and context• Framework for quality classroom assessmentFramework for quality classroom assessment• PD model - the assessment portfolioPD model - the assessment portfolio• Research design - focus on one research toolResearch design - focus on one research tool

Small groups• Sample assessment portfoliosSample assessment portfolios• Research tool: Practice Profiles Research tool: Practice Profiles

Reconvene• Revisions of assessment portfolioRevisions of assessment portfolio• Preliminary research findingsPreliminary research findings• PD model - assessment leadershipPD model - assessment leadership• Q & AQ & A

4

Purposes of R & DPurposes of R & D

Classroom assessment expertise

PD strategy - assessment portfolio

Research tools

Framework to guide PD and research

New forms of collaboration

5

BackgroundBackground

Psychometric standards

• Student Evaluation StandardsStudent Evaluation Standards (2002) (2002)

• Standards for Educational and Standards for Educational and Psychological Testing (1999)Psychological Testing (1999)

• Buros (1990)Buros (1990)

6

Standards for the teaching profession

• National Board for Professional Teaching National Board for Professional Teaching Standards (NBPTS)Standards (NBPTS)

• California Standards for the Teaching California Standards for the Teaching Profession (CSTP)Profession (CSTP)

• Performance Assessment for California Performance Assessment for California Teachers (PACT)Teachers (PACT)

7

Conceptions of Classroom practice

• General models: General models: Stiggins….Stiggins….

• Assessment that supports learning:Assessment that supports learning: Black & Wiliam, Shepard, Wiggins …. Black & Wiliam, Shepard, Wiggins ….

• Cognitive models:Cognitive models: Knowing What Knowing What Students KnowStudents Know …. ….

• Multi-leveled assessment systems:Multi-leveled assessment systems: Shepard, WilsonShepard, Wilson

8

Science Education Resources

• AAAS, AAAS, Benchmarks for Science Benchmarks for Science LiteracyLiteracy

• National Science Education National Science Education StandardsStandards

• NRC, NRC, Classroom AssessmentClassroom Assessment

9

CAESL MissionCAESL Mission

Center for Assessment and Evaluation of Student Learning (CAESL)

• strengthening K-12 science strengthening K-12 science assessment at all levels of the assessment at all levels of the systemsystem

10

               

        

QUALITY GOALS FOR STUDENT LEARNING AND

PROGRESS

QUALITY USE

QUALITY TOOLS

CAESL Quality Classroom CAESL Quality Classroom AssessmentAssessment

QUALITY GOALS FOR STUDENT LEARNING AND

PROGRESS

QUALITY USE

QUALITY TOOLS

Valid for Purpose

•Developmentally Sound Content

•Clear Expectations

•Reliable & Accurate

•Fair & Unbiased

Alignm

ent

Alig

nm

ent

Alignment

Support Student Learning

•Guide for Instruction

•Instructional Materials Refinements

•Assessment Refinements

•Student Involvement and Accountability

•Timely Feedback

•Equitable

Sound Interpretation

CAESL Quality Classroom CAESL Quality Classroom AssessmentAssessment

• Quality Criteria

• Quality Application

• Quality Analysis

CLASSROOM ASSESSMENT SYSTEM

• Longitudinal to track student progress

• Multileveled

•Coordinated and strategic

•Feasible

CAESL Quality Classroom CAESL Quality Classroom AssessmentAssessment

14

Assessment PortfolioAssessment PortfolioPurpose

• design, implement, evaluate an design, implement, evaluate an assessment plan for a curriculum assessment plan for a curriculum unitunit

Support for• individual learning and reflectionindividual learning and reflection• coachingcoaching• collaborationcollaboration• leadershipleadership

Team organization• cross site unit teamscross site unit teams• K-12 district teamsK-12 district teams• local coachinglocal coaching

15

Portfolio sectionsPortfolio sections

Section I: The plan

Section II: Implementation

Section III: Evaluation

16

I: The planI: The plan

Conceptual Flow

• Establish learning goals Establish learning goals

• Context for assessment Context for assessment planning planning

• Deepen knowledgeDeepen knowledge

17

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

18

Building (Identifying) the Conceptual Flow

Unit OrganizingConcept

Unit topic:Unit concept :

S ubc oncep t

Gr ade Level Concept

Lesso n Concept :

Lesso n A ct ivit y:

19

20

I: The planI: The plan

Record of Assessments in Instructional Materials (RAIM)

• Concepts assessed Concepts assessed

• Expected Student Responses (ESRs)Expected Student Responses (ESRs)

• Any need for assessment refinementAny need for assessment refinement

• Key assessments to track progressKey assessments to track progress

21

Section I A 2: Record of Assessment in Instructional Materials (RAIM2)How are assessments organized and used in the unit of instruction to make sense of and understand student learning?

Concept Assessed Type ofAssessment

Page # Use orRefine

Expected Student Response

HIGH•What kinds of understandings do you

expect from students?

Expected Student Response

MEDI UM and LOW•What kinds of understandings do

you expect fr om students?Note possible alternative conceptions

Pre• The structures of plants a re

roots, stems, leaves, andflowers

Observati onsDrawing

Not intext book

New Parts should be lab eled correctly.Roots in dirt. Drawings are in color,detail ed, and looks like plants.Complete sentences about the plantthat describe color, leaves, names,etc.

Assessment #2• The structures of plants are

roots, stems, and leaves.• Plants need water, sun, air,

and something to grow in

booklet Not intext book

NewPg 149inCompleteBookofScience

All parts of a plant are labeledcorrectly and all jobs of plants a recorrectly l abeled.

All parts of a plant are labeledcorrectly . Some of the jobs arecorrectly l abeled.

Assessment #3• Plants can change

depending on theenvironment. They havecharacteristics that are thesame and different a nd growin different environments.

22

II: ImplementationII: Implementation

Selected assessments

Documentation of:• Concepts assessedConcepts assessed• Expected Student Responses (ESR)Expected Student Responses (ESR)• Analysis of student workAnalysis of student work• Instructional follow-up and feedbackInstructional follow-up and feedback

23

III: EvaluationIII: Evaluation

Prompts to guide review and revision

Revised tasks

Revised criteria

24

Research QuestionsResearch Questions

How does assessment expertise develop?

How does the portfolio support the development of assessment expertise?

25

Design and Data SourcesDesign and Data Sources

• Longitudinal

• Nested design All participants:All participants:

SurveysSurveys Assessment Assessment

portfoliosportfolios

Cases:Cases: ObservationsObservations InterviewsInterviews

26

Research tool: Research tool: Practice ProfilesPractice Profiles

Purpose: Capture development of assessment expertise over time

Unit of Analysis: Teacher practice regarding one assessment

27

Designing the Profiles Designing the Profiles • Audience

• Comprehensiveness

• Detail

• Assumptions

28

Draft Profile LevelsDraft Profile Levels

• Draft levels of emerging assessment expertise

Inexperienced -> Inexperienced -> Developing ->Developing ->Maturing -> Maturing -> ExpertExpert

QUALITY GOALS FOR STUDENT LEARNING AND

PROGRESS

QUALITY USE

QUALITY TOOLS

Valid for Purpose

•Developmentally Sound Content

•Clear Expectations

•Reliable & Accurate

•Fair & Unbiased

Alignm

ent

Alig

nm

ent

Alignment

Support Student Learning

•Guide for Instruction

•Instructional Materials Refinements

•Assessment Refinements

•Student Involvement and Accountability

•Timely Feedback

•Equitable

Sound Interpretation

CAESL Quality Classroom CAESL Quality Classroom AssessmentAssessment

• Quality Criteria

• Quality Application

• Quality Analysis

30

Sound InterpretationSound Interpretation

• Draft components

Quality criteriaQuality criteria

Quality application of Quality application of criteriacriteria

Quality analysisQuality analysis

31

Small groupsSmall groups

• Review sample portfolio

• Review draft profile for Sound Interpretation

32

Portfolio RevisionsPortfolio Revisions

I: Greater focus and integrationRAIM fewer assessmentsRAIM fewer assessmentsIntegrated CF & RAIMIntegrated CF & RAIM

TradeoffComprehensive <- -> FocusedComprehensive <- -> Focused

33

Plants have structures that enable them to live and grow.

Thestructures ofplants areroots, stemsand leaves.

Seeds help the plantreproduce.Roots provide support andcarry the water andnutrients into the stem.Stems provide the supportand carry the water andnutrients. Leaves makethe food for the plantthrough photosynthesis.

Plants needwater, sun, airand somethingto grow in(water, soil)

How do the plantstructureshelp them live?

What do these plants needto live?

Chapter 1, Lesson 2,A8-A11Supplement

What happens to a plantwhen it does not get it’sneeds met?

Plants can changedepending on theenvironment.

Chapter 1, Lesson2, A12-A13

How are plants alikeand different?

Plants have similarand differentcharacteristics.

Plants grow indifferentenvironments.

What kind ofenvironments doplants grow in?

Plants grow in thedesert, themountains, inhomes, in thevalley, etc.

Chapter 1,Lesson 3,A15-A17.

Chapter 1, lesson 1SupplementaryMaterialsPlants (Evan-Moor)

Chapter 1, lesson 3SupplementaryMaterials.

Carnation andfood coloringPlants pg. 23

Magic SchoolBus- Going toSeed

Living/non-living pg. A4

Write the needs of plants tolive and grow. Write aboutthe job of each part of aplant. Label the parts of aplant.

Plant parts havefunction Plantspgs. 16-29

Observe 4differentplants. Drawand write aboutyourobservations ofeach of theplants. Labelthe parts of theplants.

Sort items into living andnon-living. Showunderstanding of sensesused in science,vocabulary of differentenvironments, and whatliving things need and do.

Discuss seeddispersal - Plantspg. 44-49

EnvironmentsPocket Chart Science:Animals and Habitats

Nature Walk Parts of plantsPocket ChartScience: Plants

Compare/Contrastdifferent seeds

Plants needsPlants pgs.59-64

34

Portfolio RevisionsPortfolio Revisions

II: Scaffolding the analysisPreliminary criteria for H M LPreliminary criteria for H M LCriteria revision using student workCriteria revision using student workModels of whole class analysisModels of whole class analysisModels of target student analysisModels of target student analysis

TradeoffResources <- -> PrescriptionsResources <- -> Prescriptions

35

Research QuestionsResearch Questions

How does assessment expertise develop?

How does the portfolio support the development of assessment expertise?

36

Findings from ‘03-’04 Findings from ‘03-’04 surveysurvey

Quality of assessment tool

Assessment use

37

Teachers feel most comfortable with:

• Using assessment results as evidence of student progress to design developmentally appropriate instruction

• Evaluating the alignment of assessments with learning goals

Teachers feel less comfortable with:

• Evaluating scientific soundness of assessment content

• Judging whether assessments capture full range of alternative conceptions

38

Leadership ContextLeadership Context

• Districts committed to K-12 science

• Districts committed to reform/inquiry-based science

39

Challenges of Challenges of LeadershipLeadership

• Various, developing levels of assessment expertise

• Various, developing levels of leadership expertise

• Science not the priority, erosion of reform

• Quality assessment of reform science really not the priority

40

Goals of LeadershipGoals of Leadership

• Growth & improvement of individual practice in instruction & assessment

• Leading/guiding individual growth in others

• Influencing systemic change in districts: developing/implementing quality assessment systems

41

Dimensions of Dimensions of LeadershipLeadership

(Program Elements (Program Elements Matrix)Matrix)

• Professional Development• District Policy• Parent/Community

Involvement/Support• Develop/Implement Quality

Assessment System• Leadership Development &

Capacity Building

42

Leadership ActivitiesLeadership Activities• CF & RAIM with Reform Unit with a

Buddy (or a few)• Participate in FOSS ASK Project• Serve on and influence district

committees• Presentations to school board,

district administrators, parent groups

• Develop benchmark assessments(!?)