GLCE and YOU! Focus on Assessment January 12 and 13, 2005.

GLCE and YOU!Focus on Assessment

January 12 and 13, 2005

GLCE and YOU-Focus on Assessment 2

Assessment Crisis

Assessmentof

Learning

Assessmentfor

Learning


Crucial Distinction

Assessment OF Learning:How much have students learned as of a particular point in time?

Assessment FOR Learning:

How can we use assessment to help students learn more?


Key Questions

Are our current approaches to assessment

improving student learning? helping teachers improve

their craft?


A comparative view

Assessment Focus:

OF Learning State standards/GLCE

FOR Learning Achievement targets that underpin standards/GLCE


Assessment FOR learning

Provides timely information that helps teachers teach and students learn

Is currently the weak link in our quest for continuous improvement and our effort to “leave no child behind”


When Assessing for learning:Teachers… understand and articulate achievement

targets before teaching. inform students of targets in student

friendly terms. create assessments that accurately reflect

student achievement. use assessments positively to build student

confidence in themselves as learners provide frequent, descriptive feedback to

students to help them improve. continuously adjust instruction based on

classroom assessment results.


What we really need to do…

Plan carefully for assessment use.

Understand what information we need from assessment.

Determine if an assessment can provide the information required for the intended purpose.


Research demonstrates clearly… Strengthening formative

assessment practice results in significant, substantial learning gains.

Improved formative assessment helps the lowest achievers most.

The common feature of studies showing large effect sizes accurate, high quality formative assessments.


1. Does better formative assessment lead to higher test scores?

2. Does formative assessment need improving?

3. If so, what improvement is needed?

Black & William Questions:Black & William Questions:


What They Found Yes - better formative

assessment did lead to higher test scores.

Yes – the development and use of formative assessments greatly need improvement.

Accuracy, descriptive feedback, student involvement are areas most in need of improvement.

GLCE and YOU-Focus on Assessment

12

Research - Effects of Major Initiatives

.5 to 1.8**Rodriguez (2004)

.7 to 1.5Meisels, et al. (2003)

S.D. Gains Study

1.0 to 2.0 *Bloom (1984)

* Rivals one-on-one tutorial instruction

.5 to 1.0**Black and Wiliam (1998)

** Largest gains for low achievers


1.0 Standard Deviation equals: 30+ Percentile Points On ITBS

(middle of score range) 3-4 Grade Equivalents 100 SAT Score Points 5-6 ACT Score Points U.S. TIMMS Rank From 23rd to Top

5 Potential Elimination of Score GapsUnprecedented Achievement Gains


Largest effects when teachers…

increased the accuracy of increased the accuracy of formative assessmentsformative assessments

enhanced the quality of enhanced the quality of formative assessmentsformative assessments

increased descriptive feedbackincreased descriptive feedback increased student involvementincreased student involvement


Think-Pair-Share

Think of a time when you’ve used formative

assessment really well in your classroom.

How did this effect your teaching?

How did this impact student learning?

16

Standards Of Assessment Quality

Standard 1Assess what? Are targets clear

and appropriate?

Standard 2Why assess?

Are users and uses clear?

Standard 3Assess how? Is the assessment method

appropriate?

Standard 4How much evidence? Is achievement

adequately sampled?

Standard 5Assess accurately? Has bias been

minimized?


Standard 1: Clear and Appropriate Learning Targets

What’s the Target?A learning target is any achievement expectation we hold for students. It’s a statement of what we want the students to know and be able to do.


Kinds of Achievement Targets

Master Factual and Procedural KnowledgeSome to be learned outrightSome to be retrieved using reference materials

Use Knowledge to Reason and Solve ProblemsAnalytical or comparative reasoningSynthesizingClassifyingInduction and deductionCritical/evaluative thinking


Kinds of Achievement Targets, cont.

Demonstrate Mastery of Specific SkillsSpeaking a second languageGiving an oral presentation Working effectively on a teamScience process skills

Create Quality ProductsWriting samplesArtistic productsResearch reportsShop projectsScience exhibits


Acquire Positive Affect/DispositionsPositive self-concept

Desire to learn/read/think criticallyPositive attitude toward schoolGood citizenshipRespect toward self and others FlexibilityPerseverance

Kinds of Achievement Targets, cont.


Is This a Target? Senior project Bird feeder Use a band saw safely State report Diorama



A Mathematics Example Math Decimals Page 152 in the book Going on a “decimal hunt” Read decimals and put them in

order



Examples of learning targets taken from a 4th grade social studies text:

Knowledge Targets (recall)• Name at least two regions of which Washington is a part.• Tell at least one way that location has affected Washington's

history.• Name five main parts of Washington State and describe key

land and water forms of each. • Define the term natural resources and give examples of those

found in Washington. Examples of key words: tell, understand, name, describe, list, identify, give examples.

Knowledge Targets (procedural)• How to use map scales to measure distance-• How to use latitude and longitude to locate places on a map

or globe. Examples of key words: how to (followed by some procedure).

What’s the Target?


Examples of learning targets taken from a 4th grade social studies text

Reasoning TargetGive examples of differences between coastal and plateau

cultures and relate these to differences in the natural environment.

Examples of key words: classify, compare, contrast, analyze, synthesize, determine, evaluate, justify, construct support, draw conclusions.

Skill TargetUse map scales to measure distance.Examples of key words: read, speak, assemble, operate,

use, demonstrate.

Product TargetMake a relief map of any region of the whole state, or make

maps of products, points of interest, or land uses. Examples of key words: create, design, make, produce,

write, draw.

What’s the Target? continued…


Standard 1 – Features of Clear & Appropriate Student Learning Targets

Fast-Tracked Targets are stated,

selective, and easy to find Targets are important –

worth the assessment time devoted to them.

Targets are related clearly to district/state standards/outcomes.

There is an effort to define targets: examples of student work, references to definitions, references to performance criteria, and/or a table of specifications; it is clear that the author understands that such references help users define the targets.

The targets are clear enough that educators would more or less interpret them the same.

Target descriptions and definitions reflect an understanding of best thinking in the field.

There is an appropriate mix of targets and/or there is evidence of long-term thinking – how the targets in the current assessment fit with the plan for the year.



On Track, But Needs Work Targets are listed, but they might

be stated differently in different places, scattered, or require some work to find.

Some of the targets are essential, but there also seems to be some dead wood that might profitably be cut; some targets seem to have been chosen because they were easy to assess.

Targets seem to be retrofitted to an already existing test; as a result, one might feel somewhat dissatisfied that the assessment is not as well thought out, comprehensive, or as focused as it might have been had the targets been identified first.

Although targets are stated, there is some question as to their meaning-different educators might define the targets differently

At first glance there appears to be a connection between stated targets and local content standards, but on closer examination the connection is not clear.

Although the author provides local content standards, rubrics, etc., it is not clear that he or she knows that these help users understand the nature of the targets being assessed.

Rubrics only partially help define the targets.

Some of the targets represent the best thinking in the field; others do not.



Side-Tracked Stated targets are broad,

general and vague; there is little attempt at clarification

No targets are stated. There is little focus;

everything is listed. Statements of targets

ramble; the author lists one and later seems to list others; targets have to be inferred from the assessment itself.

Targets are stated, but seem trivial; why spend time assessing this?

The description of targets doesn’t reflect an understanding of best thinking in the field.

There is a poor mix of targets; the author might, for example, have chosen only the easiest targets to asses; or, there is little evidence of long-term thinking how the targets in the current assessment fit into the overall plan for the year.

There is no connection made to district and state standards or outcomes.

Targets and tasks are mixed up.


Standard 2: Clear and Appropriate Users and Uses

Who will use assessment results and for what purpose? Classroom- students, teachers, parents Instructional Support - curriculum

specialists, principals, counselors Policy – – superintendent, school board,

state department of education, legislators, taxpayers


29

Assessment User

Assessment for Learning

Assessment of Learning

Students Am I improving over time?Do I know what it means to succeed?What should I do next?What help do I need?

Am I succeeding at the level that I should be?Am I capable of success?How am I doing in relationship to my classmates?Is the learning worth the effort?

Teachers What does this student need?What do these students need?What are student strengths to build on?How should I group my students?Am I going too fast? Too slow? Too far? Not far enough?

What grade do I put on the report card?What students need to be referred for special service?What will I tell parents?

Parents What can we do at home to support learning?Is my child learning new things?

Is my child keeping up?Is this teacher doing a good job?Is this a good school? District?

Principal Is instruction producing results?Are our students ready for the workplace or the next step in learning?How shall we allocate building resources to achieve success?

Superintendent Are our programs of instruction producing desired results?Is each building producing results?Which schools need additional resources?How shall we allocate district resources to achieve success?


Think-Pair-Share

Think of a time when you’ve used formative

assessment data effectively. What did you do that made your

use of the data effective? How did this effect your teaching? How did this impact student

learning?



Fast-Tracked It is clear who the intended users &

uses are Users and uses are appropriate and

focused. There are statements relating

assessment design to users and uses.

It is clear how the assessment can be used to inform instruction.



On Track, But Needs Work Author has considered several users/uses,

but does not clearly determine how assessment design should differ for these audiences.

Users and uses are implied but not clearly stated..

Implications for instructional decision making are there, but must be inferred.

Users and uses are stated, but there is a question of appropriateness.

Users and uses are stated, but the author doesn’t seem to understand the importance of stating them.


Standard 2: Clear and Appropriate Users and UsesSide-Tracked There are too many users and uses; cannot

satisfy all stated purposes in single assessment. No purposes are stated; it is not clear why

assessment is being given. The stated purpose doesn’t match the

assessment. The only purpose, ever, is grading. The author doesn’t appear to be aware that

assessments should be designed with users and uses in mind.

It is not clear how results would inform future instruction.


Standard 3: Target-Method-Match

Determining most efficient and effective

way to assess targets: Identify critical targets Meet purpose of instruction Review possible methods Identify rationale for methods

selected


Possible Assessment MethodsSelected Response/Short Answer

○ Multiple Choice ○ True/False○ Matching ○ Fill in the blank○ Label a diagram ○ Sentence

Essay Assessments – extended written response

Performance Assessments○ Demonstrating skills○ Developing a product

Personal Oral Communication○ Question and Answer ○Conferences○ Interviews ○ Oral Examinations


36

Target-Method Match ActivitySelected

Response & Short Answer Essay

Performance Assessment

Personal Communication

Knowledge

Reasoning

Skills

Products


Standard 3 – Features of Target-Method MatchFast-Tracked The assessment method matches purpose and

target. There’s a reasonable rationale for the method used. An assessment outline clearly shows how each

target is to be measured and its relative importance. There is a balanced and appropriate mix of

assessment methods. The author carefully chose when and how to use performance assessments; using simpler methods when possible.

There is a good match between instructional emphasis on targets and assessment emphasis on targets.


Standard 3 – Features of Target-Method MatchOn Track, But Needs Work Author has used a variety of assessment

methods, but it is somewhat unclear why. Some of the assessment methods could be

improved. Author overuses performance assessment

when a simpler method might be cleverly applied.

Target/method matching are not clearly explained and must be inferred.

There are some mismatches among targets, instruction and/or assessment methods.


Standard 3 – Features of Target-Method MatchSide-Tracked The assessment method is inappropriate for

the learning target. Only one type of assessment is used for

every target. There is no apparent rationale for the

assessment methods used. Over-reliance on assessing only the higher-

level skills, without consideration for assessing prerequisite skills which might require a simpler assessment method.

Mismatch between the target being assessed and the targets emphasized in instruction.

Many missed opportunities for assessment.


Standard 4: Sampling - The Basics

Any assessment is a sample of the possible items/questions/tasks a teacher could use to assess a target.

To determine how much is enough, ask:“Does my assessment cover all important aspects of the target and give students enough chance to show what they know?”


Example 1: How many landings, in which types

of conditions, would it take for you to certify a pilot’s competence?

Example 2: How many writing samples, of

which types, would it take for you to certify that a student can write well for different audiences and purposes?




We have sampled enough when…

we are confident we can predict how student will do on the next item/test/task based on previous responses to items/tests/tasks of the same target.


Standard 4– Features of SamplingFast-Tracked The author has defined the target/domain from

which the specific assessment tasks have been sampled.

The sample of student performance will accomplish the purpose.

There are enough samples of student work to get a good estimate of achievement.

There are not too many tasks, nor too few. The tasks cover the learning target/domain well. The sample matches the breadth of the target

and/or the importance of results.


Standard 4– Features of Sampling

On Track, But Needs Work Author seems to have covered the learning

targets/domain well, but has not made a clear enough description of domain to know for certain.

The author has students doing more tasks than necessary to get a good estimate of achievement.

There is good coverage of the domain of skills needed to make a stable estimate of achievement, but assessment would benefit from a few additional tasks.

Although sampling might be ok for some uses, the stakes are such that additional samples would be beneficial.

Sampling seems acceptable, but is not explicitly addressed and must be inferred.

Some outcomes are sampled well, some are not.


Standard 4– Features of SamplingSide-Tracked There are not enough tasks to draw the

desired conclusion. The tasks do not cover the ground well. The author doesn’t seem to be aware

that all assessments sample from a domain and that the domain must be defined.

The sample doesn’t match the breadth of the target nor the importance of the results.


Standard 5: Avoid sources of bias and distortion

Bias – A test or item is said to be biased when a certain students or groups have an unfair advantage over others.

Distortion- Biased tests or items cause results to inaccurately reflect student ability.


Brainstorm potential sources of bias -

Test format/content

Testing environment

Student input


Standard 5: Avoid sources of bias and distortionBarriers common to all assessment

methods:Occur within the student:

Language barriers Emotional upset Poor health Physical handicap Peer pressure to mislead assessor Lack of motivation at time of assessment Lack of understanding how to take tests Lack of personal confidence leading to evaluation

anxiety


Occur within the assessment context: Noise distractions Poor lighting Discomfort Lack of rapport with assessor Cultural insensitivity in assessor or

assessment Lack of proper equipment



Arise from the assessment itself (regardless of

method): Directions lacking or vague Poorly worded questions Poor reproduction of test questions Missing information



Possible barriers with multiple choice tests:

Lack of reading skills More than one correct response Incorrect scoring key Incorrect bubbling on answer sheet Clues to the answer in the item or in

other items



Potential barriers with essay tests: Lack of reading skills Lack of writing ability No scoring criteria Inappropriate scoring criteria Scorer untrained in applying scoring criteria Biased scoring due to stereotyping of

respondent Lack of time to read/score carefully Scoring criteria not communicated to

students



Standard 5– Features Of Avoiding Bias and DistortionFast-Tracked It is clear what students are to do during the assessment;

instructions are clear. Tasks match the targets and criteria – complex target,

complex task; simple target, simple task. Possible sources of bias and distortion are described or

acknowledged and caveats on use are given. Author has sought out assessments with certain features to

minimize bias and distortion. Performance criteria are clear, well defined and cover the

most salient features of a performance. Paper and pencil methods adhere to standards of quality. Students of equal ability will have equal opportunity to

excel regardless of racial, cultural or gender differences. Tasks and exercises are feasible, student can complete

them successfully. Reader cannot identify any obvious sources of bias and

distortion.


Standard 5– Features Of Avoiding Bias and Distortion

On Track, But Needs Work Author is aware that bias and distortion can be a

problem, but doesn’t completely address potential problems in the assessment.

Although task might have a few vague or confusing features, they are generally sound and just require some adjustments.

Although criteria may be a bit vague or confusing, they are generally sound and just require some adjustments.

Seems to have covered the learning targets/domain well, but has not made a clear enough description of domain to know for certain.

The assessment might work well for one group of students, but might need to be reworked for use with other groups.

Information about bias and distortion is included, but has to be inferred.


Standard 5– Features Of Avoiding Bias and Distortion

Side-Tracked Tasks are vague or confusing and it is difficult to see

how they might be fixed. Tasks (multiple choice to performance based) do not

adhere to standards of quality. It would be hard to do the task successfully. Author seems unaware of possible sources of bias &

distortion. Reader can readily identify several possible sources of

bias distortion. Criteria for performance assessments don’t cover

important elements of performance, are vague or confusing, miss the point, or are missing entirely.

Tasks and procedures might unfairly cause different groups of students to do poorly even when skills and knowledge are the same.


Key Challenges Classroom assessment often encourages rote

or superficial learning versus understanding, Quantity and presentation typically

emphasized over quality. Collection of marks, grades to meet reporting

requirements often higher priority than ability to discern student learning needs.

Assessments, typically not common within grades.

Assessments typically not reviewed to determine if they accurately/adequately assess intended targets.


High Quality Classroom Assessments

Maximize efficiency because – Clear targets set boundaries on

content, reducing amount to be assessed.

Eliminate redundancies across grades, reducing amount to be assessed.

Help identify most efficient methods of assessment for target(s) and content.

Efficient sampling minimizes amount of items to be tested.


Excellent resources to learn more or to access tools and activities:

Stiggins, Richard J., Judith Arter, Jan Chappuis, Stephen Chappuis. 2004, Classroom Assessment of Student Learning. Assessment Training Institute, Inc. Portland, Oregon

Stiggins Richard J. 2005 Student-Involved Assessment for Learning. 4th Edition Upper Saddle River, NJ: Merrill/Prentice Hall.

Assessment Training Institute website: www.assessmentinst.com

Northwest Regional Educational Laboratory Assessment Tool kit 98; www.nwrel.org/assessment

http://www.assessmentinst.com/

http://www.nwrel.org/assessment


In Summary

In your own words, write down what each standard means.

When you have finished, please share with your neighbors

60

Standards Of Assessment Quality

Standard 1Clear and appropriate targets

Standard 2Clear and appropriate uses and users

Standard 3Target-Method-Match

Standard 4Adequate Sampling

Standard 5Avoid bias and distortion

GLCE and YOU!Focus on Assessment

Additional Resources


62

Kinds of Achievement Targets – Key Words5

Target Type Explanation Content Standards Key Words Examples

Knowledge/ Understanding

Some knowledge/facts/ concepts to be learned outright; some to be retrieved using reference materials

Explain, understand, describe, identify, tell, name, list identify, give examples, define, label, match, choose, recall, recognize, select

VocabularyMeasurement conceptsUS Government structurePatterns of growth and development

Reasoning Thinking proficiencies- using one's knowledge to solve a problem, make a decision, plan, etc.

Analyze: components, parts, ingredients, logical sequence, steps, main idea, supporting details, determine, dissect, examine, orderCompare/contrast: discriminate between; alike and different, distinguish betweenSynthesize: combine into, blend, formulate, organize, adapt, modifyClassify: categorize, sort, groupInfer/deduce: interpret, implications, draw conclusions, predictEvaluate: justify, support opinion, think critically, appraise, critique, debate, defend, dispute, evaluate, judge, prove

Think criticallyAnalyze authors' use of language Solve problemsCompare forms of governmentSelf-evaluationAnalyze health information

Skills Behavioral demonstrations; where the doing is what is important; using one's knowledge and reasoning to perform skillfully

Observe, focus attention, listen, perform, do, question, conduct, work, read, speak, assemble, operate, use, demonstrate, measure, investigate, model, collect, dramatize

Read fluentlyOral presentationsPlay an instrumentUse laboratory equipment Conduct investigations

Products Where the characteristics of the final product are important; using one's knowledge, reasoning, and skills to produce a final product

Design, produce, create, develop, make, write, draw, represent, display, model, construct

WritingArtistic productsResearch reportsMake a mapPersonal fitness planMake a model that represents a scientific principle


63

Indicators of Sound Classroom Assessment Practice*1. Why Assess? a. Teachers understand who the users and uses of classroom assessment information are

and know their information needs.

b. Teachers understand the relationship between assessment and student motivation and craft assessment experiences to maximize motivation.

c. Teachers use classroom assessment processes and results formatively (assessment for learning).

d. Teachers use classroom assessment results summatively (assessment of learning) to inform someone beyond the classroom about students’ achievement as of a particular point in time.

e. Teachers have a comprehensive plan over time for integrating assessment for and of learning in the classroom.

2. Assess What?

Assessments Reflect Clear and Valued Student Learning Targets

a. Teachers have clear learning targets for students; they know how to turn broad statements of content standards into classroom-level targets.

b. Teachers understand the various types of learning targets they hold for students.

c. Teachers select learning targets focused on the most important things students need to know and be able to do.

d. Teachers have a comprehensive plan over time for assessing learning targets.

3. Assess How?

Learning Targets Are Translated into Assessments That Yield Accurate Results

a. Teacher understand what the various assessment methods are.

b. Teachers choose assessment methods that match intended learning targets.

c. Teachers design assessments that serve intended purposes.

d. teachers sample learning appropriately in their assessments.

e. teachers write assessment questions of all types well.

f. Teachers avoid sources of bias that distort results.

4. Communicate How?

Assessment Results Are Managed Well and Communicated Effectively

a. Teachers record assessment information accurately, keep it confidential, and appropriately combine and summarize it for reporting (including grades). Such summary accurately reflects current level of student learning.

b. teachers select the best reporting option (grades, narratives, portfolios, conferences) fro each context (learning targets and users).

c. Teachers interpret and use standardized test results correctly.

d. teachers effectively communicate assessment results to students.

e. Teachers effectively communicate assessment results to a variety of audiences outside the classroom, including parents, colleagues, and other stakeholders.

5. Involve Students How?

Students Are Involved in Their Own Assessment

a. Teachers make learning targets clear to students.

b. Teachers involve students in assessing, tracking, and setting goals for their own learning.

c. Teachers involve students in communicating about their own learning.*Sound classroom assessment practice=Skill in gathering accurate information +effective use of information and proceduresClassroom Assessment for Student Learning 2004


64


Unpacking Content Standards Content Standards Classroom Learning

Targets Subject: ______________________ Standard: __________________________

Instructions: Identify the Knowledge, Reasoning, Skill, and Product learning targets underpinning the standard. Remember, not all standards have all of these as underpinnings.

What knowledge or understanding is required to become competent on this indicator?

What reasoning (if any) is required to be competent on this indicator?

What performance skills (if any) are required to demonstrate competence on this indicator?

What product competencies (if any) are required by this indicator?


65

Nine Principles of Sound Assessment

1. Teachers understand and articulate achievement targets in advance 2. Students are informed of targets regularly in student friendly

language3. Students can describe targets and what comes next in their learning4. Teachers can transform targets into dependable assessments

yielding accurate information5. Classroom assessment is used by teachers and students to revise

and guide teaching and learning6. Students are given descriptive, frequent, constructive and

immediate feedback to aid them in improvement7. Students are actively involved in the assessment process8. Students actively communicate their achievement status and

improvement9. Teachers “get” the connection between assessment and student

motivation and use assessment to build confidence


66

Key Questions about Standards/GLCE• What do my students need to know and

understand to be ready to meet this standard expectations?

• What patterns of reasoning must my students have mastered to be ready to meet this standard/expectations?

• What performance skills must my students have mastered, if any, must my students have mastered to be ready to meet this standard/expectation

• What product development capabilities, if any, must my students have mastered to be ready to meet this standard/expectation?


Michigan Curriculum Framework:Assessment Standards

Organization of Information: The task asks students to organize, synthesize, interpret, explain or evaluate complex information in addressing a concept, problem or issue.

Consideration of Alternatives: The task asks students to consider alternative solutions, strategies, perspectives, or points of view in addressing a concept, problem or issue.



Disciplinary Content: The task asks students to show understanding and/or use ideas, theories, or perspectives considered central to an academic or professional discipline.

Disciplinary Process: The task asks students to use methods of inquiry, research, or communication characteristic of an academic or professional discipline



Elaborated Written Communication: The task asks students to elaborate on their understanding, explanations, or conclusions through extended writing.

Problem Connected to the World Beyond the Classroom: The task asks students to address a concept, problem or issue that is similar to the one that they have encountered or are likely to encounter in life beyond the classroom.



Audience Beyond the School: The task asks students to communicate their knowledge, present a product or performance, or take some action for an audience beyond the teacher, classroom, and school building. Newman, Secada and Wehlage, Wisconsin Center for Educational Research. A Guide to Authentic Instruction and Assessment: Visions, Standards and Scoring


Michigan Curriculum Framework:Teaching and Learning Standards

Higher-Order Thinking: Instruction involves students in manipulating information and ideas by synthesizing, generalizing, explaining or arriving at conclusions that produce new meaning and understandings for them.



Deep Knowledge: Instruction addresses central ideas of a topic or discipline with enough thoroughness to explore connections and relationships and to produce relatively complex understanding



Substantive Conversation: Students engage in extended conversational exchanges with the teacher and/or peers about subject matter in a way that builds an improved and shared understanding of ideas or topics.



Connections to the World Beyond the Classroom: Students make connections between substantive knowledge and either public problems or personal experiences.

GLCE and YOU! Focus on Assessment January 12 and 13, 2005.

Documents

Transcript of GLCE and YOU! Focus on Assessment January 12 and 13, 2005.