IITE Professional Development CourseLucknow University (6/4/2010)

Professor Tim Keirntimkeirn@csulb.edu

Module 6: Assessment

A Review: Standards-Based Approaches and Learning OutcomesProgramme learning outcomesCourse learning outcomesProgram curricular map w/ sequenced

papers for two certificationsPhysical Science: Teacher Ed, Chemistry,

Mathematics and PhysicsBiology/Life Science: Teacher Ed, Chemistry,

Botany and Zoology

Review ContinuedLearning outcomes for each paperDesign an example of a lesson within a

paper that is:Inquiry-basedAligned to a paper specific learning outcomeEngages students with materials from the

web

Review ContinuedDesign an assessment that is aligned to the

inquiry-based lesson and the specified paper learning outcome

Design a rubric for the aforementioned specific assessment

Publish materials to the portal on the web

General Introduction to AssessmentDo students learn what faculty believe

they are teaching? How do you know? On what evidence do you substantiate your claims?

As an employer of a candidate with an upper second B.Sc from Lucknow in e.g. Botany -- what do I know ‘they know’ and what do I know ‘they can do’?What more do they know and what more can

they do than someone with a ‘lower second’ and compared to some one with a ‘first’?

Introduction to Assessment

Can I assume that someone who did the same paper with Vivek ‘knows and can do’ the same as a student of Nalini?

If so -- how can you substantiate these claims?

Think-Pair-Share Strategy: Identify and discuss the origins of three weaknesses in the current means by which students are assessed at Lucknow University This may not be an exhaustive list!

3 Weakness of Current Assessment

Description Impact of Weakness

Exams testing factual knowledge and asked to reproduce knowledge

The exams are the same each year; responding without a deeper understanding of the concepts; not training/ developing skills

Evaluation of the exam is effected by the readers mood, quality of other papers

Unreliable evaluation

Evaluation is not continuous and comprehensive, reliable and valid

Students and employers don’t have a reliable confidence in what a student could actually do

Traditional AssessmentTraditional assessment is inseparable

from traditional modes of teaching and learningPH.D. provides discretion as to what is taughtStand and deliverDesign assessment to measure knowledge

retentionAssign marks based on the ‘volume’ of

knowledge retainedPH.D provides discretionary authority to

assess the ‘volume’ itself

Problems with Traditional AssessmentServe to discriminate between students

as opposed to demonstrating competencies

Almost always measures the reproduction of factual knowledgeLittle if any variance in both the method of

assessment and the modality of learningAssessment is never deployed as a

learning toolThe secret handshakeBlame the learner, not the teacher

Problems with Traditional Assessment (Cont)Assessment is infrequent and heavily

weighted (high stakes)Summative over formative assessment

Limited measurement of teaching efficacy:Did the instructor get the content ‘across’? Did the students read and ‘remember’ the

book?

Alternative Forms of AssessmentStandards-, disciplinary- and inquiry-

based approaches to teaching and learning require a different approach to assessment

Seek to measure:Thinking and skill > factual retentionProduction and application of knowledge >

reproduction of knowledgeWhat is learned (aligned to SLO) > What is

taught

Alternative Assessment (Cont)Standards-based assessments:

Are designed to measure task competence and degrees of proficiency > ranking and discriminating between students

Are done in multiple forms to measure multiple modalities of learning

Are learning tools in support of instruction and are transparent to students

Are on-going and used to support reflection and improvement in teaching practice

Alternative Assessment PracticumIn disciplinary groups -- design a draft of

both a formative and summative assessment aligned to specific student outcome from a paper in the programmeSpecify the SLODiscuss what dimensions of a task are

specifically measured in your standards-based assessments

SLODemonstrate

Different Forms of Assessment and MethodologiesFormative Assessments

Aligned to learning outcome and to summative assessment

Should provide appropriate feedback to student in preparation for the summative assessment

Provide appropriate feedback to instructor about the efficacy of the pedagogic methodology

Monitoring for comprehension in lectureThink-pair-shareShort prompts

Other types of formative assessmentMultiple-choice quizzesShort exercises and promptsMeeting the challenge of marking

Be specific about nature of feedback and limits of time

Peer evaluationRubrics

Multiple Choice Questions Design questions that assess thinking and

skill > factual contentBloom’s taxonomyDevelop ‘justified’ multiple choice questions

that demonstrate thinking and processDevelop distracters that demonstrate &

identify student (mis)understandingsQuestions that task students to substantiate

or challenge claims

Bloom’s TaxonomyBloom’s pyramid and active verbs

Recall (list)Application (show)Analysis (compare)Synthesis (predict)Evaluation (dispute and/or substantiate)

Authentic AssessmentPerformance assessments tied to

authentic disciplinary-tasks -- students produce knowledge as opposed to reproducing knowledgeLaboratory practicumResearch projectsAssessment constructed as a problem Evaluating the validity of different

interpretations and conclusions and their evidentiary basis

Counterfactual questions and prompts

Rubrics - A Scoring Guide that Provides Criteria to Describe Levels of Student Performance

The advantages of using rubrics:Instructors marks more accurately, reliably and

quicklyRequires greater accuracy about the criteria of

student performanceServes as a learning tool and provides better

feedback to students and makes the standard of performance explicit

Creates better reliability across sections

Challenges to Using RubricsInitially time-consuming (but in long-run

saves time)Difficulty to find exact language that

distinguishes between levels of performance and establishes criteria

May require revision in initial implementation

Rubric PracticumIdentify the dimensions of competence in

the task that can be both delineated and demonstrated in the student performance (aligned with SLO)Holistic versus analytic (and the advantages

of the latter within limits)Weight and scale the dimensions within the

task

Rubric Practicum Cont.Establish criteria for competent

performance of each specified dimension of the task

Establish a scale of criteria performanceHow many clearly identifiable scales? E.g.,

Competent and Not CompetentNot Proficient, Proficient, ExcellentNot Proficient, Developing, Proficient, Beyond

Proficient, Exemplary# of scales needs to be justified by clearly

delineated performances of each dimension of the task

Rubric Practicum ContinuedThe ideal process

Create draft of rubricImplement and refine with evaluation of

samples of student workCalibrate with other facultyMark!

Rubric ExerciseIn disciplinary groups -- create a draft

rubric for a laboratory practicum with three scales of performance for each dimensionTeacher education faculty -- to do the same

but for a pre-service teacher’s design of a laboratory practicum

SLO: Laboratory PracticumDimensions Criteria

Not Proficient Proficient/Baseline skills

Exemplary

Lab preparation

DESCRIPTION DESCRIPTION DESCRIPT

Execution of methodology