SUMMATIVE EVALUATION

Prof. K.T. MolyPrincipal

Amrita College of Nursing

CONTENTS Introduction Significance Definition Existing Pattern Steps for improvement Criticisms Conclusion

INTRODUCTION Assessment and evaluation are

essential components of teaching and learning.

Without an effective evaluation program it is impossible to know whether students have learned, whether teaching has been effective, or how best to address student learning needs.

SIGNIFICANCE“Doing Evaluation and doing it well matters in pragmatic terms because bad products and services cost lives and health, destroy the quality of life and waste the resources of those who cannot afford waste. In ethical terms, evaluation is a key to in the service of justice, in programmes as well as in personnel evaluation.” Scriven (1991)

SIGNIFICANCE (Cont…..)

The outcomes of summative evaluation decisions have serious implications

For students in terms of self-esteem and protection of their rights to pursue a lively hood

For patients with regard to protection from unsafe practice and

For teachers concerned with potential charges of unfair evaluation practices and the legal repercussions of decisions where students are wrongly failed (or passed)

DEFINITIONSummative evaluation Determining the extent to which a

student has achieved the course objectives.

Can occur during a term as well as at the end of instruction.

Summative assessment methods are the most traditional ways of evaluating student work.

Summative EvaluationPurposes. To evaluate learner success in

retaining content knowledge through types of assessment such as traditional paper and pencil tests.

A tool used to evaluate the curriculum and training process.

Formative Assessment Assessment for learning

Taken at varying intervals throughout a course to provide information and feedback that will help improve

the quality of student learning the quality of the course itself

Summative Assessment Assessment of learning

Generally taken by students at the end of a unit or semester to demonstrate the "sum" of what they have or have not learned.

"Good summative assessments must be demonstrably reliable, valid, and free of bias" (Angelo and Cross, 1993).

Formative Summative

‘… often means no more than that the assessment is carried out frequently and is planned at the same time as teaching.’ (Black and Wiliam, 1999)

‘… provides feedback which leads to students recognizing the (learning) gap and closing it … it is forward looking …’ (Harlen, 1998)

‘ … includes both feedback and self-monitoring.’ (Sadler, 1989)

‘… is used essentially to feed back into the teaching and learning process.’ (Tunstall and Gipps, 1996)

‘…assessment (that) has increasingly been used to sum up learning…’(Black and Wiliam, 1999)

‘… looks at past achievements … adds procedures or tests to existing work ... involves only marking and feedback grades to student … is separated from teaching … is carried out at intervals when achievement has to be summarized and reported.’ (Harlen, 1998)

The Garden AnalogyIf we think of our children as plants …

Summative assessment of the plants is the process of simply measuring them. It might be interesting to compare and analyze measurements but, in themselves, these do not affect the growth of the plants.

Formative assessment is the equivalent of feeding and watering the plants appropriate to their needs - directly affecting their growth.

Existing Pattern in Nursing

Written Examination

Practical Examination

Oral Examination

Written Examination Educational objectives centered?

Content coverage?

Types of questions?

Time (duration) ?

Practical / Clinical Evaluation “Any institution wanting an overflow

audience for a symposium need only plan a programme called evaluation of nursing students in the clinical area” (Woolley 1977)

These words are equally relevant today and perhaps more so as educators struggle with issues of clinical evaluation.

Practical / Clinical Evaluation Nursing educators face the dilemma of

how to determine a student’s competence level while at the same time realising that this cannot be known directly, exactly and for all time because of the indeterminate, evolving and contextual nature of clinical competency.

Combining multiple methods adds breadth and depth to an investigation.

Performance check list Written exams, etc.

Practical Examination - Issues Patients Number of students per day Examiners (Internal / External) Environmental factors Subjectivity and inconsistency of

clinical evaluation practices

Practical ExaminationStandardized methods Objective structured clinical examination

(OSCE) Designed to objectively assess clinical skills

performance in simulations of actual clinical situations

The triple-jump exercise or examination (TJE) Uses case studies to assess a students ability

to identify potential and real patient problems and develop a plan of care. It provides a means for observing how a student handles new knowledge and formulates the information into a management plan.

Oral Examination - Issues

Whole syllabus / case or both ?

Questions - more theory /

applications ?

Procedure book / Log book / Subject

file ?

Same questions or different ?

Steps for Improvement

Clearly defined objectives/outcomes Clearly defined evaluation criteria Authenticity - Assessment that is

aligned with the classroom objectives and that reflects real-world applications

variety of assessment techniques.

Practice Standards for Nurses in India (by INC) Used as a parameter for evaluation of

performance and as a measure of quality control

Must be used in conjunction with the code of ethics and professional conduct for nurses India.

Classified under six practice areas consistent with areas for code of ethics and professional conduct

Six Practice Areas1. Professional responsibility and

accountability2. Nursing practice3. Communication and interpersonal

relationship4. Valuing human being5. Management6. Professional Advancement

Area no. 2 Nursing Practice - Standards

2.1 Nursing care reflects adherence to practice standards

2.2 Delivery of Nursing care reflects nursing process approach

2.3 Nursing care is provided in a safe environment

Clinical Evaluation as Inquiry

Clinical evaluation is the sister of research

The purpose of clinical evaluation as a form of research is the discovery and verification of the process and product of the teaching and learning of nursing practice.

Strengthening Teacher Expertise Evaluation systems should capitalize

on the teachers professional expertise.

Expert nurse educators should know what quality clinical practices entails and can judge both the quality and appropriateness of a students practice in relation to individual client situations

Standard of Nursing Practice – 2.3 Nursing care is provided in a safe environment

Performance Criteria2.3.1 Ensure safe and therapeutic

environment in care settings2.3.2 Adheres to standard safety measures2.3.3 Follows guidelines for biomedical

waste management2.3.4 Sensitizes co-workers, individuals and

groups about the importance of safe environment

CRITICISMS Students are encouraged to be involved in the

formative evaluation process while only teachers are considered to be eligible to participate in evaluation decisions of the summative type. (Best et.al 1990, Arthur 1995)

The assumption is that summative evaluation require expertise that only resides within the teacher.

Summative evaluation can be easily compromised by the close teacher student relationships that develop in clinical teaching. (Lenburg and Mitchell 1991)

CRITICISMS (Cont…)

Evaluation is a continuous and ongoing process.

Formative and Summative evaluation are similarly seamless; there is a formative element in any summative evaluation (Schoenhofer and Coffman 1994)

CRITICISMS (Cont…) Findings from medical education studies

suggest that performance in one problem or case does not necessarily predict performance on other problems or cases, particularly in situations that test problem solving and clinical reasoning ability.

Several studies have demonstrated a low correlation between performance course on simulations and marks from written and oral tests of knowledge and global ratings of performance in the clinical settings.

CRITICISMS (Cont…)

Fourth generation evaluation – Evolved from a concern that the previous forms did not fairly and adequately involve stakeholders in the design of an evaluation and implementation of its results.

It also requires that educators believe that students can be knowers of quality nursing practice and should be involved in summative decisions.

Conclusion

Evaluation is a threatening process and steps are required to address this reality or else students cannot be expected to willingly develop a professional valuing of the evaluative process.

Thank Thank youyou

Assessing and Evaluating Learning

Definition of assessment: Assessment is the process of gathering

information on student learning.

Definition of evaluation: Evaluation is the process of analyzing,

reflecting upon, and summarizing assessment information, and making judgments and/or decisions based on the information collected.

Evaluation is both a process and a product; the word evaluation refers to the process of systematically and objectively determining the merit, worth and value of things and it also denotes the product of that process.

Scriven (1991)

Aim of Student Evaluation Incentive to learn Feedback to student Modification of learning activities Selection of students Success or failure Feedback to teacher Protection of society

The artificial separation teaching – evaluating and formative – summative creates a dichotomy that serves to maintain distance between teacher and student and perpetuate a belief that students are unable to be ‘knowers’ of quality practice and their own learning.

The issue is how to deal with the teachers negative feelings when a student’s performance is substandard and how to counter the reluctance of teachers to deal definitively with poor performance.

FOURTH GENERATION EVALUATION According to Guba and Lincoln (1989), the

field of evaluation can be seen to have passed through three distinct phases or generations.

Early evaluation – strictly technical and based in measurement.

The second generation – expanded to include description

Third Generation - The purpose of measurement and description was to inform decisions and judgments about the merit and worth of the evaluand (the object of the evaluation that is the person, programme or product)