Evaluation as a tool for quality improvement in education IEFE workshop, February 18, 2013 Riyadh, Saudi Arabia Agi Csonka, PhD Executive Director The Danish Evaluation Institute

Denmark: Population: 5.000.000 1.300 elementary schools

EVA is an independent institution under The Danish Ministry of Children and Education

•  EVA undertakes mandatory evaluation of teaching and learning at all levels of the Danish education system

•  EVA accredit higher education programmes •  EVA is a knowledge centre for evaluation of education

The big question:

How can evaluation:

examine whether schools comply with laws and government initiatives?

AND

at the same time facilitate professional development

and quality improvement?

Agenda

1)  Evaluation and quality assurance: Some concepts and definitions

2)  Evaluation as a tool for quality improvement: Presentation of three types of evaluation

3)  – conclusions and perspectives

1) Evaluation and Quality Assurance - some concepts and definitions

Definition of Evaluation

•  Evaluation is an applied inquiry process for collecting and synthesizing evidence that culminates in conclusions about the state of affairs, value, merit, worth, significance or quality of a program, product, person, policy, proposal, or plan.

(Encyclopedia of Evaluation, 2005)

Keywords

•  ”…conclusions about the state of affairs, value…..or quality…” •  It is the normative aspect, distinguishing evaluation

from other types of inquiry

•  ”…applied inquiry process…” •  Systematic data collection with focus on use in practise

(empirical aspect)

•  ” …a program, product, person, policy, proposal, or plan…” •  A broad subject area

Institutional level

Organization, development, cooperation

Classroom level Teaching activities

Classroom management

System level

- Implementation of law - Interaction between national and local level

Level of evaluation

Student level student outcome

What is Quality Assurance?

•  Quality assurance may by defined as a planned and systematic set of activities that provide confidence that a product or service will fulfill requirements for quality.

•  In order for quality assurance to be effective, it is necessary to hold specific actors accountable in practice.

* Source: Encyclopedia of Evaluation (2005), Sandra Mathison (ed.), Sage Publications

Quality Assurance

•  Quality Assurance is a specific type of evaluation, which is characterized by:

•  Focus on an individual institution or education •  Assessment of whether the institution or

education fulfils the standards that are set •  The institution is provided with recommendations/

orders •  In special cases sanctions may be imposed on the

institution and/or it may be deprived of privileges

Institutional level

Organization, development, cooperation

Classroom level Teaching,

Classroom management

System level

- Implementation of law - Interaction between national and local level

Level of evaluation

Student level student outcome

QUALITY ASSURANCE

2) Presentation of three types of evaluation

Three examples of evaluation as a tool for quality improvement

a)  Accreditation - an external quality assurance activity

b)  Multi method programme evaluations

c)  School level evaluation of teaching activities

a) Accreditation is a quality assurance method which:

•  Is based on pre-defined criteria

•  Is an assessment of whether the •  institution meets the criteria

•  leads to an authoritative decision

Ministry forms criteria

EVA facilitates Process and form panel of experts

Self evaluation at institutions

Site visit by EVA and experts

Final report by expert panel

Final decision by independent council

The accreditation process

Pros and cons of external quality assurance

Pros

•  A systematic review of all schools

•  Points out strengths, weaknesses and improvement points in particular schools

•  Provides an overview of the quality of schools

Cons •  Costly and bureaucratic

•  Is perceived as control

•  Does not provide tools for quality improvement

b) Multi method programme evaluations

Broad in scope: •  Evaluation of implementation of government

programmes •  Evaluation of common challenges (retention, teaching

methods, teacher qualification, use of information technology etc.)

•  Evaluation of specific subjects (math, history etc.) Methods: •  Based on quantitative and qualitative scientific

methods •  Effects, benchmarking, best practices

Evaluation of the introduction of young teachers to the workplace

Purpose of evaluation: Identify challenges and best practices of introducing new teachers to the workplace.. …in order to promote effective teaching and retention

Problem: Too many young teachers leave the profession because of ”reality chok” Poor introduction to school as a workplace

Evaluation of new teachers

EVA evaluation team

Advisory board 8 teachers

Survey

School leaders

Interviews with teachers

Interviews with

drop-outs

Deskresearch Evaluation design

Design Research activities report

Survey

Graduates

Analysis of data

Recommendations

Best practises

Dissemination to schools by Seminars, workshops etc

Pros and cons of programme evaluations

Pros

•  Provides evidence of strengths, weaknesses and improvement points in general

•  Disseminates knowledge on effects and best practices

Cons •  Does not enhance

accountability

•  Does not point out what the single school should do

c) School level evaluation of teaching practices

Planning teaching practices

Performing teaching practices

Collecting data on

achievements and outcome

Analysing and discussing

insights

Five questions for self evaluation

•  Are we doing the right things? •  Are we doing these things right? •  How do we know, we are doing the things right? •  Do others outside of the school agree, that we are doing the right things?

•  What are the next steps?

-

3) Conclusions and perspectives

A sound system of evaluation for improvement must consider:

•  Balance between accountability and improvement

•  Different types of evaluation carries different advantages and disadvantages.

•  Use a variety of evaluation models and concepts for different purposes

Thank you for your attention

My e-mail: agi@eva.dk

Our website: english.eva.dk