Key Issues for Educational

Assessment in Europe

Eduardo Cascallar and Christina Wikström,AEA-Europe Council Members

Kiev, Ukraine – July 2007

Overview

Standards with different purposes Current issues in assessment research and

development Trust in the Assessment Process Fairness for all Participants in the Process Appropriate Use of Information Technology

in Assessment Adequate Measurement in Performance

Assessments

Standards in assessment

Standards are ”hot” in educational assessment today, but: The term 'standard' is often used in ambiguous and

confusing ways Also in assessment theory, ”standards” is used in many

contexts: Standard-based assessment, Standardized assessment, Standard setting etc.

Formally, a standard is ”something considered by an authority or by general consent as a basis of comparison; an approved model” (could be anything that meets this criteria)

Three types of standards in assessment systems in Europe

1) Focus is internal (standards-based assessment)

2) All conditions are common (Standardization)

3) Focus is external: format and administration

1) Standards-based assessment

Standards-based assessment is more formally known as criterion-referenced assessment/measurement.

The idea is to measure what the student can do, not how other students are doing (relative/norm-referenced assessment).

The criterion is of central importance, often linked to the educational curriculum.

Popular (and necessary) for formative assessment, but also for summative and formative purposes – value added measurement, often related to (school) accountability

Examples of countries with recent reforms with such purposes in mind; USA (the No Child Left Behind-Act), UK (as discussed by Chris Whetton), Sweden and Norway (new grading system).

2) Standardized assessment

Standardized assessment has existed for a long time (roots in psychological testing), and is becoming more common in certain areas, for instance in university admission and in certification/licensing.

• Admissions are becoming more school oriented – changing approach from the psychometric to being more ”edumetric”. More diversified sub-tests (subject oriented).

• Licensing/certification tests more directed towards performance assessment, assessment of skills and higher cognitive aspects rather than basic knowledge.

A standardized test is designed in such a way that the "questions, conditions for administering, scoring procedures, and interpretations are consistent” and are "administered and scored in a predetermined, standard manner" (Popham, 1999)

3) External Standards (without content) The third type of evolving standard-related assessment program in

Europe has no criterion and does not meet the requirements for a standardized instrument.

The standard is limited to format and administration, will not contribute to high reliability and validity (internal variations).

Example: ECTS (the European Credit Transfer System)Purpose: comparability within/across European countries, transparency.

Format: Common grading scale A-F, norm referenced but with additional ”Diploma Supplement” to serve as criterion.

ECTS – an international standard? ECTS Transfer system one year full time studies equals 60 credits.

ECTS Grading system Norm-referenced Scale: A-F: A (top 10%), B (next 25%), C (next 30%),

D (next 25%), FX (FX means “fail- some more work required to pass” and F means “fail – considerable further work required).

The purpose of the Diploma Supplement is to provide a standardized description of the nature, level, context, content and status of the studies. The idea is that the DS will provide transparency and facilitate academic and professional recognition of qualifications.

ECTS (Cont’d)

The ECTS Grading system is now on its way to be implemented in a number of European universities.

Relevant questions are: Is the diploma supplement a criterion? What type of assessment is the ECTS? Is ECTS a standard?

Issues for research and discussion1. How to achieve comparability in standards-based assessment

within and across schools, regions, countries – same criterion, different teachers, schools, raters, systems etc.

2. How to ensure equity when different student groups are included (especially problematic with multi-ethnicity)

3. How to measure achievement over time, especially when tests and test scores are public – what instruments to use, and statistical techniques

4. Validity aspects – how evaluate the validity of these systems?5. Do programs like the ECTS have a chance in Europe – and

what will the consequences be?

”When you want to measure change, do not change the measure” (R. Zwick)

“A true education requires far more than prepackaged tests and a box of No. 2 pencils.” (Hall, 2004)

Jacquelyn Hall

President

Organization of American Historians.

Some identified action lines in European education: Adoption of a system of comparable

degrees Promotion of mobility Promotion of European cooperation in

quality assurance Lifelong learning Promotion of the European dimension

(particularly in higher education)

Important Issues to Achieve Goals in Educational Assessment:

Trust in the Assessment Process Fairness for all Participants in the Process Appropriate Use of Information

Technology in Assessment Adequate Measurement in Performance

Assessments (including ‘Soft Skills’ and Values: such as team working, problem solving, etc)

Trust in the Assessment Process Has to represent the interests of all

stakeholders (examinees, teachers, school-district-national authorities, society at large, etc.)

Has to represent the values and educational traditions of each society.

Must be perceived as fair to all participants.

Fairness and Equity

Must be real, not just the perception of. Has to take into account differences in

culture, language, disabilities, and other issues unrelated to the constructs being measured.

Must have quality standards for the instruments used, including levels of reliability, validity, relevance, control of bias, impact.

Use of Information Technology

Appropriate for the assessment tasks. Must take into account issues of fairness. Should avoid introducing extraneous

factors in the assessment. In general: “use the lowest technology

necessary for the assessment task, not the most novel or ‘trendy’.”

Use of Information Technology: Some new possibilities New skills can be measured adequately:

speaking proficiency in languages, complex reasoning tasks, fine motor skills, problem solving processes

Use of new methods, like predictive technologies, to diagnose and

classify (assessments beyond testing) with the use of intelligent systems.

Performance Assessments: some initial considerations Are they desirable in the assessments? What are the prerequisites needed to

guarantee their quality and applicability? When should they be used? What are the consequences of their use

(budgetary, impact, biases, etc.)? Is the infrastructure available for their

implementation? Do they guarantee validity, reliability and

fairness?

Quality Considerations for European Educ. Assessments Learn from mistakes in Europe and

elsewhere. Consider the political and policy

implications of accountability systems. Prepare citizenry to the needs of a

changing and interconnected world. Adequate funding of national educational

priorities, including assessment to improve the learning process.

Looking to the Future of European Educational Assessment:

Should be in line with the desired value system and model of citizenship adopted by each society.

Should promote common European standards to facilitate mobility

Should include systems of accountability and quality assurance

Thank you!

Christina WikströmChristina.wikstrom@edmeas.umu.se

Eduardo Cascallarcascallar@msn.com

www.aea-europe.net