StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion...
Transcript of StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion...
![Page 1: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/1.jpg)
Statewide High School Exit Exams and Data-‐Based School Improvement:
A European Story Esther Dominique Klein
BEAR Seminar – UC Berkeley – February 23, 2016
![Page 2: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/2.jpg)
Program
What is a “statewide exit exam”?
Why are statewide exit exams so popular?
One size doesn’t fit all -‐ Different statewide exit exam formats
Statewide exit exams and school improvement
State intenTons…
…and school responses: Does the message “hit home”?
![Page 3: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/3.jpg)
What is a “statewide exit exam”?
![Page 4: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/4.jpg)
![Page 5: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/5.jpg)
Statewide exit exams...
(Defini'on adapted from Kellaghan and Madaus‘, 2003, defini'on of public exams)
are exams taken at the end of a course of studies that are…
mandatory for graduaTon
based on prescribed syllabi in curriculum
taken in a common test situaTon
controlled by an agency external to the schools
based on tasks or assignments that are set by an agency external to the schools
usually administered by a naTonal or state government
![Page 6: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/6.jpg)
Exit Exams in Europe (as of 2010)
state-‐wide exams only
state-‐wide and school-‐based exams
school-‐based exams only
no exit exams or no common procedure
![Page 7: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/7.jpg)
Germany before 2005 and aDer 2008
Länder with statewide Abitur exams (Zentralabitur) Länder with school-‐based Abitur exams Länder with a mix of both
before 2005 a^er 2008
![Page 8: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/8.jpg)
Curriculum-‐based statewide exit exams 2002
New York Texas*
2008
Mississipi New York Tennessee Virginia
2012
Alabama* Arkansas Indiana Louisiana Maryland Massachusegs* Mississippi New York Oklahoma Virginia
(Zabala et al., 2008) (McIntosh, 2012)
![Page 9: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/9.jpg)
Why are statewide exit exams so popular?
![Page 10: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/10.jpg)
In recent years educa'onal policy makers have recognized what was known by the Imperial Chinese court over a thousand years ago (…): that a
na'onal external examina'on system can be a powerful instrument for moving schooling in a desired direc'on.
Noah & Eckstein, 1992
10
![Page 11: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/11.jpg)
SWEE as potenJal “allrounders”
Achievement
Accounta-‐bility
InnovaTon
Professional learning
• Formal: e.g., cerTficates, comparability
• Contents: e.g., standards assurance, control of the delivered curriculum
For both students and teachers/schools
ImplementaTon of new curricula or new methods of delivery and assessment • Teachers: improving
diagnosTc skills, reduce uncertainTes, iniTate reflecTve thinking
• Schools: External validaTon and evaluaTon, improving collaboraTon and other organizaTonal rouTnes
![Page 12: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/12.jpg)
Statewide exit exams are perceived as homogeneous
SWEE as a tool for school improvement
Heterogenous exam formats
Research focuses on the effects of student achievements and instrucJon
![Page 13: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/13.jpg)
Research Project „Forms, FuncJons, and Effects of Statewide Exit Exams“, funded by the German Research FoundaJon
• Interview Study • Document Analysis
State IntenTons
• QuesTonnaire Survey
School ImplementaTon
Review of Exam Procedures in OECD countries ● Document Analysis ● Email Interviews
Exam Features
![Page 14: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/14.jpg)
One size doesn‘t fit all:
Different Statewide Exit Exam Formats
![Page 15: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/15.jpg)
Analyzing different SWEE formats
• e.g., structure of secondary educaTon, historical context, development of exam tasks, student choice, exam supervision, marking procedures, use of exam results, ...
Exam procedures
• e.g., specificity, standardizaTon, focal / core themes, ...
Curriculum
• e.g., task formats, complexity / difficulty, curricular validity, ... Tasks
• e.g., specificity, standardizaTon, and bindingness of marking schemes, actual compliance with marking schemes, ...
Marking
Document analysis
Interviews
Curriculum / Task analysis
ObservaTon of marking behavior
...
![Page 16: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/16.jpg)
DescripTon of statewide exit exam procedures at the end of ISCED 3A
17 OECD countries in Europe**
→ 31 naTonal or state authoriTes
A. content analysis (Eurybase, countries‘ relevant websites, legal documents)
B. expert interviews in some countries
(email, unstructured)
** no federal countries except Germany
Cycle 1 Study Procedures
(Klein et al., 2009; Klein & van Ackeren, 2011)
![Page 17: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/17.jpg)
What sub
jects • statewide vs.
school-‐based • core vs. specializaTon
• students‘ choice? W
hat type • wrigen vs. oral
• students‘ choice?
How diffi
cult • different
difficulty levels?
• role of difficulty level for final grade
• students‘ choice?
Differences in the organizaJon
![Page 18: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/18.jpg)
Who
sets th
e pape
rs
• private vs. state-‐run • bogom-‐up vs. top-‐down
• unit sexng the papers
How are papers set
• guidelines and specificaTons • quality assurance • standards assurance
• procedures
Development of exam tasks
![Page 19: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/19.jpg)
Who
doe
s it • marking stages
• role of class teacher • role of second marking • external marking • marker choice • marker training
How is it don
e • control • marking schemes • Standards assurance
Marking
![Page 20: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/20.jpg)
Handling of results Final grade
• weighTng • role of school-‐based exams
• role of conTnuous assessment
Feed
back and
use
• System monitoring • Control and Accountability
• Benchmarking and ProfessionalizaTon
• Student selecTon
High or low
stakes?
![Page 21: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/21.jpg)
Statewide exit exams as a tool to support improvement?
![Page 22: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/22.jpg)
Purposes of statewide exit exams
Achievement
Accounta-‐bility
InnovaTon
Professional learning
• Formal: e.g., cerTficates, comparability
• Contents: e.g., standards assurance, control of the delivered curriculum
For both students and teachers/schools
ImplementaTon of new curriculua or new methods of delivery and assessment • Teachers: improving
diagnosTc skills, reduce uncertainTes, iniTate reflecTve thinking
• Schools: External validaTon and evaluaTon, improving collaboraTon and other organizaTonal rouTnes
![Page 23: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/23.jpg)
“Educa@onal Governance” – Managing the mulJ-‐level school system
Macro level
Meso level
Micro level
§ Not one single “governing unit”, but instead mulTple actors with their own logics
§ Impulses from other actors are “recontextualized” (Fend, 2008)
§ Interdependencies between different actors
§ “Actor constellaTons” à acTons must be coordinated between different actors
(e.g., Altrichter & Maag Merki, 2010; Kussau & Brüsemeister, 2007)
External actors
Inter-‐mediary actors
How can quality and effec@veness be assured and improved in a system like this?
![Page 24: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/24.jpg)
Actual Outcome
Target Outcome
Comparison of target and actual
outcome
NaTonal tests Statewide exit exams
School processes
ReflecTon Strategic decisions
for school development
Standards
Reinforcement
If congruent
If not congruent
InterpretaTon RecontextualizaTon
ModificaTon
Expected cause and effect model (Maag Merki, 2010; shortened version)
24
Accountability, support, incenTves
Individual, school, society, and policyfactors
![Page 25: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/25.jpg)
“Technology” of the exams
Use of the exams (School, department, teacher level)
OrganizaTonal rouTnes (School, department, individual level)
Intended and unintended effects
of the exams
FuncTons and strategies
ImplementaTon process
Governance system, System of quality assurance
Local administraTon, district
Exam Use Model (Klein, 2013, adapted from Visscher, 2002)
![Page 26: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/26.jpg)
Research quesJons IntenJons of educaJon policy • How do policy makers in different cultural and poliTcal sexngs want state-‐wide exit exams to affect schooling?
Responses of school actors • How do school actors respond to the exams, and what factors on system, school and subject level influence their responses?
26
Ø InternaTonal comparaTve case study of statewide exit exams at the end of upper secondary educaTon (ISCED 3A)
Ø MulT-‐method/mulT-‐perspecTve approach (Yin, 2009) Ø Exploratory approach
Context-‐free research quesTon à DescripTon à JuxtaposiTon and Comparison à Hypotheses (Philips & Schweisfurth, 2007)
![Page 27: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/27.jpg)
The three cases Finland Ireland Netherlands
Exam Ylioppilastutkinto Leaving Cer'ficate Examina'ons Eindexamen vwo
Exam type Statewide exit exams
Statewide exit exams
Statewide (50%) und school-‐based (50%) exit exams
StandardizaTon of the exam procedures
moderate high SWEE: high SBEE: low
Governance* LegiTmacy type with Local Empowerment
LegiTmacy / Bureaucracy type with School Empowerment
LegiTmacy / Efficiency type with School Empowerment
*Schmid et al., 2007; Glager et al., 2003
![Page 28: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/28.jpg)
State intenJons
![Page 29: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/29.jpg)
Analyzing state intenJons
QuanJtaJve: • e.g., correlaTon between exam type and demographic data or achievement data
QualitaJve research: • Document analyses (websites, press releases, educaTon codes)
States with statewide high school exit exams have a higher unemployment rate and more diversity among students à SWEE are a means to counteract
disadvantages and inequiTes in the acquisiTon of cerTficates and access to higher educaTon
(Warren & Kulick, 2007)
SWEE are used for „a higher transpa-‐ rency of the requirements, a higher comparability of the achievements, and more equity in the allocaTon of cerTficates. [...] Peformance of the educaTon system as a whole: [...] Based on the data from SWEE, improvement measures can be iniTated.“ (h]p://www.standardsicherung. schulministerium.nrw.de/zp10/ziele/ ; transla'on by me)
![Page 30: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/30.jpg)
DescripTon of state intenTons connected to statewide exit exams
Content analysis
(countries‘ relevant websites, legal documents, brochures, literature)
Expert interviews with representaTves of the exam
bodies (N = 9) semi-‐standardized interview
protocol
Cycle 2a Study Procedures
(Klein 2013)
![Page 31: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/31.jpg)
FuncJons of the exams
Finland • Formal qualificaTon of students
• No agempt to influence school processes
• No school inspecTon • Data feedback as offer • No development expectaTon
Ireland • Formal qualificaTon of students
• Completely external • No agempt to control schools
• School inspecTon with low stakes for schools
• No development expectaTon
Netherlands • 50% statewide, 50% school-‐based exams
• Exam results as benchmark for student achievements
• MulT-‐dimensional internal and external evaluaTon system
• Clear development expectaTon
![Page 32: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/32.jpg)
School responses:
Does the message “hit home“?
![Page 33: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/33.jpg)
• Monitoring
• Policies • Strategic use • Professional Development
• Support conversaTons • ReflecTon • Curricular and instrucTonal decisions
(Schildkamp et al., 2012)
Different users with different logics
• AdministraTon / data-‐wise Leadership
• Teachers as the “deliverers” of instrucTon
• ...
Different types of data use by schools
![Page 34: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/34.jpg)
Use of SWEE data for school improvement
Perceived usefulness and use of data
• England: Perceived benefit of retroacTve feedback is low (Saunders & Rudd, 1999)
• Germany: Perceived usefulness of data is higher for SWEE than for other tests (Wurster et al., 2013), but the actual use of the data is not higher and is restricted to decisions on the curriculum (Wurster et al., 2013; Wurster, 2015)
OrganizaJonal factors
• School organizaTonal factors lead to differences in uTlizaTon (DeBray, 2005; DeBray et al., 2000; Saunders, 2000; Saunders & Rudd, 1999; Schildkamp & Kuiper 2010; Schildkamp et al. 2012)
• Schools need support for data-‐based school development (DeBray et al., 2000)
• Even with training the use of data from exit exams is low (Tymms, 1995)
![Page 35: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/35.jpg)
Is the school actors’ percepTon of the exam funcTons coherent with
the intenTons of the state?
How do school actors uTlize the exams for school and classroom
development?
Cycle 2b Study Procedures
(Klein 2013)
Standardized quesTonnaire survey
35 upper secondary university-‐bound schools • 15 schools in Finland • 13 schools in Ireland • 7 schools in the Netherlands • N = 35 principals • N = 385 teachers
![Page 36: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/36.jpg)
Data use at school level
0%
25%
50%
75%
100% Benchmarking
Monitoring
Vergleich
Rechensch_L
EvaluaTon_L
EvaluaTon_SL
PublikaTon
U-‐entwickl.
Kursdiff.
U-‐ressourcen
Finland
0%
25%
50%
75%
100% Benchmarking
Monitoring
Vergleich
Rechensch_L
EvaluaTon_L
EvaluaTon_SL
PublikaTon
U-‐entwickl.
Kursdiff.
U-‐ressourcen
Ireland
0%
25%
50%
75%
100% Benchmarking
Monitoring
Vergleich
Rechensch_L
EvaluaTon_L
EvaluaTon_SL
PublikaTon
U-‐entwickl.
Kursdiff.
U-‐ressourcen
Netherlands
N = 15 N = 12
N = 6
Comparison Comparison
Comparison
Teacher Acc. Teacher Acc.
Teacher Acc.
Teacher Eval. Teacher Eval.
Teacher Eval.
Principal Eval. Principal Eval.
Principal Eval.
PublicaTon PublicaTon
PublicaTon
Instr. Develop. Instr. Develop.
Instr. Develop.
Streaming Streaming
Streaming
Instr. Resources Instr. Resources
Instr. Resources
(Klein, 2013)
![Page 37: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/37.jpg)
Teacher collaboraJon in the context of the exams
2,0
2,5
3,0
3,5
4,0
A1 A2 A3 B1 B2 C1 C2 A1 A2 A3 B1 B2 C1 C2 A1 A2 A3 B1 B2 C1 C2
Finnland Irland Niederlande
Means and
95%
con
fiden
ce inervals
5 point Likert scale; 1 = not at all true; 5 = totally true
A: IntegraJng exams in lessons (exchange of experience, synchronize teaching of exam content) B: InstrucJonal improvement (using exams and data to improve teaching)
C: StandardizaJon (use of older exam tasks, adjustment state standards)
(Klein, 2013)
![Page 38: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/38.jpg)
Exam use at individual level
3,0
3,5
4,0
4,5
5,0
Improvement Assessment Improvement Assessment Improvement Assessment
Finland Ireland Netherlands Finland
Means and
95%
con
fiden
ce intervals
Use of exam tasks and results for...
5-‐point Likert scale; 1 = strongly disagree; 5 = strongly agree
*** p < .001; ** p < .01; * p < .05
MA-‐B = 0,29 t (128) = 3.11**; d = .31
MA-‐B = 0,89 t (148) = 8.77***; d = .92
MA-‐B = 0,20 t (53) = 1.78, n.s.; d = .26
(Klein, 2013)
![Page 39: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/39.jpg)
JuxtaposiJon and comparison
Finland Ireland Netherlands
FuncTon of the exams
Formal qualificaTon and development
Formal qualificaTon
Formal qualificaTon, moTvaTon, and development
Relevance in schools lower higher higher
School data use
Individualized rouTne
One-‐dimensional purposive
MulT-‐dimensional systemaTc
(Klein, 2013)
![Page 40: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/40.jpg)
Discussion
• Heterogeneous Exams à Heterogeneous Inten@ons à Heterogeneous Responses
Ø The “governing effect” of the exam system becomes visible through the internaTonal comparison
• SWEE are only used for data-‐based school improvement under certain condi@ons
Ø Different funcTons and purposes Ø Data are used at the organizaTonal level, but to a lesser extent at the individual level
Ø Aspects of the macro level might help change collecTve rouTnes
• Variance within countries indicates school-‐specific use Ø Needs more in-‐depth analyses
![Page 41: StatewideHighSchoolExit( Exams(and(( Data5BasedSchool ......Statewide’exit’exams... (Definion adapted)from)Kellaghan)and)Madaus‘,2003, definion )of)public)exams)’ are’exams’taken’at)theend](https://reader033.fdocuments.us/reader033/viewer/2022051805/5ff9632fc11f3122c93d1db0/html5/thumbnails/41.jpg)
Thank you!