OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - School...
-
Upload
oecd-education -
Category
Education
-
view
1.425 -
download
0
description
Transcript of OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - School...
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes
School Evaluation in the Flemish Community of Belgium
Brussels, 7 December 2011
Claire ShewbridgeAnalyst, Education and Training Policy [email protected]
Structure of the presentation
• The OECD review in general and the OECD review of school evaluation in the Flemish Community
• Policy trends in the Flemish Community and the analytical approach used by the OECD review team
• The OECD review team’s assessment of strengths and challenges in the current approach to school evaluation
• The OECD review team’s policy recommendations to build on the current approach to school evaluation in the Flemish Community
• Further information and OECD review outputs
THE REVIEW IN GENERAL AND IN THE FLEMISH COMMUNITY
The OECD review of School Evaluation in the
Flemish Community of Belgium
OECD reviews of evaluation and assessment: School evaluation in the Flemish Community of Belgium (1)
• The Flemish Community is one of 12 systems being reviewed– 12 other systems providing analytical reports, i.e. total 24 systems– Presenting evidence and including stakeholder views
Country review strand
OECD review visit Country report
OECDReport
Australia June 2010 Y Y
Belgium (Fl.) January 2011 Y Y
Chile November 2011
Czech Republic March 2011 Y
Denmark October 2010 Y Y
Luxembourg June 2010
Mexico First quarter 2012
New Zealand August 2010 Y
Norway December 2010 Y Y
Portugal February 2011
Slovak Republic First quarter 2012
Sweden May 2010 Y Y
Analytical strand
Country Report
Austria
Belgium (Fr.) Y
Canada
Finland
France
Hungary
Iceland
Ireland
Korea
Netherlands
Poland
Slovenia Y
OECD reviews of evaluation and assessment: School evaluation in the Flemish Community of Belgium (2)
• Focus of international review is on how to best use evaluation and assessment to improve school outcomes– Designing a coherent framework for student assessment, teacher
appraisal, school evaluation and system evaluation– Paying attention to use of results, evaluation capacity, procedures and
implementation
• Background report developed by University of Antwerp Edubron Research Group and the Ministry of Education and Training
• 6 days of review, including interviews of all major stakeholders and visits to Brussels, Antwerp, Vilvoorde and Sint-Niklaas– Seeking a broad cross-section of information and opinions on school
evaluation policies and how to improve these• OECD review team = Claire Shewbridge and Deborah Nusche (OECD),
Marian Hulshof (Netherlands) and Louise Stoll (UK/Canada)
POLICY TRENDS IN THE FLEMISH COMMUNITY AND OECD’S ANALYTICAL APPROACH
The OECD review of School Evaluation in the
Flemish Community of Belgium
Major policy developments influencing school evaluation in the Flemish Community
• Inspection focus on “output”• Differentiated inspection to target schools/areas with
greater quality concerns (2009)• Publication of school inspection reports (2004, 2007)• Schools legally responsible for their quality (2009)• Consultation platforms within the Policy on Equal
Educational Opportunities (GOK) (2002)
Increased focus on school
quality and reducing inequities
• GOK funding evaluation requirements (2002)• School innovation projects• Feedback to schools from National assessment• Participation Decree (2004) ensures stakeholder voice• Promotion of “school communities”• Inspection judges school “policy-making capacity”
Stimulating school self-evaluation
Analytical framework for the review of school evaluation in the Flemish Community
Governance: a framework for school evaluation
Effective evaluation procedures
Competencies for evaluation /
feedback
Use of evaluation
results
STRENGTHS AND CHALLENGES
The OECD’s assessment of the current approach
to school evaluation
Governance: a framework for school evaluation
Effective evaluation procedures
Use of evaluation
results
Competencies for evaluation/
feedback
• Schools have the major responsibility for the school improvement process• The Ministry fosters a degree of common understanding of basic quality • An increasingly information rich environment for school evaluation• Availability of robust student assessment tools for primary schools• Increased focus on the importance of engaging all stakeholders in school
evaluation• The Ministry does not mandate or steer school self-evaluation• School evaluation is not well embedded in a larger vision for evaluation
and assessment• Miss potential synergies between different evaluation approaches• Insufficient emphasis on improvement/excellence in attainment targets• High degree of variation in school policy-making capacity• Lack of information flow impedes school evaluation efforts
Governance: a frameworkfor school evaluation
• Considerable use of technology to support self-evaluation• The Context-Input-Process-Output (CIPO) inspection framework is
empirically grounded and comprehensive• Collection of evidence during the inspection can stimulate school self-
evaluation activities• There is a legal basis for inspection, including the examination of
school policy-making capacity• Lack of clarity around purpose of school self-evaluation and minimum
quality• Inadequate information base for risk assessment prior to inspections• Communicating school understanding and uptake of the new
inspection methodology• Judging schools’ implementation of attainment targets
Effective evaluationprocedures
• Recognition of the importance of a school’s policy-making capacity• Network support to promote and develop school self-evaluation
capacity• Identified teacher competencies to support school evaluation• Emerging collegial relations within and between schools to support
competency development• Efforts to improve the capacity of the Inspectorate to conduct coherent
inspections• Variation among school leaders in policy-making capacity• A need to strengthen educators’ evaluation literacy• Teachers’ orientation to research and enquiry• Clarifying and making more uniform inspectors’ judgements on quality
Competencies for evaluation / feedback
• Students have a voice in school policy, including growing involvement of students in self-evaluation
• Good balance of school responsibility and external pressure to use inspection results for improvement
• Publication of inspection reports• Examples of schools using feedback from national and international
assessments in their self-evaluation activities• Examples of primary schools using network test results as part of
whole-school evaluation• Lack of strategic and consistent use of self-evaluation• There is room for improvement in the public use of inspection results• Schools only receive Inspectorate profile information when being
inspected• School self-evaluation results not necessarily shared with inspectors
Use of evaluationresults
Some challenges to implementing school evaluation for continuous improvement
Lack of objective information to assess “output” prior to inspection
Varying levels ofself-evaluation capacity among schools.Lack of strategic use of evaluation results for school improvement.
Focus on minimum only. Hard to judge how schools implement them.
Continuous
improvement
Flemish learning
objectives
Inspectionfocus onoutput /
differentiation
Schoolsresponsiblefor quality
!
! !
POLICY RECOMMENDATIONS
The OECD’s assessment of how to build on
the current approach to school evaluation
Suggested priorities in going forward
Governance: a framework for
school evaluation
Competencies for evaluation/
feedback
Use of evaluation
results
Effective evaluation procedures
Clarify the goals of school evaluation and how different types of evaluation fit together
Continue to invest in school leader and
teacher capacity to conduct evaluation
and use its results for improvement
Increase the objectivity of evaluation procedures and ensure they promote improvement and excellence
Increase the use of available information
(collected at school and central level) in
both internal and external
school evaluation
• Further clarify common goals and expectations with a view to encouraging excellence and continuous improvement
• Strengthen consistency and coherence of different elements of school evaluation– Articulate appropriately with teacher appraisal– Reinforce links with school leadership appraisal– Better integration between self-evaluation and inspection
• Promote the use of evaluation and assessment tools by schools for improvement– Comparative evaluation of externally developed self-evaluation tools for
schools– Promote examples of effective practice
• Continue to embed self-evaluation requirements in new policies/programs– Schools should link this to school quality assurance/improvement plan– Provide guidelines on how to map this to CIPO inspection framework
Governance: a frameworkfor school evaluation
• Evaluate potential to develop set of criteria for learning progressions– Reference to assess student progress in different subjects
• Clarify the criteria for inspection judgements on the quality of education– Define clear criteria for all components used to determine quality– School self-evaluation capacity is a core component of its quality
• Go further in improving the inter-rater reliability of inspection reports– Concrete rating scale; build on inspection of school groups
• Strengthen the commitment of inspection and schools to the implementation of Flemish attainment targets
• Extend collegial practice both within and among schools– Further develop and coordinate emerging critical friendship
• Promote further involvement of students in self-evaluation activities– Critical feedback for learning, teaching and school improvement
Effective evaluationprocedures
• Further strengthen professional development for effective school self-evaluation– Prioritise support to building school policy-making capacity– Conduct external review of PBD tools and services
• Recognise and strengthen key role of school leaders in self-evaluation– Flemish framework for leadership competencies– Develop new leadership roles– Refine leadership training– Ministry project on school evaluation competency development
• Increase teacher understanding of policy-making capacity– Strengthen links between teaching quality and self-evaluation– Build competencies in data use, research and innovation– Fund collaborative teacher research projects with cross-school peer
review
Competencies for evaluation / feedback
• Strengthen information flow on key indicators from and to schools– Provide schools with access to information in Data Warehouse– Schools to provide performance information to the Inspectorate– Improve availability and use of objective output measures
• Ensure regular feedback to schools on key CIPO inspection framework indicators– Help schools with clear goals and measurable objectives– Work with schools to build shared language of quality/ evaluation
• Promote the availability and use of appropriate self-evaluation resources– Comparative overview of tools available to schools– Assess school demand for other self-evaluation tools
• Devise ways to improve public use of inspection results– Simplified language; dedicated website; reports for school groups
Use of evaluationresults
Ensure the quality of teaching and learning is at the heart of school evaluation
• Clarify the goals of school evaluation and how different types of evaluation fit together
• Continue to invest in school leader and teacher capacity to conduct evaluation and use its results for improvement
• Increase the use of available information in both internal and external school evaluation
• Increase the objectivity of evaluation procedures and ensure they promote improvement and excellence
ImprovementTeacher
appraisal /development
Self-evaluation / inspection
Support to schools (PBD)Leadership /evaluation competenciesCollaboration
Learning progressions
Inspection criteriaTools for
improvement
School access to central dataSchool performance data to Inspectorate
Quality of teaching
and learning
Outputs available on the Review’s website:www.oecd.org/edu/evaluationpolicy
OECD Reviews of Evaluation and Assessment in Education:
AustraliaDenmarkNorwaySweden
Country Background Reports: 9 out of 24 available
Literature reviews /conceptual papers including:
formative and summative assessment; teacher evaluation; school evaluation; education standards; use of results...