EVALUATING THE QUALITY OF EARLY CHILDHOOD EDUCATION PROGRAMS

Robert MyersPresentation at CIES February 27, 2007

ORGANIZATION OF THE PRESENTATION

• Edging up to an operational definition of “educational quality” for preschools

• A Scale for Evaluating the Quality of Preschool Centers

• Results of the application of the scale in an evaluation of the Mexican Schools of Quality Program

• Some final reflections

WAYS TO APPROACH A WORKING DEFINITION OF EDUCATIONAL QUALITY

CONTRASTING FRAMEWORKS

Quality as

universal, uniform, constant, objetive, tidy (inherent and discoverable quality)

or relative, diverse, changing, subjetive,

disorderly (negotiated quality)

The Intersectorial Project(Negotiating a definition of quality)

• Participants in the negotiation from: education, health, family welfare, NGOs university

• Purpose: to define indicadors de well being for children under six

• Working groups on:– The condition of children– Family and community contexts

– Early education programs: their quality

A review of research and evaluations

• Research– What conditions and processes most help children

learn? – How should one organize and administer an

institution?

• Evaluaciones : Cuatro dimensiones:• Resources• The educational process• Management• Relationships with parents and communithy

ASKING MEMBERS OF THE EDUCATIONAL COMMUNITY

• Personal experience of participants in the project (experts?)

• Interviews with educational “authorities”• Interviews with directors and educadors about what

should be done to improve learning• Results:

– Varied views, with emphasis on classrooms and on the interactions between the teacher and the child as well as emphasis on improving the resource base.

– Context is important

THE VIEW OF THE SOCIETY IN WHICH WE WANT TO LIVE

• Characteristics of the society– Democratic, multi-cultural, respectful of rights,

equitable, productive,……

• What type of citizen does that require? What type of child should we be educating?– Literate, informed, participatory, critical, tolerant,

empathetic, ……

• ¿What type of education, school and educator is required?

The Nacional Education Program, 2001-2006, in Mexico

• The educational system: Equitable, flexible, descentralized, democratic in its modes of operation, accountable

• Schools: Offering equal educational opportunity to all, taking into account cultural variations, colegial, open to participation by families and community members, capable of evaluing and communicating results

• Type of educador: Technically competent, capable of motivating and facilitating active learning and development, participatory, respectful of differences, egalitarian, colegial, able to work with parents as well as children, ….

Establish criteria

– Active learning: The child constructs knowledge

– Integral: in the definition of development and learning applied

– Relevant: to the life/interests of the child– Pertinent: within a multi-cultural environment– Participativa: including families/community– Flexible: adjusts to differences in children

and contexts (María Victoria Peralta, 2000)

A SCALE FOR EVALUATING THE QUALITY OF PRESCHOOL

CENTERS

THE CHALLENGE

Convert the foregoing into indicators of educational quality in order to evaluate and monitor quality.

Develop a methodology and instruments that can provide the information needed to utilize the indicators.

First Steps

• Establish dimensions of quality

• Within each dimension, identify indicators,

• Examine methods and instruments in use (in Mexico and in other countries)

• Construct questions and guides for systematic collection of information through conversations and observations

DECISIONS

• Four main dimensions (at first): resources, management, the educational process, relation to families and community

• Center and classrooms• Utilize a continuum (instead of yes-no options)

de 5 categorías:– 1 = inadecuade– 2 = incipient– 3 = básic- a kind of minimum all centers should be

able to obtain on each indicator– 4 = good– 5 = excellent

Example of a scale item:The educational process

Indicator: The process fosters and permits children to chose topics, activities and materials and to put in practice their own ideas

Inadecuate 1The teacher: - chooses the theme, activities and materials. - does not accept suggestions or variations

Incipiente 2The teacher:- chooses the theme, activities and materials- sometimes accepts suggestions or variations

Básic 3There are moments during the day when the teacher proposes the theme and the children choose the activity and/or materials

Bueno 4There are moments during the day when the teacher and children work together to choose the theme, activities and materials

Excelente 5 The teacher forsters and helps the children to choose the theme, activities and materials to be used throughout the day

Observations:

EVOLUTION OF THE SCALE

• The Quality Scale is now in its fifth version, each version incorporating adjustments based on what was learned from applying the scale and as a result of a continuous dialogue with the intersectorial working group.

• In its first version the scale had 40 indicadors (and corresponding items); in the third, 45 and in the present version 54.

• Along the say, three additional dimensiones were added: accompaniment, health, and children with special needs.

• The main increase in the number of indicators was in the dimension of the educational process giving it more weight in the scale.

Results from the application of the Quality Scale (Version 3) in an evaluation of the Mexican Schools of Quality Program

The Schools of Quality Program(PEC)

• Objectives: improve school quality, autonomy and management, with parental/community involvement

• Modest funding to individual schools based on: Diagnosis—Strategic Plan---Proposal

• Participation voluntary• Funding from federal government, matched by states• Annual review, possible funding up to 5 years• Participation at level of about 5% of preschools in 2004-

2005

Characteristics of the study• Evaluation and accompaniment• Two samples:

– S 1• 4 centers in each of 26 states• Visits at the beginning and end of the 2003-2004, school

year and at the end of the 2004-2005 andy 2005-2006 school years (tres años)

– S2 • 2 additional centers per state and two additional states. • Visits atthe beginning and end of 2004-2005, and at the end

of 2005-2006 (two years)

Metodología

• 3-person teams: PEC, Preschool education and the Pedagogical University

• 2-day visits: observation and interviews based on the ECCP plus field diaries, fotos (and in some cases videos)

• Analysis of results at the national level and for each center

• Accompaniment with feedback (3 sessions) aimed at improving plans and actions

Some Results

PEC contributed to improving the quality of preschools participating in the study

JN Inicio 2003

10%

62%

28%

Más de 4Más de 3Menos de 3

JN Fin 2006 MI

43%

54%

3%

Más de 4Más de tresMenos de tres

PEC has a greater influence on quality at the level of the center than in classrooms

Begin 2003-4 End 2005-6 Difference

Total 3.27 3.91 +.64

Center 3.34 4.15 +.81

Resources 3.27 4.16 +.89 Ed. Process 3.35 4.18 +.83 Ed. Management 3.41 4.23 +.82 Family/community 3.34 3.85 +.51

3º Year classrooom 3.12 3.80 +.68 Resources 3.20 3.70 +.50 Ed. Process 3.19 3.86 +.67 Ed. Management 2.97 3.58 +.61

STRENGTHS

• Improved infrastructure and materials

• Change management style – more colegial

• Improved attention to health and hygiene

• Improved statements of mission and vision preparations that support the educational process (mission and vision, diagnoses and planning)

AREAS REQUIRING IMPROVEMENT

• Identification and incorporation of the interests of the children; opportunities for children to propose ideas and act upon them.

• Activities that promote active learning: exploration, manipulation and reflection

• Attention to cultural diversity• Resolution of conflicts• The school´s relationship with parents and the

community• Supervision and accompaniment

After three years, the gap in quality between centers that entered with high and low quality has started to narrow

Comparación entre JN (ECCP Total)

2

2,5

3

3,5

4

4,5

5

Inicio 2003 Final 2004 Final 2005 Final 2006

Altos

Bajos

REFLECTIONS

• When defining quality it is important to think beyond the availability of schools and resources and beyond narrowly defined child outcomes.

• The most important dimension of quality is the educational process focussed on the interaction of teachers and students, mainly in the classroom.

• This process should be consistent not only with recognized principles of learning but should also with anagreed-upon (negotiated) vision of the society in which we want to live.

Reflections (2)

• Successfully changing the educational process from one that is mainly instructive to one that is constructive is a long-term project.

• A key element in obtaining that change (and others) is a process of human and technical accompaniment that goes beyond inspection or “supervision”, treating directors and teachers as subjects rather than objects.

REFLECTIONS (3)

• The Quality Scale helps the educational community reflect on and reexamine policies and programs at national, state and school levels by providing diverse and systematic information on a variety of dimensions.

REFLECTIONS (4)

A major challenge for the Mexican educational system is to close the quality gap between higher and lower quality centers. This challenge is greatest in rural areas, often indigenous, where the conditons that support quality education are not in place.