Evaluating and Developing the Early Education Pilot

for Two Year Olds

Gail Peachey and Frances Miller

DCSF Research Conference 2009

Evaluating and Developing the Early Education Pilot

for Two Year Olds

Gail Peachey and Suzanne Lunn

DCSF Research Conference 2009

Overview

– Acknowledgements

– Background

– Methodology

– Key messages

– Implementation

Acknowledgements – Contractors:

• NatCen: Ruth Smith, Susan Purdon, Ivana La Valle and Caroline Bryson

• University of Oxford: Sandra Mathers and Kathy Sylva

• University of London: Eva Llloyd

– DCSF Research Team– Policy: Suzanne Lunn, Frances Miller,

Background• Policy

o Between 2006 and 2008 pilot provided free early years education for over 13,500 disadvantaged two year olds across 32 local authorities

o Available in a variety of settings

Aim of policy was to: o improve children’s social and cognitive outcomes,

e.g. social confidence and independence and their verbal and reasoning skills

o have a positive impact on parents and family, e.g. parent-child relationship; parents’ emotional wellbeing

Background (cont’d)• Evaluation design

o To assess whether or not the policy had achieved its aims a programme of evaluation was developed including:

o Mapping study looking at implementation and targeting

o Quality assessments of pilot settings o Impact study using matched comparison group to

measure impact on pilot children and familieso Qualitative interviews exploring parents’

experiences of the pilot and perceptions of impact

Mapping Study

• Why?o Local Authorities were given the flexibility to implement the

pilot in a way they thought would best tackle local issues this raised problems for an overall pilot evaluation

• Aim of the Mapping Studyo Gather detailed information on local implementation

strategies to inform the development of the evaluation framework

• Methodologyo Contacted all 32 local authoritieso 45 minute interviews conducted with key local authority staffo Issues covered included:

o How they chose their target groupso Types of settings offering the piloto How the free hours were being delivered in their local authority

Quality Assessments• Why?

o Quality of childcare is associated with differential child outcomes and the quality of care offered by different settings could influence the impact of the pilot

• Aim of the Quality Assessmentso To collect information on the quality of the different settings offering

the pilot o Link setting quality information to individual children participating in

the impact study• Methodology

o Collected setting characteristic information from all participating settings

o Carried out observations at 75 settings that displayed a range of characteristics (14% of settings; covered 38% of impact study sample)

o Observations carried out using tried and tested rating scales:o Infant-Toddler Environment Rating Scales (ITERS-R)o Early Childhood Environment Rating Scale (ECERS-R)o Caregiver Interaction Scale (CIS)

Impact Study• Objective

– To support the future development of the two year old offer it was important to understand exactly what impact the pilot had had on the participating children and their families

• Methodology– A quantitative, longitudinal, matched comparison study was

undertaken• Compared results for the pilot group versus a

comparison group from other deprived areas of England• Undertook face to face baseline interviews when the

child was aged 2 and in pilot areas close to when the child took up their pilot place

• Undertook face to face follow-up interviews at age 3• Cases were matched across a wide range of baseline

characteristics including child cognitive development at age two

Impact Study (cont’d)Sampling

• Pilot Areas– Local authorities collected the details of all parents.– Details of those who had not opted out of the evaluation were

passed to NatCen.– Baseline: All 2,186 eligible parents were followed up resulting

in 1,400 interviews.– Follow-up: 1,386 followed up; 1,116 completed interviews.

• Comparison Areas– The comparison sample was selected from Child Benefit

Records with a skew towards non-pilot disadvantaged areas.

– Baseline: Following an opt out exercise 2,782 records were issued and 1,821 interviews were conducted.

– Follow-up: 1,748 followed up; 1,376 completed interviews

Impact Study (cont’d)Matching

• Propensity score matching was used to make sure that the pilot and comparison samples are as similar as possible at baseline.

• Achieved by modelling the difference between the two samples on a wide range of variables using logistic regression and recording the likelihood of each person being in the pilot group.

• The treatment group individuals are then matched to the comparison group individuals so that the final matched samples have similar propensity score profiles i.e. they are equally likely to be in the treatment group.

• The types of variables in the analysis included:– cognitive social behaviour e.g. size of English vocabulary, level of

parental concern about how the child is learning and growing up– Other variables e.g. use of childcare, receipt of income support,

degree of TV watching, lone parenthood

Qualitative Study• 54 qualitative interviews were carried out in

January to February 2009 • Interviews lasted between 60 and 90 minutes• Conducted with a range of parents (mainly

mothers) to explore:– influences on take up– perceptions of the impact of the pilot on children,

parents and families.• Data was interpreted using a thematic

approach developed by NatCen.• Provided insight to the broader impact of the

pilot.

Key Messages • The final report drawing together the key findings from

the study published in July 2009 concluded:• Implementation

o The pilot successfully targeted children experiencing different types of disadvantage.

o However, the study found that around half of the control group were experiencing childcare by the end of the pilot clearly raising questions about deadweight. How many of the children in the treatment group would have experienced childcare without the pilot?

o To reach the key target, the most economically disadvantaged children who, as a group were less likely to be able to access good quality childcare, local authorities would need to receive clearer guidance on targeting.

• Impacto To have a positive impact on language ability and parent-child relationships

children would have to attend higher quality settings – Ofsted rated as ‘Good’• Quality of settings

o The quality of provision in target areas needs to be improved – the majority of provision in the pilot was ‘adequate’ (77%); only one fifth was ‘good’ (21%)

• Parents’ experiences and perceptions of impacto Overall positive: staff approachable and friendly, high satisfaction with feedback

on child’s progress, positive effect on child’s developmento Positive impact on parenting skills, physical health and mental wellbeing

Policy Development - Wave 1 LAs• Original pilot

o In 2006, 32 LAs start delivering a targeted offer of free early education to disadvantaged 2 year olds (No cap on numbers – at LAs discretion)

o LAs took individual approaches in delivering the offer.

o Definition of “Disadvantage” varied across the pilot authorities.

o Different experiences of Quality in settings.

• Impacto Findings from the evaluation influenced future

policy development bringing changes in the implementation and delivery of the offer across the Wave 2 (31 LAs) and Wave 3 (89 LAs)

Wave 2 Development• Changes

o Instituted high level National eligibility criteria based on economic disadvantage.

o Equalising the offer more – giving a harder focus on low income families.

o Set delivery target – offer to reach15% of the most deprived families in each Local Authority identified using the IDACI (the Income Deprivation Affecting Children Index ).

o Wave 1 LAs working to bring numbers down and some to increase numbers, to ensure offer reaches the 15% target group

o The offer is increased from 7.5 hours to 15 hours per child per week over 38 weeks of the year across the Wave 1 & 2 LAs

o Quality criteria is reviewed and given a stronger focus

Policy Changes (Contd) o To be eligible for the offer all families must meet the primary

criteria - be in receipt of at least one of the following:• Income Support• Income-based Jobseeker's Allowance• Child Tax Credit at a rate higher than the family element• Extra Working Tax Credit relating to a disability• Pension Credit

o Local Authorities may also apply secondary criteria to meet local priorities:

• SEN / Teenage Parents / Lone Parents etc

o Specific funding is issued to all LAs for Outreach / Family Support Services targeting the hardest-to-reach families, and those who will benefit most from an offer of free childcare.

Wave 3 and the National Offero Wave 3

o The Prime Minister wanted to move the offer further towards being a universal offer for all two year olds

o Commitment to roll an offer of free early learning and childcare “stage by stage” to all two year olds across the country – starting with the most disadvantaged.

o Qualityo The final evaluation report highlighted a key component for

the offer to impact on child outcomes required all provision to be of good quality

o Quality section in the Guidance is strengthened further (July 2009) to help LAs inform their local quality arrangements:

o Settings must have an Ofsted ‘Good’ or ‘Outstanding’ rating

o LAs may only use settings with a ‘Satisfactory’ rating if they can evidence they are actively working towards a good rating - to ensure sufficiency and standards to deliver the offer

Implementation

o DCSF has provided guidance to LAs and support via Government Offices, to run regional learning and knowledge sharing workshops with LAs to share good practice and drive delivery

o September 2009 saw the remaining 89 (Wave 3) LAs start delivery of the offer

o Wave 3 LAs are offering 10 hours per child, per week over 38 weeks of the year

o Currently providing funding for more than 20,000 children per year and their families.

o The extended offer will be data-driven to ensure effective targeting of the hardest-to-reach families, and those who will benefit most from an offer of free childcare.

o All 152 local authorities across the country have been funded to deliver a targeted offer to 15% of their most disadvantaged 2 year olds by 2010-11

Next Stages

o Secure take up of the number of places allocated per Local Authority; identify and share good practice in local delivery

o Further extend the number of places for the most disadvantaged 2 year-olds beyond 2011

o Improve access amongst Families with SENo Look further at what high quality provision for two year olds

means in practice - parents perceptions of quality childcare/early years education and its broader impact

http://www.dcsf.gov.uk/everychildmatters/earlyyears/

localauthorities/lapractice/pilots/twoyearoldsoffer/