Aspire to STEM Independent Process Evaluation
63
Page | 1 Aspire to STEM Independent Process Evaluation 24 th November 2020 Final Evaluation Report
Transcript of Aspire to STEM Independent Process Evaluation
Page | 1
Aspire to STEM
Independent Process Evaluation
24th November 2020
Final Evaluation Report
Page | 2
Contents
Executive Summary ............................................................................................................ 3
1 Introduction ................................................................................................................ 10
2 Evaluation Purpose and Aims ................................................................................... 11
3 Evaluation Method ..................................................................................................... 12
4 The Aspire to STEM Process Journey ...................................................................... 12
5 Senior Management Perspectives ............................................................................ 13
6 Classifying Partnerships ........................................................................................... 18
7 Preliminary partial process evaluation results ........................................................ 19
8 Key Findings .............................................................................................................. 20
8.1 Partnerships and Partnership working - including school recruitment .......... 20
8.2 The role and impact of a customisable programme and bespoke CPD .......... 31
8.3 Role of Educational Leads to support Partnership activity ............................. 38
8.4 STEM Capital, community engagement, and enrichment activities ................ 45
8.5 Barriers to sustained engagement .................................................................... 51
9 Conclusions ............................................................................................................... 58
10 Technical Annex ..................................................................................................... 59
Page | 3
Executive Summary
Evaluation Purpose
STEM Learning has commissioned an independent process evaluation of its Aspire to STEM
Programme in order to ensure that other programmes (such as Enthuse Partnerships), and
future bids can benefit from its legacy and learning lessons.
Evaluation aims
- To gain an understanding of the keys to sustained engagement with disadvantaged
schools;
- To collate evidence of and how to undertake successful community activity;
- To understand lessons learnt from the content of the Programme;
- What lessons can be learned from Partnerships.
Topics of inquiry
The process evaluation investigated seven topics (bold text denotes a topic indicated as a priority by STEM Learning)
1. Partnerships and Partnership working – including school recruitment*
2. The value of the needs analysis / assessment process
3. The value of action planning
4. The role and impact of a customisable programme and bespoke CPD activity
5. The role of Educational Leads to support Partnership activity
6. Science capital, community engagement and enrichment activities*
7. Barriers to engagement*
Method
This report is based on a survey completed by 14 out of 22 invited Educational Leads (ELs)
who each supported between 1 and 4 schools for c15 days over a two year relationship
period; contrasted with results from depth telephone interviews with 7 participating schools
from 5 different Partnerships selected using an agreed sampling frame. Fieldwork was
completed between the 14th of September and the 9th of November 2020. Given the
challenges schools faced as they returned to school completing fieldwork at this time with
schools was difficult, despite a range of participation methods and financial incentive being
offered.
Process checklist
The process journey for the Aspire to STEM Programme was mapped out with the Senior
Management responsible for the Programme and research materials were designed to
assess the experiences and learning from each of the 10 identified ‘steps’ seen from the
perspectives of both Educational Lead and the school (see main report).
Page | 4
Partnerships and Partnership working - including school recruitment
One of the defining aspects of the Aspire to STEM programme was that it was delivered to
school Partnerships rather than individual organisations. The importance of effective
selection and recruitment of schools therefore creates the conditions for all that follows.
Schools that already had a history or affiliation to one another were quicker to develop.
Partnerships comprising schools with common needs were also more likely to cohere more
quickly. In fact, 71% (10 of 14 Educational Leads) agreed that the success of a Partnership
was in part determined by whether schools had common or divergent needs and priorities.
However, there were examples of how AtS has brought outside schools ‘into the fold’ of
Partnerships which has resulted in collaborations that have been sustained which is
potentially where the evidence of additionality is the strongest.
Schools (left) and Educational Leads (right) highlighted the importance of developing
Partnerships built around the following qualities:
Due diligence by STEM Learning when recruiting schools for future programmes should
seek to assess the school’s level of readiness for a new programme like AtS1, and seek buy
in at multiple levels, starting with the Head Teacher / MAT leadership team.
The value of needs analysis and action planning
13 of 14 Educational Leads (93%) considered the needs analysis and planning to have a lot
or some impact on the success of the Partnership, and 64% thought these processes had a
lot or some impact on the schools’ ability to build trust with one another and their EL. These
processes gave access to Senior Leadership Teams that helped to secure early buy in.
Action planning that was regularly revised and responsive to emerging needs was seen to
be more valuable than a more fixed process. STEM Learning programmes taking place
since Covid-19 may require shorter planning cycles.
1 Typically, an assessment of how likely it is the school can cater for any new initiative that requires staff time for 2 years and whether it can directly complement prevailing priorities (e.g. Ofsted inspection improvement challenges).
Trusting relationships at different levels
Willingness to share and
receive good practice with other schools
Capacity to engage
Senior buy in, including MAT
Trusting relationships
Commitment to the
programme
Capacity to engage
Tenacity to persevere and adapt
Page | 5
The role and impact of a customisable programme and bespoke CPD activity
The bespoke, rather than generic, offers were seen to be more valued. While schools did
observe variability in quality between training delivered locally compared to at the STEM
Learning Centre, on balance the training was very well regarded. It inspired and empowered
staff to become STEM champions within their schools.
“Not all Aspire to STEM CPD was as strong as others, but some created a wow
reaction, for example, ‘science through stories’. It was a lovely way to do it.”
The autonomy of the Educational Lead to customise the Programme to meet school needs
was important, and the skill of the EL to then open up training to other schools was a
strength of the Programme. Even greater discretion to respond to the needs of under-
performing Partnerships was suggested as a ‘top tip’ for the future.
This sample of schools, drawn from higher- and average-performing Partnerships2,
considered that the improvements in profile and practice of activities to develop Science
capital and connect young people to potential careers were the outcomes they would have
been least likely to achieve anyway. Schemes of work that include the Gatsby benchmarks
are part of the legacy from the Programme for some schools, while the focus on quality of
STEM leadership and teaching is continuing for those schools that have gone on to become
part of an ENTHUSE Partnership.
Educational Leads were most likely to shine a positive light on the bespoke CPD to both
leaders and teachers, and the community engagement and enrichment activities. The RAND
preliminary results (December 2019) found that schools could not always identify STEM
specific subject CPD they wanted but could not access. Our sample of schools reported, on
the contrary that the ability of the Education Lead to respond to their needs and identify
relevant training was a key strength of the programme. A stand-out message from interviews
with schools and the EL surveys is to simplify the offer of generic CPD and increase the
promotion of this type of training.
The role of Educational Leads to support Partnership activity
Notwithstanding Multi Academy Trusts (MATs) and local connections that existed prior to
Aspire to STEM, it was the EL who provided the energy and tenacity that Partnerships
needed to thrive. They acted as trusted consultant, evangelist, administrator, and facilitator.
Most reported investing more of their time than the 15 days allocated per Partnership, and
their responsiveness and consistency was warmly appreciated by schools. It is interesting to
note that the two Educational Leads who assessed their Partnerships as under-performing
both spent less than their 15 days with the schools, and shared the role with another person.
Top tip: A single Educational Lead, with an increased allocation of days to support
Partnerships will likely create better conditions for success.
Reflecting on the entirety of the Aspire to STEM journey to March 2020, Educational Leads
were more likely to identify successes in the Partnership formation and communication
(aspects they led on). There were fewer examples of successes for aspects which were led
by schools. The most learning came in how to support schools to communicate with one
another, for example to share resources, CPD and learning. Nevertheless, these are the
outcomes that schools were most keen to share – pointing to improved access to STEM
resources and embedded knowledge sharing protocols in place.
2 A subjective assessment made by each Educational Lead reflecting on their experiences.
Page | 6
Science capital, community engagement, and enrichment activities
AtS afforded the time and budget to enable schools to reach out to parents and, to a lesser
extent), the wider community. ‘Bringing parents into the triangle’ with pupil and school
helped to establish or secure relationships between departments and home. From a school
standpoint this had other benefits too, for example increased engagement with homework.
The one-off conferences and STEM days were memorable and brought teachers, students
and parents closer, which subsequently helped when only remote contact was possible
during the 2020 national lockdown.
The legacy (still unfolding) is an appetite and willingness to invest time in these types of
activities in future. A project to develop science capital within communities was regarded as
a longer term priority, and was a less urgent priority for the schools interviewed over the 18
months of Aspire to STEM.
Links to STEM careers was the least of the priorities for schools, but there were some
examples of how schools have now taken the learning and impetus from AtS to translate into
their schemes of work that are used daily to connect students to potential careers3. This
kind of good practice ‘as it happens in schools’ during a programme such as ATS could be
celebrated and propagated more widely as a function of any STEM Learning programme
management function in future.
Even schools in small, rural communities (where the assumption that engagement is strong)
valued the opportunity to engage more with parents, or to bring in different parents, and
shining a light on other career pathways that were perhaps less visible.
Barriers to engagement
Staff capacity and the associated challenge of releasing staff to attend CPD were the most
significant barriers for Educational Leads to sustaining engagement by the schools they
supported during AtS.
Associated changes in senior leadership and the shifts in priorities that can follow were also
inhibitors to consistent performance amongst schools within Partnerships. For schools, the
senior buy in (both at school and, where applicable, MAT level) was the most critical
element, and therefore risk to the chances of success if that is not secured early are
greatest. Staff turnover is the other significant detractor from incremental Programme
evolution or expansion within a school.
Reflecting back on the training content, a clear suggestion was to spend more time
considering how practically to support schools that are busy, under pressure and in flux.
Drawing on past successes tackling challenges and pitfalls would inspire confidence in
future Educational Leads. This cohort of Educational Leads feel they could help develop the
content for future STEM Learning programmes based on their Aspire to STEM experiences
and therefore opportunities to bring these people together in a learning environment focused
on programme design may reap benefits. Similarly, co-designing any future programmes
with willing schools that participated in AtS and / or have matured to becoming ENTHUSE
Partnerships would be even better.
3 Example schemes of work from one school was shared with STEM Learning for review as a consequence of the evaluation interview that took place in October 2020.
Page | 7
What have we discovered about the keys to sustained engagement with
disadvantaged schools?
The role of the Educational Lead appears to be fundamental to securing and sustaining
schools. They led their schools through the needs analysis and action planning processes,
provided ideas, suggestions and support in a customised fashion. They also led on decision
making and prioritisation of Partnership Delivery Plan activities and budget deployment.
To sustain engagement it is vital for SLTs and MAT leaders to recognise that Aspire to
STEM could support the achievement of existing school priorities.
Partnership working is by no means the norm or natural state for schools as they focus on
urgent, inward-looking priorities linked to inspections and improvement this sustaining
engagement with a partnership element can be very challenging for some schools.
Even schools facing multiple challenges though were still able to engage on their own terms
with the Programme, although less active or engaged by the partnership aspects. There was
less buy in from schools and leaders who had not chosen to take part and this reflected their
further behaviours during the Programme.
Early meetings with the Education Lead helped to make schools more aware and open to
the Programme; in turn more likely to release staff / budget when asked.
The EL’s ability to rapidly appraise the school’s situation was essential. For example.
schools already part of MATs were already further advanced with Partnership working, while
rural schools felt they needed to work harder and together to access the same opportunities
that their counterparts in urban areas routinely benefited from.
What have we discovered about how to undertake successful community activity?
While overall less of a priority than the quality of STEM leadership and teaching ambitions of
AtS, successful community events featured prominently when ELs and school interviewees
were asked to consider stand out moments and to produce case studies. For those in a state
of readiness to commit to this kind of activity, it was fun, engaging and inspirational.
The Educational Lead was key to knowing the school’s improvement context in order to
identify high impact community activities. Where activities were already taking place, ELs
were able to amplify the outcomes by drawing in more schools.
Events that provided CPD for staff, enrichment for students and also involved parents were
highly valued; creating a number of benefits stemming from a shared, enjoyable science
based experience. For schools, nurturing relations with parents can be resource intensive
and challenging, so these events provided an additional hook to reach out with.
Neither schools nor Government measures the levels of science capital in communities, so
programmes that look to develop this pose measurement challenges. Qualitatively, schools
are aware of local STEM employers, but may not have the capacity (or known parent
connections to make an approach), and events can make such connections more overt.
Some schools were able to build on face to face events to virtually support families during
the Covid-19 lockdown. Support to start or restart STEM clubs enabled other members of
staff to benefit from the Programme and, once safe to do so, a working model that can be
sustained. The costs of community events led by external companies makes them more
challenging to continue.
Page | 8
ELs nor schools appear to have had the capacity to reach out and work with other
community groups as part of AtS. Schools reported mixed experiences of the availability (in
person) and quality of STEM ambassadors. One senior Education Lead reported that
schools were not very strategic in how they used STEM Ambassadors, and future
programmes could make better use of virtual ambassador engagement. Since the Aspire to
STEM programme, the STEM Learning team consider that the training and deployment of
STEM Ambassadors has improved. COVID-19 has accelerated the use of online
engagement which has led to shorter, more impactful engagements. Virtual engagement
was also creating opportunity to introduce students to industries and careers that were not
visible locally. Schools identified that working with STEM Ambassadors was a current (i.e.
2020-2021) KPI for staff, which suggests engagement will continue and broaden. One
school noted the importance of the STEM Ambassador understanding the local or regional
context, ideally being based in the same region.
The relationship with parents and the wider community, whilst important, is one that schools
often lack capacity to foster. Targeted support to build science capital, over a longer period
(2-5 years as a minimum) would be unlikely to displace or duplicate activities schools could
do for themselves.
What lessons have we learnt from the content of the Programme?
The processes of needs analysis and action planning appear to be important components of
a programme such as AtS. 93% of Educational Leads suggest that these particular planning
processes correlate with the ability for a Partnership to achieve success. It is appreciated by
schools but they rarely lead the process, which has been driven by the Educational Lead,
who also took much of the administrative burden from schools.
The action plan is most appreciated if it is iteratively created and adaptable to reflect the
changing circumstances in each school – especially if personnel change during the
Programme’s lifetime.
The ability to combine different CPD to meet school needs is valued. Bespoke CPD is
perhaps unsurprisingly especially appreciated – offering development opportunities for
STEM staff who would be less likely to receive external training. For Education Leads, there
was most learning in the delivery of generic CPD for STEM teachers, compared to the other
Programme elements for example in how it was badged and marketed to schools.
There were only a few ingredients that were less valued that offer some potential for
rationalisation in future (see main report).
Having a unique cancellation waiver for this Programme may have been unnecessary.
It was natural for ELs to target their support towards senior staff, where the opportunity to
positively influence departments was greatest. However, this created a vulnerability in that
the Programme then relied upon the SLT member / HoD having the time and influence to
cascade knowledge internally. Ideally, in the words of one EL, future programmes should
“ensure that the Partnership works not only with subject leaders but all staff within the
schools and the leadership teams.”
Training that was local and tailored was an easier sell internally for schools than remote
support that appeared more generic. Similarly, ‘tried and tested’ offers, especially when
scheduled after school were an easier proposition for Partnerships to take up compared to
new approaches that took staff out of the school day over a number of weeks.
Page | 9
What lessons can be learned from Partnerships?
Partnerships, rather than 1 to 1 support, created an additional layer of working practices for
schools by cascading learning, amplifying CPD outcomes and extending peer networks. At
the same time, they also created additional meetings and discussions to agree activities
suitable for more than one school. Without the Education Lead, schools reported that they
would have been unlikely to form or sustain these networks for themselves.
Where schools had a history of working together, Partnerships moved more quickly from
forming to performing. MAT structures could increase the reach of the Programme but also
potentially slow partnership working owing to more centralised decision making / budgeting
processes.
Schools needed to be ready to work in partnership; i.e. stable enough to commit to the
Programme, and with sufficient buy in to allow senior staff time away to devote to Aspire to
STEM. AtS has to ‘land’ at the right time for a school in its wider improvement journey.
Feedback from lower performing Partnerships may point to the need for greater due
diligence by STEM Learning before signing them up to a programme. Their state of
‘readiness’ is something to define further for future programmes. Early engagement and buy
in from schools enables faster progress, but greater exposure also helps the EL to more
confidently assess a school’s priorities (and if necessary push back).
Learning about past successes tackling challenges and pitfalls would inspire confidence in
future Educational Leads. This cohort of Educational Leads felt they could develop the
content for future programmes based on their Aspire to STEM experiences.
The drive for the Programme to be non-prescriptive, flexible and customised, may have
claimed primacy over other alternative programme design considerations such as the need
for consistent approaches in processes to encourage more uniform outcomes. On the one
hand this derives positive advantages i.e. demonstrating flexibility within the context of being
a member of a local school partnership. On the other hand this causes challenges owing to
the effort required to achieve such a level of flexible customisation per individual school.
The range of experiences that ELs chose to highlight exemplifies the diversity of outcomes
that Aspire to STEM was able to generate (not the focus for this report though). In pursuing
four overall aims, this gave schools broad scope to identify their priority goals and also to
achieve unexpected benefits most commonly around developing STEM careers. A narrower
focus would arguably have created deeper and more consistent outcomes across
Partnerships, but at the expense of the aims linked to STEM careers and science capital.
Creating the conditions where schools take on greater leadership and accountability will
support the sustainability of activity beyond the Partnership in future. There is a danger that
a dependency is created on EL resources and this would ideally be given a long enough
time period (minimum 3-5 years) to be transferred steadily into school as part of their
business and usual practices.
The facility to pay for cover was the most commonly identified suggestion for improvement,
as fees to cover absence was a barrier in some schools, then to schedule sessions later in
the day, ideally in the ‘twilight period’ immediately after the end of the teaching day.
Looking beyond the funded period of a programme like AtS, Educational Leads advised
ongoing communications, the creation of virtual Partnerships, CPD offers and updates, the
offer of further funding, and removing the stipulation to work in Partnership – perhaps a
contentious, yet important insight for future STEM Learning programme design.
Page | 10
1 Introduction
Aspire to STEM was a two-year Programme funded by the Department for
Education’s Teaching and Leadership Innovation Fund between 2018 and 31st March 2020.
It enabled target schools across England to access free CPD and support.
The Programme aimed to:
- improve leadership to support STEM teaching
- develop great teaching of STEM subjects
- increase science capital within disadvantaged communities
- increase teachers’ confidence in advising pupils about STEM careers and the
transition from school or college to employment.
40 Partnerships consisting of over 200 primary and secondary schools in areas designated
as Opportunity Areas (OAs) or in Local Authority Districts (LADs), with a category of 5 or 6,
and which had an Ofsted rating of 3 or 4 were involved in Aspire to STEM. Schools worked
with a dedicated STEM Learning mentor known as an Educational Lead to create a
bespoke, flexible and focused selection of free CPD and support for the teachers and
leaders in their school or Partnership. Schools worked with STEM Learning for two years,
with support given throughout to ensure the positive impact on teachers, leaders and
students was long-term and sustainable.
In April 2020, Skyblue Research Ltd provided an independent evaluation of the intermediary outcomes emerging from the Programme and found evidence in sampled schools of:
Improved subject leadership Enhanced teacher retention Improved quality of teaching Improved student engagement and progress Community events engaged students and families, boosting STEM uptake.
However, the outcomes were not uniform. Independent evaluation by RAND on behalf of the
Education Endowment Foundation will report in 2021 on the impact of the Programme on
educational attainment outcomes, but in the meantime, STEM Learning wished to develop
an understanding of whether the processes of the Programme could offer up any further
insights about how to create the conditions for successful school Partnership models of
working in future.
Page | 11
2 Evaluation Purpose and Aims
Process evaluation purpose
STEM Learning commissioned an independent process evaluation of its Aspire to STEM
Programme in order to ensure that other programmes (such as Enthuse Partnerships), and
future bids can benefit from its legacy and learning lessons. The process evaluation would
be completed by Skyblue Research Ltd between 1st September and 30th November
Evaluation aims
- To gain an understanding of the keys to sustained engagement with disadvantaged
schools;
- To collate evidence of and how to undertake successful community activity;
- To understand of lessons learnt from the content of the Programme
- What lessons can be learned from Partnerships.
Evaluation topics of inquiry
The process evaluation investigated seven topics (bold text denotes a topic indicated as a priority by STEM Learning).
*Partnerships and Partnership working
*STEM Capital, community engagement, and enrichment activities
Value of needs analysis
Value of action planning
Role of Educational
Leads to support Partnership
activity
*Barriers to engagement
Role and impact of a customisable
programme and bespoke CPD
activity
Page | 12
3 Evaluation Method
A mixed methods approach was adopted for this process evaluation comprising learning
reviews with senior management, the design of formal bespoke research materials (online
self-completion survey and telephone interview topic guide) and associated fieldwork that
yielded responses from 14 / 22 Educational Leads and 5 school leads (representing 7
schools) from 4 AtS Partnerships.
Step Evaluation Activity
1 Scoping out the Aspire to STEM process journey
2 Facilitating an initial learning review with AtS senior management
3 Mapping of process evaluation aims to RAND study and other relevant secondary sources
4 Recommendations for an appropriate primary research phase - Partnership / schools sampling frame considerations (see appendices) - Financial incentive agreement given challenging time of fieldwork
5 Design, administration of self-completion online survey to ELs
6 Design, recruitment, delivery of depth interviews with schools
7 Analysis of available data and triangulation where possible
8 Report and review process - Draft report supplied on completion of EL fieldwork - Second draft supplied on completion of school fieldwork - Client review prior to supply of final report and summary
Fieldwork was completed between the 14th of September and the 9th of November 2020 with
ELs and schools. Given the challenges schools faced as they returned to school completing
fieldwork at this time with schools was difficult, despite a range of participation methods and
financial incentive being offered (£100 for a completed telephone interview and £40 for a
self-completed online survey4).
4 The Aspire to STEM Process Journey
Working with STEM Learning’s management team for this evaluation (Daniel Pledger, Erin
Gray and Wayne Jarvis), Skyblue’s evaluation team mapped out the process journey for the
Aspire to STEM Programme. Ten process steps were identified.
Step
1 Pre-recruitment processes/activity by STEM Learning
2 Educational Leads are recruited to the Programme
3 Schools are identified as eligible for the Programme
4 Recruitment of schools / Partnerships to the Programme
5 Partnership formation
6 School agrees to participation / engagement5
7 Schools complete a needs analysis (mandatory)
8 An action Plan is agreed with the Educational Lead for each school6
9 Schools agree on the activities for their Partnership (typically 3-10 schools)
10 Schools engage in the way that feels right for them; flexible CPD and support
4 No schools elected to complete the online survey despite the incentive in place. 5 Immediate access for schools to: STEM Learning network and resources; STEM ambassador volunteers; STEM Club infrastructure and community groups i.e. online forums on the STEM Learning website and the STEM Learning Resource eLibrary. 6 at which point several types of activities and resources become available to that school available as part of the intervention support package.
Page | 13
5 Senior Management Perspectives
Initial learning review with Aspire to STEM Senior Management
Facilitated by Skyblue over a series of ‘workshop’ virtual focus group calls in September
2020, the management team provided their personal experiences of, and insights about,
each process step. This helped to extract their tacit knowledge about the processes
involved, provide some working theories as to why certain processes may have well or not
so well, and informed the evaluation materials that would be designed.
Educational Leads background
- The Educational Leads recruited to the Programme were already known to STEM
Learning (SL) through their network activities. Almost all were very experienced
consultants and some Leads were part of SL’s internal Education Team. The focus
was around people SL knew that could do the work due to their experience and
where in the country they were based – so they were familiar with the local education
landscape.
- STEM Learning ran 4 Educational Lead ‘general guidance’ days and regular email
updates to support a consistency of knowledge and approach amongst the selected
Educational Leads.
- Educational Leads were contracted for approximately 15 days over 2 years with a
view to their role being ‘very much a touch base with schools every so often with
some “things to do” to move things forward.’ Proportionally therefore ELs spent very
little time with the schools and would also be working on other things, depending on
their background, for example, delivering CPD, triple science support as well as their
own consultancy and advisory work (if appropriate), teaching if they were Science
Learning Partnership (SLP) Leads delivering and developing other programme
materials if they were Education Team members internally. 7
- Most ELs will have had experience of, or have been able to support the setting up of
a STEM Club or more importantly know where to go for support if they didn’t.
Recruitment of schools to the Programme
Recruitment was described as complex during the scoping sessions.
- Some schools were approached directly by SL’s senior management – based on the
use of data downloaded for schools with Ofsted ratings and LAD (5/6). “This (cold
calling) was generally not successful.”
- Opportunity Area leads8 were engaged to recruit schools but this was described as
having “limited success due to the amount of support in these areas.”
- SLPs and ELs working in the areas also recruited schools. In some cases ELs were
approached ahead of being contracted to lead a group of schools that they pulled
together that met the criteria.
7 STEM Learning estimated that 9 ELs were part of an Science Learning Partnership, with 4 being SLP leads 8 These are the people associated with the Opportunity Areas and were often seconded from Local Authorities to the DfE teams working with the areas to improve the social mobility. They were not associated or known to STEM Learning prior to AtS and there is said to have been a retention issue of these staff within OA teams.
Page | 14
- Some contacts came as a result of previous engagement with STEM Learning
programmes and activities.
In all cases Head Teachers and Science leads were the main contact points for recruitment.
- “Uptake was variable – some responded immediately and actively helped to onboard
other local schools to them, others took ages to get recruited and in place. Typically it
probably took 2-3 months on average from initial contact to agreeing.”
- “Even the contract signing was fraught (with issues)”. The main issue was around
GDPR which emerged at the time some schools were engaging and Head Teachers
not wanting to sign over the rights to data.
- The other issue was around the pressure of having to start delivery to make progress
against DfE KPIs.
- “With the short lead in time we had to start delivery before contracts were signed and
all agreed this was the right thing to do. But having started the delivery, the lever to
get contracts back was considerably shortened.”
Partnership Formation
- Most initial contact between schools that were brought together in a Partnership was
via a network meeting initiated by the Educational Lead allocated to that Partnership.
- Following that network meeting emails were used to correspond with schools in each
Partnership.
- Many Partnerships used network meetings to inform the delivery plan at regular
points during delivery.
- Some Partnerships were brought together for the first time driven by the DfE funding
and eligibility criteria, whilst others had some prior existence and experience of
working together. This dynamic was expected to have a bearing on the likely success
(performance), or at least pace of progress, made by different Partnerships.
Access to resources on sign up to the Programme
- Schools simply needed to sign up to the STEM Learning website if they did not
already have an account and this gave them immediate access to the resources
contained therein.
Needs Analysis / Assessment and Action Planning with Schools and Partnerships
- The aim was for the Head Teacher and relevant subject leads in each school to
complete their needs analysis with the Educational Lead. This was considered by
STEM Learning to have taken place in the majority of cases, although a small
number were completed at partnership rather than school level.
- Two EL days were allocated for this process including needs analysis and action
planning.
- ELs did individual needs analysis with schools to ensure all needs were identified for
a programme to be put in place.
- This process didn’t always happen owing to:
o time pressures
Page | 15
o other non-STEM based priorities in school, meaning AtS was A ‘nice to have’
but not top priority (e.g. Ofsted identified a weakness in literacy across the
school and the schools needing to focus on rapid improvement in this area)
o ELs needing to move the Programme forward with more pace so in some
cases doing a Partnership wide needs analysis instead
- Where schools did not complete a needs analysis they were involved in meetings
around the content on the Partnership Delivery Plan instead.
- The needs analysis process was variable in terms of the time it took to complete. “In
schools where the relationships were already in place with us, this was probably
quicker to complete.”
Agreeing and prioritising Partnership activities
- Educational Leads generally used the intelligence they acquired through the earlier
process steps to draw up a plan that met the needs of each school as well as the
needs of a Partnership as a whole.
- “This was important in terms of delivery modality – i.e. sending one teacher on a local
course or running a bespoke, Partnership-wide session. This enabled efficiency in
delivery.”
- Educational Leads – rather than schools in a Partnership – led the process of
prioritising activities and allocating the available Partnership budget (c£25,000 for
each of the 40 Partnerships). Whilst this could be interpreted as being an approach
that isn’t particularly empowering for the participating schools, the management team
suggested that another explanation is important to consider given the context for the
schools that were taking part in AtS: “ELs did this because we did not want to burden
the schools with admin at any stage. The focus [for them] was school improvement
not admin tasks around the Programme.”
Accessing bespoke, flexible CPD and support
A range of CPD and support was available to schools on a customised basis to meet needs.
It was made available for free.
Asked if there were any regrets about making the training completely free, the scoping
session consensus was: “No, I don’t believe so – on the whole the participants really valued
the support received and comments have been received that without the funding they
wouldn’t have been able to access the CPD, held enrichment and community events, etc.”
- Suitable activities from the STEM Learning offer was identified, and then staff from
the school were required to book on to those activities
- CPD was booked by different staff in schools including teachers who would be
attending themselves, Heads of Department (HoDs) on behalf of their team or
finance or admin staff.
- In some Multi Academy Trusts (MATS) CPD was booked centrally by their CPD
Coordinator.
- CPD could be booked directly via the STEM Learning website if the CPD was
national or with a network hub (Science Learning Partnership) if the CPD was
regional or local
Page | 16
- Participants used a ‘discount’ code which meant fees for the selected were CPD
charged to the Programme and SL could track each school’s participation.
- Bespoke sessions put on specifically to meet a school or Partnership need were
generally set up on SL’s booking system as a bespoke AtS course so they could be
booked via the website and SL could track participation
- A small number of activities were organised locally, particularly bespoke coaching
sessions so school staff would liaise directly with the provider to schedule. In some
(but not all) cases the provider of additional bespoke coaching was the EL (i.e. where
the coaching need was something specific and went beyond the EL role, approved in
advance).
- There were inevitably CPD/course cancellations during the Programme, but the
management team did not believe this was due to any of the processes.
“Cancellations were due to busy schedules, changes in priorities, workload, capacity
and / or absence. The fact that it was free to the school may have meant that there
was no financial implication of cancelling. This is unique to AtS in our programmes –
we do have a cancellation policy on all our CPD, but this was generally waived for
AtS participants as long as there was some explanation for why participants were
unable to attend and no one else could attend in their place.”
- Asked why the different waiver for AtS the scoping session revealed some possible
learning for future programmes:
“In hindsight it may have been better to uphold our general cancellation policy so
there was some financial obligation from the participating schools. This would have
been useful where there was less buy in from school leadership or perceived value
attached to the Programme. The difficulty was that the support package was all free
– to then be landed with a bill for not attending risked damaging the longer-term
relationship with the school and experience of the programme.”
Sustained engagement
- The processes for agreeing the regularity or style of communication between ELs
and schools was left to the ELs to manage and feedback to SL as required.
Communication was generally email with Partnership network meeting touchpoints
as agreed. The processes for agreeing the regularity or style of communication
between ELs and schools in a Partnership as a whole was the same. -
- Network meetings were the main mechanism for encouraging the sharing of learning
across each Partnership. This is an important point only in so far as one of the
underlying theories of Aspire to STEM is that a successful Partnership is one that
‘uses a model where clusters of between 3-10 schools in the same
geographical area share resources and support to form a Partnership, in order
to build a sustainable community of practice’ in STEM, adapted to their local
needs and context’9.
- The processes whereby schools in their Partnership came together or worked
together, learn and made decisions were determined on a Partnership basis.
9 Study Plan for Aspire to STEM RAND Europe (page 4)
Page | 17
“Again network meetings were probably the main mechanism for this. Some bespoke
CPD with Partnership schools in attendance helped sharing of practice.”
Community engagement and enrichment activities
“The Partnerships were expected to include some community engagement activity – funding
was specifically ring-fenced for this. Because of the broad nature of what community
engagement could be it was left to the ELs to support schools / Partnerships to do this.”
- Careers guidance events for teachers were organised with support from the EL,
perhaps making introductions or suggesting activities or training to meet needs but
the events themselves would have been booked by schools. Time frames between a
school expressing interest and an event happening “will have varied.”
- For inspiration and enrichment activities ELs shared ideas of providers or types of
activities between each other, which were passed onto the schools as suggestions
but schools organised this by themselves on the whole (generally coordinated by a
key member of staff). They either organised an event which just involved their school
and families, or liaised directly with external providers, feeder primaries, schools in
their Partnership, uniformed groups / youth groups, etc to host events.
- In some cases, one school organised an activity or provider that then was replicated
and took place at all schools in the Partnership, or brought a number of Partnership
schools together at one location.
Key points from working with the Senior Management Team
The management team were able to describe each of the chronological processes
involved in an Aspire to STEM journey, and help the evaluators understand why and
how some of the processes differed for schools and Partnerships.
The drive for the Programme to be non-prescriptive, flexible and customised, may
have claimed primacy over other alternative programme design considerations such
as the need for consistent approaches in processes to encourage more uniform
outcomes.
The role of the Educational Lead appears to be fundamental to the journey, given
that they have led schools through needs analysis and action planning processes,
provided ideas, suggestions and support in a customised fashion, and led on
decision making and prioritization of Partnership Delivery Plan activities and budget
deployment.
There is perhaps a slight disconnect between AtS’ desire to minimize administrative
burden and the role they necessarily had to take in organising and booking CPD,
community engagement and enrichment activities.
Whilst free CPD is seen to have been an important lever to engage schools, there is
some doubt that applying a unique cancellation waiver to this Programme (unlike any
other STEM Learning programmes) has been an appropriate ingredient.
Page | 18
6 Classifying Partnerships
Skyblue facilitated discussions to help identify the management team’s views on how best to
classify the 40 Partnerships. Consequently, a subjective classification was agreed as
illustrated in the diagram.
This was a useful exercise in so far as it:
Enabled the AtS senior managers to identify and express their own thoughts about
which, and why, Partnerships differed in their performance over the Programme
period as defined
Supported the evaluation sampling methodology for selecting Partnerships to include
in the process evaluation allied to a number of other agreed variables (please see
appendices)
Enabled us to use this classification in the self-completion survey with Educational
Leads so that the learning lessons they shared with us could be contextualized within
their perceived assessment of each Partnership’s performance
Could be used to triangulate the views of the STEM Learning Centre senior
management team and ELs from a sample of the Partnerships.
Higher Performing
Very successful, meeting if not
exceeding your expectations and
offering some exemplary practice
Average Performing
Performed ‘OK’, it did fulfil AtS
requirements. Could learn from higher performing
Partnerships.
Aspire to STEM
Partnership Categories
Lower Performing
Disappointing compared to its
potential and for a range of reasons was not especially
successful
Page | 19
7 Preliminary partial process evaluation results
Research completed for STEM Learning by RAND Europe for December 2019 identified
some preliminary learning as follows, some of which resonate with the learning shared by
the management team (seen in section 4 of this report):
This new (later) process evaluation by Skyblue affirms and extends the initial findings from
this earlier RAND study – except, as noted earlier for the finding on accessing specific CPD,
which was not repeated in our work. Further though, this more recent process evaluation
conducted 6 months after the AtS Programme finished enables a more detailed reflection by
ELs and sampled schools about what worked well or not so well to inform future STEM
Learning programme design and incremental improvement.
Page | 20
In order to preserve their anonymity each school is referred to as ‘school 1’ or ‘school 2’
throughout the narrative.
8 Key Findings
8.1 Partnerships and Partnership working - including school
recruitment
In this section we examine the following topics:
How to establish relationships with schools and build trust
How to establish an aim for Partnerships – strategies for understanding the needs of Partnership schools and how to capture this e.g. doing a survey, data, observations, conversations with HoDs
Recruitment of schools.
Context
STEM Learning have delivered school-centred Partnerships for a number of years with great
success. Using this model, and acknowledging potential concerns due to the nature of the
target audience, each Partnership is allocated a dedicated Educational Lead. Both the
schools and Educational Leads have access to a number of support and assistance
services.
School characteristics
3 of the 5 schools interviewed were part of a Multi-chain Academy Trust (MAT)
2 out of the 5 schools interviewed had previously worked together prior to Aspire to STEM. For example, school 2 is part of “5 schools (all part of [Trust name) since 2017, but we have worked together since 2013. Rural schools need to work together to access opportunities.”
School 1 was already part of a Multi-Academy Trust, although two, not all, of the schools
from the MAT were involved in Aspire to STEM as they did not fit the eligibility criteria. As a
result, these Trust schools also engaged with other secondary schools in the town.
“[school] is an associate school, so it has links with the [location] MAT. As a result of this programme we now share more information with [location] - they are part of our shared knowledge hub.”
School 3 did not have previous links with the other secondary schools involved in their Partnership. In this average-performing Partnership, engagement from the schools was mixed:
“It felt very mixed - the Head here is very supportive. Some of the other schools weren't as fully engaged. For example, we'd arranged a knowledge swapping session, and they didn't turn up.”
Schools 4 and 5 did not have connections to others in their Partnerships, and welcomed the opportunity.
“No, there was no chance of us working together - but now we have formed friendships.”
Page | 21
Experiences of working together varied. A consistent theme however was that when one or more schools were unable to attend a meeting, then the others would continue rather than reschedule.
“[School] were going through an Ofsted, then had a change of both school leadership and department leaders. They missed meetings, but we took the view that 4 out of the 5 would press on.”
School 5 described itself as “struggling on all fronts” as Aspire to STEM began. Consequently, the importance of working with the other schools was a secondary consideration.
“The meetings were hard because nothing overlapped between the schools. Others were looking at transition or other narrow points of focus. We did take some ideas away e.g. around using stories. We all talked about our needs - then everything was tailored within a flexible framework.”
Key Learning
Schools already part of Multi Academy Trusts were already further advanced with
Partnership working, but this did not necessarily filter down to networks at
department level.
Rural schools feel they need to work harder and together to access the same
opportunities that peers in urban locations routinely benefit from.
Schools noted the following vital ingredients to working in Partnerships that have the ability
to develop trust over time.
The extent to which schools joining a Partnership have common or divergent priorities
(including STEM, but more widely) is likely to be a factor in the extent to which schools take
up and sustain their involvement in a discretionary programme like Aspire to STEM. Two of
the four schools considered that their needs were shared.
“We are all in the same town, so it makes sense for us to share learning and
resources.”
“Communication is key. Secondary
schools that already collaborate together -with primary schools.”
“Strong leadership -STEM leads who
have the passion for the subject.”
“Honesty at meetings.”
“Educational Leads to share practice.”
“High quality mentors and trainers.”
Page | 22
School 2 said that rural and urban schools might share similar ambitions but were starting
from very different places on every level.
School 3 made the observation that differences not only affected how schools related to one
another, but also departments within schools.
“We were in very different positions as schools. Different departments were also in
different positions within schools. For example one science department was strong,
but the technology department kept going back from meetings and not getting the
support internally. Some schools were more open to learning and improving than
others, depending on the culture and the wider challenges they were going through.”
Recruitment challenges
86% (12 of 14 Educational Leads) said the way schools were recruited had
either ‘a lot’ or ‘some’ impact on the Educational Lead’s ability to subsequently
build trust
3 out of the 5 schools would prefer to work with a slightly different set of
schools for future programmes.
Key Learning
Schools that had been told to take part in the Programme took longer to engage and
build trust than those that had opted in.
Schools that found it harder to engage in the Partnerships could still engage in the
Programme on different terms.
Starting at the beginning of the Aspire to STEM process journey, and taking a ‘what works’
approach, Educational Leads pointed to the following tips for recruiting schools. The most
commonly identified factor was seeking out schools that already had a history of working
together, with Multi Academy Trust that clearest example.
“It is unfair to compare all Aspire to STEM Partnerships - those that were based in existing MATs were always going to find this much easier.”
Top tips
Do differently?
“Targeting MATs as an existing school
partner group” “Existing networks that collaborate and
then bring in new partners to build the reach.”
Using a ‘snowball’ approach “Asking Head teachers recruited to get in touch with others”
“Use personal knowledge of the schools” (and connections)
“Consider geographical location.”
Avoid presuming all trust schools will want to join and act ‘as one’ “It felt like the schools had had this forced on them by the trust without any real consultation. Some schools could not commit the time needed and as a consequence withdrew from the programme.”
Less chasing of unresponsive schools
Avoid programme fatigue – “Not tried to re-engage schools from previous projects such as 'priority primary' schools”
Emphasise the importance of collaboration between schools: “STEM Learning needs to ensure that there is a genuine desire to collaborate across the Partnership”
Page | 23
Timing is such an important factor to gain traction and build strategy
Feedback from schools and Educational Leads suggest that it is also important to consider
the state of readiness of a school to take on additional opportunities including partnership
working. To establish this, the early engagement of the Head and senior teams was deemed
to be essential.
“There has to be the willingness to trust, and collaborate with each other; without that
you don't really have a Partnership.”
“Getting the Head Teacher’s buy in at the start was the key thing for us.”
School 2 joined Aspire to STEM having completed a major restructure of their curriculum. If
the Programme had been pitched to them while this process was taking place, the
conditions would not have been in place to embrace the opportunity.
“The timing of ATS was a key factor in that we were ready for it. We had just
completed a major overhaul of the curriculum in the two years before ATS, when we
would not have been able to take advantage to the same extent.”
School 5 had just promoted an existing member of staff to be Head of Science. The external
support she received as she established herself in this new role was critical.
“We were struggling on all fronts - the Programme was just what we needed. There
isn't anywhere else I could have got this support. Every time [mentor] walked in the
door, I knew that it meant loads of work for me. I'd see other schools talking about
practice that we couldn't get near - I felt quite demoralised…”
In the second example below, School 4 is taking this learning forward into their 2020
ENTHUSE Partnership.
“This time we have got the other Heads on board early. AtS worked here because the
school was ready. The timing was right - we'd focused before this on sport and
drama. It came from listening to our pupils. I’ve realised that I’m not getting to all of
them - and in primary we need to find everybody's thing. They told me they loved
building and making things, and it began there.”
Educational Leads identified the following additional challenges to encouraging partnerships
in the context of a programme such as Aspire to STEM. These are arguably preconditions
that need to be in place for a programme such as AtS to thrive to its optimum.
Schools may already be receiving support from multiple actors, so need to have
sufficient leadership capacity to take on new programmes and see them through to
completion.
Teachers are empowered to take part in CPD away from the classroom
Staff turnover is stable or improving
The case can be made that improving STEM is an important aim in its own right, and
may not directly be picked up by the Ofsted inspectors.
“Many already felt they were at capacity and didn’t see engagement with this
Programme was necessarily going to improve the Ofsted rating.”
To assess the programme’s potential in terms of value, rather than expenditure
Page | 24
“The cover was a cost that SMT moaned about, but I explained that we got far more in
return. In the end, we covered absence from within the department, which also meant
that teaching continued.”
Top tips from Educational Leads
“{Focusing a programme on] fewer fully committed schools who are made fully aware
of the time needed would work better than more schools who just sign up and hope
for the best.”
“Coasting schools would likely have more capacity to respond to a programme like
Aspire to STEM, compared with schools facing more immediate challenges.”
“With hindsight I would engage the schools even earlier. It takes time to start those
conversations. More time upfront too getting into the schools more. Otherwise you
have to take needs on face value – but with greater knowledge you can explore and
challenge more. The need to get buy in often leads you to say yes.”
“Need schools that naturally collaborate with themselves - that works best.” (school
response)
To be eligible for support through the Aspire to STEM Programme (TLIF funding), schools
needed to be either Ofsted rated ‘Requires Improvement’ or ‘Inadequate’ (grade 3 or 4) or
located in one of the 12 areas designated as Opportunity Areas (OAs) or in Local Authority
Districts (LADs) which were rated lowest in England (identified as LADs 5 or 6). ELs
reflected that there are a number of challenges to supporting these kind of schools.
“How do they share best practice if they are in category 3?”
Operating in areas facing multiple disadvantage posed additional challenges when seeking
to recruit schools to the Programme too. Most commonly identified was that schools facing
‘myriad issues’ were likely to already be receiving support from other sources, so have
limited capacity to take on new programmes, particularly in areas where the direct link to
improved Ofsted results is lacking.
“Primary schools have other priorities, and access to other support for those areas.
They haven't got time or supply budget to do everything and unfortunately Literacy
and Maths tend to come first.”
“Capacity, where schools are deeply struggling, they don't have capacity to look
deeply at science. Lack of staff meeting time, resourcing, and a leader to be
strategic. Too busy fighting fires”
The quality and stability of the leadership team (for example whether the Head is in an
interim role) provide clues to the likely value the school will gain from being involved, at that
particular stage of their journey.
“Poor leadership is often a characteristic of an Ofsted category 3 or 4 school and
good leadership is needed for successful Partnerships.”
Key learning
Feedback from lower performing Partnerships may point to the need for greater due
diligence by STEM Learning before signing them up to a programme. Their state of
‘readiness’ is something to define further for future programmes.
Page | 25
Partnership challenges
All 5 schools agreed that a lack of buy in (either at an early stage or following the departure
of a Head) was the most demotivating factor in working in Partnership. It was also apparent
if staff, regardless of their position had been told to attend, rather than choosing to come.
Top tips
Keep Heads copied in and share the accountability
Have a Head attend the network meetings to keep them on track
While still considering their Partnerships to be successful overall, the examples from schools
(below) point to challenges and areas where performance was still sub-optimal.
These challenges were attributed to a lack of delegated authority to the Head of science,
which resulted in challenges, together with, or perhaps the result of, wider cultural issues
within schools that were less engaged. A lack of senior level buy in was the likely main
cause, with SMT too removed from the Programme to ‘see it other than a budget line’.
Examples from schools in average-performing Partnerships
“One school in the Partnership did not use resources e.g. funds allocated for community events. The Head of science in one school wasn't allowed to let a group of us in to do a short learning walk to observe teaching - the staff were up in arms, rather than seeing it as a chance to get constructive feedback. This Head of science was also unable to make decisions without getting sign off.” “I came in at the back end of the Programme. The Educational Lead is a terrier at keeping us going. XXX (Primary Lead) kept us training locally. We had a STEM with Maths training session that was so well received I signed up whole staff for a 4-week block. But it was not so good - but I spoke up and to their credit they then improved it in weeks 2, 3 and 4.”
In addition, School 2 highlighted how schools within a Partnership can be in very different
financial positions, owing to the number of Pupil Premium Pupils within their schools.
“We come from very different perspectives - the disparity in budgets and resource
across schools can be demotivating.”
School 4 noted the challenges in sharing good practice amongst Partnership members.
“While the Programme worked well when staff went to York, we needed to work
better as a Partnership - training wasn't always disseminated effectively between us,
but that was down to us [not the Educational Lead].”
Key learning
There is financial disparity between schools in a Partnership despite them being
located in an opportunity area; this appears to affect motivation levels within a
Partnership context.
Page | 26
Prevailing models of power, authority and structure
A challenge identified by the Educational Leads in positioning AtS within target schools, but
not unique to the schools or areas that were the focus for this Programme, was delivering
training and support into a MAT which are larger structures that devolve less responsibility to
individual schools.
“In the case of MATs, leadership and staff development is often a top down model -
and any Partnership work has to fit in with their existing schedules”
The additional challenges posed working with ‘requires improvement’ or ‘inadequate’
schools relate to capacity issues at SLT, or for one of the Partnership schools to take more
of a leadership role.
More fundamentally, some schools were not (yet) ready to work with others:-
“These schools are struggling to work on an individual basis. Working as a
Partnership is a step too far.”
There may be additional challenges in communication within schools which can slow the
process, further disrupted by staff turnover.
“Even neighbouring schools (especially secondary) may compete. There can also be
different attitudes towards cluster working between groups of senior leaders, and
groups of middle leaders who tend to end up 'doing the work' despite a lack of
consultation in the initial stages.”
Key Learning
The Educational Lead was key to knowing the school’s improvement context in order
to identify high impact leverage activities.
Early meetings helped to make schools more amenable to the Programme.
Partnership working is by no means the norm or natural state for schools as they
focus on urgent, inward-looking priorities linked to inspections and improvement.
In areas where schools were working together for the first time, there was a recognition that
the pace of gaining trust and developing the collaboration would be slower.
“Many schools were not previously working in Partnerships and so relationships were
slow to establish - in fact to be honest, the collaboration was at best very limited
throughout the Partnership - this may have been compounded in this particular
Partnership by geography and distance between schools.”
Page | 27
Planning Processes
Following recruitment and establishing the conditions for effective Partnership working, our
focus now turns to the planning stages of the Aspire to STEM project.
Context
Individual schools undergo a needs analysis with the help of the STEM Learning educational
lead in order to identify their bespoke support needs. Following this, Partnership leads agree
on an action plan for the Partnership. Resulting in a combination of individual school and a
wider Partnership support package. It is envisaged that each Partnership (40 in total) has a
focus specific to its STEM-related needs.
This section explores the learning from the planning processes, which were most
pronounced at the start of the Programme, then typically updated termly or even half termly
at Partnership network meetings.
School perspectives
3 of the 5 schools considered the aims of Aspire to STEM to be a good match
for their school, while the remaining two said ‘a fair match’
Schools were asked to reflect on how well matched the four aims of Aspire to STEM were to
their specific context. They shared a desire to focus on STEM leadership and STEM
teaching, while ‘the extent of fit’ the outcomes relating to science capital and STEM careers
was less obvious.
Reporting by exception, the planning processes were not discussed at length within the
school interviews. The opportunity to spend time with the Head teacher was highlighted as a
positive, but the process was ‘fine’. The learning was seen to be in the extent to which the
plans were updated.
With hindsight, the planning process did leave gaps:
“It would have been lovely as part of the CPD to have seen more of STEM happening
with children – so we could then model. This was not really part of the Partnership
plan. Teachers learn and respond to training differently to children”
“Most of the time was spent with the Head of science. If we did it again now I’d get
the other staff involved directly.”
Page | 28
Educational Lead Perspectives
71% (10 of 14 Educational Leads) agreed that the success of a Partnership was
in part determined by whether schools had common or divergent needs and
priorities.
Thirteen of 14 Educational Leads (93%) considered the needs analysis and
planning to have a lot or some impact on the success of the Partnership.
Nine of 14 (64%) considered the needs analysis and planning to have a lot or
some impact on the school’s ability to build trust with one another and the
Educational Lead.
Nearly two thirds (64%) of the Educational Leads had spent either a little (6) or
significantly more (3) than the 15 days allocated per Partnership10.
Analysis of the number of open responses provided by Educational Leads to the questions
‘what worked well about ‘x’ and ‘what would you do differently?’ is illustrated below. It shows
an interesting pattern as the Programme moves from a focus on individual schools to a
Partnership wide view. The number of examples of what went well decreases while the
learning lessons for the future increase.
Educational Leads fed back their ‘top tips’ on what they had learned about overcoming the
challenges of being so flexible, specifically, how to successfully plan a programme of activity,
pathways and packages of support to meet the identified needs of schools and their
Partnerships.
10 It was noted that in addition to the Educational Lead responsibilities to co-ordinate, facilitate, organise, and support, some were also involved in delivering training and support as part of this package of days.
10
9
6
4
10
7
0
2
4
6
8
10
12
The process of producing anindividual school action plan
The process of needs analysis /assessment
The process of developing apartnership delivery plan
Planning Strengths and Opportunity to do differently
What worked really well? Do differently?
Page | 29
Top tips
“Keep it simple, do what you can with what you've got, work closely with SLT”
Initial individual face to face needs analysis is important - helps to build trust and
ensures that the delivery plan properly reflects the needs of the school. This is also
essential for the buy in of the schools.
Scheduling a clear programme of courses that takes place at the schools is important
for keeping momentum going
“Schools need support turning a plan into booked-on places for teachers on CPD”
“Ensure the WIFM11 is clear… Clear communication with the schools so they
understand why packages have been put together and how it benefits them.”
Consider working on a shorter planning period, 6-8 months ahead rather than 24
months, especially on longer duration programmes, to ‘bake in’ the ability to respond
to the changes that are likely to take place within schools over a multi-year period.
Flexible deadlines
Quick wins “Definitely going along with a core of CPD suggestions and a list with
descriptions, getting a whole staff CPD in early to establish good communication and
then being adaptable to change the Programme as the school begins to understand
how things work.”
Ask schools to lead on agreeing shared CPD dates
The value of action planning
In this section we examine the following topics:
Creating an action plan – the process, what can help, how can it support success?
The process of reviewing action plans and reflecting on needs and progress made Schools valued the iterative nature of the Partnership Plan, and the availability and responsiveness of the Educational Lead throughout the journey.
“[The needs assessment and action planning] was excellent - this gave me data and
access to the Head Teacher.”
Feedback from Educational Leads highlighted the iterative nature of effective action planning – typically looking 1 to 1.5 terms ahead in order to respond to the changing needs emerging in schools as they progressed.
“It’s like getting a menu, it’s hard to know what the issue is initially. You think it’s
about (a) then realise actually we need (b) and (c). So, the plan was adjusted.”
Educational Leads described the importance of being prepared to adapt the plan.
“The action plans were used at the start. After that we met as a cluster and asked
‘What’s going on?’ ‘What do you want to work on as a group?’ and I had the action
plan document as a reference.”
11 ‘What’s in it for me?’
Page | 30
Building in time to reflect was also considered an important part of the planning process:
“At a network meeting in July 2019, schools were asked to come with their own
presentation and notes on what has happened. One of the Heads attended – then
she was evangelical about the Programme. It was fascinating to listen to how the
Programme’s impacts had gone further, to students, changing the whole atmosphere
for science around the school.”
“Reflection is definitely taking place. The case studies were excellent, and they had
stories to tell.”
Key Learning
The processes of needs analysis and action planning appear to be important
components of a programme such as AtS. It is appreciated by schools but they rarely
lead the process, it needs driving by the Educational Lead.
The action plan is most appreciated if it is iteratively created and adaptable to reflect
the changing circumstances in each school – especially if personnel change during
the Programme’s lifetime.
Creating opportunities for these needs and plans to be shared as a group can lift their
profile and importance especially if school representatives present them to one
another in a setting with peers in their Partnership.
93% of Educational Leads suggest that these particular planning processes correlate
with the ability for a Partnership to achieve success.
Page | 31
8.2 The role and impact of a customisable programme and bespoke
CPD
In this section we examine the following topics:
How to plan a programme of activity to support schools / Partnerships with ‘X’ identified need - suggested pathways of CPD and packages of support.
How to organise CPD to allow different schools to attend and meet local needs.
The value and effectiveness of internal CPD. What worked? What didn’t?
Organising CPD attended by different schools
Educational Leads identified various factors that would encourage positive outcomes from a
customised CPD and support package, stressing the importance of:
Location (where CPD could be accessed)
Proactive and persistent administration
The optimal timing of CPD to suit school preferences.
Looking forward offering remote / virtual meetings was seen to be a positive development in
any future programme support package offered by STEM Learning.
Top tips
“Set a central location (within reach of the others) or rotate around venues.”
“Arrange dates at network meetings and go with the majority, when others see the
success, they will make changes to be involved.”
“Get staff to book on courses in meetings”
“Mailing out early and then sending reminders and calling subject leads.”
“We discovered afternoon into twilight sessions worked best. No one is released in
the morning.”
“Schools did not have the budget for supply staff” and impractical for one or two
lessons out of a day.”
“Be prepared to change the plan”
“Clear plan of local courses that reflect the needs analysis is essential.”
“Move to more remote offer will be very helpful.
Work closely with SLT - does it meet their needs too?”
Page | 32
Most valued aspects of the support package by schools
Schools were asked to consider the aspects of the Programme that they felt they had made
the most progress in. STEM leadership and teaching ranked equal first.
Interestingly, when asked to consider ‘what if your school hadn’t got involved at all, how
much progress do you feel you would have made, regardless of Aspire to STEM?’ the
pyramid above is inverted. Schools considered they would have made the least progress in
increasing teacher confidence about STEM careers, followed by (joint) increasing science
capital and developing great STEM teaching. Improving leadership to support STEM
subjects, identified elsewhere as perhaps the key factor in effective, sustained engagement
with a programme like Aspire to STEM, was the aspect schools felt they would have made
most progress with anyway.12
Context is important here:
School 1 noted that the aims relating to STEM capital and STEM careers were not the main
priorities at the start of the Programme, but nevertheless there had been gains and this was
now an area being actively pursued:
“Leadership capacity has really improved. I don't need to drive the agenda now, the
schools are doing that. Aspirations had taken a back seat, but now we more aware of
this- and they now feature in schemes of work across the MAT - but there is more to
do. Careers are too often an afterthought but should be embedded. We have more
trips now, at a recent one, we had a young girl from a very disadvantaged area say
afterwards that she wants to be a medical physicist which was really fantastic.”
12 Please treat this finding with caution, as it is based on a small sample of five schools.
Most progress achieved
Improve leadership to support STEM teaching
Develop great teaching of STEM subjects
Increase science capital within disadvantaged communities
Increase teachers’ confidence in advising
pupils about STEM careers and the transition from school or college to
employment
Page | 33
The senior leader from school 1 was also responsible for 2 other schools, 3 in total involved
with Aspire to STEM.
My response is nuanced. XXX [school 1] was probably less successful due to the
timing of a maternity leave - so they had less engagement than XXX [school 2] y or
XXX [school 3]. The Programme forms part of an overall plan across the MAT to
improve leadership and teaching. Results in English and Maths have improved
significantly.
The schools have a high percentage of Pupil Premium pupils - which can make
science capital work challenging. However, this is now more of a focus - and the
presence of Gatsby benchmarks etc. on schemes of work are to a large extent due to
Aspire to STEM. The whole Programme has been great, it got all the teams
involved.”
School 2 (a primary) could point to the measurable gains for the Head of STEM, which has
in turn supported an increased STEM profile:
“The biggest benefits from the Programme are to the STEM leader, particularly her
understanding of the subject and ability to then support other staff. She felt
empowered to talk to staff - it was a lot of work (she works 3 days a week) but she
managed, and she was given plenty of management time (I am a teaching Head, so
covered). We can see how the profile of science is increasing as our new NQT
already received science training. The profile of STEM is higher - pupils receive 2hrs
of science per week. We came across teachers with passion and that made a mark
on [STEM Leader]. In our school, science takes on the same level of importance as
English and Maths -the last 18 months has given a real focus for this. We look to
build it into our curriculum, for example we have a new forest schools programme
running.
One example that is important for me is that we now call things by the right names-
science is science! (not lost under 'projects'). Part of a group of 14 schools across
[the county] - working together across all these schools. We all have access to Trust
CPD - so the online resources were useful for talking to other staff. It’s lovely to share
resources and ideas, for example STEM and maths leads sharing ideas and driving
the agenda in multiple schools. D's passion was wonderful - we all got the
information from the Programme. As a teaching Head I took over the cover. HLTAs
also helped.”13
School 3 was attracted to the Programme by the CPD opportunities:
“We were looking for opportunities for CPD for staff and the best learning
experiences for students - so the quality of teaching and learning was our uppermost
priority. We didn't do much about careers before, but we do now include links to
relevant careers in our schemes of work. We now have a Gatsby slide for every
lesson.”
“We identified student literacy to access exams as a weakness and training was then
put in place.”
13 This is triangulated in the school action plan: ‘Teachers say that 'Inspire to STEM training has improved their subject knowledge, supported their planning for science lessons and signposted to useful resources.'
Page | 34
School 4 singled out the quality of the CPD delivered at the STEM Learning Centre in York.
“York courses were wonderful. Fantastic facilities and dialogue between colleagues.
Came back empowered - and ready to teach- knew more and access to great
resources. Two years on we have a brand new curriculum in place (we’ve gone back
to national curriculum). We didn't probably teach science properly, it got lumped into
‘topics’ in the afternoon. They wouldn't have known they were doing science.”
STEM teaching is working well in Years 3-6, but the ‘STEM way of working’ is still
developing in KS1. For me, this means building and making alongside gaining
knowledge and using equipment. We do have a focus on literacy - our pupils still
have progress to make here. The difference is now, we've flipped it on its Head, so
science is a way into writing. We write about science more than we ever did before. I
don't know how the staff do it sometimes, but they link it up to the necessary parts of
the curriculum.
Finally, school 5 highlighted the quality of the mentor assigned by the Educational Lead.
“Quality of the mentors – [A] was utterly amazing. [B] kept us on track. It had a
periodicity to it, regular meetings and a consistent point of contact. However, I think
once [B] was introduced we would have got as much out of it without [A] to be
honest.”
Here too, the profile of science was both higher and more positive than before.
“The profile of science has completely changed. Before it was always something
science hadn't done, but now there are things we are leading the school on.”
Schools all agreed that Aspire to STEM felt customised to their needs.
“It could feel generic but we really felt that our needs were being used to shape the
training”
“Anything I wanted we were able to have.”
The degree to which schools attached importance to this customisation varied, but it was
valued:
“The more targeted to a school's needs the better, but we also have to be realistic
about the availability and cost of that kind of support.”
“It made it easy for us and other schools to stay involved. The EL really listened and
tried to respond.”
“This was a wonderful aspect of the Programme.”
Key learning:
The skill of the Education Lead to respond with specific CPD, tailored to a school’s
needs was highly valued. Moreover, if this CPD could be opened up to other schools
there was scope to gain further outcomes. (perhaps from a school’s perspective
unexpected)
Page | 35
Educational Leads were also asked which aspects they felt were most highly valued by the
schools and Partnerships they supported, and in particular what Aspire to STEM
Partnerships offered schools over and above their existing networks (for example with Multi-
Academy Trusts).
Their responses accentuate the pedagogical aspects of Aspire to STEM, particularly the
CPD for middle leaders.
The bespoke, rather than generic offer was seen to be more valued generally.
“The middle leadership CPD because they gained new knowledge and skills and had
an opportunity to discuss these with other science leads and put plans into practice
via gap tasks
“Bespoke CPD, DT and CS support as well as science, face to face department
sessions.”
Lesson observations and feedback led, in time to change at school 5.
“We had practice that wasn't consistent. [Y] pulled no punches. I described it to the
team beforehand that it was going to be like a spa day, where we could sit back and
reflect. Instead they said later it was more like going for a Thai massage, you left
feeling bruised and beaten up. When getting feedback, my team were initially very
negative and defensive. Then they'd reflect and soften, then they'd change. This
feedback had to come from someone other than me.”
Practical training was well received, and more easily ported back into both the staff room
and the classroom.
“The practical shared CPD, whether in science or subject leadership because it
inspired and built a shared understanding and confidence amongst the staff. Being
ab le to share CPD and talk about what went well with schools outside the rather
'controlled' environment of the MAT.”
By contrast, one Educational Lead reflected that:
“Known entities were a safer bet e.g. conference was well attended. However, a new
initiative – bite sized physics (6 sessions during school time) was less well attended
as staff couldn’t come to them all.”
Combination of offers
It is interesting to observe how Educational Leads selected a combination of offers, which
were seen to work together to good effect.
“Residential CPD backed up with coordinated in-school support.”
“In-school support/CPD/consultancy; AND money to spend on activities they simply
would not have been able to undertake without Aspire to STEM”
This last comment about funding was affirmed by interviews with schools (March 2020)
where the availability of funding was an important factor in being able to engage.
Bringing schools together
Educational Leads also highlighted aspects of the Programme that enabled schools to come
together, to train, or to engage with the wider community (of both schools and parents).
Page | 36
“Ability of other schools outside the Aspire to STEM network to join CPD activities;
community events were hugely valued.”
“Working together on similar events/ Science week and CPD.”
“The conference that was organised was really outstanding.”
The Programme created the conditions where schools felt confident enough to start (or re-
start) STEM clubs, or in the example below, to connect up technicians into a network that did
not previously exist.
“The technician network didn't exist and was a real need across the Partnership and
beyond, sustainable and supported the work of the SLT too.”
Learning points around offering generic CPD were to simplify the offer and ensure STEM
leads were involved in the selection of training.
“Have a simple 'set menu' of CPD opportunities at the start.”
Less valued aspects of the support package by schools
By contrast, aspects of the support package offered to schools that involved travel and
overnight stays were more challenging to fill. Schools that did take up this option were very
positive about the impact attending had on participants.
It is interesting to note there was divergent experiences for different STEM departments
within schools – with Science and Maths receiving more attention.
“The ‘T’(echnology) and ‘E’(ngineering) of STEM - most of the focus from leadership
teams and development plans was on S(cience) and M(aths).”
In some schools, it proved challenging to divert SLT focus away from the subjects that
Ofsted inspectors look at first, although comments later suggest that changes in the
inspection framework should reduce this challenge in future programmes.
“This may be as a result of their current position - rarely would an Ofsted report
mention STEM - issues would be around individual subjects. SLTs generally wanted
things to get better but didn’t create the time to engage themselves and often didn't
see that they could be part of the solution to move a department forward.”
“I think perhaps now that the Ofsted framework is different some attitudes will have
changed. I wonder how we get Heads more accountable in processes such as ATS?”
For one Educational Lead, selling leadership packages ‘at a level higher than the subject
leader’ proved difficult in a low performing Partnership.
Again, although by no means a uniform experience, the (arguably) longer term ambitions to
build science capital and improved Careers Education, Information, Advice and Guidance
systems were less of a priority than the focus on STEM leadership and teaching.
“STEM clubs, STEM ambassadors - these were all seen as add-ons and schools
didn't have time to engage with them.”
“We have set a KPI for a member of the team to look at STEM ambassadors.
Building science capital is achievable, even in a struggling school, and we now
include links to careers in our schemes of work.”
Page | 37
Key learning:
Training that was local and tailored was an easier sell internally for schools than
remote support that appeared more generic. Similarly ‘tried and tested’ offers,
especially when scheduled after school were an easier proposition compared to new
approaches that took staff out of the school day over a number of weeks.
Encouraging schools and Partnerships to engage with Aspire to STEM
4 out of 5 said their expectations had been either fully met, mostly met or
exceeded.
3 of the 5 said they had ‘fully’ made the most of the Aspire to STEM
opportunity, while 2 said ‘to some extent’.
The figure on the next page shows how Educational Leads were most likely to shine a light
on the bespoke CPD (to both leaders and teachers) and the community engagement and
enrichment activities. There were learning points identified across all of the processes, with
the greatest number around the provision of generic CPD to STEM teachers.
7
10
3
5
6
5
4
3
7
6
4
2
4
2
4
2
4 4
0
2
4
6
8
10
12
AtS processes: Successes and Opportunities to do differently
What worked really well? Do differently?
Page | 38
Key Learning
The ability to combine different CPD to meet school needs is valued. Bespoke CPD is
perhaps unsurprisingly especially appreciated. For Education Leads, there was most
learning in the delivery of generic CPD for STEM teachers, compared to the other
programme elements. It was more difficult for staff from smaller STEM departments to
access training directly, given the greater priority accorded to Maths and Science.
Success was more likely to be realised by working closely with STEM leaders (also
perhaps closer to the skill sets of the ELs), who were then empowered to support
their teams.
There were only a few ingredients that were less valued that offer some potential for
rationalisation in future. Having a unique cancellation waiver for this Programme may
have been unnecessary.
8.3 Role of Educational Leads to support Partnership activity
“The Educational Lead was fundamental, we would not have sustained the
Partnership without her.”
This section examines how to:
Build positive relationships with Partnerships and schools.
Set up and run a good networking meeting-what does a good meeting look like?
Keep the momentum going – strategies to maintain continued engagement in the programme over time
Keep focus – how to support schools to maintain their focus on the target cohort.
Share good practise between schools – how to facilitate schools effectively sharing impact of CPD with each other.
The Educational Lead role had different facets, for example: consulting; assessing then
diagnosing needs and priorities; building relationships and trust; leading when required;
making decisions; having a strategic view for a Partnership; facilitating conversations;
sharing learning; brokering solutions; administrating; facilitating Partnership network
meetings.
Schools all praised the Educational Lead – highlighting their availability, responsiveness and
flexibility. The tone and frequency of communication (a mix of face to face, telephone, text
and email) met the schools’ requirements, and senior figures appreciated being reminded
and prompted – which served to maintain the Programme’s impetus.
“[Education Lead] was an absolute credit to the programme…She was the key. Her
communication was first class and her targeted support really helped us to get the
best out of the programme, despite the different pressures we were facing.”
“The Educational Lead was absolutely lovely; she was available and that was the big
thing. Always well prepared and responsive to emails and texts.”
“[Education Lead’s] involvement was brilliant, she made us aware of all the
opportunities. She looked at alternative ways of delivering sessions so we could
spread the benefit wider.”
Page | 39
“[Education Lead] was a terrier at keeping us on track. I really valued his seriousness
(so unlike me) and passion for the subject. He kept us on track.”
It is noteworthy that 4 of the 5 schools could not identify an improvement for the Educational
Lead to consider for future programmes. School 3 made the single suggestion below:
Top tip
“Meet the middle leaders earlier in the process to get them onboard.”
To put this suggestion in context, this senior leader described how he was a number of
priorities and could not always respond as quickly as he would have liked.
“There was a dilution effect, through me, rather than a direct voice to department
Heads. As lead contact last year, I was stretched across my commitments. We all
have so many plates to spin - and colleagues need to prioritise.”
Key Learning
It was natural for ELs to target their support towards senior staff, where the
opportunity to positively influence departments was greatest. However, this created a
vulnerability in that the programme then relied upon the SMT member / department
lead having the time and influence to cascade knowledge internally. Ideally, in the
words of one EL, future programmes should “ensure that the Partnership works not
only with subject leaders but all staff within the schools and the leadership teams.”
Page | 40
Educational Leads
As set out in figure 3 below, analysis of Educational Lead’s open responses shows they
were most positive about the engagement and relationship aspects of their role, but
interestingly also identified the greatest number of suggestions for further improvement in
this area.
There were fewer positive responses on sharing good practice and keeping schools focused
on the target cohort (disadvantaged students, communities, or under-performing
departments).
Some of the challenges in developing and fostering Partnerships can be traced by to the
original Educational Lead training and induction which is the subject of the next section.
14 14
10
8
5
10
5
76
4
0
2
4
6
8
10
12
14
16
Building positiverelationships withpartnerships and
schools
Sustainingengagement byschools in theprogramme
Setting up andrunning an
effective networkmeeting
Sharing goodpractice / CPD
learning betweenschools
Keeping schoolsfocused on thetarget cohort
Successes and Opportunities to Do Differently (Part 1)
Went well Do differently
Page | 41
Educational Lead Training and Induction
Nine out of 14 (65%) considered that the training and induction they had
received prior to commencing work as an Educational Lead were effective in
equipping them to fulfil their role.
Reflecting back on the training content, a clear suggestion was to spend more time
considering how practically to support schools that are busy, under pressure and in flux.
“Clear ideas for how to overcome issues faced in engaging with disadvantaged
Partnerships such as these. Sessions from people who've done it before what went
well, what didn't etc. Strategies for working with busy schools, who have a lot of
immediate issues and this is just another thing that they are being told to do. Peer
reviewing a network course wasn't useful for leading a successful a Partnership.
Training did get better later into the Partnership, but this was too late as over a year
through it.”
Top tips
Involve Educational Leads earlier in the Programme, prior to the recruitment of
schools
Practical how-to’s for engaging and re-engaging busy schools
Differentiate training for consultants (from outside STEM Learning)
Provide opportunities for Educational Leads to share practice and learning as the
Programme progresses
Additional feedback related to timing and communication:
“Time was always an issue, and we were asked to support the programme some time
after it had commenced. This meant we were continuously playing catch-up and
trying to fit everything into a short amount of time.”
“Some of training was spent looking at creating training packages, rather than giving
ideas on how to support my Partnership. I didn't feel like I gained much from the
training days. Also, there were some mixed messages over the years which caused
issues when planning support.”
Key Learning
Learning about past successes tackling challenges and pitfalls would inspire
confidence in future Educational Leads. This cohort of Educational Leads felt they
could develop the content for future programmes based on their Aspire to STEM
experiences.
Education Leads joining partnerships that had already formed required more support
and handover to enable them to quickly establish themselves and renew activity.
Page | 42
Inspirational Moments
All Partnerships produced inspirational moments. Educational Leads were asked to
describe one such moment.
It is interesting that Educational Leads most commonly highlighted examples where schools
took ownership of the programme or the delivery of training (as a result of CPD undertaken),
and stand out moments that demonstrated how the mindsets and behaviours of staff were
being positively influenced; thus taking the outcomes from the programme beyond the
individual and towards the department and whole school levels.
Schools taking ownership and leading CPD
“CPD led for all schools from one Partnership school lead who did CPD and then
invites other schools from the trusts to access to so broadening the scope, leads role
to coach the school lead and engage all in the activity“
“When one of the schools took the lead and coordinated the community day for the
rest of the Partnership.”
Behaviour and mindset change
“Retelling the impact of particular CPD through what staff did and said afterwards.”
“September inset day 2019. Seeing all teachers as the asset. They received good
quality science inset… that made a big impact.”
Specific CPD
“Aspiring leaders CPD with 2 of the Partnership schools - individuals with fantastic
potential - very inspiring to work with.”
“The conference was outstanding. The AQA exam Head came and shared new
insight. This transformed the way we deliver practicals. Before we just used to go
through the motions really. Now we have booklets, with key questions highlighted,
and we really understand why we are we doing it.”
Being together
“Having key stakeholders in a room to explore options and match the programme to
school needs.”
Engagement and science capital
“I went into one school during their science showcase - at the end of a term when
they'd been working on their projects they had a celebration event…I hosted the
panel; the student engagement through the whole event was excellent.”
Key learning
The range of experiences that ELs chose to highlight exemplifies the diversity of
outcomes that Aspire to STEM was able to generate. In pursuing four overall aims,
this gave schools broad scope to identify their priority needs and also to achieve
unexpected benefits (most commonly around developing STEM careers). A narrower
focus would arguably have created deeper and more consistent outcomes across
partnerships.
Page | 43
Network meetings
A learning point identified in the previously completed outcomes evaluation study (April
2020, Skyblue Research Ltd) was that effective network meetings included CPD for
attendees as well as operational planning. Rotating meeting locations also reduced barriers
to attendance
“I went to get reassurance - I felt like a lone voice at school.”
Sustained engagement
Six months after the Programme ended, 9 of 13 Educational Leads (69%)
considered that most schools did demonstrate sustained engagement (i.e. they
were engaged in the needs analysis, planning, and taking part in activities on
offer).
Participating schools continue to access the online training resources. Schools reported
increased knowledge sharing between schools (in addition to MAT protocols)
“[school name] is an associate school, so it has links with the MAT. As a result of this
programme we now share more information with them - they are part of our shared
knowledge hub. The relationship is now strong.”
With greater maturity in subject leadership, school 1’s trust is extending their support to
primary feeders:
“Leaders in the secondaries have also developed and I am now able to spend more
time within primaries… There's a much greater need in primary schools where
science can be seen as a non-core subject. The pressure to get outstanding results
narrows the focus - instead of weaving English and Maths through all subjects.”
Yes, most of the schools showed
sustained engagement behaviours but not all
69%
No, most of the schools did not show
sustained engagement behaviours
15%
No, none of the schools showed
sustained engagement behaviours
8%
Something else: 8%
Page | 44
Supported by improved connections between the science department and the community,
school one was also able to share activities with pupils during lockdown.
“We were also able to connect families to experiments they could do at home during
lockdown using Twitter.”
School 3 observed that its focus on careers (linked to the Gatsby Benchmarks) and support
for disadvantaged pupils across the school had improved.
“We've rewritten our KS3 scheme of work to include Gatsby at every lesson.
ENTHUSE was due to start but is now delayed - this will continue the focus on great
teaching. 'Educake'14 is a programme we now use to support most disadvantaged
pupils. It works out where they've performed worse, which allows us to plan extra
lessons in response.”
The Educational Lead for this Partnership also noted a behaviour that could be sustained:
“CPD led for all schools from one Partnership school lead who did the Foundation for
Professional Development (FPD) and then invited other schools from the trusts to
access the training, so broadening the scope.”
School 5 could also point to new schemes of work as tangible evidence of change.
Moreover, as a department, there was a clear sense of purpose.
“Before the programme, the department had no real vision, management or end
goals. So what we have now has much of 'a flavour of [Education Lead] in it'. She
helped us to decide how to prioritise. Two years on, we have clear expectations that
you can see in our schemes of work. We think hard about how you introduce and
apply knowledge. Every member of the team was helped.”
Top tip
Including Foundation for Professional Development training creates the conditions
where schools can take greater ownership of the delivery and cascading of training.
As a result of COVID-19 Education is always in flux, so planning for the future is not being
attempted.
How to respond to competing, multiple initiatives? School 4:
“We are making sure we maintain the momentum - other priorities come into schools
on a regular basis. However, we are ploughing our own course through this. We keep
asking, 'what's the right thing for our school?' Don't let go of the wider curriculum -
with time limited programmes, there is always the risk that the progress falls away
unless the passion continues.'
We were also able to connect families to experiments they could do at home during
lockdown using Twitter
School 2 has a whole school plan objective to ‘reframe science as a core subject in the
curriculum.'15
14 https://www.educake.co.uk/ 15 Within this overall objective is the target to School will: ‘Run termly Science moderation as part of staff meetings using ASE plan (exemplification materials)’ and to ‘provide an increased level of release time for science leaders in schools this year’. http://islschool.co.uk/school-development-plan/
Page | 45
Top tips to sustain engagement
Lead time to gain buy in before the project starts
Shared vision and clear links to the school’s needs
Good personal relationships built at start of programme
More than one contact in each school
Understanding and appreciation of school’s context and challenges
Regular face-to-face appointments, ideally led by the schools
Regular takeaways of new learning
Regular emails
“Communication to HT /MAT leads about impact and progress termly”
“Actually it isn’t about what we do it is about Head teacher attitudes”.
“Offer quick wins at start - money that can be spent flexibly, free resources”
Flexibility
Access to bursaries.
8.4 STEM Capital, community engagement, and enrichment activities
“The single biggest problem is the dislocation between work and education. We need to
bring together work and education from quite a young age.” Charles Clarke, former
Secretary of State for Education16
In this section we consider how the Aspire to STEM programme:
Successfully engaged the community
Harnessed the power of enrichment activities – ways to link to STEM Clubs-how to
promote the formation of a STEM club
Worked with employers – i.e. use of STEM Ambassadors (and also how did
ENTHUSE Placements support Partnerships?)
Facilitated opportunities to help students make links to careers/further STEM study.
Celebrated success and raised the profile of efforts (by subject / school) – with
consideration of the power that positive press can have with the community / parents,
as well as staff.
As noted earlier in this report, the Aspire to STEM aims linked to the fostering of science
capital, enrichment activity and developing community engagement were the least
developed, but also the aspects schools were unlikely to progress without this kind of
external impetus.
A divergent view from one Educational Lead was that engagement and enrichment events
were going on anyway, but Aspire to STEM provided the opportunity to combine and deliver
on a larger scale.
“We asked, how can we go above and beyond – that’s how we build science capital
in the area.”
From this starting point, schools did report how they have embedded new behaviours linked
to these outcomes.
16 From the podcast: https://3me.sounder.fm
Page | 46
“Education generally is weakest at working with the community - you have the
triangle of school, child and parents. Parent engagement events pulls them into the
triangle. The events [Educational Lead) worked with us on went well (e.g. astronomy
visit). Individuals within communities vary dramatically - and it’s all about buy in. We
have now set up an event - with a termly science prize for all the family, not just the
pupil. Some parents are not that interested...apathy at home is a key barrier. Shifting
science capital is a 5 year plus project.”
Taking part in Aspire to STEM also (unwittingly) helped to prepare schools for dealing
remotely with children and families during the March 2020 COVID-19 national lockdown.
“[Parental engagement] has been added to the ‘Turnkey’ software the Trust uses as
its online learning platform.”
When I joined the school, the community had a very negative attitude towards the
science department, as the teaching and results were poor. We built that back by
events that brought staff, parents and children together positively. Now we can send
them texts, like we did in lockdown, and we get feedback.”
The view that the enduring value of such events comes not only from reach but through
repetition was affirmed by both the Educational Lead, and in another area of the country,
School 4.
The Royal Society did a ‘wowy!’ event to kick started the programme in 2019/2020.
Parents and families came and it cost us £700. Was it worth it? (long pause). It was
probably worth it, as we used it re-launch science. We want to do more here, we’ve
signed up to Primary Futures17 and involved with the SSE windfarm near us.
Curriculum and staff came first, we are now ready for the next phase. Our curriculum
is so sorted now. The Goblin car club has people coming along – and some of the
parents are getting a bit too carried away.”
The Educational Lead offers
“In the local area, we have British Steel and health care employers, but we are an LA
district 5 and there were barriers to aspirations. Historically British Steel have done
engagement events, mostly at secondary or post 16, less at primary. Primary schools
have no formal teams to engage with employers, so it is done by word of mouth.
Enrichment events offer fresh perspectives and sight of career pathways that aren’t often
followed locally.
“Because of our rural location, people assume communities are all together and see
things the same way, but we don't. Therefore, getting us all together matters. Seeing
there is other stuff beyond the farming community that surrounds us. These events
also strengthen the connection between the family and the school which helps in
other areas too, e.g. attendance or completing homework.
Unfortunately, this type of event, for the second cohort was likely to be cancelled or
postponed due to COVID-19.
17 https://primaryfutures.org/
Page | 47
Educational Leads
The balance of Educational Lead feedback was to identify actual benefits for schools, staff
and communities as a result of community engagement and enrichment activities, offset by a
smaller number of comments that either described potential value or did not ascribe benefits
to this part of the Programme. (Figure xx below)
“Enrichment events were going on anyway (at individual schools) so we asked how
can we go above and beyond – that’s how we build science capital. [Before
lockdown] we had a massive Knowsley Big Bang planned with parents involved.”
The positive value of community engagement
“Activities resourced and started that have become annual events or allowed access to
more pupils e.g. science club resources of transition days
“We sent pupil premium pupils on a trip to Coventry University. If it gives them the idea that
university is a viable option for them in the future, then we've been successful.”
“Building connections and Partnerships with people in science related work outside of
normal school contacts. Primary school teachers have little awareness of career pathways
available through science.”
“Lots of different things. Some individuals who went on residential courses spoke of
profound change to them and their ability to work within the school on that particular aspect
(leading technology for example). This will help them in the future and has set a solid
bedrock for that subject. Some ongoing bespoke work with schools has influenced their
ability to develop the working scientifically skills with students of all year groups.”
“Inspirational enrichment which will be remembered for a long time to come and will
hopefully sow the seeds of aspiration. One school in the Partnership has an iconic image of
their high-altitude weather balloon with school logo as the front page of their website
available for staff, students, parents, and the wider community to see. Hopefully, parents and
children will see science as being for them. Value science in everyday life.”
“It is a realistic ambition to raise science capital. We include careers within schemes of work.
Three year groups will visit the zoo this year, but it is hard to get parents involved.”
The potential (but as yet unrealised) value of community engagement
“The activities that took place were all great successes and schools valued them highly.
Whether they will endure is yet to be seen - finance, drive from Ed Lead, other factors? A
single event may be memorable for some individuals but if we are to win hearts and minds
for the long term, there needs to be a consistent diet of opportunities.”
“It was easier for the schools to go for one off spends on single events - good but not long
term.”
Value of community engagement not realised through the Programme
“No visible value - seemed to be a priority of the MAT lead, less of the schools themselves”
“We didn’t do a lot on community engagement or enrichment. We focused on teacher CPD
because they hadn’t had any before and this was the glaring area of need. Would be good to
work on this but a limiting factor is capacity.”
Page | 48
Maximising the value of science capital, community engagement and community
enrichment activities
School 1 had also developed a new resource designed to make it straightforward for
teachers to connect their teaching to possible STEM careers.
Time constraints meant that we couldn't involve other groups. STEM ambassadors
can work well, but we need to make more 'virtual use' of them. As a direct result of
the Programme we have [developed?] a 'finger tips' resource that helps teachers
make links to local employers and careers.
“STEM ambassadors are on the plan for this year, COVID-permitting.”
The availability and quality of STEM ambassadors was also noted, but also a realization that
a virtual approach – by design or necessity was an option (WI-FI permitting).
“We are looking for opportunities to investigate and we need to know about this area.
Online could work (e.g. with a STEM ambassador) but harder with poor Wi-Fi
reception here. Remote training needs to show an awareness and knowledge of our
area - so would ideally be delivered from somewhere in our region.”
The sharing of resources and good practice enabled through Aspire to STEM could have a
further legacy, if schools continue to share in the organising and running of events.
“Shared events between schools. Teachers now knowing one another and we are
actively sharing Physics specialists across schools.”
At the time of writing this process evaluation report, schools involved in ENTHUSE
Partnerships were hoping that this kind of activity would be taken up through this avenue.
“Parents are pretty supportive here (not a given across the other schools) and we
want to get more family learning days - e.g. to get shared enthusiasm from children
and parents for Maths.”
With the exceptions of the organisations commissioned to deliver CPD and community
engagement events, none of the schools had involved other community groups – with lack of
time the chief barrier.
“There are always options, but overall it's limited by the time that can be spent by the
individuals in the school - overall we've had a really positive effect on a large number
of these teachers - and we can already see its effect on the students.”
Educational Leads identified the following resources and activities that can be used to
leverage greater value from events and activities designed to develop science capital:-
1. Quality marks and recognition
2. Ambassadors
3. Crib sheets
4. Support to STEM clubs
Page | 49
Top tips
“Ensure full support and engagement from SA hub.”
“At home science kits and parental activities e.g. events to share with parents what
science/STEM is like these were valued by the schools.”
Challenge stereotypes and pre-conceptions “More work could go in to making
careers in Science for those students going into the job market at 16 and 18 rather
than only focusing on "proper" scientists - i.e. with A levels and degrees. If Science is
so valuable because of the skills it gives students, what jobs does that open up?
Students think that they must be bright enough to do A levels and the only thing they
can do with Science is be a full-blown scientist.”
“The availability of age specific STEM ambassadors able to support primary
education in opportunity areas. Challenging finding someone that children can relate
to.”
“Support STEM Clubs with CREST and ambassadors working together.”
“A list of 10 examples on one sheet of ways that Stem Ambassadors have interacted
with schools. Not a case study but just 2-3 lines per item. Also for employer and
parental involvement in school to give ideas for similar contacts and events.”
“All schools are signed up to PSQM18 which I hope will continue this trend.”
“Support for STEM Clubs and easy access to STEM Ambassadors would be
beneficial.”
“There's a value in inter-school competitions which is the next step and follow on
from establishing strong extra-curricular provision. We didn't get chance to fully
realise this before the end of Aspire to STEM but will be a focus for emerging
Enthuse Partnerships.”
Key Learning
In Aspire to STEM schools reported mixed experiences of the availability (in person)
and quality of STEM ambassadors.
The more strategic use of STEM ambassadors offers the most realisable opportunity
to maximise the value of enrichment activities for future programmes. Enhancing /
promoting virtual opportunities to engage with STEM ambassadors would ease
booking challenges.
18 https://www.stem.org.uk/psqm
Page | 50
Designing a Programme to develop science capital in disadvantaged communities
“Blimey, this is hard…”
This innocuous comment is revealing in that it highlights that developing science capital is
outside the typical experience and expertise of the Educational Lead, and suggests that
different actors may be required to take the aspirations below forward.
This was seen to be a longer term commitment, but important as parents remain the single
biggest influence on their children.
“We were a programme in an area of disadvantage. It would look like this but go on
for longer. Year two we got a little deeper in our discussions and how we were
unravelling the issues... Imagine where we could have got in years three, four or
five?”
Changing the recruitment criteria would be important in developing such a programme:
“Base future programmes on the percentage of disadvantaged students (Pupil
Premium / FSM).”
In the long term, influencing curriculum design to include science capital raising activities
would elevate and amplify science capital in schools.
The ‘component pieces’ are recreated below:
Key Learning
The relationship with parents and the wider community, whilst important, is one that
schools often lack capacity to foster. Targeted support, over a longer period, 2-5
years as a minimum, would be unlikely to displace or duplicate activities schools
could do for themselves.
Multiple
Offsite trips
Day /evening
£200 for parents to
run science events
Narrow STEM
or science focus
Led by STEM
ambassador hub
Sustained
over
2-5 years
Strategic use of
STEM
Ambassadors
Hands-on
events, family
learning
Day / evening
so families can share
homemade creations
Page | 51
8.5 Barriers to sustained engagement
This section:
Considers how the Programme overcame barriers to engagement; and
Identifies strategies for overcoming challenges.
When asked about the key barriers to sustained engagement, schools and Educational
Leads agreed on 3 of the 4 top priorities. For schools the barriers were (in descending
order):
1. Staff capacity
2. Changes in senior leadership
3. Shifts in priorities
For Educational Leads, the top ranked barrier was also staff capacity and the associated
challenge of releasing staff to attend CPD. These were followed by changes in senior
leadership and the shifts in priorities that can follow. The figure below shows the overall
ranking of barriers calculated by giving a score to 1st, 2nd, 3rd choices and so on.
Schools then considered how best these barriers could be mitigated.
Leadership buy in and shifts in priorities
“Making sure we maintain the momentum - other priorities come into schools on a
regular basis. However, we are ploughing our own course through this. We keep
asking, 'what's the right thing for our school?' Don't let go of the wider curriculum -
with time limited programmes, there is always the risk that the progress falls away
unless the passion continues.”
“With a Head to keep them on track they now send me apologies if they can't make it.
As Heads we are keen to talk up the programme when it suits us, so we are
introducing more accountability.”
Page | 52
Staff release time
“Barrier 1: Has to relate to how training readily related to the classroom - relatability
to the curriculum. Barrier 2: Breakdown in communication. Barrier 3: Time constraints
- when and how training occurred. Education is always in flux, so planning for the
future is not being attempted.”
“Timing of meetings and support from SLT to allow staff to attend.
“Lower staff turnover, collective need and more people across the school involved.”
“The department had such a steep mountain to climb. SMT did moan about the cost
of supply cover, so it was covered by the team.”
School top tips
“In an ideal world, paying for cover. Post COVID, recording the training events so others
can watch later would increase the reach. We are doing this for school meetings
already.”
“Afternoon meetings mean staff needed to leave school at lunchtime. After school /
twilight meetings are better.”
“It helps if STEM leaders proactively share successes with me / their SMT sometimes - I
was blown away when I found out what had gone on, I would have liked to know even
sooner.”
Top tips for sustained engagement beyond Aspire to STEM
“Continue to provide a named liaison, able to meet regularly with key staff.”
“Engage in existing local area networks e.g. HT, SL or early career.”
“Establish virtual community groups for each Partnership - a ‘go to’ platform where ideas
could be discussed and resources exchanged.”
“Provide specific updates on resources designed to support pupils learning in STEM.”
“Offer discounts for local CPD courses and online ones.”
“Drop the need to work as Partnerships where schools will work better on their own.”
“Offer bursaries.”
“Support them into an ENTHUSE Partnership with local industry and network leads.”
Top tips: what can schools do for themselves?
“Link up programmes they are involved in and increase number of staff involved at
different times.”
“Engage with networks and communities”
“Collaborate and share CPD with neighbouring schools, even if from a different MAT.”
“They are working as hard as they can, lets respect schools as they are... they make
strides when they see if working...”
“Actively seek out CPD”
“Develop a long-term plan not just looking for a quick fix, recognise the effectiveness of
collaboration and allow time and resources for it.”
Page | 53
Key Learning
Creating the conditions where schools take on greater leadership and accountability
will support the sustainability of activity beyond the partnership
“Begin to lead network meetings and community groups for themselves”
Overall reflections
Reflecting on the entirety of Aspire to STEM journey to March 2020, Educational Leads were
more likely to identify successes in the Partnership formation and communication (aspects
they led on). There were fewer examples of successes for aspects which were led by
schools. The most learning came in how to support schools to communicate with one
another, for example to share resources, CPD and learning.
Partnership performance was variable, and Educational Leads were asked to consider the
characteristics of high and lower performing Partnerships, borne out of their own experience.
6 6
4 4
33
1
6
2
4
0
1
2
3
4
5
6
7
Forming the localPartnership
Partnership networkmeetings
Communicationbetween schools
Working togetherdespite different
needs and priorities
Encouraging schoolsto share resources,CPD and learning
Educational Lead Successes and Opportunities toDo Differently (Part 2)
What worked really well? Do differently?
Page | 54
Higher performing Partnerships
Schools were asked to assess the degree to which their Partnership was successful. Their
responses highlight the persistence of the Educational Lead and the unifying aim of raising
the profile of STEM. Schools that said AtS was very successful did so because:
“[The] Educational Lead was brilliant - her communication was first class - that
happened straight away. We thought about our primary feeders - Science [sic
Engineering] Fairy tales was arranged at one of the [Trust] schools. She was
absolutely relentless.
“Having that joint focus on STEM was very powerful. Our priority was raising the
profile of STEM. 'Equity not equality' across the schools depending on where they are
and their resources.”
Top tip
“There's a much greater need [for programmes like Aspire to STEM] in primary
schools where science can be seen as a non-core subject. Pressure to get
outstanding results narrows the focus - instead of weaving English and Maths
through all subjects.”
Schools from higher performing partnerships were asked to consider examples of where
their Partnership was working at its most effective. Their responses show how school needs
were met, good practice shared, networks strengthened and a positive view of the
Programme was gained, then shared.
“The Trust schools worked closely together and with [another school]. Towards the
end, the EL asked if there was any extra support needed, and we couldn't think of
something - which I think is revealing.”
“The joint training, we had the opportunity to sit down and network.”
“Sharing good practice was a highlight. We did the science show for 70-80 students,
so we then shared that. As a result, others then followed. We shared our experienced
technician with a more junior person at another school, so he could gain tips and
experience. Not to mention the chance to talk to another technician in a similar sized
school.”
“A few weeks prior to the presentation (June 2019) one of my teachers left, and I had
to ask questions about Aspire to STEM and thought 'bloody hell this is brilliant'. So it
was easy to be passionate about the Programme to others. I found out that pupils
were buying MAPLIN technology sets for Xmas rather than X-boxes.”
Key Learning
Earlier identification of both high and under-performing partnerships would enable
STEM Learning to share (or draw in) good practice taking place elsewhere. A neutral,
safe space for ELs to share their challenges with one another would similarly
encourage a culture of honesty and mutual learning.
Page | 55
Educational Leads
Analysis of Educational Lead insights has highlighted seven aspects of high performing
Partnerships.
Strong leadership and ownership of the project
“They have a team that has a clear Partnership lead who keeps everyone engaged
and moves the project forward.”
“Regular personal and face to face interactions between the Ed Lead and the middle
and senior leaders of the school. The Ed Lead regularly offers advice, troubleshoots
issues and flexes delivery plan according to emerging needs and priorities. The
schools themselves feel as though they own the delivery plan and that STEM
Learning are servants to them rather than the other way round. Staff from schools
meet regularly as a collective via network meetings.”
Reach
“I think it is about reach, ensuring that the Partnership works not only with subject
leaders but all staff within the schools and the leadership teams. As in primary, the
Heads are the keepers of the keys, they decide if developing science is a priority.”
Acceptance of the time required to embrace the opportunity
“Willingness to commit members of staff to take a lead role with the project and to be
given time to share with staff at INSETs and department staff meetings. Being able to
adapt and see opportunities beyond the initial CPD, and to give staff time to take next
steps in developing stem teaching and learning.”
Shared need and willingness to accept outside support
“A shared vision and working collaboratively towards it, support from SLT and buy in
from staff.”
“The Partnership that worked the best took on the support from STEM and scheduled
regular support.”
Collaborative mindset
“All members of the Partnership share expertise and collaborate to meet project
aims.”
Willingness to consider a wider view
“Looking beyond the opportunity as simply a way to reinforce what they already do.
Looking at a bigger picture - although this is where capacity to engage becomes
critical.”
Staff turnover
“Lower staff turnover, collective need and more people across the school involved.”
Page | 56
Less successful Partnerships
Educational Leads were asked to reflect on the learning gained by working with Partnerships
that they self-rated as lower performing. The importance of proactive and clear
communication was identified as was recognising that travel times (especially in rural areas
or larger Partnerships) was a limiting factor.
“Start quickly…and get a commitment early on - and then keep talking to those
people.”
“We need to have one person who is a regular point of contact for me and have the
backing of the Lead team.”
The flexibility to create a bespoke package of support is arguably more important in lower
performing Partnerships or those that are considered at risk of under-performing, particularly
early on in the Programme to secure greater buy-in.
“A bespoke solution can help with some engagement, but travel time was important
pre-Covid-19.”
One EL given the opportunity again give greater focus to school level objectives rather than
supporting individual teachers. From his response flow the following top tips.
Top tips
“Bring key stakeholders together to explore options and to match the programme to
school needs”
“Picking from a clearly defined programme of local, date specific CPD”=
“Offer a pathway of CPD to make course prioritisation easier.”
“Less focus on individual teachers at early stages - more school-level objectives.”
“Have a limited requirement for collaboration, as schools are motivated most by their
own priorities.”
Identify a shared need which allows a particular group of staff from across the
Partnership to come together on a regular basis for CPD and mutual support
Their comments were a mixture of affirmation for the resources readily available, but also the
challenges of gaining early buy in from, then dealing with the departure of key staff, whose
successors may have other priorities.
Key Learning
For lower performing Partnerships, consideration should be given to provide an
incentive for schools to manage absence or departure of key staff.
As noted above, there are additional steps required to secure the effective buy in and
engagement of schools within a MAT, to avoid situations like the example from an
underperforming Partnership: “Senior leaders were overly concerned about efficient use of
the budget across the MAT, which ultimately meant that a significant amount of available
funds was never spent. It took over a year for them to get their CPD programme and release
time for teachers ratified by their CEO. Too bureaucratic with too many admin hurdles to
clear before the Programme could get running.”
Page | 57
Practical ways to overcome barriers for future programmes?
ELs were asked to consider what steps could be taken to overcome or mitigate their top
three barriers. The table19 provides pointers to sustain engagement with schools beyond the
end of a funded programme, and suggestions for what schools can do for themselves.
Engaging schools and SLT Working with SLT Arranging CPD Other tips
Early buy-in from MAT senior team (one step removed from the programme) to reduce administrative delays and risk of under-spend / reduced engagement
“When to approach schools at the start - May, July and September are tricky.”
“Recommendations from other
influential MATs/Partnerships - sometimes more powerful for senior leaders to hear recommendations from their peers than from third party organisations such as STEM Learning”
“Looking at success stories and
how STEM can benefit other priority areas.”
“Clear escalation route for non-
contact.”
“Assess the stability of the SLT. Are they interim Heads?”
“Discussions with SLT to ensure they understand what’s involved.”
“Identify the effective key decision makers”
“Coaching to support leadership skills for staff involved with the Partnership”
Get SLT support to enable training days or other creative ways to open up time”
“Partnership
with a supply agency to reduce overhead on schools arranging staff time outside of the classroom.”
“Use twilights/
PPA time” “Remote CPD”
“Focus on a small, limited number of repeated actions, not a long list of different ones”
“More than one
person from each school involved (bring a friend)”
“Try to include
activities which reduce workload.”
“Staff capacity is tricky and again an external factor we were sensitive to the timings
of events. I think ideally supply cover should be included in funding if we are to help
attendance.”
Key Learning
The facility to pay for cover was most commonly identified suggestion, then to
schedule sessions later in the day, ideally in the ‘twilight period’ immediately after
school.
Looking beyond the funded period of the programme, Educational Leads advised ongoing
communications, the creation of virtual Partnerships, CPD offers and updates, the offer of
further funding, and removing the stipulation to work in Partnership.
Key Learning
The attributes listed in this section describing what schools can do for themselves
could be regarded as more desirable when considering the criteria for schools
applying for future programmes.
19 The table only presents tips that have not been shared previously in this report.
Page | 58
9 Conclusions
This process evaluation has provided practical insights that should assist STEM
Learning with existing programme implementation that has a model of working with
schools in clusters or partnerships, and future programme design and funding
opportunities in relation to these aspects:
1: How to create the conditions for engaging with disadvantaged schools, and then
supporting them to sustain their engagement over a two year project intervention period.
There are lessons for how to and which schools to recruit giving greater consideration to
where they are on their improvement journey continuum when a programme such as Aspire
to STEM is introduced. Buy-in from the Head Teacher, SLT and MAT leaders is essential to
ensure programmes have a good chance of success, and the sustained engagement of a
school appears to be heavily reliant on the role and capabilities of the Educational Lead. Not
all disadvantaged schools will be ready for a programme such as AtS, however, those that
perform well and achieve valuable outcomes appear to exhibit the characteristics of having
pre-existing priorities and challenges that Aspire to STEM could more readily align with and
contribute towards.
2: How to create the conditions for successful community activity. Compared to the
other Aspire to STEM Programme aims, this strand of activity was less of a priority, however,
when it was delivered by schools, they often provided those memorable, stand-out moments
that have the potential to embed new ways of thinking, attitudes and pride. For those in a
state of readiness to commit to this kind of activity, it was fun, engaging and inspirational.
The ways in which Aspire to STEM funded community activity helped some schools reach
parents in novel ways has been a breakthrough for some schools as evidence by case
studies supplied to STEM Learning during the Programme’s lifetime. Mixed experiences of
the use of STEM ambassadors suggests more work can be done to improve the potential of
this asset, and more generally, schools lack capacity and time (2-5 years would be needed
for targeted support) to develop this kind of activity as much as they might like.
3: How to undertake valuable processes and provide CPD content and support to a
diverse set of schools across the UK. The ability to combine different CPD to meet school
needs is valued. Bespoke CPD is perhaps unsurprisingly especially appreciated – offering
development opportunities for STEM staff who would be less likely to receive external
training. It was natural for ELs to target their support towards senior staff, where the
opportunity to positively influence departments was greatest. However, this created a
vulnerability in that the Programme then relied upon the SLT member / HoD having the time
and influence to cascade knowledge internally.
4: How to support partnership working amongst a local cluster of schools.
Partnerships differed in their performance, something ELs were able to assess and
substantiate through their reflective process as part of this evaluation. Higher performing
partnerships are more likely to have exhibited traits such as having worked together before;
and having common needs and priorities identified during the needs analysis and action
planning phases of their journey. Those with more diverse needs that were less ‘ready’
(owing to more urgent improvement challenges in their school) to incorporate a programme
such as Aspire to STEM into their daily practices and norms would rely even more heavily on
their Educational Lead for support, without which they would have been unlikely to form or
sustain local networks for themselves. Encouragingly, some schools have since gone on to
become an ENTHUSE Partnership.
Page | 59
10 Technical Annex
Methodology Notes
Evaluation guides were produced and cross-referenced to the RAND national evaluation
materials supplied.20
Thematic analysis of open text responses has been carried out, and themes are presented
in descending order i.e. the most observed theme is shown first.
Partnership fieldwork sampling approach
AtS comprised 40 Partnerships. The evaluators wished to engage with up to 9 schools from
3 Partnerships of 8 schools from 4 Partnerships given the parameters of the process
evaluation, equivalent case study method employed by RAND earlier in 2020 (where they
engaged 4 schools from an initial target of 8 hindered by the first national lockdown) and in
order to provide a mix of Partnerships classified by subjectively assessed performance.
Assessed by Partnership performance
1: Best practice / very successful / high performing
2: Typical, average-performing but with potential to be more successful in future
3: Unsuccessful / disappointing compared to its EL-perceived potential / lower performing
Secondary criteria included:
1: Rurality and distinct disadvantage and barriers to overcome – at least one should be
a rural Partnership owing to the theory posited in the brief that engaging rural schools may
have attracted distinct barriers compared to more populous or urban schools.
2: Sustained engagement – e.g. based on our conversation a Partnership deemed
successful / high performing would exhibit this trait and potentially fall within the subset of
the 19 Partnerships that have gone on to become an Enthuse Partnership. The average- or
lower performing Partnership would likely not have had this trait.
3: Pre-existing / new Partnership
Erin felt that success of a Partnership may also have been influenced by the schools’ pre-
existing willingness to work together (‘share resources and support[i]’) in a local cluster as
opposed to if they formed ‘from scratch’ as a direct result of fitting the geographical Ofsted /
LA rating eligibility criteria of the Aspire to STEM Programme[ii].
4: Length of time involved in the Aspire to STEM Programme
It may be useful to ensure that within the selection there is at least one Partnership that
joined at the very start.
5: Size and composition of the Partnership
Clusters were originally intended to be anywhere between 3 and 10 schools, a mix of
primary and secondary schools in a locality identified as being punctuated by
disadvantage[iii].
20 Head Teacher and classroom teacher surveys, focus group prompts and an evaluation report of the preliminary, partial process evaluation findings (December 2019)
Page | 60
6: ‘Dosage’
One of the common units of data monitored by the Aspire to STEM Programme has been
the number of CPD hours accumulated by each school. If the team theorises that there is a
relationship between the success of a Partnership and the amount of CPD they received this
may be a useful criterion, but this would need aggregating at Partnership level to be really
useful.
7: Similarity or difference of needs of schools in the Partnership
The sampling frame used by RAND Europe to complete their case studies with 8 schools
from 4 Partnerships is as follows:
8: Pragmatism existing rapport
Schools already engaged in Skyblue’s prior evaluation work in 2020 and who may therefore
be more amenable to interview in October, and where it may be possible to provide a more
rounded view of reasons for success based on the additional insights we have about their
context and intermediate outcomes. Information about their journeys is significantly
advanced compared to any fresh selections made for this process evaluation.
Through careful consideration of these criteria, a list of Partnerships was selected to invite to
participate in the process evaluation. Following a robust consent process, the evaluators
were enabled to engage with selected schools from 4 Partnerships; and ultimately
interviewed 5 individuals in schools across these Partnerships, with 1 interviewee being
directly responsible for 3 schools. Thus whilst 5 interviews were completed their coverage is
slightly larger as they detail experiences for 7 separate schools, across 4 Partnerships.
Page | 61
Partnership characteristics
The four Partnerships included in this process evaluation had the following characteristics.
Partnership Interview volume and
school stage
Performer rating
(subjective assessment)
Demography & ‘Place’
Size of Partnership
Engagement and dosage
Preconditions Enthuse Partnership
AtS Cohort
Similarity or difference in needs
(senior management assessment)
Blackpool 2 interviews covering 4 secondary
schools
Average performing
Coastal 5 schools Engaged Pulled together as a Partnership for
Aspire to STEM and characterised by pulling together
wider / additional schools e.g.
Fleetwood outside of the LA
No 1 Diverse
Ipswich 1 primary school
Higher performing
Rural 10 schools Very engaged Pulled together as a Partnership for
Aspire to STEM but also part of the Trust so some pre-existing
connections
Yes 2 Mixture
St. Helen’s 1 secondary school
Higher performing
Urban 5 schools Engaged EL responsible for pulling the schools
together for the Aspire to STEM
Partnership
Yes 2 Diverse
Scunthorpe 1 primary school
Average performing
Mix (urban, coastal, rural)
4 schools Very engaged Pulled together as a Partnership for Aspire to STEM
Yes 2 Thought to have had fairly common needs
and priorities
Page | 62
Educational Lead Classification of Partnership performance
14 Educational Leads took part in the evaluation, from a total of 22.21
ELs supported between 1 and 4 Partnerships each in their role between 2018 and 2020.
Collectively, these 14 ELs reflected on their experience of supporting 26 Partnerships and
classified them using performance definitions supplied in their self-completion survey. This is
a considered, but subjective assessment, and has been triangulated with the views of the
STEM Learning Centre team and teachers from a sample of the Partnerships.
Higher performing Average performing Lower performing
1. Doncaster / Barnsley Astrea 1
2. Milton Keynes Primary 1 3. Scunthorpe 4. St Helens 5. Ipswich (Suffolk) 6. Warrington
1. Doncaster / Barnsley Astrea 1
2. Doncaster Barnsley Astrea 2
3. Corby/ Kettering 4. Coventry Primary 5. Coventry Secondary 6. Delta 1 7. Delta 2 8. Delta 3 9. Luton 10. Milton Keynes Primary 2 11. Milton Keynes Secondary 12. Norwich 13. South Humberside
(Delta) 14. Stoke-on-Trent 15. Telford 16. Warrington
1. Sheffield Astrea 2. Doncaster Barnsley
Astrea 2 3. Dudley 4. Wolverhampton
It is interesting to observe divergence when considering Doncaster / Barnsley Astrea 1 and 2
when the respective Educational Leads provided their assessments.22
21 There was a total of 26 Educational Leads covering 40 Partnerships. 22 were contacted to take part in this process evaluation. 22 Given this variance, care should be taken when labelling Partnerships as higher / medium / lower performers’ using this assessment.
Page | 63
End notes [i] Study Plan for Aspire to STEM RAND Europe, p4 [ii] The programme will see STEM Learning support 40 Partnerships of schools located either in 12 areas designated as Opportunity Areas (OAs) or in Local Authority Districts (LADs) which are rated lowest in England (identified as LADs 5 or 6). OAs were announced in 2016-2017 as areas “identified as the most challenged when it comes to social mobility” by the Department for Education (DfE). These areas were identified drawing on the Social Mobility Index and the Achieving Excellence Areas Index (DfE, 2017: 1). The Opportunity Areas include 12 locations: West Somerset, Norwich, Blackpool, North Yorkshire coast, Derby, Oldham, Bradford, Doncaster, Fenland and East Cambridgeshire, Hastings, Ipswich, and Stoke-on-Trent. LADs rated 5 and 6 by DfE (within 1 to 6 range) are local authority districts with the lowest scores on composite indicators that include measures of pupil attainment, such as Progress 8 scores, and DfE “capacity to improve indicators” such as the number of teacher trainees per 10,000 pupils (DfE, 2016:10-11). To be eligible for TLIF, schools in these areas have to be rated ‘requires improvement’ or ‘inadequate’ (3 or 4) in their most recent Ofsted inspection.3 Schools rated as ‘good’ schools can join the programme on DfE approval of the rationale. The recruitment process also aimed to keep all schools within an education authority together as these schools are likely to have existing collaborations. [iii] Aspire to STEM includes both primary and secondary schools. There are differences between how the intervention operates in primary and secondary schools. In primary schools, it is common for teachers to be generalist and degrees in STEM subjects are not expected. In secondary schools, teachers are expected to have relevant education and/or experience to be able to teach STEM subjects. Therefore, teachers in primary and secondary schools differ in the type of STEM CPD support they need. Given the differences between science teaching and, as a result, Aspire to STEM delivery, in primary and secondary schools, we can think of the Aspire to STEM in primary and secondary schools as two versions of the programme. This suggests a need to conduct analyses separately for primary and secondary schools. Furthermore, a combined average effect would not be applicable to either primary or secondary schools.
Disclaimer: The information in this draft evaluation report is presented in good faith and is thought to
be accurate at time of publication (24th November 2020). However, the authors cannot accept any
responsibility for errors or omissions.
