Welsh Assembly Government · Appendix C. Study characteristics of the primary studies 51 C.1...

Welsh Assembly Government

Outcomes-based Allocation of Educational

Resources

Rapid Evidence Assessment

Final report

22 December 2009

REA: Educational resources and outcomes

Matrix Evidence | 20 January 2011 2

Disclaimer In keeping with our values of integrity and excellence, Matrix has taken reasonable professional care in the preparation of this report. Although Matrix has made reasonable efforts to obtain information from a broad spectrum of sources, we cannot guarantee absolute accuracy or completeness of information/data submitted, nor do we accept responsibility for recommendations that may have been omitted due to particular or exceptional conditions and circumstances.

Confidentiality This report has been prepared for the client within the terms of our contract, and contains information which is proprietary to Matrix and confidential to our relationship. This may not be disclosed to third parties without prior agreement. Except where permitted under the provisions of confidentiality above, this document may not be reproduced, retained or stored beyond the period of validity, or transmitted in whole, or in part, without Matrix Evidence‘s prior, written permission. © Matrix Evidence Ltd, 2009 Any enquiries about this report should be directed to [email protected]

mailto:[email protected]



Abstract

We conducted a rapid evidence assessment (REA) of the literature on the relation between

educational resourcing and various outcomes. Of nearly 4,000 abstracts assessed, 60 studies

were identified and synthesised using systematic REA methods. Overall, the evidence suggests

that there is a small positive association between educational inputs (such as per-pupil

expenditure) and attainment. There is some evidence for improvements in educational

progression (typically measured as pursuing higher education) and reduction in likely crime-

related behaviour, although fewer studies measured these outcomes. The study concludes that

there might be sufficient grounds for using attainment and educational progression in an

outcomes-based educational resource allocation formula, but a well-developed dataset with

pupil-level data would be required to establish the strength of the relation in the Welsh context.

Keywords

Economics of education

Educational resources

Educational expenditure

Student-teacher ratio

Class size

Educational grants

Educational outcomes

Attainment



Contents

1.0 Key implications 5

2.0 Executive summary 6

3.0 Main report 9

3.1 Context 9

3.2 Approach 12 3.2.1 Research questions 13 3.2.2 Searching 13 3.2.3 Screening 14 3.2.4 Data extraction 15 3.2.5 Data synthesis 15 3.2.6 Summary 16

3.3 Results 16 3.3.1 Studies located 16 3.3.2 Summary of key study characteristics 17 3.3.3 Resources and student attainment 17 3.3.4 Resources and other outcomes 31 3.3.5 Summary of the findings 36

3.4 Implications 37

3.5 Knowledge gaps 39

3.6 References 42

Appendix A. Search Strategy, inclusion criteria, and inclusion

figures 46

A.1 Search sources 46

A.2 Search strategy 47

A.3 Screening 48

Appendix B. Flow of literature through the review 50

Appendix C. Study characteristics of the primary studies 51

C.1 Descriptive statistics 51

C.2 Limitations of the Research Field 11

C.3 Limitations with the individual studies included in this review (an annotated

bibliography) 52

Appendix D. Data extraction tool 58

Appendix E. Summary of included studies 60



1.0 Key implications

In the context of considering options for developing and refining funding formulae for schools in

Wales to better reflect desirable educational outcomes, this study set out to review the evidence

for links between school funding and pupil attainment, as well as a range of other pupil

outcomes including university entrance, truancy, criminal behaviour, and future earnings. Key

policy implications arising from the review findings are summarised below:

There is evidence for a link between school funding and outcomes. However, it is most

meaningful to consider this association in relation to pupil progress (i.e., ‗value-added‘

attainment);

Evidence shows clearly that pupil attainment is also influenced by individual, social, and

economic factors over which schools have no control. For that reason, value-added

approaches to funding formulae are likely to be more equitable than approaches based

solely on pupil attainment;

Since links between funding and pupil attainment are much less consistent for primary

schools, it may be that different funding formulae would be appropriate for the primary

and secondary sectors;

Given that the evidence suggests the strength of links between funding and pupil

attainment varies across subject areas, consideration should be given as to the

measures of attainment included in an outcome-based funding formula;

The strength of evidence for links between funding and pupil attainment varies across

countries. The development of a funding formulae should be informed by reference to

data from Welsh schools. Following from this, it would be important to establish that the

relevant datasets are fit for purpose;

The rate of progression to higher education might be improved by increased resourcing,

according to the UK evidence;

Truancy appears to be reduced and attendance improved by increased resourcing,

according to a small number of studies;

The likelihood of future involvement in crime-related behaviours, such as juvenile

delinquency, appears to reduce with increased educational resourcing, according to a

small number of studies;

Future wages, occupational or educational aspirations, and social outcomes are not

consistently related to resourcing, according to the few studies reporting such outcomes

that were reviewed here.



2.0 Executive summary

The review described in this report set out to establish the extent to which research evidence

supports the development of a new funding formula for schools in Wales based on various

outcomes. Specifically, the review examines the evidence for links between school funding and

pupil attainment, as well as a range of other pupil outcomes including university entrance,

truancy, criminal behaviour, and future earnings.

The review identified, retrieved, and reviewed studies from the academic literature using an

approach known as rapid evidence assessment (REA). REAs involve transparent and explicit

methods applied in a standardised and systematic way, producing a synthesis of the evidence

that is, as far as possible, accurate and free from bias.

Searching yielded nearly 4,000 studies of possible relevance. The abstracts and titles of these

studies were then assessed on their relevance to the review question, which narrowed the

sample down to 298 studies. The full-text version of these studies were then retrieved and

screened. The final result was a sample of 60 studies produced since 1999 that were deemed

to be both relevant and to have sufficient information to warrant synthesis.

The review team extracted data from these 60 studies including the country in which the study

was conducted, the educational input type and the findings in relation to pupil outcomes. They

then synthesised the extracted information to summarise the results of the studies and the

extent to which we can be confident in the findings. We present a summary of the findings

below.

Before summarising the results, it is important to understand the limitations of this review. Some

of the evidence we have reviewed has been extracted from studies low on methodological

rigour, which inevitably reduce confidence the robustness of their findings. Importantly, many of

the studies in the review are correlational in nature. Such research designs do not allow causal

attributions to be made. In other words, very few of the studies included here allow us to

definitively say that increased educational inputs cause improved outcomes.

More generally, it is important to understand that pupil attainment can is inexorably influenced

by a complex and wide-ranging set of factors, many of which fall outside of the school

environment. Relationships between funding and pupil outcomes will be subject to the influence

by any number of external or mediating factors. For example, Bramley and Watkins (2007)

noted that the best predictor of academic attainment is prior attainment; examining only the link

between funding and attainment without considering students‘ previous academic attainment

will lead to an overestimation of the contribution of funding to subsequent attainment. Numerous

other variables, such as the social and economic status of the student, or their belief in their

own academic ability, are also likely to play a role in determining future attainment (Marsh &

O‘Mara, 2008) and might also mediate the resource-attainment link.



Summary of findings in relation to educational attainment

Fifty of the studies examined pupil attainment as an outcome. UK studies tend to support a

small, positive association between per-pupil expenditure and attainment. However, both UK

and international research suggests that the impact of funding on attainment may be subject-

dependent. In general, spending has a greater impact on attainment in maths and science than

on English/verbal attainment and general school grades. The evidence also suggests that

although funding increases might benefit secondary-school students, the evidence for primary-

school children is more mixed.

Findings on the impact of class size and attainment are mixed. UK research suggests that

improving the pupil-teacher ratios (PTR) has a positive impact on science and possibly maths

attainment, but not on English attainment. However, research from other countries has

generally not found a significant relationship between PTR and attainment.

Increasing funding for ICT and library facilities might improve attainment, although findings are

mixed. Grant initiatives, in which funding is dedicated to broad improvement programmes,

appear to be associated with small but positive effects on attainment.

More generally, the evidence around how funding might influence pupil attainment needs to be

interpreted with care. Learning and attainment take place in the context of complex social and

individual circumstances. In particular, evidence has shown that pupil attainment is influenced

by prior attainment, by the pupils‘ social and economic circumstances and by the quality of the

teaching that they receive.

Summary of findings in relation to other outcomes

Evidence concerning the impact of funding on other pupil outcomes is relatively rare, making it

harder to draw firm conclusions. Four UK studies on educational progression (e.g., continuing to

higher education) suggest that it might be improved by increased resourcing. However, four US

studies failed to find evidence linking improved financial resources to educational progression.

The small number of studies looking at truancy and attendance suggest increased resourcing

can lead to positive pupil outcomes. Similarly, a small number of studies suggest the risk of

committing crime-related behaviours (e.g., juvenile delinquency) might be reduced by increasing

resources—particularly where those resources are used to reduce the pupil-teacher ratio.

Evidence from three studies reviewed found no clear relation between school resourcing and

pupils‘ future earnings. Similarly, no conclusions can be drawn regarding the relation between

resourcing and pupils‘ aspirations or social outcomes.

How confident can we be that resource inputs cause outcomes?

Causality in research methods typically refers to the level of confidence in our findings that

differences in an outcome are caused by a particular input (e.g., ―increased expenditure causes



improved pupil outcomes‖). Experimental methods, in which individuals are randomly assigned

to a ―treated‖ group or ―untreated‖ group, are the strongest designs for evaluating the causal

effect of an input on an outcome. Very few of the studies in this review employed experimental

research methods. Most of the studies examined associations between resourcing and

attainment over time (i.e., longitudinal modelling). Such methods make it difficult to claim

confidently any causal relationship between inputs and outcomes because longitudinal research

designs cannot fully take into account initial differences between individuals, or other variables

that might influence the relation between input and outcome. As such, the findings presented

herein generally do not verify a causal relation, but rather suggest that there may be an

association between these variables over time that appear to have some causal ordering (that

is, prior inputs can predict future outcomes).

Implications for developing an outcomes-based funding formula

The review has found sufficient evidence to support the view that levels of educational funding

can influence pupil attainment; it also found that funding might have an impact on educational

progression and crime-related behaviours, although the pool of evidence on these outcomes is

smaller. However, when determining the viability of a funding formula based on this evidence,

the Welsh Assembly Government needs to be aware that the relationship between funding and

attainment is inevitably influenced by pupil characteristics, staff quality and programmes

available at the school (such as Reading Recovery literacy programmes or special educational

needs units).

Key to predicting the impact of spending on attainment is considering how funding is spent once

it is obtained. The Welsh system is quite devolved, in that schools have a high degree of

autonomy over what they do with their funds. Should an outcomes-based formula be

introduced, it might be useful to offer guidance to schools on the types of initiatives that they

could implement to maximise the effectiveness of their additional resources. A review of the

literature on successful educational intervention might be useful in developing such advice.

Furthermore, although we have established that there appears to be some evidence of a

relation between resourcing and certain outcomes (attainment, educational progression and

crime-related behaviours), we have not established the strength of this relation. This could be

confirmed for Welsh students through the statistical analysis of large-scale databases (e.g.,

PLASC - Pupil Level Annual School Census) and possibly supplemented with quantitative meta-

analysis. In other words, to implement an outcomes based school funding model it would need

to be established how much additional funding per pupil with a given level of disadvantage is

needed in order to get their attainment to a defined level. Bramley and Watkins (2007) produced

a strong report that takes us closer to this goal1.

1 This report is very strong and uses appropriate multilevel model analyses. However, they acknowledge that the

unavailability of some data (e.g., school capacity and capital stock) could be a limitation of the analyses. As such, some updating of the analyses may be required when such data are available and adequate.



3.0 Main report

3.1 Policy context

This report summarises the evidence produced since 1999 on the relationship between

educational resources for primary and secondary students and various outcomes2. The

objective of the study was to provide context for the possible future development of a new

outcomes-based resource allocation formula for the Welsh Assembly Government, by

establishing whether resources are sufficiently linked to outcomes. The report does not seek to

determine the how much additional spending is required to obtain a desired improvement in

student outcomes, which could be better achieved through sophisticated statistical modelling

such as in Bramley and Watkins (2007).

The Welsh Assembly Government provides funding to local authorities for pre-16 provision in

schools in Wales, mainly through the local government revenue settlement (Revenue Support

Grant). The other main sources of funding for local authority education budgets are council tax

income and non-domestic rates income.3 A 2006 report of the National Assembly for

Wales School Funding Committee4 recommended that funding through the local government

revenue settlement be reviewed. This led to the commissioning of a report that investigated

alternative methodologies that could be used to predict relative demand/need for local

government services, particularly education, with an emphasis on the importance of outcomes-

based distribution formulas.5 Therefore, the present report aims to determine the link between

resource distribution and outcomes (such as cognitive attainment6) to best inform the new

student finance delivery formula.

A 2000 review of the UK literature by Vignoles et al. indicated that some educational resources

seem to have an impact on outcomes.7 That report cited a positive relationship between the

school-level pupil-teacher ratio and attainment, but noted that there is almost no UK evidence

that smaller class sizes lead to better outcomes. This review now requires updating, as recent

2 Outcomes in this review include pupil cognitive attainment, educational progression, and future wages. We also

include some outcomes that can be seen as both mediating variables and outcomes in their own right, such as school climate and educational aspirations. However, we do not include ―throughputs‖—variables that have little intrinsic value on their own, but rather have value from the way in which they mediate or moderate outcomes. A key example of such a throughput is teacher quality. 3 Welsh Assembly Government website. School funding. Accessible at

http://playlearngrowwales.gov.uk/topics/educationandskills/learningproviders/schools/schoolfunding 4 The School Funding Committee report can be found on the National Assembly for Wales website, see:

http://www.assemblywales.org/N0000000000000000000000000045329.pdf 5 Bramley, G., and Watkins, D. (2007). Alternative resource allocation models for local services in Wales. Final report for

the Welsh Assembly Government. 6 Attainment in this study is used to refer to cognitive attainment unless otherwise specified, and is typically

operationalised in the included studies as school grades or standardised academic test results. This is distinct from other forms of attainment measured in pupils such as affective attainment. It is also distinct from academic achievement, which often refers to educational progression—typically operationalised by measures such as high school completion or higher education enrolment. However, there is some inconsistency in the literature regarding the use of the term achievement. Where academic achievement is used in the included studies to denote school grades or test results as opposed to educational progression, it will be referred to as attainment in the present review to avoid confusion. 7 Vignoles, A., Levačić, R., Walker, J., Machin, S. and Reynolds, D. (2000). The relationship between resource

allocation and pupil attainment: A review (DfEE Research Report 228). London: DfEE.

http://www.assemblywales.org/N0000000000000000000000000045329.pdf



moves towards cost-effective education has prompted a surge in studies looking at the

association between school funding and educational outcomes.

Figure 1 shows the process model within which this report is couched. The model posits that

policy and funding decisions determine the allocation of funds. This in turn leads to an

investment in resources (e.g., reduced class sizes, better facilities, better trained teachers) or

investment in educational interventions (e.g., ICT training, reading interventions). It is then

hypothesised that these resources or interventions lead to an improvement in outcomes, such

as improved student attainment or decreased poverty. Some research projects test only the link

between policy decisions or allocation of resources and outcomes; these links are represented

by dotted lines in Figure 1 as they are considered to be indirect effects8 on the outcomes.

The present report focuses on the white boxes in Figure 19, although the various other paths will

be occasionally referred to throughout the report. As such, the key aims of this research are to:

Establish the evidence base for the effect of increased resource allocation on improved

outcomes (e.g., attainment).

Relate the findings from other educational systems to the Welsh context, and evaluate

their relevance.

Establish the most important outcomes through which an outcomes-based resource

distribution formula could be developed.

Figure 1. Process model of funding and outcomes within which the present report is

situated.

Note. Dotted lines represent hypothetical relationships that were not examined in this review. Grey boxes

indicate factors that were not included in the evidence review.

8 Indirect effects are those effects on a variable by another that are mediated by other (third) variables. For example,

school funding is likely to have an effect on achievement indirectly through the equipment or staffing improvements that the additional money can buy, rather than as a direct effect of the money itself. 9 Although they have been discussed in similar reviews on the topic (notably Vignoles et al., 2000), educational

interventions such as Reading Recovery programmes, will not be reviewed in this report. In the Welsh system, decisions to conduct such interventions lie within the school, in contrast to inputs such as per pupil expenditure that are largely under the remit of the Welsh Assembly Government. We have limited the review to those inputs that are more clearly under the influence of the WAG to enhance conceptual clarity and due to limitations of time; however, we do acknowledge the importance of educational interventions in broader debates about improving student outcomes.



3.2 Research Context

The unit of analysis used in quantitative studies in this field of research is an important

consideration. Educational systems can be thought of as multilevel models, in which students

are nested within classes, classes within schools, schools within local education authorities

(LEAs), and LEAs within counties or even countries (Hox, 2003). Each level in the system has

different features that can be measured by certain variables. Some variables pertain to the

student level, such as individual attainment or aspirations to attend university. Other variables

are measured at the school level, such as the number of teachers in the school, or the type of

school (e.g., primary school, secondary school). Many studies aggregate (average) data

collected at lower levels in the hierarchy to form higher-level variables. The most common is

attainment: the educational attainment of individual students is commonly aggregated to obtain

estimates of school-level attainment. The problem with this is that aggregating data loses

important information about the variance between students within in a school. As a result, when

school level variables (e.g., the size of the school) are correlated with school-average

attainment, estimates of the relation may be biased. Thus, using aggregated data can reduce

our confidence in the findings of some studies—often referred to as aggregation bias.

The most appropriate method for dealing with aggregation bias is to use multilevel modelling,

which does not use aggregated data. Rather, multilevel modelling estimates the amount of

variance both within and between schools (i.e., taking into account variation between students

and schools), which allows for more accurate estimation of relationships (Goldstein, 2005). The

educational resourcing literature is rife with analyses based on aggregated data that do not use

this multilevel modelling approach (see Vignoles et al., 2000, for a review), and therefore need

to be interpreted with some caution. However, this is typically due to the way in which large-

scale datasets are created. Often only school- or district-level datasets are available, making

this problem difficult to overcome.

A further problem in this literature is the issue of endogeneity. In simple terms, this refers to the

problem stemming from the way in which resources are distributed to schools; i.e., resources

are likely to be distributed non-randomly across the school system. For example, Jenkins et al.

(2006) noted that, ―In the English school system this [endogeneity] could arise through the

compensatory funding mechanism whereby the per pupil funding received by a school is related

to indices of social deprivation such as free school meals (FSM) status, which are in turn

inversely related to pupil attainment‖ (p. 4). In other words, factors beyond the inputs and

outcomes under examination might be fundamentally interlinked with this relationship.

Endogeneity can be partly resolved by including additional explanatory variables in the model of

the effect of inputs on outcomes. For example, including student socioeconomic status (SES) in

the model can help to explain the non-random distribution of resources associated with

additional funding for low performing schools. Thus, studies that include various background



variables (such as SES, community poverty, etc.) will generally allow greater confidence in the

findings relating inputs to outcomes.

Linked to this is the notion of value-added models, which look at incremental improvements in

attainment over time. Failing to statistically control for students prior attainment can lead to

inflated estimates of the influence of resourcing on subsequent attainment. Although many of

the studies in this review do include a measure of prior attainment in their models (particularly

the more recent ones), not all studies in this field take this into account.

Finally, it is important to note that many studies in this field of research implicitly adopt a

unidirectional approach, with the assumption that resourcing inputs lead to changes in pupil

outcomes. However, as Figure 1 suggests, it is possible that outcomes might drive the resource

allocation in such a way that outcomes lead to changes in the inputs. For example, low

achieving schools are often given financial boosters based on their low performance on

standardised tests. It is likely that a reciprocal effects model, in which resources affect

subsequent outcomes, which in turn affect future resource allocations, is likely to exist (see the

causal loop in Figure 1). Levačić (2007) refers to this as two-way causality10

.

3.3 Approach

Matrix Evidence has used rapid evidence assessment (REA) methods to search for and analyse

data on educational resourcing. The REA process consisted of four stages:

Searching using electronic databases and other sources of literature;

Screening the literature identified in the search stage for relevance, using inclusion

criteria that are clearly defined and set in advance of screening;

Assessing the literature for methodological quality and extracting data using

standardised forms; and

Synthesizing the data to provide an overview of research findings.

The REA method meets the research needs as it is less biased, more thorough and more

transparent than a traditional literature review. The emphasis on transparency and replicability

in the reporting process minimises threats of bias often associated with traditional reviews, and

therefore builds greater confidence in the research findings.

A guide to conducting REAs can be found on the UK Government Civil Service website (Civil

Service, 2009). Below is an overview of the methods employed by Matrix Evidence (see Figure

2). For full technical details related to this report, please see the Appendices.

10

Levačić, R. (2007). The relationship between pupil attainment and school resources. In T. Townsend (ed.), The

International Handbook on School Effectiveness and Improvement (Chapter 22, 395-410). Dordrecht: Springer.



Figure 2. The REA workflow

3.3.1 Research questions

From the broader aims described above, the main research questions explored in this report

are11

:

1. What evidence is there that resources have a positive effect on outcomes such as

student attainment levels?

2. From this, what outcomes might be useful for consideration for inclusion in resource

funding formulae?

3. How relevant is the international evidence to the Welsh context?

3.3.2 Searching

We searched 10 electronic databases using three main clusters of search terms. A list of the

databases and other sources searched can be found in Section A.1 of Appendix A. The clusters

contained terms relating to

Schools and education

11

Taken from the tender specification document.



Funding or resources

Outcomes

Further restrictions were placed on the searches where possible. These

were:

Date restriction ―1999 – Current‖ to maximise the relevancy of the

evidence

Restriction to English language and Welsh language abstracts

The initial draft of the search strategy was constructed at King‘s College,

London in consultation with Matrix Evidence and Professor Rosalind

Levačić of the Institute of Education. A brief scoping review of the topic area was conducted and

the strategy was further developed into a preparatory working draft. Relevant officials from the

Welsh Assembly Government were consulted on this draft and their suggestions were

incorporated into the strategy.

This draft was tested across a couple of days, first to ensure that the relevant material was

being tracked and secondly, to assess the volume of returns that the project could anticipate.

This volume was initially large, which is understandable given that broad, popular terms such as

―funding‖ and ―attainment‖ were included in the strategy. We reviewed the strategy further,

looking at the thesauruses in the Education Resources Information Centre (ERIC) and the

British Education Index (BEI) again to track our terms. In addition, we looked to localise terms

within clusters as well as rooting phrases using speech marks.

The final strategy was again reviewed by Levačić and the client before searching commenced.

The searching started towards the end of September 2009 using a selection of resources

selected by the Centre for Evidence and Policy in consultation with Matrix and our experts. It

was necessary to run a shorter version of the strategy in some databases that not permit longer

search syntax. The searching was completed by the middle of October 2009.

3.3.3 Screening

Through the searching process, we identified 3,752 hits. Screening was

conducted on the titles and abstracts of these hits by assessing each

record against the inclusion criteria, below.

Inclusion criteria

1. The study is available in English or Welsh language.

2. The study refers to the use or allocation of resources or funding.

3. The study refers to educational settings—specifically, primary

and/or secondary schools (i.e., educational institutions covering

students that were within the age range 4-16 years at the time of attending). Note that

this does not mean that the participants are limited to those age ranges, but rather that

the resource allocation referred to in (2) had to occur while the students were in schools

Systematic searching maximises the

likelihood of locating relevant evidence. It also helps to ensure that the sample of

studies is representative of the wider population of

studies that could be included in a

particular review.

Applying consistent inclusion criteria to

each piece of evidence reduces the potential for bias that

is associated with ‘hand-picking’ studies to include in a review.



for 4-16 year olds. Thus, any resource allocation to universities, for example, would fail

this criterion.

4. The study includes the measurement of outcomes such as attainment, drop-out, school

exclusion, post-school participation in education/training.

5. The study provides quantitative or qualitative data on the outcomes mentioned in (4), or

is a systematic review (or meta-analysis) of the relevant literature. Non-systematic

literature reviews, ―think pieces‖, or policy pieces should not be included.

6. The study was published in 1999 or after.

7. The study was carried out in a context that ensures some level of transferability. Those

studies that, for example, are concerned with expanding the coverage of compulsory

education should be excluded. Studies from developing countries should also be

excluded.

3.3.4 Data extraction

The studies were data-extracted by two researchers. Piloting of the tool

ensured a good level of inter-rater agreement. We used a standardised data

extraction tool for each study to facilitate comparisons across different

studies (see tool in Appendix D). The tool included

Contextual data relating to the services provided, the sample and

population, and other relevant background information;

Data relating to the outcomes;

Methodological data on the aims and design of the study; and

Information on the methodological quality of the primary research and report.

3.3.5 Data synthesis

We brought together findings from the studies in a synthesis report. This involved grouping the

findings by outcomes. Within outcomes, the studies are presented by input type. UK studies

received higher precedence in the synthesis than international literature. The order of

presentation is summarised in Figure 3, below.

There are some limitations to the literature presented in this report that should be taken into

account. A methodological note regarding problems more generally with quantitative evidence

in the field of educational resourcing research can be found in Section C.2 of Appendix C. A

brief annotated bibliography of each of the included studies can be found in Section C.3 of

Appendix C. These should be considered when assessing the strength of the conclusions of this

report.

Using a data extraction tool

ensures that similar information is

extracted from each study. This facilitates comparisons across

studies.



Figure 3. The structure of the research synthesis in the present report

Note. The studies in the report are not given a quantitative ―confidence‖ ranking. Rather, the

methodologically stronger studies (―high‖ confidence) are presented towards the start of each section,

while weaker studies (―low‖ confidence) are presented at the end of the section.

3.3.6 Summary

In summary, the method used involved searching, screening, data extraction, and synthesising

the research evidence. The methods are detailed in Appendix A, which enhances transparency

about the decision to include some evidence but not others. The emphasis in the synthesis on

confidence in the research findings and transferability to the Welsh context help to ensure that

the evidence reported herein is useful for its intended purpose.

3.4 Results

3.4.1 Studies located

The searches identified 3,736 documents, to which 16 references provided by the experts were

added. A diagram setting out the flow of literature through the review can be found in Appendix

B. After applying the exclusion criteria, we were left with 60 primary studies. Details on the

inclusion and exclusion figures broken down by each database can be found in Appendix A.



3.4.2 Summary of key study characteristics

This section summarises the characteristics of the 60 primary studies on educational resourcing

that were included in the REA. A summary of each study is presented in Appendix E, and a note

on their methods are available in Section C.3 of Appendix C.

Searches were limited to literature published since 1999. Within this time frame, references

tended to be fairly recent, with more than half concentrated on the last five years (n = 32).

Regarding the country of study, most references came from the US (n = 30), followed by the UK

(n = 25). Four studies contained Welsh-specific data. Nine studies were carried out in urban

settings, and one in a rural setting. The rest of the studies were either a combination of rural

and urban sites or did not specify the type of location.

Educational inputs were mainly operationalised in the form of per-pupil expenditure, pupil-

teacher ratio, and grant funding12

. Educational outcomes included attainment (e.g., test scores),

educational progression (e.g., access to higher education), and truancy/attendance. Non-

educational outcomes tackled in the included studies covered future wages, aspirations and

crime-related behaviour.

3.4.3 Resources and student attainment

Fifty-three of the 60 studies included in this review included some measure of student

attainment. This could either be in the form of school grades or performance on a standardised

test. Below, we provide a brief summary of each study included in the review, organised by the

resource type. Within each subsection of the review of attainment evidence, the studies are

roughly presented in descending order of methodological quality (i.e., the strongest studies are

presented first). Table 1 provides a summary of the age range (e.g., primary school students)

and level of data (e.g., pupil-level data) for the 39 studies the included specific resource

allocations in this review; the remaining 14 studies are discussed later.

12

Grant funding refers to general grants and funding initiatives that are not measured by a specific variable (such as

PTR); rather they measured student attainment before and after a new funding policy or initiative was introduced.



Table 1

Country, school age of pupils, and data level for the 39 studies examining specific resourcing

variables with attainment

Citation Country School age of pupils Data level

Addonizio (2009) USA primary school level

Babcock & Betts (2009) USA primary school level

Betts & Morell (1999) USA secondary school level

Betts et al. (2003) USA primary, middle, and secondary

pupil level

Blatchford et al (2002) UK - England primary pupil level



Darling-Hammond (2004)

USA secondary district level

Dustmann et al. (2003) UK - England and Wales

secondary pupil level

Fuchs & Wößmann (2007)

31 countries secondary pupil level

Graddy & Stevens (2005)

UK private secondary school level

Greene et al. (2007) USA secondary school level

Grubb (2008) USA secondary school level

Haimson (2000) USA primary school level

Holmlund et al. (2008) UK - England primary pupil level

Hoxby (2004) USA primary and secondary district level

Iacovou (2002) UK primary pupil level

Ilon & Normore (2006) USA primary school level

Jacob (2003) USA primary school level

Jenkins et al. (2006) UK - England secondary schools school level

Koshal et al. (2004) USA high school district level

Leuven et al. (2007) the Netherlands primary pupil level

Levacic & Hardman (1999)

UK - England and Wales

primary and secondary school level

Levacic & Marsh (2007) UK - England secondary pupil level



Citation Country School age of pupils Data level

Levacic et al. (2005) UK - England secondary pupil level

Loubert (2008) USA elementary school level

Machin et al. (2006) UK - England primary and secondary district level

Michie (2009) USA elementary and secondary

school level

Miller-Whitehead (2003) USA elementary unclear

Muijs & Reynolds (2003)

UK - England primary school level

Payne & Biddle (1999) USA secondary district level

Pugh et al. (2008) UK - England secondary school level

Ram (2004) USA elementary and secondary

state level

Richmond (2008) USA middle school level

Steele et al. (2007) UK secondary pupil level

Waldfogel & Zhai (2008) 7 countries preschool and primary pupil level

West et al. (2001) UK - England secondary district level

Wilson et al. (2002) UK - Scotland primary school level

Wößmann (2001) 39 countries secondary pupil level



Expenditure per pupil

Of the 54 studies reporting student attainment as an outcome, 23 examined

per-pupil expenditure as a predictor. Of these, eight reported on UK data.

Overall, the eight UK studies indicated that expenditure had a small but

positive association with attainment. The remaining 15 studies were based

on international data (mostly from the US). The international findings were

more mixed than the UK literature, with some studies even reporting small

(albeit generally non-significant) negative relations with attainment. In

general, across all 23 studies, the effects tended to be more positive for

maths and science attainment than for English/verbal attainment or general

school grades. Details of the studies are below

UK studies

Eight studies reported data from UK students. In general, these showed a

positive relation between per-pupil expenditure and attainment.

In one of the better quality studies, Levačić et al. (2005) examined the

relation between educational inputs and students‘ attainment at Key Stage 3

in English secondary schools. They found that higher levels of expenditure

per pupil were associated with higher test scores in maths and science, but

not in English; these effects were small. This finding was echoed by Steele

et al. (2007); they reported a small but statistically significant increase in

mathematics and science attainment with increased expenditure per pupil in

UK secondary schools, while expenditure had a slightly negative effect (yet

significant) relation with English attainment.

In combination, these two high-quality studies suggest that maths and science attainment could

be significantly affected by increased expenditure, whereas English might not be linked to

funding. Looking at final school grades more generally (rather than subject-specific attainment),

Jenkins et al. (2006) investigated the impact of additional school resources on pupil attainment

in the GCSE scores. They used data from the 2003 National Pupil Database, and found that

small gains in attainment were associated with higher per-pupil spending.

Levačić and colleagues have examined the expenditure issue in relation to specific school

types. Levačić and Marsh (2007) focused on secondary modern schools in England. They used

a national data set at the pupil level for GCSEs sat in 2001. They found that modern secondary

schools, which typically had lower per-pupil expenditure than comprehensive or grammar

schools, had lower average attainment. This follows the general trend of the Jenkins et al.

(2006) study, but it is difficult to isolate the effects of the school type from the effects of

additional funding. In contrast, Levačić and Hardman (1999) compared the performance from

1991 to 1996 of grant-maintained (GM) and local educational authority (LEA) primary and

secondary schools in England and Wales. They reported that the average real school budget

per pupil had a significant negative effect on GCSE scores, for both GM and LEA schools. The

authors suggested that this counterintuitive finding might be the result of greater funding being

In general, the relation between per

pupil spending is more positive for

maths and science achievement than

English/verbal achievement and general school

grades.

UK studies tend to support a small,

positive association between per-pupil expenditure and

attainment—depending on the subject. However, one study found a small, significantly

negative relation with overall grades.



allocated to schools with a greater proportion of special educational needs students or more

free school meals, whose students tend to also be academically disadvantaged. However, this

study did not have pupil level data and did not sufficiently control for pupil background effects.

Holmlund et al. (2008) reported a positive and significant effect of school expenditure on

English, maths, and science attainment at Key Stage 2. However, school and pupil deprivation

influenced this relation. Increases in funding have a greater effect on English in deprived

schools, while the opposite is true for science attainment.

Graddy and Stevens (2005) differed from most of the other studies in the review by focusing on

UK independent schools. They found that capital spending per pupil had a statistically

significant positive effect on the proportion of A-grades that a school received. In another study

focusing on the proportion of certain GCSE grades attained, this time in state schools, West et

al. (2001) analysed data on 96 LEAs in England. Spending per pupil was positively associated

with national test and examination performance after statistically controlling for poverty. This

relationship was not statistically significant for Key Stage 1 results but was a significant

predictor of the number of students with five or more A*-C grades at Key Stage 4 (GCSE) level.

Pugh et al. (2008) analysed school-level performance at GCSE in England. They found a small

but significant positive effect of per-pupil spending on both the average points score and the

number of GSCE grades at A*-C level. They observed that the estimated effect of expenditure

in specialist schools was lower than in non-specialist schools.

In summary, UK evidence suggests that maths and science attainment, and the proportion of

students receiving A*-C levels at GCSE, are positively related with per-pupil expenditure.

English attainment does not appear to be related to expenditure.

International studies

Fifteen studies were located that used non-UK data. Overall, these also

supported the positive link between expenditure and attainment.

The strongest international studies come from analyses of large international

databases. Fuchs and Wößmann (2007) conducted an analysis of data from

31 countries using the OECD‘s Programme for International Student

Assessment (PISA) 2000 dataset of 15-year-old students. They found that

educational expenditure per student was strongly and positively correlated

with maths test results, but was not significant for science and reading

attainment. In contrast, Wößmann‘s (2001) study reported analyses on data

from 39 countries, with the aim of estimating the effects of family background, resources and

institutions on mathematics and science secondary school performance. They concluded that

the estimated relationship between the overall amount of resources and student performance

was not significantly positive. Interestingly, it was observed that school autonomy regarding

purchasing supplies positively predicted student attainment.

Focusing on a younger age group, Waldfogel and Zhai (2008) looked at maths and science

outcomes for students in seven countries. They found that public preschool expenditures have

The international literature also

indicates that the positive association between per-pupil expenditure and attainment might depend on the

subject.



small but statistically significant positive effects on fourth graders‘ science scores, while

expenditure in primary education has a smaller but significantly negative effect on subsequent

attainment. These findings indicate that the level of schooling might be important in determining

the impact of expenditure on attainment. Importantly, they also noted that children from low-

resource homes might gain more from increased public preschool expenditures than other

children.

Hoxby (2004) analysed 30 years of national school-level data from the US. They found that test

scores have remained relatively unchanged over that period, but expenditure per-pupil had

doubled. This led Hoxby to argue that educational productivity had decreased, which explains

the absence of improvement despite the injection of funds.

Darling-Hammond (2004) explored disparities in educational access in the US states of South

Carolina13

and Massachusetts. They analysed existing school district data and found that

Massachusetts‘ net school funding was not a significant predictor of English results once race,

socioeconomic status, and teacher characteristics were taken into account. In contrast, school

spending was a significant predictor for mathematics attainment. Similarly, Ram (2004)

conducted an analysis of elementary and secondary schools using national data in the US.

Expenditure had a significantly positive relation with maths scores and SAT scores, but a

negligible (non-significant) positive effect on verbal scores. These findings reflect that of the UK

literature; that is, the school subject is an important factor. However, in contrast, Richmond

(2008) did find a positive association between per-pupil expenditure and verbal student

attainment. A significant relation was also found with maths attainment for middle school

students in the US state of Virginia.

Several studies have found a small but positive relation for secondary students in the US.

Koshal et al. (2004) examined data from 576 school districts in Ohio. They found that

expenditure per pupil was linked to small but ―practically meaningful‖ increases in attainment.

Jacob (2003) found that both total expenditures and the fraction spent on instruction are

positively related to student mathematics attainment in Chicago, although the magnitude of the

effects was quite small. Payne and Biddle‘s (1999) study aimed to evaluate the relation between

poor school funding, child poverty, and maths attainment for eighth grade students in 32 states

of the US. They found that district-level per-student funding was a significant positive predictor

of maths attainment, even after controlling for child poverty, ethnic minorities, and average level

of curricular instruction.

In opposition, further studies on primary school students in the US are not as promising.

Addonizio (2009) found a significant positive relation between expenditure and attainment for

Grade 5 students but a non-significant relation for Grade 3 students in Minnesota. Similar to the

Waldfogel and Zhai (2008) study above, Ilon and Normore (2006) found that expenditures per

student are negatively related to attainment in elementary school students (this study was

conducted in Florida).

13

This variable was not included in the South Carolina models because adequate data were not available.



Other studies on elementary school students have found small or non-significant findings.

Loubert (2008) analysed the data from a natural quasi-experiment. They examined elementary

school attainment before and after increases in funding occurred in Dallas County, Texas in

1993. They reported that overall school attainment increased slightly with increased funding,

although it was not clear if the results were statistically significant or not. Miller-Whitehead

(2003) reported a study on Grade 5 students in Tennessee. They noted that per-pupil spending

was not a significant predictor of science attainment.

In summary, the international literature generally supports a small positive association between

per-pupil expenditure and student attainment in secondary schools. The relation is weaker—or

even negative—in younger grades. This is perhaps because of the different resource needs of

primary school students14

. Also, a difference in subject-specific attainment gains was observed,

in that some studies found no support for improvement in English but did for maths attainment.

Class size

The relation between class size and attainment has received mixed support.

Nine studies reported on the relation between class size and attainment.

Three of these were reviewed for the per-pupil expenditure, above. Fuchs

and Wößmann (2007), a relatively high-quality study, reported a positive

relation between class size and attainment in maths and science, but no

relation with reading attainment. That is, as class size increased, so did

attainment in maths and science. Miller-Whitehead (2003), a poorly reported study, found that

class size significantly and positively predicted Grade 5 science attainment. Ilon and Normore

(2006), a weaker study, reported a negative relation between class size and attainment.

In one of four UK studies on the relation between class size and attainment, Dustmann et al.

(2003) found a significantly negative relation between class size and exam results (O-levels) in

English and Welsh secondary school data. Iacovou (2002) found that smaller class size has

positive effect on reading at ages 7 and 11 when school size and type were controlled for, but

found no effect on maths attainment. However, ―small‖ class sizes at the period in which this

study was conducted would be large by today‘s standards: approximately 80% of the students

were in classes of more than 30 students.

Blatchford et al. (2002) found positive effects of smaller classes for literacy and numeracy

during the first year of schooling. In an important demonstration of the need to include prior

attainment in the modelling, they reported the strongest effects were found for pupils who had

the lowest baseline scores on school entry. Blatchford et al. (2004), found no effects of smaller

class size on year 4 or 5 maths and literacy attainment, or year 6 science and maths attainment.

However, contrary to expectations, a positive relationship between class size and literacy was

found in year 6.

14

This should be considered in comparison with the UK literature, which found mixed and occasionally positive relation

between resourcing (class size) and attainment.

The findings for class size and attainment

are mixed and inconclusive.



Similar to Blatchford et al. (2004), Betts et al. (2003) reported no significant relation between

class size and attainment in high schools in San Diego in the US. In the same study, they also

looked at elementary and middle schools, with different findings. In elementary schools, when

controlling for teacher characteristics, they found a significantly negative relation between class

size and reading attainment, but no relation between class size and maths attainment. For

middle schools, they found a significantly positive effect of class size on maths attainment, but

no significant relation with English attainment. Babcock and Betts (2009) also examined

elementary schools in San Diego. However, they found that there were generally no significant

relations between class size and maths or reading attainment.

The findings for class size are inconsistent, and so firm conclusions cannot be drawn. However,

there appears to be a trend across studies towards higher attainment in smaller classrooms.

Pupil-teacher ratio

Fourteen studies report the relation between pupil-teacher ratio (PTR) and

attainment. Ideally, the relation should be negative. This means that as the

ratio of students to teachers decreases (that is, there are fewer students per

teacher) then attainment increases.

UK studies

Four of the 14 studies report UK data. Overall, the UK data indicate that

lower PTRs might be linked with higher attainment, depending on the

subject.

Graddy and Stevens (2005) found a small negative relation of PTR with

GCSE scores; Steele et al. (2007) reported a negative relation with maths

and science but not English attainment; Jenkins et al. (2006) found a

negative relation between PTR and science but not with maths or English; and Levačić et al.‘s

(2005) study mirrored those of Jenkins et al. (2006). In summary, the UK studies seem to

indicate some negative relation between PTR and attainment for science and perhaps maths,

but not English.

International studies

Ten international studies examining the relation between PTR and attainment are included in

the review. These almost unanimously conclude that there is no significant relation between

PTR and the attainment measures considered.

Five of the ten international studies have already been discussed above (Addonizio, 2009;

Darling-Hammond, 2004; Jacob, 2003; Richmond, 2008; Wößmann, 2001). These five studies

found no or mixed support for the relation between PTR and attainment. The further five

international studies are described below.

Two further international studies found no relation between PTR and attainment. Betts and

Morell (1999) examined the secondary school characteristics that are determinants of college

In the UK, decreases in the pupil-teacher

ratio (PTR) appear to be related to

increases in science and possibly maths attainment, but not

English attainment. In the international

literature, research tends to suggest no

relation between PTR and attainment.



grade point average in San Diego. They reported no significant relation between PTR and

subsequent attainment at college. Greene et al. (2007) analysed data from secondary schools

in New Jersey in the US. Class size, student-teacher ratio and student-aide ratio were not

significant predictors of grades in languages, arts or mathematics.

Leuven et al. (2007) compared the effects of two subsidies (more personnel in the school and

more funding for ICT equipment) targeted at schools with large proportions of disadvantaged

pupils on attainment in the Netherlands. They concluded that both resources led to ―harmful‖

outcomes.

Two studies found some benefits of reduced PTR on attainment, although both of these studies

were quite weak. Grubb (2008) examined the effectiveness of an ―improved‖ school finance

system on secondary school attainment using a United States nationwide dataset. More pupils

per teacher were associated with reduced maths scores; other attainment measures were not

associated with PTR. Haimson (2000) assessed a class-size reduction programme on teacher

perceptions and student performance after one year in New York. Unlike most of the other

studies, Haimson focused on interview data and reported that principals and teachers felt

attainment had improved. However, as this was based on teacher perception as opposed to

actual attainment on tests or school grades, this finding should be viewed very cautiously.

Facilities/books

Three studies examined the effects of improved facilities or library resources

on attainment15

. The results are mixed, but indicate that there might be some

improvement in attainment with improved ICT or library facilities.

Machin et al. (2006) examined English data to test the impact of ICT

investments in student attainment at primary and secondary levels. They

found that ICT funding led to significant improvements in English and science

test scores—but not maths—at age 11. In contrast, the Dutch study by

Leuven et al. (2007) found that funding for improved ICT facilities had no

effect on attainment.

Michie (2009) analysed data from 24 US states to evaluate the Improving Literacy Through

School Libraries programme in 2005. That study suggested that an increase in the number of

books in a school might increase attainment.

15

Other studies were located that focused on ITC or reading training programmes, which are not reviewed here. The

included studies differ in that the main focus was on how increased funding on these facilities improved attainment, rather than on the effectiveness of an intervention.

The evidence suggests that there might be a positive

effect of improved ICT and library facilities

on achievement, although findings are

mixed.



General grants and funding initiatives

Fifteen studies included in the review were based on grants and funding

initiatives. These differed from the abovementioned studies in that they were

not measured by a specific variable (such as PTR); rather they measured

student attainment before and after a new funding policy or initiative was

introduced. Because of the heterogeneity in the range of initiatives

introduced, we present the aims of the 14 studies along with the features of

the grant and the attainment outcomes in Table 2. Overall, the variety of initiatives seemed to

be associated with small but positive changes in student attainment.

Grant initiatives appear to be

associated with small but positive effects on

attainment.



Table 2

Description of 14 studies reporting the effects of grant/initiatives on attainment

Citation Country School level Study aim Features of the grant/ initiative

Attainment outcomes

Bradley & Taylor (2002) UK - England

secondary To investigate the effects of the quasi-market on efficiency and equity in the secondary education sector in England during the 1990s

Funding, directly related to age-weighted pupil numbers and control over financial resources which has been devolved from the local education authority to the schools themselves

Positive effect on exam results

McNally (2005) UK - England

secondary To evaluate the Pupil Learning Credits (PLC) policy initiative

Schools with more than 50% of pupils known to be eligible for Free School Meals obtain extra funding

PLC increases maths performance, but has no effect on English attainment

Noden & Schagen (2006)

UK - England

secondary To evaluate a Specialist Schools programme

Additional (one-off) capital funding and recurrent grants for four years

GCSE results is positively and significantly correlated with the Specialist Schools programme.

PricewaterhouseCoopers (2003)

UK - England

primary and secondary

To investigate the effects of both family and school capita on math and reading test scores for a sample of elementary and middle-school age children

Funding In relation to primary schools, capital investment had some positive effect on attainment. For secondary schools, the capital investment was also positive.




Attainment outcomes

Tikly et al. (2002) UK - England


To evaluate the effectiveness of the Ethnic Minority Achievement Grant (EMAG)

EMAG grants: mentoring, administrative support, and monitoring

The overall improvement rate from 1998-2000 in the proportion of pupils achieving Level 4 and above for the 81 LEAs which provided data for the three years was 11.9 percentage points in English and 14.9 percentage points in mathematics.

Estyn (2008) UK - Wales primary and secondary

To review the impact of funding with a particular focus on how effective it was in helping schools and local authorities to tackle the underachievement of pupils with socio-economic disadvantage

Funding, spent in several different ways

Positive effect on GCSE results

AEL Inc (2000) USA primary To evaluate the impact of 10 years of the Kentucky Educational Reform Act

Improved school facilities, new support programs, technology resources, and educational materials

Some positive improvement in test scores

Bacolod et al. (2007) USA primary and middle

To examine California's accountability system; focused on how schools used financial rewards

Governor‘s Performance Award: money awarded to schools (per pupil) for good performance on standardised tests

Little improvement in test scores or other measures of attainment




Attainment outcomes

Baskett et al. (2004) USA kindergarten To test whether full-day or half-day kindergartens are associated with improved educational and development scales

Half-day versus full-day kindergarten

Mixed results on reading, literacy, and maths attainment

Cramer (2006) USA secondary To assess the impact of smaller learning communities on student outcomes

Grants provided to implement smaller learning communities

Some positive effect on academic attainment

Kinnucan et al. (2006) USA ns To examine the relationship between state aid and student performance

Funding Increased state aid can have important benefits for students if directed toward improved teacher pay and not to reductions in class size.

Klaauw (2008) USA elementary and middle

evaluation of the impact of Title I funding of compensatory education programmes on school finance and student performance in New York City public schools

Funding Title I has not led to better student outcomes

Parcel & Dufur (2001) USA primary To evaluate a Specialist Schools programme

Additional (one-off) capital funding and recurrent grants for four years.

School financial capital has a positive effect, with children who attend schools with higher per-pupil expenditures experiencing increases




Attainment outcomes

in math attainment

Reynolds et al. (2001) USA primary To conduct cost-benefit analysis of a federally financed, comprehensive early childhood programme (The Title I Chicago Child-Parent Centers)

Educational and family support services to low-income children

Participation in the programme led to significantly better school attainment in most measures analysed.



3.4.4 Resources and other outcomes

In total, 17 of the 60 studies examined the relation between resources and non-attainment

outcomes. These studies were diverse in both the educational inputs measured and the

associated outcomes. We grouped the outcomes into six categories:

Educational progression . This is typically measured either by participation in school

after compulsory education finishes, completion of non-compulsory education or

enrolment at a higher education institution.

Truancy/ attendance. Truancy is measured as the number of unauthorised absent days,

while attendance refers to the number or proportion of days attended in the school year.

Future wages. Typically measured as post-school or post-higher education earnings.

Aspirations. In this review, two studies examined educational aspirations (i.e., the desire

to pursue higher education) and one measured occupational aspirations (i.e., the desire

to pursue professional careers).

Crime/ behaviour. These outcomes were a mixed bag of variables such as incarceration

risk, juvenile delinquency and disciplinary referrals.

Social outcomes. Again, this was a mixture of measures including the student‘s social

function, teacher morale, and school climate.

Table 3 overleaf summarises the studies by outcome. Studies marked as ―good‖ indicate that

increased resources were linked to improvements in the specific outcome. Studies marked as

―neutral‖ found no significant relation between resourcing and outcome. Studies with ―mixed‖

results were internally inconsistent; e.g., positive impact for some students but not others. The

one study marked as ―bad‖ had a negative effect of increased resourcing on the outcome. The

studies are detailed below, with reference to the resourcing type (e.g., per-pupil expenditure).

Educational progression

The UK literature is supportive of the relation between resourcing and educational

progression—regardless of the type of resourcing—but the data from the US suggest less of a

link.

Dustmann et al.‘s (2003) analysis of English and Welsh data suggests a

small but negative effect of class size on the decision to continue with

post-16 education after controlling for other variables. In a study of a

Scottish high school, Brown (2004) found that grant money improved rates

of higher education access. That is, more students progressed to further

education or training after the school received the grant funding. McVicar‘s

(1999) study on high schools in Northern Ireland found that the reduced

PTR and higher expenditure both have significant desirable impacts on staying at school. From

the four studies reviewed here, it seems that UK studies support the impact of resourcing on

improved educational progression.

The UK evidence suggests that educational

progression might be improved by

increased resourcing.



Studies from the US are less promising. Betts (2001) measured educational

progression as the latest observation available on years of schooling

obtained for each woman in the US sample. High-school resources as

indicated by pupil-teacher ratio, spending per pupil, teachers‘ salary and

books per student did not significantly predict educational progression.

Cramer (2006) found that differences in drop-out rates and preparation for

post-secondary education were not statistically significant after high schools

received federal grant money. Grubb (2008) found that high PTRs reduce

the likelihood of completing a standard academic programme and of continuing to a four-year

college; however, it also increases the likelihood of graduating and going to a community

college. More encouragingly, grant-funded participation in a Chicago programme was

significantly associated with higher rates of high-school completion in Reynolds et al.‘s (2001)

study. These four US studies indicate no or mixed evidence of a relation between educational

resourcing and educational progression.

There is little or no evidence in the US

literature that educational

progression might be improved by

increased resourcing



Table 3

Relationship between educational resources and various outcomes by study

Outcomes

Study

Educational

progression

Truancy/

attendance

Future

wages Aspirations

Crime/

behaviour Social

AEL Inc (2000) Improved

Anderson et al. (2008) Mixed

Arum & LaFree (2008) Improved

Baskett et al. (2004) Improved

Betts & Roemer (2005) Neutral

Betts (2001) Neutral Mixed

Bradley & Taylor (2002) Declined

Brown (2004) Improved

Cramer (2006) Neutral

Dustmann et al. (2003) Improved Improved Improved

Greene et al. (2007) Neutral

Grubb (2008) Mixed Improved

Haimson (2000) Improved Improved

Jones et al. (2006) Improved

McNally (2005) Neutral

McVicar (1999) Improved

Reynolds et al. (2001) Improved Improved

Note. ―Improved‖ indicates increased resources were linked to improvements in the outcomes; ―neutral‖ indicates no

significant relation between resourcing and the outcome; ―mixed‖ indicates that some of the finding supported the

relationship; ―Declined‖ indicates that resources had a negative relation with the outcome.


Truancy and attendance

Only three studies assessed the benefits of resourcing on reducing truancy or

improving attendance. They were generally in favour of increased resourcing.

Dustmann et al. (2003) found that an increase in class size increases the

probability of truancy in England and Wales. In an English sample, McNally

(2005) found that reduction in absences after the introduction of the Pupil

Learning Credits (PLC) policy initiative was quite small. This could be

because the programme simply involved providing money to the school,

without any obvious change in factors that might improve attendance (such as student

incentives).

In the US, Jones et al. (2006) found that Texan high schools had a small but possibly practically

important change in daily attendance with the change in class size. Larger enrolments led to

lower attendance rates; this is similar to the English and Welsh findings of Dustmann et al.

(2003).

Future wages

Overall, it is not clear that high-school resourcing improves the chances of

higher post-secondary wages. However, only three studies examined this

relation.

Dustmann‘s (2003) analysis of English and Welsh data was promising: The

indirect effect of class size on wages (mediated by ―staying on decisions‖)

was negative. In other words, larger classes lead to a slight decrease in future

wages. In a national survey in the US, Betts and Roemer (2005) found only negligible increases

in subsequent wages with increased high school spending. In a mixed result of US data, Betts

(2001) found that white women's future wages were positively related to the high-school PTR

and negatively to books per student, while black women's wages were negatively related to the

PTR and positively related to books per student.

Aspirations

There are only two studies that refer to the effects of resourcing on

educational and occupational aspirations. Given their different findings, it is

not possible to glean whether aspirations are enhanced by increased

resourcing. Greene et al.‘s (2007) study of New Jersey high school students

did not find a significant effect of resourcing on educational aspirations.

Specifically, class size, student-teacher ratio, and student-aide ratio were not

significant predictors of aspirations to attend college. In contrast, Grubb‘s

analysis of national US data found that teacher salaries enhanced both

occupational aspirations and the intention to continue in schooling.

The small number of studies examining

truancy and attendance seem to indicate the benefits

of increased resourcing.

From the three studies reviewed

here, there is no clear relation between

resourcing and future wages.

No conclusions can be drawn regarding the relation between

resourcing and aspirations from the

two studies considered here.


Crime and behavioural outcomes

All three studies measuring crime and behavioural outcomes focused on US

data. They each suggested benefits of resourcing for improving these

outcomes.

Haimson (2000) interviewed teachers and principals in New York elementary

schools that had received funding to reduce the PTR through reduced class

size and/or ―floating teachers‖. They reported that disciplinary referrals

dramatically reduced in the first year of the programme. However, this study shold be given low

weighting as it was based on teacher perceptions as opposed to actual attainment on tests or

school grades. Also examining the PTR, Arum and LaFree‘s (2008) US national survey found

that students from schools with lower PTRs have a lower risk of incarceration. However, they

also noted that states that invest more in reducing the PTR also tend to spend more on social

welfare programmes, which could explain some of this variation. In another project that could be

evidence of co-varying social phenomena, Reynolds et al. (2001) measured the impact of

educational and family support services to low-income children in Chicago. They reported

improvements in juvenile delinquency and child maltreatment rates.

Thus, improved resourcing seems to improve crime-related behavioural outcomes in US

studies. However, these findings are likely highly interrelated with other social services.

Social outcomes

Studies reporting social outcomes are mixed in terms of both the outcomes

measured and the impact of resourcing. It is difficult to draw firm conclusions

from these studies.

Bradley and Taylor (2002) reported an English study in which funding reform

was directly related to age-weighted pupil numbers. Control over financial

resources was also devolved from the local education authority to the

schools. They concluded that these reforms increased competition between

schools. Although this increased attainment (reviewed above), they also

cited evidence of a widening gap in the social composition of schools. Haimson‘s (2000)

interviews with teachers and principals of New York elementary schools indicated positive

perceptions of reduced class sizes. Specifically, they report that smaller classes led to higher

morale among teachers, less staff turnover, improved parent-teacher relationships, and more

collaboration between teachers.

Another positive finding came from a study in New Zealand. Anderson et al. (2008) reported

that additional funding for school nurse and social worker services improved school climate.

This is likely to be a mediating variable between resourcing and final outputs such as

attainment, although it can be argued that school climate is a desirable outcome in its own right.

The risk of committing future crime

behaviours might be reduced by increasing

school resources.

No conclusions can be drawn regarding the relation between resourcing and social outcomes, in part due

to the diversity of social outcomes

explored.


On a staff school climate survey, significant positive increases were found on scales including:

Improvements in Last 12 Months, Support for Ethnic Diversity, Innovation Culture, and Students

are Friendly. In the student school climate survey, year 10 students showed more positive mean

scores than students in comparison schools on the three scales of Satisfaction with School,

Support for Achievement, and Support for Ethnic Diversity. Overall, these indicate some

improvements in school climate for both staff and students.

With an emphasis on parental perceptions, AEL (2000) reported on the effects of funding

increases in Kentucky. According to the authors, these funds improved school facilities, and

introduced new support programs. After the reforms were introduced, AEL reported that parents

felt the schools were preparing students for adult life. However, given that these are perceptions

of improvement rather than more objective measures, it is difficult to determine how

substantively meaningful such improvements are.

In a different type of resourcing issue, Baskett et al. (2004) compared half-day with full-day

kindergartens in Maine, USA. They reported that social functioning had small but significant

differences in favour of students in full-day kindergarten programmes.

In summary, the diversity of outcomes makes it difficult to review these studies. The findings are

quite mixed, although this could be due to the range of inputs and outcomes. Promisingly, some

studies did indicate positive social outcomes.

3.4.5 Summary of the findings

Summary of findings in relation to educational attainment

Fifty studies in the review examined attainment as an outcome. UK studies tend to support a

small, positive association between per-pupil expenditure and attainment. However, both the

UK and the international literature indicate that the positive association between per-pupil

expenditure and attainment might depend on the school subject. In general, the relation

between per-pupil spending is more positive for maths and science attainment than for

English/verbal attainment and general school grades.

The findings for the relation between class size and attainment are mixed. In the UK,

decreases in the pupil-teacher ratio (PTR) appear to be related to increases in science and

possibly maths attainment, but not to English attainment. However, in the international literature,

research tends to suggest no relation between PTR and attainment.

There might be a positive effect of improved ICT and library facilities on attainment, although

findings are mixed. Grant initiatives, in which funding is dedicated to broad improvement

programmes, appear to be associated with small but positive effects on attainment.

Overall, the studies indicated a small positive relation between educational resourcing and

attainment. However, the strength of the relationship depends largely on the school subject


being measured: resourcing appears to have a stronger relation with science and maths

attainment than with English or verbal attainment. Still, the relation between attainment and

resourcing needs to be considered in the context of several student background and

instrumental variables. In particular, student prior attainment and socioeconomic status, and the

characteristics of the teachers (qualifications etc.) are important.

Summary of findings in relation to other outcomes

The evidence for non-attainment outcomes was sparser. This makes it harder to draw firm

conclusions. The four UK studies on educational progression (e.g., continuing to higher

education) suggest that it might be improved by increased resourcing. However, there is little or

no evidence for this from the four US studies.

The small number of studies examining truancy and attendance seem to indicate the benefits

of increased resourcing. Also with a small number of studies in the review, it appears that the

risk of committing crime-related behaviours (e.g., juvenile delinquency) might be reduced by

increasing resources—particularly reducing the pupil-teacher ratio.

The remaining outcomes examined here did not yield sufficient evidence. In particular, from the

three studies reviewed, there is no clear relation between resourcing and future wages. In

addition, no conclusions can be drawn regarding the relation between resourcing and

aspirations, or resourcing and social outcomes.

3.5 Implications

This REA was conducted according to systematic review principles. However, it was completed

in a short time and, because we limited the number of databases searched, we did not search

the literature in a comprehensive fashion. No authors were contacted for their data, and we did

not carry out any re-analysis of published data. However, the use of specific search strategies

and well-defined inclusion criteria ensured that no relevant study that we found was excluded

with the intention to support a specific hypothesis. Therefore, as much as possible, this work is

free from bias.

The ultimate aim of this exercise is to establish the link between resourcing and outcomes in

order to support the use of outcomes in determining a funding formula (see Figure 4). The

above research indicates that, to some extent and for some inputs and outcomes, increases in

resources can lead to improved outcomes. However, many of the studies above are

correlational in nature, such that they determine whether high levels of funding are associated

with higher levels of an outcome. This does not allow causal attributions to be made. In other

words, very few of the studies included here allow us to definitively say that increased resources

cause improved outcomes. As with any evidence synthesis, our findings reflect what is available

in the literature and the specific factors that primary study authors have chosen to investigate,

and so stronger causal relations can be established only by conducting more primary research.


Figure 4. A hypothesised link of this research and the application to a funding formula

Note. S = the numerical strength of the relation between funding and outcomes, as per the Bramley and

Watkins (2007) report.

The studies included in the REA are heterogeneous in terms of their setting and context and the

particular factors investigated. It can be problematic taking the evidence from one context and

applying it to another context, as the same findings might not apply. In the present study, we

hoped to gain some insight into the educational resourcing issue that can be informative for the

Welsh context. However, most of the evidence that was located comes from the international

literature, particularly the US. Some of the important considerations with respect to

transferability to the Welsh context are listed below:

The policy context. In Wales, funding for compulsory education (up to 16 years of age)

is under the Local Education Authority‘s (LEA) control. In other countries, funding is

partly at the state level and partly at the district level. The source of funds might

influence how decisions are made about the expenditure of available resources. In

Wales (and the UK), for example, ―closing the gap‖ in terms of attainment between low

and average high school leavers is a high policy priority, and so resources could be

geared towards this.

The geographic context. Wales has a substantial number of rural (sparsely populated)

areas, which poses challenges to educational resourcing. These could be increased

costs due to the lack of economies of scale, or through the higher costs for school

transport. Thus, whether rural locations were included in the study was recorded during

data extraction.

The social context. Each country differs in its racial, ethnic, and socioeconomic make-

up. Given the established strong relation between attainment and socioeconomic status

(in particular), it is important to consider these background variables. Where possible,

we note whether such variables were included in the analytical models of the included

studies, which helps to determine how much of the change from the inputs improves the

outcomes above and beyond the background variables.


The research context. As mentioned above, the data that are used in educational

resourcing studies tend to come from national and large-scale datasets collected by the

educational administration rather than the researchers themselves. In the US, this

typically means data aggregated to the district or even state-level, since the country has

such a vast population. However, the equivalent to district-level in Wales is the LEA.

Using LEA-level data would lose a lot of meaningful variation as Wales has only 22

LEAs. As such, Welsh and UK studies tend to analyse pupil-level data. Thus, there is a

mismatch between the level of analysis between Welsh/UK and US studies (see earlier

discussion about the problems with analysing data at higher levels; see also Vignoles et

al., 2000). We note which data level were used for each study.

Particularly important to the Welsh context, given the devolved educational system in place, is

the issue of autonomy of budget. Fuchs and Wößmann (2007) concluded, ―Student

performance is higher with external exams and budget formulation, but also with school

autonomy in textbook choice, hiring teachers and within-school budget allocations‖ (p. 433).

This supports the system currently in place in Wales. However, other studies noted that simply

giving funding to schools without providing guidance on effective spending practices could lead

to wastage or misspending (e.g., Leuven et al., 2007). Although leaving the decisions in the

hands of the schools is appropriate, it could be beneficial to offer advice and information on

established effective practices that the funding can be spent on, such as reading interventions.

Good uses of funding could be identified trough further evidence reviews.

3.6 Knowledge gaps

Establishing causality and direction of effects

Experimental methods are the strongest designs for evaluating the causal effect of an input on

an outcome, yet very few of the studies in this review employed experimental research

methods. This is likely due to the ethical and social implications of such methods in educational

research—it is difficult to offer increased funding to one school and not another unless one is

substantially deprived in one way to begin with. As a result, most of the studies examined

associations between resourcing and attainment over time (i.e., longitudinal modelling). Such

methods make it difficult to claim confidently any causal relationship between inputs and

outcomes because longitudinal research designs cannot fully take into account initial

differences between individuals, or other variables that might influence the relation between

input and outcome. As such, the findings presented herein generally do not verify a causal

relation, but rather suggest that there may be an association between these variables over time

that appear to have some causal ordering (that is, prior inputs can predict future outcomes).

Hand-in-hand with this issue, it is important to note that many of the studies implicitly adopt a

unidirectional approach, with the assumption that inputs lead to outcomes. However, as Figure

1 suggests, it is possible that outcomes will drive the political agenda (or funding formula) in

such a way that outcomes lead to changes in the inputs. Consider, for example, the case where

low achieving schools are given financial boosters based on their low performance on


standardised tests. It is likely that a reciprocal effects model, in which resources affect

subsequent outcomes, which in turn affect future resource allocations, is likely to exist (see the

causal loop in Figure 1). This has been termed two-way causality (Levačić, 2007).

This two-way causality can be modelled in longitudinal studies where measures of inputs and

outputs at three or more time points are taken, and analysed using longitudinal structural

equation models. Recent developments in software such as MPlus now allow modelling of

multilevel structure data with structural equation models. This is an exciting new approach to

analysing such data, and should be adopted in future longitudinal studies on this issue.

Pupil-level data

Most of the studies in the review included data that had been aggregated (averaged) to the

school or district level, rather than analysing individual student data. This can lead to errors in

the analytical modelling (see technical discussion in Appendix C). More research should be

conducted using individual student-level data for variables such as attainment in order to gain

more accurate estimates of the relationships.

Teacher characteristics

In the educational resourcing literature, the number of years of teaching experience, the

qualifications of the teachers and teacher salaries are the most commonly measured teacher

characteristics. We argue that teaching experience and the qualification of the teachers are not

necessarily linked to financial resources—rather, they are indicators of the quality of education.

This issue is beyond the scope of the current review and so we do not review these variables

here. However, it is inevitable that the quality of the teachers, their morale, and their

relationships with their students will affect the learning experience. A separate review on the

relationship between resources, quality of teaching, and student outcomes could disentangle

this issue.

Educational interventions

As with teacher characteristics, it is likely that what actually happens in the classroom affects

student outcomes. This in turn is linked to characteristics of the school and its students. For

example, a school with student behavioural problems that spends its money on developing

disciplinary procedures could see more benefits than if it had spent its funding on new library

books. Thus, the effectiveness of educational interventions could be determined by the

characteristics of the sample.

A related consideration is the choice of educational intervention. In our review, we came across

many studies looking at the effectiveness of various programmes, such as improved curriculum

or reading interventions. These were a very heterogeneous set of studies, and beyond the

scope of this review. However, such studies could be reviewed to provide guidance to schools

on possible effective uses of the funds that they received.


3.7 References included in the review

Addonizio, M. F., Mullin, C. M., Brown, K. S., Verstegen, D. A., Driscoll, L. G., Alm, J., et al.

(2009). X-Efficiency and Effective Schools: A New Look at Old Theories. Journal of

Education Finance, 35(1), 1–25.

AEL Inc. (2000). Notes from the field: KERA in the classroom. Notes from the field: Education

Reform in Rural Kentucky, 7(1), 1-23.

Anderson, A., Thomas, D. R., Moore, D. W., & Kool, B. (2008). Improvements in school climate

associated with enhanced health and welfare services for students. Learning

Environments Research, 11(3), 245–256.

Arum, R., & LaFree, G. (2008). Educational Attainment, Teacher-Student Ratios, and the Risk

of Adult Incarceration Among US Birth Cohorts Since 1910. Sociology of Education,

81(4), 397–421.

Babcock, P., & Betts, J. R. (2009). Reduced-class distinctions: Effort, ability, and the education

production function. Journal of Urban Economics, 65(3), 314–322.

Bacolod, M., DiNardo, J. E., & Jacobson, M. (2009). Beyond Incentives: Do Schools use

Accountability Rewards Productively? NBER Working Paper.

Baskett, R., Bryant, K., White, W., & Rhoads, K. (2005). Half-day to full-day kindergarten: an

analysis of educational change scores and demonstration of an educational research

collaboration. Early Child Development and Care, 175(5), 419–430.

Betts, J., & Roemer, J. E. (2005). Equalizing opportunity for racial and socioeconomic groups in

the United States through educational finance reform. Department of Economics,

UCSD.

Betts, J. R. (2001). The impact of school resources on women's earnings and educational

attainment: Findings from the National Longitudinal Survey of Young Women. Journal of

Labor Economics, 19(3), 635–657.

Betts, J. R., & Morell, D. (1999). The determinants of undergraduate grade point average: The

relative importance of family background, high school resources, and peer group

effects. The Journal of Human Resources, 34(2), 268–293.

Betts, J. R., Zau, A., & Rice, L. (2003). Determinants of student achievement: New evidence

from San Diego. Public Policy Institute of CA.

Blatchford, P., Bassett, P., Browne, P., Marin, C., & Russell, A. (2004). The effect of class size

on attainment and classroom process in English primary schools (Years 4 - 6) 2000-

2003. DfES Research Report, RRx13 -04.

Blatchford, P., Goldstein, H., Martin, C., & Browne, W. (2002). A study of the class size effects

in English school reception year classes. British Educational Research Journal, 28, 169-

185.

Blatchford, P., Russell, A., Bassett, P., Brown, P., & Martin, C. (2007). The role and effects of

teaching assistants in English primary schools (Years 4 to 6) 2000-2003. Results from

the Class Size and Pupil-Adult Ratios (CSPAR) KS2 Project. British Educational

Research Journal, 33(1), 5–26.

Bradley, S., & Taylor, J. (2002). The effect of the quasi-market on the efficiency-equity trade-off

in the secondary school sector. Bulletin of Economic Research, 54(3), 295–314.


Brown, J. (2004). Integrated community schools: At the heart of the community. Children Now,

(23 November).

Cramer, K. (2006). Making schools smaller: Do smaller learning communities improve student

outcomes? (PhD thesis). The George Washington University.

Darling-Hammond, L. (2004). The color line in American education: Race, resources, and

student achievement. Du Bois Review: Social Science Research on Race, 1(02), 213–

246.

Dustmann, C., Rajah, N., & Van Soest, A. (2003). Class size, education, and wages. The

Economic Journal, 113(485), 99–120.

Estyn. (2008). The impact of RAISE. Evaluation of the impact of RAISE funding on schools'

effectiveness tackling the link between socio-economic disadvantage and

underachievement. An interim report after the first 18 months. Cardiff: Estyn.

Fuchs, T., & Wössmann, L. (2007). What accounts for international differences in student

performance? A re-examination using PISA data. Empirical Economics, 32(2), 433–464.

Graddy, K., & Stevens, M. (2005). Impact of School Resources on Student Performance: A

Study of Private Schools in the United Kingdom, The. Industrial and Labor Relations

Review, 58(3), 435-431.

Greene, G. K., Huerta, L. A., & Richards, C. (2007). Getting real: A different perspective on the

relationship between school resources and student outcomes. Journal of Education

Finance, 33(1), 49.

Grubb, W. N. (2008). Multiple Resources, Multiple Outcomes: Testing the "Improved" School

Finance with NELS88. American Educational Research Journal, 45(1), 104.

Haimson, L. (2000). Smaller is Better: First-hand reports of early grade class size reduction in

New York City Public Schools. Educational Priorities Panel.

Holmlund, H., McNally, S., & Viarengo, M. (2008). Impact of School Resources on Attainment at

Key Stage 2, Research Report DCSF-RR043.

Hoxby, C. M. (2004). Productivity in Education: The Quintessential Upstream Industry. Southern

Economic Journal, 71(2), 208–232.

Iacovou, M. (2002). Class size in the early years: is smaller really better? Education Economics,

10, 261-290.

Ilon, L., & Normore, A. H. (2006). Relative Cost-Effectiveness of School Resources in Improving

Achievement. Journal of Education Finance, 31(3), 17.

Jacob, B. A., Courant, P. N., & Ludwig, J. (2003). Getting inside Accountability: Lessons from

Chicago. Brookings-Wharton Papers on Urban Affairs, 41–81.

Jenkins, A., Levačić, R., & Vignoles, A. (2006). Estimating the relationship between school

resources and pupil attainment at GCSE (No. RR727). London: Department for

Education and Skills.

Jones, J. T., Toma, E. F., & Zimmer, R. W. (2008). School attendance and district and school

size. Economics of Education Review, 27(2), 140–148.

Kinnucan, H., Zheng, Y., & Brehmer, G. (2006). State Aid and Student Performance: A

SupplyDemand Analysis. Education Economics, 14(4), 487–509.

Koshal, R. K., Koshal, M., & Gupta, A. K. (2004). Students Academic Performance: An

Interaction of Inputs from the Students, Schools, and Voters. Perspectives on Global

Development and Technology, 3(3), 375–394.


Leuven, E., Lindahl, M., Oosterbeek, H., & Webbink, D. (2007). The effect of extra funding for

disadvantaged pupils on achievement. The Review of Economics and Statistics, 89(4),

721–736.

Levačić, R., & Hardman, J. (1999). The performance of grant maintained schools in England: an

experiment in autonomy. Journal of Education Policy, 14(2), 185–212.

Levačić, R., Jenkins, A., Vignoles, A., Steele, F., & Allen, R. (2005). Estimating the relationship

between school resources and pupil attainment at Key Stage 3. London: Department for


Levačić, R., & Marsh, A. (2007). Secondary modern schools: are their pupils disadvantaged?

British Educational Research Journal, 33(2), 155–178.

Loubert, L. (2008). Increasing Finance, Improving Schools. The Review of Black Political

Economy, 35(1), 31–41.

Machin, S. J., McNally, S., Silva, O., Street, H., & Street, G. (2006). New technology in schools:

is there a payoff? (No. IZA DP No. 2234). Discussion Paper Series. Bonn: The Institute

for the Study of Labor (IZA).

McNally, S. (2005). Economic evaluation of the Pupil Learning Credits pilot scheme (No.

Research Brief RB696). London: Department for Education and Skills.

McVicar, D. (1999). School quality and staying-on: resources, peer groups or ethos?. Northern

Ireland Economic Research Centre.

Michie, J. S., Chaney, B. W., & Rockville, M. (2009). Second Evaluation of the Improving

Literacy Through School Libraries Program. No Child Left Behind. Jessup, MD: U.S.

Department of Education Office of Planning, Evaluation and Policy Development.

Miller-Whitehead, M. (2003). Compilation of Class Size Findings: Grade Level, School, and

District. Presented at the Annual Meeting of the Mid-South Educational Research

Association, Biloxi, MS.

Muijs, D. (2003). The effectiveness of the use of learning support assistants in improving the

mathematics achievement of low achieving pupils in primary school. Educational

Research, 45(3), 219–230.

Noden, P., & Schagen, I. (2006). The specialist schools programme: golden goose or conjuring

trick? Oxford Review of Education, 32(4), 431–448.

Parcel, T. L., & Dufur, M. J. (2001). Capital at home and at school: Effects on child social

adjustment. Journal of Marriage and the Family, 63(1), 32–47.

Payne, K. J., & Biddle, B. J. (1999). Poor school funding, child poverty, and mathematics

achievement. Educational Researcher, 28(6), 4.

PricewaterhouseCoopers. (2003). Building Better Performance: an Empirical Assessment of the

Learning and other Impacts of Schools Capital Investment. London: Department for


Pugh, G., Mangan, J., & Gray, J. (2008). Resources and attainment at key stage 4: estimates

from a dynamic methodology (DCSF-RB056). Research Brief. Department for Children,

Schools and Families.

Ram, R. (2004). School expenditures and student achievement: evidence for the United States.

Education Economics, 12(2), 169–176.

Reynolds, A. J., Temple, J. A., Robertson, D. L., & Mann, E. A. (2002). Age 21 cost-benefit

analysis of the title I Chicago Child-Parent Centers. Educational Evaluation and Policy

Analysis, 24(4), 267.


Richmond, P. N. (2007). Wealth and achievement gaps: An examination of Virginia middle

schools (PhD thesis). Old Dominion University.

Steele, F., Vignoles, A., & Jenkins, A. (2007). The effect of school resources on pupil

attainment: a multilevel simultaneous equation modelling approach. Journal of the

Royal Statistical Society-Series A, 170(3), 801–824.

Tikly, L., Osler, A., Hill, J., Vincent, K., Andrews, P., Jeffreys, J., et al. (2002). Ethnic minority

achievement grant: analysis of LEA action plans. Department for Education and Skills.

Van der Klaauw, W. (2008). Breaking the link between poverty and low student achievement:

An evaluation of Title I. Journal of Econometrics, 142(2), 731–756.

Waldfogel, J., & Zhai, F. (2008). Effects of public preschool expenditures on the test scores of

fourth-graders: Evidence from TIMSS. Educational Research and Evaluation, 14(1), 9-

28.

West, A., Pennell, H., Travers, T., & West, R. (2001). Financing school-based education in

England: poverty, examination results, and expenditure. Environment and Planning C,

19(3), 461–472.

Wilson, V., Schlapp, U., & Davidson, J. (2002). Classroom Assistants: Key Issues from the

National Evaluation (No. 1). Insight. Edinburgh: Scottish Executive Education

Department.

Wössmann, L. (2001). Schooling resources, educational institutions, and student performance:

the international evidence. Kiel: Kiel Institute of World Economics.

Yeh, S. S., & Ritter, J. (2009). The Cost-Effectiveness of Replacing the Bottom Quartile of

Novice Teachers Through Value-Added Teacher Assessment. Journal of Education

Finance, 34(4), 426-451.


Appendix A. Search Strategy, inclusion criteria, and inclusion

figures

A.1 Search sources

The following electronic databases were searched:

Applied Social Sciences Index and Abstracts on the Web (ASSIA). ASSIA is an

indexing and abstracting tool covering health, social services, psychology, sociology,

economics, politics, race relations, and education. It indexes over 500 journals

published in 16 different countries, including the UK and US.

British Education Index (BEI). The BEI provides comprehensive information on

educational research, policy and practice in the UK. It includes journal papers, internet

documents (including Education-line and other sources), conference proceedings, and

British doctoral theses.

EconLit. Published by the American Economic Association, EconLit covers more than

thirty years of economics literature from around the world. It indexes and holds

abstracts for journal articles, books, book reviews, collective volume articles, working

papers and dissertations.

Education Resources Information Center (ERIC). ERIC is the is the world's largest

digital library of education and education-related literature, and holds abstracts and full-

text records of journal articles, books, research syntheses, conference papers, technical

reports, policy papers and other education-related materials.

International Bibliography of the Social Sciences (IBSS). From 1951 onwards. The

essential online resource for social science and interdisciplinary research. Broad

coverage of international journals and books, incorporating over 100 languages and

countries. Abstracts are provided for half of all current journal articles and full text

availability is continually increasing.

Planex. This database holds abstracts of journal articles, books, and government

literature as well as documents such as policies, plans, strategies, and annual reports,

in the following subject areas: urban & rural generation, planning & environment,

transport, social work services, housing, education & lifelong learning, local economic

development.

Web of Knowledge. Includes the Web of Science resource, which is a multidisciplinary

collection of the world‘s leading citation databases, with information from over 10,000

high impact journals and over 110,000 conference proceedings from around the world.

Social Abstracts. CSA Social Services Abstracts provides covers research focused on

social work, human services, and related areas, including social welfare, social policy,

and community development. The database indexes over 1,300 serials publications

since 1979. It holds abstracts of journal articles, dissertations, and citations to book

reviews. As of November 2009, the database holds over 155,000 records.

SPP. Social Policy and Practice is a bibliographic database covering evidence-based

social policy, public health, social services, and mental and community health. It brings

together data from 5 databases: Planex; Acompline; Social Care Online; AgeInfo. The


database holds around 200,000 records, mainly from the UK, but also from the rest of

Europe and the US. About half the references are to grey literature, including semi-

published reports, working papers, local and central government reports, and material

from the voluntary sector and charities.

Several experts were contacted directly and asked to suggest resources. The following

responded with their suggestions:

Professor Rosalind Levačić, Institute of Education, University of London. As Emeritus

Professor of Economics and Finance of Education, Rosalind is a world leader in this

field. She has worked extensively on the issue of the economics of education, including

working as principal investigator on projects such as ―The Impact of Formula Funding

on Schools‖.

Professor Dave Adamson, OBE, University of Glamorgan. Professor of Community &

Social Policy and Director of the University of Glamorgan‘s Centre for Regeneration and

Sustainable Communities, Director of the Programme for Community Regeneration and

Chair of the Regeneration Hub, an inter-disciplinary group at Glamorgan which brings

together a number of research Centres and Units engaged with the wider regeneration

agenda. Working in this field he has had three periods as an Advisor to the Welsh

Assembly Government on issues of regeneration and worked on the design of the

‗Communities First‘ programme.

Professor Stephen Gorard, University of Birmingham. Committed to the improvement of

education in terms of effectiveness and equity, his research is ‗society-wide‘ and lifelong

in scope. He has conducted studies of primary education, early childhood, secondary

education, FE, HE, adult and continuing education, and informal learning in the home.

Professor Fiona Steele, University of Bristol. A Professor of Social Statistics, Fiona has

published on assessing the impact of school resources on pupil attainment.

A.2 Search strategy

The full record (title, abstract and keywords) was searched in all databases. The long search

strategy (for those databases that do not have strict character limitations, such as ERIC) was as

follows:

(schools or educat*) AND (funding or resourc* or invest* or cost* or

class size or physical environment* or structur* or architecture* or

teacher pupil ratio* or teacher student ratio* or classroom help* or

classroom assist* or teaching assist* or learn* assist* or book* or

instruction time or IT equipment* or Sport* equipment* or music*

equipment* or “Revenue Support Grant” or “Local Government funding” or

entitlement*) AND (attainment* or achievement* or “minimum standard”

or “league table” or test* score or exam* or social disadvant* or

“drop-out” or participation or progression or transition or social

inclusion* or social exclusion* or social isolation* or social

deprivation* or material deprivation* or inequality* or household

income* or poverty or free school meal*)) OR (Wales or Welsh)


The short search strategy (for those databases that do have strict character limitations, such as

BEI) was as follows:

(schools or educat*) AND (funding or resource or grant$ or invest$ or

cost$ or (ratio and (teacher pupil or teacher student)) or classroom

help$ or (assist$ and (classroom or teaching or learning)) or

(equipment$ and (Sport$ or music or IT)) or entitlement$) AND (attain$

or achieve$ or minimum standards or league tables or exam$ or

participation or transition or (social and (inclusion or exclusion or

isolation or disadvantage or deprivation )) or inequality or poverty)

OR (Wales or Welsh)

The full record (title, abstract and keywords) was searched in all databases. A date restriction

―1999 – Current‖ was used to maximise the relevancy of the evidence. Searches were not

restricted by country or language, although only English and Welsh language abstracts were

retrieved.

A.3 Screening

The inclusion criteria used in the screening stage were the following:

1. The study is available in English or Welsh language

2. The study refers to the use or allocation of resources or funding

3. The study refers to educational settings—specifically, primary and/or secondary schools

(i.e., educational institutions covering students that were within the age range 4-16

years at the time of attending). Note that this does not mean that the participants are

limited to those age ranges, but rather that the resource allocation referred to in (2) had

to occur while the students were in schools for 4-16 year olds. Thus, any resource

allocation to universities, for example, would fail this criterion.

4. The study includes the measurement of outcomes such as attainment, drop out, school

exclusion, post school participation in education/training

5. The study provides quantitative or qualitative data on the outcomes mentioned in (4), or

is a systematic review (or meta-analysis) of the relevant literature. Non-systematic

literature reviews, ‗think pieces‘, or policy pieces should not be included.

6. The study was published in 1999 or after

7. The study was carried out in a context which ensures some level of transferability.

Those studies that, for example, are concerned with expanding the coverage of

compulsory education should be excluded.


Table A1.

Inclusion figures by search source (including 3 review studies)

Abstract Full text Total

Ex NA EX1 EX2 EX3 EX4 EX5 EX6 IN Irretriev Dup Linked EX1 EX2 EX3 EX4 EX5 EX6 EX7 IN

ASSIA 0 0 32 11 11 0 0 6 0 0 0 0 4 0 1 0 0 0 1 60

BEI 489 1 191 29 62 1 29 44 6 1 0 0 20 0 7 1 0 1 8 846

EconLit 3 0 122 43 82 2 1 47 5 0 3 13 1 5 3 3 1 13 300

ERIC 0 0 54 4 169 7 0 61 14 0 0 0 20 0 4 14 0 2 7 295

IBSS 15 0 130 36 54 2 0 31 1 4 1 1 11 2 1 0 0 1 9 268

Planex 0 0 10 63 17 0 0 9 2 1 0 0 3 1 1 1 0 0 0 99

Soc Abs 0 0 536 65 70 7 0 24 5 1 0 0 13 0 0 0 0 1 4 702

SPP 3 0 250 110 283 3 0 30 1 0 2 0 5 1 5 3 0 0 13 679

Wok 0 9 271 86 82 9 0 30 4 1 0 0 9 0 6 5 0 2 3 487

Experts 0 0 0 0 0 0 0 12 0 1 0 0 8 0 0 1 1 0 5 12

Total 510 10 1596 447 830 31 30 298 38 9 6 1 106 5 30 28 4 8 63 3752


Appendix B. Flow of literature through the review

Figure B. Flow of literature

Excluded on abstract n = 510 EX 1 n = 10 EX 2 n = 1,596 EX 3 n = 447 EX 4 n = 830 EX 5 n = 31 EX 6 n = 30

References located: database searches n = 3,736 provided by experts n = 16

Excluded on abstract n = 3,454

Excluded on full text EX 1 n = 1 EX 2 n = 106 EX 3 n = 5 EX 4 n = 30 EX 5 n = 28 EX 6 n = 4 EX 7 n = 8

Included studies n = 60 + 3 reviews

Irretrievable n = 38

Full text retrieval n = 298

Excluded on full text n = 182

Duplicate and Linked studies n = 15


Appendix C. Study characteristics of the primary studies

C.1 Descriptive statistics

Table C1.

Study publication year (primary studies)

Year of Study n

1999 4

2000 2

2001 5

2002 5

2003 6

2004 7

2005 4

2006 7

2007 7

2008 10

2009 3

TOTAL 60

Table C2

Country of study (primary studies)

Country n

England 17

Wales 1

England and Wales

3

Northern Ireland 1

Scotland 2

UK 1

US 30

Other 2

Multi-country 3

Total 60


C.3 Limitations with the individual studies included in this review (an

annotated bibliography)

Addonizio (2009). The dataset used had a large number of missing data points as a result of

non-participation by schools in the study; schools that did not participate may be

systematically different from those that did, which could bias the results. Also, the study

used school-level data, and did not control for many other variables. This makes the

findings from these analyses questionable.

AEL Inc (2000). The authors noted that there were some problems with introducing aspects of

the reform, suggesting the changes in resources may not have been consistent across

the sample. Moreover, many initiatives were introduced at once, and the evaluation was

not rigorous. It is therefore hard to tell how much of the improvements were due to

particular changes.

Anderson et al. (2008). The reliability of some of the scales used to measure school climate

were not as high as desired. This could lead to measurement error of the instrument.

Perhaps more concerning, the time between the 2003 and 2004 student surveys was

only 5 months, in which the effects of the programme may not have had time to manifest.

This could explain the non-significant findings for this analysis.

Arum and LaFree (2008). The authors note that a limitation of their research lies in linking PTRs

and incarceration risk. ―In particular, our data do not allow us to determine the extent to

which teacher/student ratios affect incarceration risk by improving youth socialization and

decreasing criminality or through their impact on the decision-making of officials in the

legal system‖ (p. 26).

Babcock and Betts (2009). Used gain scores to account for prior attainment. They noted that

that there were larger attainment gains for disadvantaged students.

Bacolod et al. (2007). A conclusion of this study was that the reward money given to schools

was not used in meaningful way, undermining the resource allocation. Moreover, the

study treats those that did not receive the reward as a control group. This is dubious as

there may be initial differences between the groups.

Baskett et al. (2004). It was not clear how many schools were involved in the project, or the

reliability of the measures. The groups analysed were not really comparable.

Betts (2001). This study uses very old data: the school expenditure data were collected in 1968,

while data on attainment and wages was collected between 1968 and 1991. It is difficult

to tell whether this still has enough currency.

Betts et al. (2003). Data consisted of gain scores rather than raw attainment scores, which

helps to account for prior attainment.


Betts and Morell (1999). The sampling of the study makes it difficult to generalise to other

contexts; however, the inclusion of social background variables helps to isolate the

variance due to PTR.

Betts and Roemer (2005). There is limited information on the sampling strategy, making it

difficult to determine the representativeness of the sample.

Blatchford et al. (2004). This report does not adequately describe the statistical analyses used

to arrive at the conclusions.

Blatchford et al. (2002). A longitudinal design with baseline assessment was used. They also

ran models with different ―splines‖, because ―Almost all previous studies have limited

themselves to linear relationships between achievement and class size, but this may only

be true over a restricted range. Fitting simple polynomial relationships may impose too-

rigid constraints on the shape of the relationship, especially at the extremes (p. 173)‖.

Blatchford et al. (2007). The study examined relationships between TAs and the academic

outcomes for the whole class, rather than for the specific pupils that they support.

Bradley and Taylor (2002). This study notes that funding reform introduced both financial and

organisational changes, although the extent of these inputs were not assessed.

Brown (2004). There is a strong possibility of confounding variables in this study.

Cramer (2006). This study used a non-random comparison group.

Darling-Hammond (2004). These models take into account a reasonable number of background

variables, which reduces the risk of endogeneity; however, the use of aggregated data

raises concerns about the strength of the observed relationship.

Dustmann et al. (2003). Although the study did control for prior attainment, it was based on

quite old data (collected in 1974 and 1981).

Estyn (2008). The authors note that the diversity of the RAISE funded activities makes the

assessment of effectiveness difficult. ―It is often difficult to isolate one activity or

intervention as the main cause of any improvement in pupil performance. It is also difficult

to assess the impact of many RAISE activities in terms of improved standards because

many have not run for long enough to see clear trends in outcomes‖ (p. 9).

Fuchs and Wößmann (2007) controlled for numerous family-background and institutional

effects, lending confidence to the findings.


Graddy and Stevens (2005). This study controlled for prior attainment, which means that any

observed differences as a result of school characteristics should be unique to the school

(rather than attributable to differences in the initial abilities of the students).

Greene et al. (2007). The study ignores the multilevel data structure, and presents an unclear

justification for inclusion of "number of feeder schools" in the model.

Grubb (2008). The study attempts to make causal attributions from correlational data.

Haimson (2000). Although it is claimed that the schools are differentiated to reflect a variety of

implementations of the reduced class sizes funding, it is not really explained how those

specific schools were chosen. Perhaps more importantly, the outcome measure is based

on perception of teachers rather than actual measured performance.

Holmlund et al. (2008). This study took into account the multilevel structure of the data. Also,

the authors modelled a range of background variables, including deprivation.

Hoxby (2004). It is unclear how consistent the general increase in expenditure is manifested at

the school-level, which is a function of the aggregation bias issue.

Iacovou (2002) included the interaction between school size and school type ―thus freeing the

estimates of bias arising from the re-distributive allocation of educational resources faced

by the children in this sample (p. 282)‖. Although this paper uses careful longitudinal

models, the data is based on pupils that were born in 1958, and state that class sizes for

this cohort were typically much higher than today‘s classes because of the baby boom at

this time. In fact, around 80% of the students were in classes with more than 30 students.

Thus, what is considered a ―small‖ class in this sample is on a different scale to more

recent studies.

Ilon and Normore (2006) provided no information about the sample to gauge whether the results

of this study are generalisable to other contexts, and used school-level data raising

concerns about aggregation bias. Also, the counterintuitive finding about the negative

relation between expenditure and attainment could be due to the simple regression

analysis used, because other variables could explain the effect (e.g., class size and

teacher experience could have some confounding with expenditure).

Jacob (2003). The estimates produced by the analyses had large standard deviations, which

suggests that there might be differences between schools that are not accounted for in

the analytical modelling. Overall, we can have low confidence in transferring findings from

this study to the Welsh school system.

Jenkins et al. (2006). This study appropriately controlled for endogeneity by including various

control variables, such as prior attainment, school size, and political control of the LEA.

This also minimises the likelihood that results are due to aggregation bias associated with

using school-level data.


Jones et al. (2006). There are a large number of predictor variables in model—perhaps more

than the amount of data can sufficiently model. They appear to have averaged findings

across 9 years, and the test for time effects does not seem adequate to detect changes.

Kinnucan et al. (2006). This article tests various explanatory models, although it is not clear to

what extent they're related to the actual empirical data.

Klaauw (2008). The study does not adequately report what the funding was spent on.

Koshal et al. (2004) demonstrated the importance of accounting for background and

instrumental variables in educational resourcing studies.

Leuven et al. (2007). The discussion in the paper suggests that seems that money was not

spent on its intended purpose, which could explain the effects. However, no data were

collected to confirm this suspicion. There were also some concerns with the how the

sample was determined, as disadvantage was not clearly defined.

Levačić et al. (2005) based their analyses on a good pupil-level database, which increases

confidence in the findings.

Levačić and Hardman (1999). This study used carefully selected instrumental variables

(including student background and school governance variables) in the modelling, which

helps to distinguish how much of the variance in attainment across schools is actually

due to the different funding mechanisms. However, this study did not have pupil level

data and did not sufficiently control for pupil background effects.

Levačić and Marsh (2007). Analysis could not be conducted that brought budgetary variables

and attainment variables into the same model, and so it is difficult to determine whether

this relation was significant or not.

Loubert (2008) used data from 1990 and 1997 for an interrupted time series analysis design,

which they acknowledged was a limitation given the dramatic change in the racial make-

up of the population during that time.

Machin et al. (2006). This study uses good representative sampling and well-developed

datasets.

McNally (2005). This has an odd design with 2 control groups, one of which seems to include

the other (see p. 7-8).

McVicar (1999). This study posits causal relations although it used a cross-sectional survey

design.


Michie (2009). The data are insufficient, with lots of missing data—only 40% of the schools

completed the baseline attainment test.

Miller-Whitehead (2003). Being a conference paper, this study lacks a lot of detail for fully

interpreting the findings.

Noden and Schagen (2006). As with many of the studies in this review, establishing a link

between funding and attainment does not determine whether improvements are due to

the characteristics of the programme or to the additional funding provided.

Parcel and Dufur (2001). This uses a strong dataset and models both school and family

variables.

Payne and Biddle (1999). In addition to concerns with using aggregated data, there is a problem

with the gap between the time that student attainment data was gathered (1982) and the

school/district information was collected (1988-1990). It is not possible to know whether

the school conditions observed in 1988-1990 are equivalent to those experienced when

the student attainment data were collected. This makes any interpretation of the findings

dubious.

PricewaterhouseCoopers (2003). Difficult to disentangle funding effects from the effect of actual

initiatives implemented as a result of the extra capital.

Pugh et al. (2008). The estimates pertaining to some school types are ―subject to a greater

margin of error‖ (p. 2).

Ram (2004). The use of the state-level average expenditure might mask the intrastate

(between-school and between-student) variance in spending and student attainment.

Reynolds et al. (2001). The design used a non-random control group comparison (i.e., quasi-

experimental design). Some of the estimates were based on projected rather than

measured effects.

Richmond (2008). The data do not enable causal conclusions to be drawn.

Steele et al. (2007) incorporated a number of instrumental variables, but the datasets they used

did not allow them to model the effects of ‗home‘ variables. The authors acknowledge that

such variables might have been useful in explaining the odd negative effect of increased

funding on English attainment.

Tikly et al. (2002). There were some serious gaps in the data.

Waldfogel and Zhai (2008). This study is strengthened by the use of a strong international

database; however, the authors note that there may be factors across the countries that

they did not control for that might explain differences in student attainment.


West et al. (2001). In addition to concerns about using district-level aggregated data, the nature

of the data do not allow causal attributions to be made.

Wößmann’s (2001) findings were based on cross-sectional data, which does not allow

prediction of prior effects on future outcomes.


Appendix D. Data extraction tool

Study information

1. Author/s & year. Use the format: Surname (year). E.g., Matrix & King (2009).

2. Type of publication. E.g. Journal article, government report, etc.

3. Funder of research.

4. Author(s) organizations(s).

5. Country of study

6. Specific location

7. Urban/rural. Choose one if specified.

8. Wales specific data? Y/N

Methodology and quality assessment

9. Aims of study. Stated objectives of the study.

10. Type of school. (e.g., primary, secondary, public, private)

11. Number of schools. How many schools were measured in total?

12. Number of students/participants. How many people were measured in total?

13. Special sample characteristics. (e.g. low-achieving students; focus on an ethic

grup)

14. Sampling method. Note any details about the sampling method (e.g., random

sampling, purposive sampling)

15. Sampling process described? Y/N. Does the study adequately describe the

sampling process, sufficient for replication?

16. Sample: socio-demographic data? Y/N. Is there a description of background

variables such as age, gender, race, or SES for the sample as a whole? Select ―Y‖

if any are present.

17. What was the study design as stated in the paper? E.g., RCT, quasi-experiment,

observational, literature review, secondary data analysis, etc. If it is not explicitly

mentioned, then infer from the methods section.

18. What is the main data type?

a. Quantitative

b. Qualitative

c. Mixed

d. Other (please note)

19. How were data collected?

e. Survey (closed and/or open questions)

f. Interviews

g. Observation

h. Existing data sets

i. Literature searching

j. Case study

k. Focus groups

l. Other (please note)

20. Source of data (e.g., name and date of survey, interviewees)


21. Data collection described? Y/N. Is there a description of the data collection

process, sufficient for replication?

Findings

22. Type of resource allocation (e.g., money, facilities, etc)

23. Source of resource allocation (e.g., local govt, central govt, etc)

24. Attainment outcomes. Note the results related to attainment or attainment.

25. Other outcomes. List all other (non-attainment) relevant outcomes and their

results.

26. Possible data for meta-analysis? Y/N. Use ―Y‖ if there appears to be sufficient data

to calculate effect sizes for the main outcomes. Note the page number where the

data can be found.

27. Key findings as stated by the authors.

28. Limitations of the study, as stated by the authors.

Other

29. Link to full text if available

30. Notes. Any other questions of interest. E.g. note external validity issues, or any

other QA-related information.

31. Coder. Who coded the record?


Appendix E. Summary of included studies

Table E overleaf summarises the resources and outcome variables reported in the included

studies.


Table E Summary of included studies (primary studies only)

Resources Outcomes

Citation Country Type of school

Per pupil expend-iture

Class size, student teacher ratio


Teaching assistants

Facilities/ books

Multi-faceted grant/ money

Attain-ment

Education progression/ participation

Truancy/ attendance

Wages/ employ-ment Other

Addonizio (2009) USA primary

AEL Inc (2000) USA primary Anderson et al. (2008)

New Zealand secondary

Arum & LaFree (2008) USA secondary Babcock & Betts (2009) USA primary

Bacolod et al. (2007) USA

primary, middle and secondary

Baskett et al. (2004) USA kindergarten

Betts & Morell (1999) USA secondary Betts & Roemer (2005) USA secondary

Betts (2001) USA secondary

Betts et al. (2003) USA primary, middle, and secondary

Blatchford et al (2002) England Infant, primary Blatchford et al (2004) England primary Blatchford et al (2007) England primary Bradley & Taylor (2002) England secondary

Brown (2004) Scotland secondary

Cramer (2006) USA secondary Darling-Hammond (2004) USA school district

Dustmann et al. (2003)

England and Wales secondary


Resources Outcomes





Teaching assistants

Facilities/ books


Attain-ment


Truancy/ attendance


Estyn (2008) Wales

primary, secondary, special education

Fuchs & Wößmann (2007)

31 countries secondary

Graddy & Stevens (2005) UK secondary Greene et al. (2007) USA secondary

Grubb (2008) USA secondary

Haimson (2000) USA primary Holmlund et al. (2008) England Primary

Hoxby (2004) USA primary and secondary

Iacovou (2002) UK Infant, primary Ilon & Normore (2006) USA primary

Jacob (2003) USA primary Jenkins et al. (2006) England secondary Jones et al. (2006) USA secondary Kinnucan et al. (2006) USA middle

Klaauw (2008) USA primary and middle

Koshal et al. (2004) USA secondary

Leuven et al. (2007)

the Netherlands primary

Levačić & Hardman (1999)

England and Wales


Levačić & Marsh (2007) England secondary


Resources Outcomes





Teaching assistants

Facilities/ books


Attain-ment


Truancy/ attendance


Levačić et al. (2005) England secondary

Loubert (2008) USA primary

Machin et al. (2006) England


McNally (2005) England secondary

McVicar (1999) Northern Ireland secondary

Michie (2009) USA primary and secondary

Miller-Whitehead (2003) USA primary Muijs & Reynolds (2003) England primary Noden & Schagen (2006) England secondary Parcel & Dufur (2001) USA primary Payne & Biddle (1999) USA secondary

PricewaterhouseCoopers (2003) England


Pugh et al. (2008) England secondary

Ram (2004) USA primary and secondary

Reynolds et al. (2001) USA primary

Richmond (2008) USA middle Steele et al. (2007) UK secondary

Tikly et al. (2002) England primary Waldfogel & Zhai (2008) 7 countries primary

West et al. (2001) England Local Education Authorities

Wilson et al. (2002) Scotland primary


Resources Outcomes





Teaching assistants

Facilities/ books


Attain-ment


Truancy/ attendance


Wößmann (2001) 39 countries secondary

Welsh Assembly Government · Appendix C. Study characteristics of the primary studies 51 C.1...

Documents

Transcript of Welsh Assembly Government · Appendix C. Study characteristics of the primary studies 51 C.1...