1 ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION Emmanuel...
-
date post
22-Dec-2015 -
Category
Documents
-
view
215 -
download
1
Transcript of 1 ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN APPLICATION OF DEA IN EDUCATION Emmanuel...
1
ALTERNATIVE METHODS FOR MEASURING EFFICIENCY AND AN
APPLICATION OF DEA IN EDUCATION Emmanuel Thanassoulis
Professor in Management Sciences
Aston Business School
University of Aston
Birmingham B4 7ET
Tel: +44(0) 121 3593611 Ext 5033
Fax: +44 (0) 121 359 5271
Email: [email protected]
2
PRESENTATION OUTLINE
How Data Envelopment Analysis (DEA) works in outline.
A conceptualisation of comparative efficiency measurement How Corrected and Modified OLS regression work. How Stochastic Frontier Analysis (SFA) works.
Decomposing pupil attainment between school and pupil effects using DEA.
Managerial information derived through decomposing pupil and school effects using DEA.
PART 1: OVERVIEW OF COMPARATIVE EFFICIENCY ASSESSMENT METHODS
PART 2: A REAL LIFE APPLICATION OF DEA ON BEHALF OF THE UK DEPARTMENT FOR EDUCATION AND SKILLS
3
PART 1: OVERVIEW OF COMPARATIVE EFFICIENCY ASSESSMENT METHODS
An Introduction to Efficiency and Productivity Analysis by Tim Coelli, D S Prasada Rao and George Battese (1998) Kluwer Academic Publishers (chapters 2, 3, 8, 9) ISBN: 0792380622
S.C. Kumbhakar and C.A. Knox Lovell (2000) Stochastic Frontier Analysis (Cambridge University Press) ISBN: 0521481848
Greene W H (1998) Frontier Production Functions in H. Peasaran and P. Schmidt (eds) Handbook of Applied Econometrics, volume 2, microeconomics (Blackwells)
Useful references on parametric efficiency assessment methods:
4
THE PROBLEM OF COMPARATIVE EFFICIENCY
As an integral part of managing organisations in the public and private sector we need to have information such as:
- How efficiently are the operating units using their resources?
- Is the industry such that there are economies of scale?
- Is the industry such that there are economies of scope?
- What are the efficient marginal costs of outputs;
- Has there over time been productivity change within the industry?
- How has a given operating unit fared on productivity change?
5
THE FUNDAMENTAL VIEW OF THE PROBLEM
Inputs OutputsTransformation
The units to be assessed transform inputs into outputs
The basic requirement is to compare the Decision Making Units (DMUs) on the levels of outputs they secure relative to their input levels.
6
MEASURES OF COMPARATIVE EFFICIENCY
Inputs OutputsTransformation
In a given operating context the measure of efficiency is normally one of:
- The distance between observed and maximum possible output for given inputs (output efficiency);
- The distance between observed and minimum possible input for given outputs (input efficiency);
7
There are two broad types of method for arriving at measures of comparative efficiency: parametric and non-parametric methods.
The parametric methods typically hypothesise a functional form and use the data to estimate the parameters of that function. The estimated function is then used to arrive at estimates of the efficiencies of units.
The non-parametric methods, best known as Data Envelopment Analysis (DEA), create virtual units to act as benchmarks for measuring comparative efficiency.
8
n1iexy i
Kk
1kikki
K inputs
Hypothesise a production function. E.g. the expression
PARAMETRIC METHODS FOR COMPARATIVE EFFICIENCY MEASUREMENT
Where y is output, xik are inputs, and ei is the residual for firm i.
It is the residual ei the captures any inefficiency.
Unfortunately, the residual also captures other random effects (e.g. omitted variables, measurement error, etc.) which makes it difficult to disentangle the component of inefficiency.
[1]
Two approaches exist: ignoring and not ignoring the random effects.
9
OLS regression assumes implicitly the ui have zero mean.
Let us treat the residual ei in [1] as capturing ONLY inefficiency ignoring other random effects. Then the model becomes
Now suppose we use OLS regression to estimate the model in [2].
ui >= 0.
Therefore we need to adjust the OLS model, to get the true residuals ui. Normally one of two types of adjustment is used: Corrected or Modified OLS.
n1iuxyi
Kk
1kikki
K inputs [2]
where
IGNORING RANDOM EFFECTS IN THE RESIDUAL
10
The OLS model we estimated when we assumed residual mean of zero was in effect model [2] implicitly modified to
They work as follows.
Let a* = [a-E(ui)] be the intercept of [3] that OLS regression yields. In order to retrieve the true underlying model in [2] we need to add E(ui) to a*.
The problem is that E(ui) is an estimate of the mean inefficiency of firms which we do NOT know.
1Kn
RSS
2
u =
To get an estimate of the mean inefficiency of firms we make an assumption about the theoretical probability density function of ui and then use the OLS regression residuals
to estimate . i
uE
i
uE
[3]n1i)u(Eux)u(Ey ii
Kk
1kikkii
K inputs
11
u
2uE
If ui is assumed to follow a half-normal distribution
COLS.
Adjust the intercept a* of the OLS model in [3] to
iuE*a
u
uE
Three estimates of normally are used as follows:
i
emax*a
i
uE
where ei is the residual for the ith firm in model [3] (OLS model).
MOLS - half normal
adjust the intercept a* of the OLS model in [3] to
where
If ui is assumed to follow an exponential distribution
iuE*a
MOLS - exponential
adjust the intercept a* of the OLS model in [3] to where
12
one-parameter probability density functions
0
0.5
1
1.5
2
2.5
0 1 2 3
random variable, u
f(u
)
f(u) exp f(u) half-normal
The exponential and half-normal assumption reflect the belief that larger values of inefficiency are less likely.
13
frontiers for Penn 90 data
6
7
8
9
10
11
12
5 6 7 8 9 10 11 12
log capital per worker
log
gd
p p
er w
ork
er
OLSLP/QP
COLSMOLS exp
MOLS half-normal
log gdp/worker
Illustrative application of COLS/MOLS to a set of data.
Adapted from Weyman-Jones lecture notes, Aston Business School
Inefficiency of country A: The colour of the arrow identifies the referent boundary
A
14
NOT IGNORING RANDOM EFFECTS IN THE RESIDUAL: Stochastic Frontier Analysis (SFA)
n1i]uv[xyln ii
Kk
1kikki
Key departure from COLS and MOLS is that we now have a composed error term
iiiuv
v is an identically distributed conventional two sided error term with zero mean. It stands for random noise, omitted variables etc.
u is an identically distributed one sided error term with a non-zero mean. It stands for inefficiency.u is typically assumed to be exponential, half-normal or truncated normal.
15
normal minus half-normal
distributions, u2 = 2.0
0
0.75
-3 -1 1 3
error term,
f()
Illustration of the composed error term i = vi - ui when ui is half normal.
Source Tom Weyman Jones, lecture notes Aston Business School.
Note the negative skew. We use a test for negative skew to decide whether there is evidence of inefficiency among the producers.
0e1n1m 3i3 If
we reject the SFA model.
16
The SFA model is usually fitted using Maximum Likelihood estimation.
We need to estimate the inefficiency of the ith producer (ui) by using its composed residual = vi - ui .
Depending on the assumption we make about the distribution of the inefficiency ui we arrive at a different formula for the conditional value
iiuE
We plug into this formula the values of and other values we derive from the to arrive at an estimate of the conditional inefficiency ui of the ith producer.
The formulae differ depending on the distribution assumed for ui but are coded in available software such as Limdep.
ii
uE
17
A GRAPHICAL OUTLINE OF THE BASIC DEA MODEL FOR ASSESSING COMPARATIVE EFFICIENCY
An introduction to Data Envelopment Analysis can be found
in:
E. Thanassoulis (2001) Introduction to the Theory and Application of
Data Envelopment Analysis: A foundation text with integrated software.
Kluwer Academic Publishers, Boston, Hardbound, ISBN 0-7923-7429-0
20
Using interpolations between observed units the set of all feasible input -output correspondences is constructed and its boundary identified
21
Using the set of all feasible input output correspondences the comparative efficiency and other information in respect of a unit (e.g. unit E) is derived as illustrated here:
Output Efficiency of E: FE/FG
Output benchmarks for E: Units C and D
Scope for output augmentation at E: EGReturns to scale (increasing, decreasing, constant): Revealed by the intercepts of the segments of the efficient boundary. Scale elasticity revealed by the slope of the segments on the efficient boundary.
Input Efficiency of E: HI/HE Scope for resource conservation at E: IE
22
Contrasting the Alternative Efficiency Assessment Methods All depend on identifying a
reference boundary relative to which efficiency is assessed.
DEA is non-parametric, can handle multiple inputs and outputs but assumes all distance from the boundary is inefficiency.
SFA allows for random noise in the distance from the boundary but needs assumptions on inefficiency distribution;
DEA reveals unit-specific peers, type of returns to scale, productivity change.
SFA and regression methods reveal industry level information.
COLS/MOLS can be very susceptible to outliers and do not allow for random noise.
2 4 6 8 10
2
4
6
8
10
12
0
Input
A
B
E
F
D
C G
H
DEA
OLS Regression
Output
SFA
COLS
24
Most studies measuring school performance use school level data and are parametric (regression-based).
One non - DEA approach which uses pupil - level data is One non - DEA approach which uses pupil - level data is Multi-level Modelling. It recognises the nested nature of school data (pupil within school, school within LEA etc.).
In a similar manner our approach uses pupil level data in a DEA framework to decompose pupil attainment into any number of components (pupil, school, type of school, gender etc.)
DECOMPOSING PUPIL ATTAINMENT USING DEA
It decomposes variance in pupil attainment into pupil, school etc. effects.
25
How the Decomposition Works
Take pupils in a number of schools.
We have a two-level model [pupil is level 1, school is level 2].
Any difference in attainment by pupils is as a result of a combination of: - Random Noise;- School effectiveness;- Differences in effort made by pupils.
Our approach allows for random noise and attempts to separate school from pupil effects.
26
How the Decomposition Works
BCD - Pupil-within-all-schools efficient boundary
BCH and GD Pupil- within-school efficient boundaries
OZ/OZ’ = Pupil-within-school efficiency measure of pupil Z.
OZ/OZ’’ = Pupil-within-all-schools efficiency measure of pupil Z.
0
5
10
15
20
25
30
35
40
45
20 30 40 50 60 70 80
GCSE scores
A -
leve
l sco
re
A
B
C
D
School 1
School 2
Z
Z'
Z''
O
E
F
G
H
OZ/OZ’’ = OZ/OZ’ * OZ’/OZ’’
OZ’/OZ’’ is a measure of school-within-all-schools efficiency at pupil Z
The graph shows data from school 1(dots) and school 2 (crosses)
27
Pupil-Within-All-Schools DEA efficiency =
Component attributable to pupil Component attributable to school
28
Application of the Methodology
Pupils of 122 schools who sat GCSE’s in 1992 and A or AS levels in 1994 were assessed.
DEA Input - Output Variables:
Contextual variables Outcome variables
Total GCSE points (GCSEpts) Total A and AS points (Apts)
GCSE points per attempt(GCSEpts_att)
A and AS points per attempt (Apts_att)
29
Estimation of DEA efficiencies
Given a set of n pupils, the DEA efficiency of pupil j0 relative to that set is 100/j0* % where j0* is the optimal value of in:
Max S.t.
n
j 1GCSEptsjj <= GCSEptsj0
GCSEpts_attjj < = GCSEpts_attj0
Aptsjj > = Aptsj0
Apts_attjj > = Apts_attj0
j = 1
n
j 1
n
j 1
n
j 1
n
j 1
j , 0, free
31
Decomposing Pupil-within-All Schools Efficiencies:
EFFwj0 - pupil-within-school efficiency
EFFij0 - pupil-within-all-schools efficiency EFFwj0
EFFsjo - school-within-all-schools efficiency
EFFij0 = EFFsjo * EFFwj0
32
Summary Statistics of Efficiency ScoresResults obtained considering all the pupils sampled
irrespective of school:
Mode Median Mean
Within-school efficiency (EFFwj0) 90,100 65 62.41
Within-all- schools-efficiency (EFFij0) 50,60 46.6 48.36
School-within-all schools- efficiency
(EFFsj0)
90,100 82.76 78.39
-The median pupil attains only 65% of the A-level scores of the benchmark pupil(s) in his/her own school;-The median pupil attains only 46.6% of the A-level scores of the benchmark pupil(s) across schools-The median pupils benchmark within own school attain only 82.76% of the benchmark pupils across schools.
33
Attribution of efficienciesSchool 60
0
20
40
60
80
100
Pupils
Efficiency
School 121
0
20
40
60
80
100
Pupils
Mean school-within-all-schools efficiency:
School 121 93.8%
School 60 60.56%.
34
- School 121 is more effective with stronger pupils
- School 2 is more effective with weaker pupils
Identifying Differential School Effectiveness
A school has differential effectiveness if it has a different effect on different groups of pupils.
0102030405060708090
100
2,2 3,2 4,2 5,2 6,2 7,2 8,2
GCSE points per attempt
Sch
oo
l ef
fici
ency
sco
re
School 2
School 121
GCSE points per attempt can be taken as indicative of the innate ability of a pupil
35
Pupil targets and peers
Pupils targets can incorporate several components of interest both to the pupil and to the school:
- - Within-all-schools targetsWithin-all-schools targets - A pupil can reach these - A pupil can reach these by a combination of reaching within school targets by a combination of reaching within school targets while his/her school becomes more effective. while his/her school becomes more effective. (Within-all-schools peers can indicate role model (Within-all-schools peers can indicate role model schools.)schools.)
- - Within-school targetsWithin-school targets - achievement of these - achievement of these targets depends only on the pupil. (Peers are within-targets depends only on the pupil. (Peers are within-school.)school.)
36
Other Decompositions Possible
We could in effect control for any number of categorical variables and estimate the impact of each one on pupil attainment: E.g:
Gender ( Estimate efficient boundaries within each gender and then pool the genders to compare the distance of the two boundaries.)
Type of school ( Estimate efficient boundaries within each type of school and then pool the types of school to compare the distance of the boundaries.)
Socio-economic factors ( Estimate efficient boundaries within each group of pupils (e.g. eligible v non-eligible for free school meals) and then pool the samples to compare the distance of the boundaries.)
Combinations of categorical variables (E.g. Gender and eligibility for free school meals) Estimate efficient boundaries within each combination of factors and then pool the samples to compare the distance of the boundaries.)
37
Conclusion The approach outlined recognises that pupil attainment is a
combination of various effects including effort by the pupil, the school and other categorical factors.
It can provide more complete information to schools, teachers and other parties in managing school and pupil performance. Eg:
- targets for individual pupils using other pupils within the school as benchmarks;
- targets for schools to raise their boundary closer to the inter-school boundary;- identification of any differential school effectiveness
with a view to its elimination or management.