ROBUST WEIGHTED COMPOSITE INDICATORS BY MEANS OF...

Statistica Applicata - Italian Journal of Applied Statistics Vol. 23 (2) 259

1 Francesco Vidoli, email: [email protected] In this paper the adjectives “composite” and “synthetic” (referred to indicators) will be used

as synonymous terms.

ROBUST WEIGHTED COMPOSITE INDICATORS BY MEANSOF FRONTIER METHODS WITH AN APPLICATION TO

EUROPEAN INFRASTRUCTURE ENDOWMENT

Francesco Vidoli1, Claudio MazziottaDepartment of Political Sciences, University of Roma Tre, Rome, Italy

Received: 14th November 2012/ Accepted: 10th July 2013

Abstract. The aim of this paper is to systematically and consistently present several recentcontributions where we have tried to combine two fields: the construction of compositeindicators and the measurement of productive efficiency by means of frontier techniques.In particular, we propose correcting the classic Benefit of the Doubt approach index bymeans of a non-compensatory approach and introducing a more robust estimator usingorder-m techniques. Suggested methods have been tested with reference to infrastructureendowment in European regions.

Keywords: Composite indicators; Data Envelopment Analysis, Order-m frontier, Condi-tional order-m frontier.

1. INTRODUCTION

This contribution is part of a branch of works whose subject is construction of compos-

ite indicators (CIs) that are statistically representative of a specific phenomenon. More

precisely, in the wake of various products by the same authors (among the most recent,

see Mazziotta and Vidoli (2009) and Mazziotta et al. (2010), where the "phenomenon"

considered was infrastructure), the specific objective of this contribution is the testing of

the robustness of some construction methods of composite indicators, particularly those

that refer to approaches like DEA, especially the Benefit of the Doubt (BoD), which has

been mainly used in the above mentioned works.

In essence, what we are trying to do - and we believe this is an attempt at "originality"

- is to combine the potential arising from two different areas of analysis: on the one hand,

the construction of synthetic2 indicators; on the other, the application of efficient frontier

techniques. In both cases it deals with research fields that have experienced significant

260 Vidoli F., Mazziotta C.

3 See, among others, Aiello and Attanasio (2004), Nardo et al. (2005), Mazziotta et al. (2010).

methodological progress in recent years (for example, see Witte and Rogge, 2009), but

which, in our opinion, have not been sufficiently evaluated for their ability to provide more

accurate and solid results - compared to other methods of synthesis of simple indicators -

if used together.

The work is structured as follows: in Section 2 we recall briefly the issues relating to

the construction of synthetic indicators and the need/opportunity to introduce weights in

aggregating simple indicators; in Section 3 we refer briefly to the methodological bases

of the frontiers of production efficiency and we attempt to combine the two search fields,

i.e. weighted synthesis of simple indicators through approaches inspired by DEA and its

additional specifications; in Section 4, and in Section 5 for the conditional measures, we

examine the possibilities that the use of so-called order-m functions (non-compensatory

approaches) using efficiency measures to be able to give more stability to the estimates

(synthetic indicators) in relation to the use of other methods of synthesis; Section 6 is

devoted to empirical verification as previously worked under the methodological profile,

implemented through the application of different approaches considered with the synthe-

sis of simple infrastructure indicators of terrestrial transport (roads and railways) in the

NUTS1 level regions of major European countries; brief conclusions in Section 7 com-

plete the project.

2. FROM SIMPLE TO COMPOSITE INDICATORS: THE PROBLEM OFWEIGHTING

The need to make comparisons between the performance achieved in varied fields within

the same country or between different countries has made increasingly frequent recourse

to synthetic indicators, i.e. indicators that seek to represent, through some form of aggre-

gation, the set of phenomena for each of which it was elaborated and quantified a simple

indicator. Synthetic indicators are widely used for comparisons in the economic, social,

environmental, and production fields: the press continuously makes categories and rank-

ings based on this or that synthetic indicator of a complex phenomenon. These forms of

communication often neglect methodological aspects underlying the construction of syn-

thetic indicators, with the consequence of conferring results of absolute certainty. How-

ever, since the close correlation between (synthesis) methods used and results obtained is

quite evident - and largely accepted in literature3 - it is worthwhile to carefully examine

the specificity of the different possible approaches to the synthesis of simple indicators. In

particular, the issue to be analysed here concerns the allocation of weights in aggregating

simple indicators. Behind this problem is the need to recognise that individual compo-

nents to be represented through the synthetic indicator of the complex phenomenon have

Robust weighted composite indicators by means of frontier methods with … 261

4 Here we refer to the determination of the infrastructure through “physical” indicators, whichinevitably face problems related to standardisation, aggregation and weighting of indicators.For more details, see infra, Section 6.

5 http://composite-indicators.jrc.ec.europa.eu/S6\_weighting.htm6 For a complete review, see Freudenberg (2003) and Nardo et al. (2005) for major applications

and papers.7 Section 3.1 shows the analogous representation (Figure 2) for the CI obtained with the BoD

approach.

different levels of importance: for example, with reference to the infrastructure (the sub-

ject of the application given at the end of this paper), one wonders if in the construction

of its synthetic indicators it is correct to give the same importance to municipal roads

than highways within the same category (road transport) or, more importantly, if it is per-

missible to attribute the same importance to different infrastructure categories (transport,

water, energy, etc.)4. If it is easy to recognise the conceptual merits of the case, and thus

join the aggregation solution "weighted" simple indicators, it is much less easy to locate

a weighted synthesis method which is statistically correct and sufficiently robust.

Along such lines the Joint Research Centre of European Commission asserts that "nouniformly agreed methodology exists to weigh individual indicators before aggregatingthem into a composite indicator".5 Much wider-ranging literature is found regarding the

aggregation methods than for weight systems; however, the two aspects are related and

interwoven and often lead to the same solutions.

Among many approaches proposed in literature6, two relate to our purposes: i) BoD;

and ii) MPCV (Method of Penalty for Coefficient of Variation) proposed by De Muro

et al. (2010). The first approach (DEA techniques) have been used, among others, for

the European labor market analysis by Storrie and Bjurek (2000); for social inclusion

policies at the European Union level by Cherchye et al. (2004) and for internal market

policies by Cherchye et al. (2007). Similarly, some authors have suggested applying

DEA techniques to the Human Development Index (HDI): see Mahlberg and Obersteiner

(2001), Despotis (2005a,b), Cherchye et al. (2008). The second approach, starting with

a linear aggregation, emphasizes the non-compensability between indicators, introducing

penalties for units that present an unbalanced basic indicators set.

For greater clarity, we have simulated for each method proposed the calculation of

the CIs for two basic indicators I1, I2 ∈ [0,1]; for example in Figure 1 the CIs value by

the MPCV method that penalizes units with higher values for I1 and a lower I2 value (and

vice versa) is represented, by rewarding the "unbalanced" set7.


8 For an explanation of all steps, see Nardo et al. (2005).

Figure 1: MPCV distribution, source: simulation results

Several steps are involved in creating CIs: investigate the structure of simple indi-

cators by means of multivariate statistics, handling the problem of missing data that may

be missing either in a random or a non-random fashion, bringing the indicators to the

same unit to avoid adding up apples and pears by normalisation, and finally selecting an

appropriate weighting and aggregation model.8

Our analysis will focus only on the weighting and aggregation phase, that in the field

of CIs is of great importance and has yet to be fully developed.

3. BENEFIT OF THE DOUBT APPROACH

3.1. THE BASIC APPROACH

In the last decade productive efficiency has been widely analysed9 and estimated more

often by using two different approaches: parametric and nonparametric techniques. The

first one specifies a priori a functional form with constant parameters which are estimated

MPCV

Indic.#1

Indic.#2

0.2

0.4

0.6

0.8

0.2 0.4 0.6 0.8

−0.8−0.6

−0.4

−0.2

0.0

0.2

0.4

0.6

−1.0

−0.5

0.0

0.5

1.0


9 For a complete survey, see Fried and Lovell (2008).10 See Debreu (1951) and Farrell (1957).

with statistical and econometric methods. Hence for each observation, the measurement

of efficiency is calculated with reference to the function estimated, and the measure ob-

tained depends strictly on the functional form specified in advance. On the other hand,

nonparametric methods allow for the determination of the relative efficiency without hy-

pothesizing a particular production process and calculating the relative Decision Making

Unit (DMU)’s efficiency through linear programming. These latter methods appear flexi-

ble and close to the CI idea when the problem is designed as a relative maximum search

in a fixed set of data.

The basic productive efficiency framework considers a production technology where

the activity of each DMU is characterized by a set of inputs x ∈ Rp+ used to produce a set

of outputs y ∈ Rq+. The production set H is the set of technically feasible combinations

of (x,y):

Ψ = {(x,y) ∈ Rp+q|x can produce y} (1)

Ψ is the so-called support of H(x,y).Some assumptions are usually made on this set, such as the free disposability of

inputs and outputs, meaning that if (x,y) ∈ Ψ, then (x′,y′) ∈ Ψ, if x′ ≥ x and y′ ≤ y.

The Farrell-Debreu10 efficiency scores, in an input-oriented framework and for a given

production scenario (x,y) ∈ Ψ, are defined as:

θ(x,y) = inf{θ |(θx,y) ∈ Ψ}. (2)

In practice Ψ is unknown and as such, must be estimated from a random sample of

production units χ = {(Xi,Yi)|i = 1, ...n}, where we assume that Prob((Xi,Yi) ∈ Ψ) (so

called deterministic frontier models). The matter is related to the problem of estimating

the support of the random variable (X ,Y ) where Ψ is supposed to be compact.

The most popular nonparametric estimators are based on the Farrell-Debreu envel-

opment ideas; in an input-oriented approach, the Free Disposal Hull (FDH) estimator

(Deprins et al., 1984) is provided by the free disposal hull of the sample points X :

Ψ̂FDH = {(x,y) ∈ Rp+q|y < Yi,x ≥ Xi, i = 1, ...,n} (3)

The FDH efficiency scores are obtained by plugging Ψ̂FDH in equation (2) in place

of the unknown Ψ.

If Ψ is convex, take the convex hull of Ψ̂FDH :


Ψ̂DEA = {(x,y) ∈ Rp+q|y <n∑

i=1γiyi for (γ1, ..γn)

such thatn∑

i=1γi = 1;γi ≥ 0, i = 1, ...,n}

(4)

The application of ideas and techniques developed in the production efficiency field

to the CI is relatively simple because, as suggested by Witte and Rogge (2009), "theBenefit of the Doubt approach is formally tantamount to the original input-oriented CCR-DEA model of Charnes et al. (1978), with all questionnaire items considered as outputsand a dummy input equal to one for all observations."

So, the Farrell-Debreu efficiency scores (input oriented) for a given production sce-

nario (x,y) ∈ Ψ when x is constant and equal to 1 may be written as:

θ(x,y) = inf{θ |(θ ,y) ∈ Ψ} (5)

and consequently the FDH estimator is provided by the particular disposal hull of the

sample points:

Ψ̂FDH = {(�,y) ∈ R1+q|y < Yi, i = 1, ...,n}. (6)

Hypothesizing the convexity of Ψ, the convex hull of Ψ̂FDH can be named Ψ̂BoD in

accordance with Cherchye and Kuosmanen (2002):

Ψ̂BoD ={(�,y) ∈ R

1+q|y <n∑

i=1γiyii for (γ1, . . . ,γn)

such thatn∑

i=1γi = 1;γi ≥ 0, i = 1, . . . ,n

}.

(7)

In Figure 2 we plot the BoD distribution; it may be readily noted that the composite

score depends exclusively on the frontier’s distance and not, contrary to the MPCV, on

the relationship between simple indicators. In Subsection 3.3 we join BoD and MPCV

methods to overcome this drawback in a non-compensatory framework.

The main drawbacks are directly linked with the DEA solution: for example since

the weights are unit specific, cross-unit comparisons are not possible and the values of

the scoreboard depend on the benchmark performance. But with respect to the specific

problem of CIs, there are three more important drawbacks we will discuss specifically

in the following sections: the multiplicity of equilibria (see Subsection 3.2), the non-

compensability between indicators (see Subsection 3.3) and the estimation robustness

(see Section 4).


11 More precisely we added an “Assurance regions”, type I (AR I), see Thanassoulis et al. (2004).

Figure 2: BoD distribution, source: simulation results

3.2. MULTIPLICITY OF EQUILIBRIA: VARIANCE WEIGHTED BOD

The first drawback discussed is the multiplicity of equilibria; equation (7), in fact, hiding

a problem of the multiple equilibria makes weights not uniquely determined (even though

the CI is unique). It is also worth noting that multiple solutions are likely to depend

upon the set of constraints imposed on the weights of the maximisation problem: the

wider the range of the variation of weights, the lower the possibility of obtaining a unique

solution. The optimisation process could lead to many zero weights if no restrictions on

the weights are imposed, so setting restrictions on weights is necessary for this method

to be of practical use. To fix this problem, many weighting schema in recent years have

been proposed (see for instance Makui et al., 2008 or Rogge, 2012) introducing additional

information (e.g. utility function) to the optimisation problem.

In 2009, following original methodology, we proposed (see Mazziotta and Vidoli,

2009) adding a particular set of weight constraints11 endogenously determined to account

for the variability of each simple indicator, in terms of sample variance of each indicator.

This solution, applied to the calculation of a synthetic indicator of infrastructure, is based

on the assumption, consistent with a policy aimed at balancing regional variability, that

BoD

Indic.#1

Indic.#2

0.2

0.4

0.6

0.8

0.2 0.4 0.6 0.8

0.1

0.2

0.3

0.4

0.50.6

0.7

0.8

0.9

1.0

0.0

0.2

0.4

0.6

0.8

1.0


12 To bypass this assumption, future developments of this methodology may involve theanalysis of the Kernel density estimate of the simple indicators and their own sample variance.

13 For a complete survey about compensatory and non-compensatory approaches, see Vansnick(1990).

between simple indicators it should be interpreted as an index (negative) of the gap be-

tween the territorial units concerned. Our basic thesis involved weighing simple indicators

by their own sample variance; thus, indicators with a high variability will strongly affect

the composite indicator. However, there are consequences to this approach: our measure-

ment has to be read as a "gap indicator" among the unit characteristics. Our preliminary

hypothesis is that every single indicator Iq, q = 1, . . . ,Q is a probabilistic variable, having

a normal distribution12:

Iq ∼ N(µIq ,σIq

), ∀ q = 1, . . . ,Q. (8)

In this way the vertical variance of each indicator can be computed in a standard

probabilistic setting and the unbiased variance confidence interval is:

P

(n−1

χ2n−1,1−α/2

S2< σ2 <

n−1

χ2n−1,α/2

)= 1−α (9)

P(lowIq < σ2 < highIq

)= 1−α. (10)

Even when the underlying distribution is not normal, the procedure is still used to

obtain the approximate confidence bounds for the variance estimated. If the distribution

is not too far from the Normal one, the procedure is sufficiently robust and usually works

well. We can use lowIq and highIq for each indicator to reconstruct the marginal rates of

substitution among indicators:

lowIihighIj

≤ wIiwIj

≤ lowIj

highIi, ∀ i, j = 1, ...,Q. (11)

Contrasting the lower limit of the variance of the confidence interval with the maximum ofanother one assumes a "benefit of doubt" attitude, by not imposing an exact relationship

among weights, but thereby establishing a range in which every unit obtains the maximum

relative weight.

3.3. NON-COMPENSATORY BOD: BOD-PCV

When proceeding to the synthesis of simple indicators, whatever the shape chosen, one

must address the issue of compensability, namely: to what extent can we accept that

the high score of an indicator go to compensate the low score of another indicator? In

the field dealt with here (infrastructure), the problem arises with reference to different


Figure 3: BoD-PCV distribution, source: simulation results

possible variabilities of indicators relating to the individual categories. In other words,

one wonders whether, and to what extent, it is correct for example that a large supply of

water resources go to offset a modest section of highways, and so on.

The problem is intertwined with the issue of attribution of weights. Munda and

Nardo (2005) affirm the "if one wants the weights to be interpreted as “importance co-efficients” (or equivalently symmetrical importance of variables) non-compensatory ag-gregation procedures must be used"13.

To overcome the "compensatory" drawback, we can easily incorporate the De Muro

et al. (2010) idea in the basic BoD model, assuming that each indicator may not be re-

placed by others or is replaced only in part. According to this hypothesis, the method

involves introducing a penalty for units that have not balanced a budget for all compo-

nents, such as:

BoD_PCVi = BoDi(1− cv2i ), ∀ i = 1, ...,N (12)

where cv2i represents the coefficient of variation for the unit i between all indicators. This

approach, therefore, allows for the penalisation of the units that, while having an equal

BoD score, have a greater imbalance among indicators (see Figure 3).

Bod−PCV

Indic.#1

Indic.#2

0.2

0.4

0.6

0.8

0.2 0.4 0.6 0.8

−0.4−0.2

0.0

0.2

0.4

0.6

0.8

−0.5

0.0

0.5

1.0


Figure 4: BoD-PCV distribution in presence of outliers, source: simulation results

With respect to the individual BoD and MPCV approaches, the BoD-PCV approach

has two advantages: (i) it consistently takes into account the benchmark units on the

frontier, and (ii) at the same time, in the case of non-compensatory issues, it penalises the

presence of unbalanced data.

It is moreover worth noting that BoD-PCV model (see Figure 4 in which we added

the presence of an outlier), is logically and strongly influenced by outliers in the simple

indicators. This problem is connected with the drawback of the estimate robustness, dealt

in the following subsection.

4. ROBUST BOD: ORDER-m METHODS APPLICATION TO CIS

The latest main drawbacks of DEA/FDH nonparametric estimators is their sensitivity to

extreme values and outliers. Even in this case we can exploit specific methods devel-

oped for the estimation of production efficiency, in particular the robust order-m method

(Daraio and Simar, 2005); in the following, the basic idea is presented and then the mo-del explained in detail.

In this context, Cazals et al. (2002) proposed a more robust nonparametric estimator

of the frontier. It is based on the concept of the expected minimum input function of

Bod−PCV − with outlier

Indic.#1

Indic.#2

0.2

0.4

0.6

0.8

1.0

1.2

0.5 1.0 1.5−0.4

−0.2

0.0

0.2

0.4

0.6

−0.6

−0.4

−0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2


order-m. Extending these ideas to the full multivariate case, Daraio and Simar (2005)

defined the concept of the expected order-m input efficiency score. They affirmed that:

"in place of looking for the lower boundary of the support of FX (x|y), as was typically thecase for the full-frontier (DEA or FDH), the order-m efficiency score can be viewed as theexpectation of the minimal input efficiency score of the unit (x,y), when compared to munits randomly drawn from the population of units producing more outputs than the levely."

We extend Daraio and Simar (2005)’s idea into CIs framework by drawing with

replacement m observations from the original sample of n observations and choosing only

from those observations which are obtaining higher performance scores (I1, I2) - grey lines- than the evaluated observation (see observation C in the Figure 5).

In other words, and practically speaking:

• We draw m observation only from those observations which are obtaining higher

performance scores than the evaluated observation C;

• We label this set as SETtm;

• We estimate BoD scores relative to this subsample SETtm for T times (see Figure

6);

• Having obtained the T scores, we compute the arithmetic average.

Figure 5: The support of unit C


This is certainly a less extreme benchmark for the unit C than the "absolute" maxi-

mum achievable level of output. C is compared to a set of m peers (potential competitors)

producing more than its level and we take, as a benchmark, the expectation of the maxi-

mum achievable CI in place of the absolute maximum CI.

More accurately, Daraio and Simar (2005) propose (for a more complete theoretical

summary see Daraio and Simar, 2007a and Fried and Lovell, 2008) a probabilistic formu-

lation of efficiency concepts, assuming that the Data Generating Process (DGP) of (X ,Y )is completely characterized by:

HXY (x,y) = Prob(X ≤ x,Y ≥ y) (13)

so HXY (x,y) can be interpreted as the probability for a unit operating at the level (x,y)of being dominated. Note that it is monotone non-decreasing with x and monotone non-

increasing with y. This joint probability can be decomposed as follows (yet in a input-

oriented framework):

HXY (x,y) = Prob(X ≤ x|Y ≥ y)Prob(Y ≥ y) = SX |Y (x|y)FY (y). (14)

Figure 6: Subsamples

An input oriented efficiency score θ̂(x,y) for (x,y) ∈ Ψ is defined for all y values

with FY (y)> 0 as

θ̂(x,y) = inf{θ |(θx0,y0) ∈ Ψ}= inf{θ |H(θx,y)> 0}. (15)


Applying Daraio and Simar (2005)’s ideas to the particular case of composite indi-

cators, equation (13) can be written:

H(x,y) = Prob(X ≡ 1,Y ≥ y) (16)

where Ψ is the support of H(x,y).

So Farrell-Debreu (input) efficiency score, since Prob(X ≡ 1|Y ≥ y) = 1 can be

written as:

H(x,y) = Prob(X ≡ 1|Y ≥ y)Prob(Y ≥ y) = FY (y) (17)

θ(1,y0) = inf{θ |(θ ,y0) ∈ Ψ}= inf{θ |H(θ ,y)> 0}. (18)

The effects of the introduction of order-m methods can be appreciated in the Figure 7and 8, respectively for BoD and BoD-PCV distribution.

Figures 7: Order-m BoD distribution in presence of outliers, source: simulation results

Order−m − with outlier

Indic.#1

Indic.#2

0.2

0.4

0.6

0.8

1.0

1.2

0.5 1.0 1.5

0.2

0.4

0.4

0.6

0.6

0.8

0.80.8

0.80.8

0.8

0.8

0.8

0.8

0.80.80.8

1.0

1.01.01.01.01.0 1.0

1.0

1.01.01.01.0

1.01.01.0

1.01.01.01.01.0

1.0

1.01.0

1.0

1.0

1.0

1.21.21.21.21.21.21.21.2

1.21.21.21.21.21.21.21.21.2

1.21.21.21.21.21.21.21.2

0.0

0.5

1.0

1.5

2.0


Figure 8: OrderBoD-PCV distribution in presence of outliers, source: simulation results

5. CONDITIONAL ROBUST BOD (CONDITIONAL ORDER-m METHODS AP-PLICATION TO CIS)

Using the probabilistic formulation, Cazals et al. (2002) also suggested a conditional effi-ciency approach, which includes external environmental factors that might influence the

production process, but are neither inputs nor outputs under the control of the producer.

Daraio and Simar (2005) extended these ideas to a more general multivariate setup and

proposed a practical methodology to evaluate the effect of environmental variables in the

production process. Later, an extension to convex nonparametric models was proposed

by Daraio and Simar (2007b) and also a significant number of works has been done to

prove the consistency and the asymptotic properties of different conditional efficiency

estimators (see Jeong et al., 2010).

The conditional efficiency approach consists of conditioning (for simplicity, we re-

port only a univariate case) the production process to a given value of Z = z, where Zdenotes a variable characterising the operational environment. The joint probability func-

tion given Z = z can be defined as:

Order−m −PCV − with outlier

Indic.#1

Indic.#2

0.2

0.4

0.6

0.8

1.0

1.2

0.5 1.0 1.5

0.0

0.50.5

0.5

0.5

1.0

1.0

1.0

1.01.0

1.01.01.01.01.0

1.01.01.01.0 1.0

1.01.01.01.01.01.01.0

−1.0

−0.5

0.0

0.5

1.0

1.5

2.0

2.5


HXY |Z(x,y) = Prob(X ≤ x,Y ≥ y|Z = z)= Prob(X ≤ x|Y ≥ y,Z = z)Prob(Y ≥ y|Z = z)= SX |Y,Z(x|y,z)FY (y|z).

(19)

Daraio and Simar (2005), by analogy with the output Farrell efficiency score, define

the conditional output efficiency measure:

θ̂(x,y|z) = inf{θ |(θx0,y0|z) ∈ Ψ}= inf{θ |H(θx,y|z)> 0}. (20)

To reduce the deterministic nature, again instead of using the full support of S(x|y,z)we can use the expected value of maximum output efficiency score of the unit (x,y), when

compared to m units randomly drawn from the population of units for which X ≤ x. Like

the unconditional order-m efficiencies, conditional efficiency measure λm(x,y|z) can be

expressed using the following integral:

λm(x,y|z) =∫ ∞

0[1− (1−SX |Y,Z(ux|y,z))m]du. (21)

Estimating SX |Y,Z non-parametrically is somewhat more difficult than it is for the

unconditional case, as we need to use smoothing techniques in z (due to the equality

constraint Z = z).

Witte and Kortelainen (2008) proposed adapting the nonparametric conditional effi-

ciency measures to include mixed (i.e. both continuous and discrete) exogenous variables

by specifying an appropriate kernel function which smooths the mixed variables.

The conditional output efficiency measure (X ≡ 1) can be expressed as:

θ̂(x,y|z) = inf{θ |(θ ,y0|z) ∈ Ψ}= inf{θ |H(θ ,y|z)> 0}. (22)

6. AN APPLICATION TO THE INFRASTRUCTURE ENDOWMENT IN EURO-PEAN REGIONS

Like previous works (Mazziotta and Vidoli, 2009; Mazziotta et al., 2010), the proposed

new approach concerning the construction of CIs has been applied to infrastructure en-

dowment. As already mentioned, the problems associated with the construction of CI

arise where it is preferable to quantify the level of infrastructure through physical indi-

cators, and not, as is frequently done, based on monetary indicators of public capital,

generally reconstructed through the perpetual inventory method. Our opinion is that the

correct approach for an estimate of infrastructure at the territorially disaggregated level,

especially in Italy with its diverging development between north and south, should be

based only on simple physical indicators (kilometres of roads, cubic meters of water,

MW of electricity, etc.). The reasons for this position are described in detail in Mazziotta

(2005) and more succinctly, in Bracalente et al. (2006).


14 The most appropriate level for an effective analysis of infrastructure should be that concerningthe NUTS2 regions (in Italy, 20 regions) or even NUTS3 (in Italy, the provinces). Here, giventhe methodological character of the application, it is considered satisfactory to operate evenat a territorially more aggregate level, such as NUTS 1.

15 http://epp.eurostat.ec.europa.eu/portal/page/portal/statistics/themes,Regional transport statistics (reg_tran).

In the present paper we have limited the analysis to terrestrial transport infrastructure

(roads and railways) for the following reasons: i) the relevance of this type of infrastruc-

ture in any regional development strategy; ii) coherence with some previous applications

(Mazziotta et al., 2010); and iii) the opportunity of following the same pattern shown in

the methodological part of this paper, in particular in Figures 1 to 8, where two dimensions

(that is, two infrastructure categories) have been considered.

Because the purpose of this paper is essentially methodological, the application and

the related results serve mostly as an empirical example, as opposed to an effective analy-

sis of infrastructure endowment. In any case, we believe the results attained are very use-

ful to verify the robustness of the proposed methods concerning the construction of CIs.

The following methods have been considered: BoD, BoD-PCV, Order-m BoD, Order-mBoD-PCV (for the meaning of these acronyms, see previous Sections).

The application concerns two main categories of terrestrial transport: roads (sepa-

rately, motorways and other roads) and railways (separately, electrified railways lines and

railways lines double). The data set includes NUTS1 regions14 of main European coun-

tries (in terms of population and GDP): France, Germany, Italy, Spain. Unlike other

works (Di Palma and Mazziotta, 2003), the United Kingdom has been omitted, due to lack

of data concerning railways. Source of data base: Eurostat, Statistics by theme, 201215.

More specifically, we used the following basic indicators:

• Motorways - Kilometres per 1000 km2;

• Other roads - Kilometres per 1000 km2;

• Electrified railway lines Kilometres per 1000 km2;

• Railway lines with double and more tracks - Kilometres per 1000 km2.

and the relative indicators have been calculated as:

IRoads =2 ·Motorways+Other roads

3

ITrains =2 ·Railway lines double+Electrified railway lines

3

(23).


16 In order to highlight the correspondence of the obtained results with the methodological partof the paper, Figures 9, 10, 11 and 12 propose the same framework that was used in previousSections. In fact, the results are represented by points corresponding to the infrastructureindicators in European regions, while the framework is only indicative of the virtual patternof each considered approach.

The main results of the application (see Figures16 9, 10, 11 and 12 and Table 1) are

the following:

• The four above mentioned approaches produce four corresponding rankings of

considered European regions. The cograduation index (Spearman Index, see Ta-

ble 1) shows the distance between such rankings. This distance is greater in the

comparison between the original BoD approach and the revised BoD approach by

means of MPCV index. Evidently, the correction introduced in the construction of

CIs by non-compensatory approach gives more changes in ranking than the cor-

rection caused by the order-m approach;

• Two regions are always on the top of the ranking: Berlin (the first region for rail-

ways) and Ile de France (the first region for roads). Their endowment level is

always much more elevated in comparison to all the remaining areas: this situation

produces a "crushing effect" that risks flattening the position of the other regions.

The correction introduced by the order-m approach mitigates this effect and high-

lights the differences among the infrastructure levels of the considered regions.

From this point of view the order-m approach can be considered an improvement

in regard to the other methods (BoD and MPCV);

• With reference to Italian regions, their ranking is not notably influenced by dif-

ferent approaches: north-east is always in the first half position, north-west and

central are at the beginning of second half, and south and the islands are located at

the bottom part of the ranking.

Table 1: Spearman index for different approaches of construction ofcomposite indicators

Approaches BoD BoD-PCV Order-m BoD Order-m BoD-PCV

BoD 1

BoD-PCV 0.78 1

Order-m BoD 0.89 0.92 1

Order-m BoD-PCV 0.70 0.93 0.93 1


Figure 9: European infrastructure endowment - BoD method,source: elaboration on EUROSTAT data

Figure 10: European infrastructure endowment - Order-m BoD method,source: elaboration on EUROSTAT data

Table 2 in Annex shows the complete results for NUTS1 regions.

Roads

Trains

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0

0.2

0.4

0.6

0.8

1.0

●●

●

●

●●

●

●●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●●●

●●●

●●

●●

GE−BWGE−BAY

GE−BER

GE−BRA

GE−BREGE−HA

GE−HE

GE−MEGE−NI

GE−NW

GE−RP

GE−SAA

GE−SACGE−SCA

GE−SCHGE−THUSP−NOR

SP−MAD

SP−CEN

SP−EST

SP−SUR

FR−PAR

FR−BAS

FR−NPC

FR−ESTFR−OUEFR−SOU

FR−CESFR−MEDIT−NOV

IT−NESIT−CEN

IT−SUDIT−ISO

Roads

Trains

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.1

0.2

0.3

0.4

0.5

0.6

0.60.6

0.6

0.6

0.7

0.7

0.70.7

0.80.80.8

0.80.8

0.8

0.80.8

0.8

0.9

0.9

0.9

0.9

0.9 0.90.90.90.9

0.9

0.9

0.90.9

0.90.90.9

0.90.9

1.01.0

1.01.01.0

1.01.0

1.01.01.0

1.01.0

1.0

1.0

1.01.0

1.01.0

1.0

1.01.0

1.01.0

1.0

1.1

1.1 1.11.1 1.1

1.1

1.1 1.1

1.11.11.11.1

1.11.1

1.1

1.1

1.1

1.11.1

1.1

1.11.11.1

1.11.11.1

1.11.11.1

1.1

1.11.1

1.1 1.1

1.1

1.21.21.21.21.21.21.21.2

1.21.21.21.21.21.21.21.2

1.21.21.21.2

1.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

��

�

�

��

�

��

�

�

�

��

��

�

�

�

�

�

�

�

�

��

��

��

��

�

GE−BWGE−BAY

GE−BER

GE−BRA

GE−BREGE−HA

GE−HE

GE−MEGE−NI

GE−NW

GE−RP

GE−SAA

GE−SACGE−SCA


SP−MAD

SP−CEN

SP−EST

SP−SUR

FR−PAR

FR−BAS

FR−NPC

FR−EST

FR−OUEFR−SOU

FR−CESFR−MED

IT−NOVIT−NES

IT−CENIT−SUD

IT−ISO


Figure 12: European infrastructure endowment - Order-m BoD-PCV method,source: elaboration on EUROSTAT data

Figure 11: European infrastructure endowment - BoD-PCV method,source: elaboration on EUROSTAT data

Roads

Trains

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

−0.4−0.2

0.0

0.2

0.4

0.6

0.8

−0.5

0.0

0.5

1.0

��

�

�

��

�

��

�

�

�

��

��

�

�

�

�

�

�

�

�

��

��

��

��

�

GE−BWGE−BAY

GE−BER

GE−BRA

GE−BREGE−HA

GE−HE

GE−MEGE−NI

GE−NW

GE−RP

GE−SAA

GE−SACGE−SCA


SP−MAD

SP−CEN

SP−EST

SP−SUR

FR−PAR

FR−BAS

FR−NPC

FR−EST

FR−OUEFR−SOU

FR−CESFR−MED

IT−NOVIT−NES

IT−CENIT−SUD

IT−ISO

Roads

Trains

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

−0.4

−0.2

0.0

0.2

0.4

0.4

0.6

0.6

0.60.6

0.6

0.60.6

0.6

0.60.6

0.6

0.8

0.8

0.80.8

0.80.8

1.0

1.01.0

1.01.0

1.0

1.01.01.01.0

1.01.01.0

1.0

1.0

1.2

−0.5

0.0

0.5

1.0

��

�

�

��

�

��

�

�

�

��

��

�

��

�

�

�

�

��

��

��

�

GE−BWGE−BAY

GE−BER

GE−BRA

GE−BREGE−HA

GE−HE

GE−MEGE−NI

GE−NW

GE−RP

GE−SAA

GE−SACGE−SCA


SP−MAD

SP−CENSP−ESTSP−SUR

FR−PAR

FR−BAS

FR−NPC

FR−EST

FR−OUEFR−SOUFR−CESFR−MED

IT−NOVIT−NES

IT−CENIT−SUD

IT−ISO


7. FINAL REMARKS

In this paper we have presented several new approaches for the construction of CIs. In

particular, two objectives were pursued: i) the correction of the BoD index by means of a

non-compensatory approach; and ii) the introduction of the order-m approach as a more

robust estimator in the field of nonparametric frontier techniques.

For the first objective, we attempted to integrate BoD the index using a non-compensatory

approach, introducing a penalty for unbalanced simple indicators in the construction of

composite indicators. The resulting approach (BoD-PCV) presents two advantages: it

takes into account the benchmark units on the frontier (peculiarity of BoD), and at the

same time penalises the presence of the unbalanced simple indicators (peculiarity of

MPCV).

For the second objective, we introduced the concept of the expected minimum input

function of order-m in the construction of composite indicators which is relevant overall

in the presence of outliers in a frontier framework. This approach has been applied with

reference to both synthesis methods indicated above (BoD and Bod-PCV), obtaining a

more robust estimation of CIs.

Finally, these approaches were tested with reference to infrastructure endowment in

European regions (terrestrial transport). The obtained results confirm the improvement in

the robustness of CIs by the introduction of order-m technique.

Robust weighted composite indicators by means of frontier methods with … 279A

NN

EX

Tabl

e2:

Eur

opea

nN

UT

S1tr

ansp

orti

nfra

stru

ctur

eby

com

posi

tein

dica

tors

(GE

=G

erm

any)

NU

TS

1A

BB

RR

oad

sT

rain

sB

oD

Ran

kO

rder

-mR

ank

BoD

Ran

kO

rder

-mR

ank

PC

VP

CV

1B

aden

-Würt

tem

ber

gG

E-B

W0,2

69977

0,0

99161

0,3

278

19

0,5

922

16

0,0

964

17

0,3

608

14

2B

ayer

nG

E-B

AY

0,3

06017

0,0

6684

0,3

313

18

0,4

933

20

0,0

106

22

0,1

726

23

3B

erli

nG

E-B

ER

0,0

29957

11,0

000

11,5

877

10,5

291

21,1

168

2

4B

randen

burg

GE

-BR

A0,1

42258

0,0

82735

0,2

213

29

0,4

529

24

0,0

890

19

0,3

206

17

5B

rem

enG

E-B

RE

00,5

58577

0,5

656

90,9

208

50,0

656

20

0,4

208

10

6H

amburg

GE

-HA

0,0

19104

0,5

77879

0,5

970

81,1

082

40,1

290

13

0,6

402

6

7H

esse

nG

E-H

E0,2

59806

0,1

36359

0,3

554

15

0,7

533

12

0,1

996

11

0,5

975

8

8M

eckle

nburg

-Vorp

om

mer

nG

E-M

E0,1

29653

0,0

3862

0,1

710

34

0,2

930

32

-0,0

995

27

0,0

226

27

9N

ieder

sach

sen

GE

-NI

0,2

32777

0,0

65871

0,2

701

23

0,4

589

23

-0,0

093

23

0,1

795

22

10

Nord

rhei

n-W

estf

alen

GE

-NW

0,3

66046

0,1

71969

0,4

646

12

0,8

723

80,2

842

80,6

920

5

11

Rhei

nla

nd-P

falz

GE

-RP

0,3

07169

0,0

83764

0,3

400

17

0,5

324

19

0,0

543

21

0,2

466

20

12

Saa

rlan

dG

E-S

AA

0,2

06084

0,1

80603

0,3

585

14

0,8

786

60,3

256

40,8

456

4

13

Sac

hse

nG

E-S

AC

0,2

27573

0,0

99669

0,2

980

22

0,5

843

18

0,1

026

16

0,3

889

13

14

Sac

hse

n-A

nhal

tG

E-S

CA

0,1

56778

0,0

90421

0,2

388

28

0,4

818

21

0,1

046

15

0,3

475

16

15

Sch

lesw

ig-H

ols

tein

GE

-SC

H0,1

88423

0,0

38911

0,2

180

30

0,3

190

30

-0,1

108

30

-0,0

099

29

16

Thüri

ngen

GE

-TH

U0,1

82386

0,0

39589

0,2

122

31

0,3

340

29

-0,1

094

29

0,0

124

28


Tabl

e3:

Eur

opea

nN

UT

S1tr

ansp

orti

nfra

stru

ctur

eby

com

posi

tein

dica

tors

(SP

=Sp

ain,

FR=

Fran

ce,I

T=

Ital

y)

NU

TS

1A

BB

RR

oad

sT

rain

sB

oD

Ran

kO

rder

-mR

ank

BoD

Ran

kO

rder

-mR

ank

PC

VP

CV

17

Nore

ste

(ES

)S

P-N

OR

0,1

63

03

10

,01

66

51

0,1

93

53

30

,26

10

33

-0,2

138

32

-0,1

463

32

18

Com

unid

adde

Mad

rid

SP

-MA

D0

,11

78

44

0,1

24

93

20

,24

34

27

0,6

25

91

50,2

288

90,6

113

7

19

Cen

tro

(ES

)S

P-C

EN

0,3

20

09

10

,00

02

66

0,3

44

91

60

,39

52

27

-0,1

543

31

-0,1

039

31

20

Est

e(E

S)

SP

-ES

T0

,22

45

99

0,0

37

34

40

,25

29

26

0,3

47

02

8-0

,1045

28

-0,0

104

30

21

Sur

(ES

)S

P-S

UR

0,2

26

07

30

0,2

54

32

50

,30

28

31

-0,2

457

33

-0,1

972

33

22

Île

de

Fra

nce

FR

-PA

R1

0,2

60

49

91

,00

00

11

,54

88

20,7

067

11,2

555

1

23

Bas

sin

Par

isie

nF

R-B

AS

0,7

25

78

60

,05

04

61

0,7

35

76

0,8

76

67

0,3

007

50,4

416

9

24

Nord

-P

as-d

e-C

alai

sF

R-N

PC

0,7

83

83

70

,14

66

13

0,7

91

73

1,2

01

53

0,4

493

30,8

591

3

25

Est

(FR

)F

R-E

ST

0,5

45

97

30

,06

76

11

0,5

62

51

10

,74

30

13

0,1

727

12

0,3

532

15

26

Oues

t(F

R)

FR

-OU

E0

,75

22

74

0,0

28

69

70

,76

13

40

,87

03

10

0,2

980

60,4

071

12

27

Sud-O

ues

t(F

R)

FR

-SO

U0

,67

21

86

0,0

14

04

40

,68

41

70

,78

86

11

0,2

046

10

0,3

091

19

28

Cen

tre-

Est

(FR

)F

R-C

ES

0,7

29

35

10

,03

74

98

0,7

39

25

0,8

71

89

0,2

881

70,4

207

11

29

Méd

iter

ranée

FR

-ME

D0

,54

76

72

0,0

25

48

50

,56

41

10

0,6

65

51

40,1

086

14

0,2

099

21

30

Nord

-Oves

tIT

-NO

V0

,27

88

08

0,0

61

89

10

,30

51

21

0,4

67

52

2-0

,0132

24

0,1

491

24

31

Nord

-Est

IT-N

ES

0,3

35

54

0,0

95

59

10

,37

13

13

0,5

87

61

70,0

930

18

0,3

093

18

32

Cen

tro

(IT

)IT

-CE

N0

,23

56

69

0,0

60

78

60

,26

74

24

0,4

10

72

6-0

,0276

25

0,1

157

25

33

Sud

IT-S

UD

0,2

80

23

30

,03

39

62

0,3

06

52

00

,43

08

25

-0,0

854

26

0,0

389

26

34

Isole

IT-I

SO

0,1

70

49

30

,00

03

78

0,2

00

73

20

,24

32

34

-0,2

971

34

-0,2

546

34


REFERENCES

Aiello, F. and Attanasio, M. (2004). How to transform a batch of simple indicators to make up a uniqueone? In: Atti della XLII Riunione SIS di Bari, CLEUP, Padova: pp. 327–338.

Bracalente, M., Palma, D. and Mazziotta, C. (2006). Federalismo, equità e sviluppo, In: Investimenti,capitale pubblico e dotazione fisica di infrastrutture nelle regioni italiane, Il Mulino, Bologna:253–288. Cazals, C., Florens, J. and Simar, L. (2002). Nonparametric frontier estimation: Arobust approach. Journal of Econometrics, 106(1): 1–25.

Charnes, A., Cooper, W. and Rhodes, W. (1978). Measuring the efficiency of decision making units.European Journal of Operational Research, 2(4): 429–444.

Cherchye, L. and Kuosmanen, T. (2002). Benchmarking sustainable development: A syntheticmetaindex approach. Technical Report, EconWPA Working Papers.

Cherchye, L., Lovell, C.K., Moesen, W. and Puyenbroeck, T.V. (2007). One market, one number? Acomposite indicator assessment of EU internal market dynamics. European Economic Review,51(3): 749–779.

Cherchye, L., Moesen, W. and Puyenbroeck, T. (2004). Legitimately diverse, yet comparable: Onsynthesizing social inclusion performance in the EU. Journal of Common Market Studies,42(5): 919–955.

Cherchye, L., Ooghe, E. and Van Puyenbroeck, T. (2008). Robust human development rankings.Journal of Economic Inequality, 6: 287–321.

Daraio, C. and Simar, L. (2005). Introducing environmental variables in nonparametric frontiermodels: A probabilistic approach. Journal of Productivity Analysis, 24(1): 93–121.

Daraio, C. and Simar, L. (2007a). Advanced Robust and Nonparametric Methods in EfficiencyAnalysis - Methodology and Applications. Kluwer Academic Publishers, Boston.

Daraio, C. and Simar, L. (2007b). Conditional nonparametric frontier models for convex andnonconvex technologies: A unifying approach. Journal of Productivity Analysis, 28: 13–32.

De Muro, P., Mazziotta, M. and Pareto, A. (2010). Composite indices of development and poverty:An application to MDGs. Social Indicators Research, 104(1): 1–18.

Debreu, G. (1951). The coefficient of resource utilization. Econometrica, 19(3): 273–292.

Deprins, D., Simar, L.L. and Tulkens, H. (1984). The performance of public enterprises: Conceptsand measurements, In: M. Marchand, P. Pestieau and H. Tulkens (Eds.), Measuring LaborInefficiency in Post Offices, 243–267. North-Holland, Amsterdam.

Despotis, D. (2005a). Measuring human development via data envelopment analysis: The case of Asiaand the Pacific. Omega, 33 (5): 385–390.

Despotis, D.K. (2005b). A reassessment of the human development index via data envelopmentanalysis. Journal of the Operational Research Society, 56: 969–980.

Di Palma, M. and Mazziotta, C. (2003). Infrastrutture, competitività e sviluppo: Il caso italiano.Economia italiana, 3.

Farrell, M.J. (1957). The measurement of productive efficiency. Journal of the Royal Statistic Society,120(III): 253–281.

Freudenberg, M. (2003). Composite indicators of country performance: A critical assessment.Technical Report, OECD Science, Technology and IndustryWorking Papers 2003/16, OECD,Directorate for Science, Technology and Industry.


Fried, H.O. and Lovell, C.A.K. (2008). The Measurement of Productive Efficiency and ProductivityGrowth. Oxford University Press, New York.

Jeong, S., Park, B. and Simar, L. (2010). Nonparametric conditional efficiency measures: Asymptoticproperties. Annals of Operations Research, 173(1): 105–122.

Mahlberg, B. and Obersteiner, M. (2001). Remeasuring the HDI by data envelopement analysis. Tech.Rep., IIASA, Interim Report IR-01-069, Laxenburg, Austria.

Makui, A., Alinezhad, A., Mavi, R.K. and Zohrehbandian, M. (2008). A goal programming methodfor finding common weights in DEA with an improved discriminating power for efficiency.Journal of Industrial and Systems Engineering, 1(4): 293–303.

Mazziotta, C. (2005). La stima del capitale pubblico a livello regionale: Una riflessione di metodo.In: M. Carlucci and G. Esposito (Eds.), Statistica economica e strumenti di analisi. Studi inmemoria di Antonino Giannone. Roma, Istituto per la Contabilità Nazionale (Iscona), ISTAT,Roma: 139–175.

Mazziotta, C., Mazziotta, M., Pareto, A. and Vidoli, F. (2010). La sintesi di indicatori territoriali didotazione infrastrutturale: Metodi di ostruzione e procedure di ponderazione a confronto.Rivista di Economia e Statistica del Territorio, 1: 7–33.

Mazziotta, C. and Vidoli, F. (2009). La costruzione di un indicatore sintetico ponderato. Un’applica-zione della procedura benefit of the doubt al caso della dotazione infrastrutturale in italia.Scienze Regionali, 8(1): 35–69.

Munda, G. and Nardo, M. (2005). Constructing consistent composite indicators: the issue of weights.Technical Report, EUR 21834 EN, European Commission.

Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A. and Giovannini, E. (2005). Handbookon constructing composite indicators: Methodology and user guide. OECD Statistics WorkingPapers 2005/3, OECD, Statistics Directorate.

Rogge, N. (2012). Undesirable specialization in the construction of composite policy indicators: Theenvironmental performance index. Ecological Indicators, 23: 143–154.

Storrie, D. and Bjurek, H. (2000). Benchmarking european labour market performance withefficiency frontier techniques. Technical Report, CELMS Discussion papers, GoteborgUniversity.

Thanassoulis, E., Portela, M.C. and Allen, R. (2004). Incorporating value judgments in DEA. In:W.W. Cooper, L.M. Seiford, and J. Zhu, (Eds.), Handbook on Data Envelopment Analysis, vol.71 of International Series in Operations Research & Management Science. Springer, NewYork: 99–138.

Vansnick, J. (1990). Measurement theory and decision aid. In: C.A. Bana e Costa (Eds.) Readings inMultiple Criteria Decision Aid. Springer, Berlin: 81–100

Witte, K.D. and Kortelainen, M. (2008). Blaming the exogenous environment? Conditional efficiencyestimation with continuous and discrete environmental variables. Technical Report, WorkingPaper Series ces0833, Katholieke Universiteit Leuven, Centrum voor Economische Studien.

Witte, K.D. and Rogge, N. (2009). Accounting for exogenous influences in a benevolent performanceevaluation of teachers. Technical Report, Working Paper Series ces0913, Katholieke UniversiteitLeuven, Centrum voor Economische Studien.

ROBUST WEIGHTED COMPOSITE INDICATORS BY MEANS OF...

Documents

Transcript of ROBUST WEIGHTED COMPOSITE INDICATORS BY MEANS OF...