Pm

PC

a

ARA

KNEOR

1

iapgttiTi

stEpbeuteftt

p

0h

Resources, Conservation and Recycling 67 (2012) 56– 66

Contents lists available at SciVerse ScienceDirect

Resources, Conservation and Recycling

journa l h o me pag e: www.elsev ier .com/ locate / resconrec

erformance assessment of refuse collection services using robust efficiencyeasures

edro Simões ∗, Pedro Carvalho1, Rui Cunha Marques1

entre for Management Studies (CEG-IST), IST, Technical University of Lisbon, Av. Rovisco Pais, 1049-001 Lisbon, Portugal

r t i c l e i n f o

rticle history:eceived 5 January 2012ccepted 21 July 2012

a b s t r a c t

In Portugal, municipalities have been entrusted with a growing number of powers and responsibilities,together with budget restrictions. Therefore, refuse collection services have gained particular interest

eywords:on-parametric techniquesfficiencyperational environment

since in most municipalities their revenues represent only a small part of the real costs. Therefore, inthis paper, we look for the sound practices and their major determinants in the refuse collection servicein Portugal. We apply distinct non-parametric techniques, such as data envelopment analysis (DEA) andthe new robust methodologies of bootstrap and order-m for this purpose. The latter was also used toevaluate the relevance of the operational environment in the refuse collection service in Portugal. Theresults show that there is room for efficiency improvement.

efuse collection

. Introduction

The waste sector in general, and the refuse collection servicen particular, have been raising increasing interest among schol-rs, economists, technical staff, etc. The growing amount of wasteroduced caused by the changing customer habits and economicrowth (Sjöström and Östblom, 2010) and the resource constraintshat have characterised the waste services recently have fosteredhe search for efficiency by these services. In this context, efficiencymprovement should be a major goal for the waste stakeholders.herefore, it is not strange that many studies have been producedn the literature on the waste sector performance in the last years.

This research relies on non-parametric techniques to mea-ure the efficiency of refuse collection services and to determinehe influence of the operational environment on performance.xogenous variables are crucial, since they widely influence theerformance of the waste services (Ronchi et al., 2002). We startedy using the traditional non-parametric techniques, such as datanvelopment analysis (DEA). In simple terms, the DEA techniqueses linear programming to build an efficient frontier (technology)hat envelops the whole data (Charnes et al., 1978). The relativefficiency is determined by the comparison between that efficient

rontier (best practices) and each utility under analysis. Unlikehe parametric approaches, such as stochastic frontier analysis,hese methodologies do not require any function or distribution

∗ Corresponding author. Tel.: +351 21 8417729; fax: +351 21 8417979.E-mail addresses: pedrotsimoes@ist.utl.pt (P. Simões),

edro.c.carvalho@ist.utl.pt (P. Carvalho), rui.marques@ist.utl.pt (R.C. Marques).1 Tel.: +351 21 8417729; fax: +351 21 8417979.

921-3449/$ – see front matter © 2012 Elsevier B.V. All rights reserved.ttp://dx.doi.org/10.1016/j.resconrec.2012.07.006

© 2012 Elsevier B.V. All rights reserved.

specification. Moreover, they have the advantage of dealing easilywith multiple inputs and outputs (Fried et al., 2008). Afterwards, weused recent methods to provide the robustness to the DEA resultsand to investigate the influence of the operational environment onefficiency. These techniques adopt the concept of partial frontierto identify outliers and extreme observations (Daraio and Simar,2007).

In addition to the contributions that this paper provides for theliterature on the waste sector performance, and, in particular, forthe refuse collection service, it computes, for the first time (to thebest of the authors’ knowledge), the robust non-parametric meth-ods of order-m and bootstrap for the waste sector. These recentapproaches (intend to) overcome the major problem of the DEAmethod which is its deterministic nature. Furthermore, to pro-vide robustness to the results obtained, we employed the bootstrapapproach and identified the outliers of the sample with the order-m method. The order-m becomes a key-issue in DEA, since thefeasibility and quality of the results are usually threatened by themisleading best practices.

This paper is also the first one (as far as we know) in the litera-ture that analyses the influence of the operational environment onthe refuse collection service through a non-parametric approach.We apply a recent methodology that takes into account the het-erogeneity and the exogenous variables based on the conditionalefficiency measures (Dario and Simar, 2005). This approach doesnot use a separability condition or any a priori assumption aboutthe environmental variable effect, having great advantages when

compared with other methodologies to incorporate the operationalenvironment (see Fried et al., 2008).

The bootstrap procedure (developed by Simar and Wilson, 1998)is, nowadays, well accepted and acknowledged as a very useful tool.

P. Simões et al. / Resources, Conservation and Recycling 67 (2012) 56– 66 57

e of th

TpDotaaeslar

arstfswoc

2

spmrs

m2vrvtbicmlw

cscspo

Fig. 1. Waste market structur

he essential idea of bootstrapping is simply to simulate the sam-ling distribution, providing the robustness that is absent in theEA model. When the re-sampling process is repeated B times,ne will be determining, each time, a particular imaginary fron-ier that corresponds to a specific set of peers (which works as

benchmark for each service). The order-m method, also knowns a partial frontier method, only includes part of the sample (mntities) to estimate the efficiency scores. Consequently, it is lessensitive to extreme data and outliers, and does not have the prob-em of the “curse of dimensionality”. The latter may occur when

sample with reduced data is used, which may lead to impreciseesults.

After this general introduction, the paper follows with the char-cterisation of the Portuguese market structure concerning theefuse collection service. In the third section, the case study is pre-ented, while the computation of the technical efficiency throughhe traditional non-parametric method of DEA is provided in theourth section. The fifth section deals with the performance analy-is by the robust non-parametric methods of bootstrap and order-mhich are used to identify outliers and examine the influence of the

perational environment. Finally, the paper ends up with the majoroncluding remarks.

. Market structure

In Portugal, the main role regarding the provision of the wasteervice is performed by the municipalities. Although there is theossibility to delegate these services to different managementodels, including the private ones, the provision of the service,

esponsibility and ownership will always be on the municipality’side.

The waste sector in Portugal is divided into three great seg-ents: collection, treatment and recycling (Marques and Simões,

008). While the waste treatment and recycling services are pro-ided by regional entities, municipalities are in charge of theesidential refuse collection service (also known as ‘retail’ ser-ice). The refuse collection service in Portugal is basically providedhrough four different management models. So, they may optetween municipal services (activities provided directly by munic-

palities), semi-autonomous utilities, municipal companies andoncessionaire companies (Guimarães et al., 2010). The first threeodels are under public management and, despite having different

evels of autonomy, are carried out by the local municipal authority,hereas the outsourced service is under private management.

In the whole country there are 274 utilities providing the refuseollection service (for 308 municipalities), including municipalervices, semi-autonomous utilities, municipal companies and con-

essionaire companies. Despite the growing presence of the privateector, mainly through outsourcing contracts (mostly through shorteriod contracts of 1–5 years), this activity continues to be mostlyperated by municipalities or their associations (intermunicipal

e ‘retail’ segment in Portugal.

entities). In 2008, the private sector represented 50% in terms of thenumber of utilities and covered 40% of the Portuguese population.Fig. 1 characterises the refuse collection service in Portugal.

3. Case study

3.1. Sample

As previously mentioned, we aim to evaluate the performanceof the refuse collection services in Portugal, by means of non-parametric techniques. In this scope, we gathered informationrelated to the service of refuse collection of 196 municipalities, rep-resenting about 84% of the Portuguese population. Data refers to theyear of 2008.

The information was obtained by means of a questionnairewhich, to clarify some particular aspects, often required a directcontact with the municipalities and operating firms (when theywere the ones in charge).

3.2. Variables and model orientation

The selection of variables is always a critical stage in this kind ofstudies, since the definition of the inputs and outputs will “speak”on behalf of the service in question. A careless selection of vari-ables may lead to distorted results. Therefore, and after reviewingthe literature, we selected the staff, the vehicles and the remainderoperational costs of the service (hereafter OOPEX) as input vari-ables; and the residential waste collected as the main describer ofthe service outcomes.

Despite some particular situations regarding the number ofresponsibilities of the municipalities, for instance the merging inthe same department of services such as street cleaning, selectivecollection (when provided) and water distribution, among others,the variables adopted here only focused on the refuse collectionservice. The staff variable refers to the number of employees inadministrative positions and the waste removal team. Usually, thisteam is constituted by 3 elements (1 driver and 2 waste removalemployees), although recently we have observed some innovationin this field, e.g. truck teams composed of 2 elements. Analogously,we considered the number of vehicles and the OOPEX, which areonly related to the refuse collection service.

Similarly to most studies in the literature, the quantity of wastecollected by the utilities, measured in tons, was adopted as a singleoutput. As a representative of the service outcomes, this variablealso indicates the presence or absence of economies of scale; andagainst the expectations, bigger is not always better (Dollery et al.,

2007). The literature on economies of scale in refuse collection ismixed, varying between the slight existence (Stevens, 1978; Dubinand Navarro, 1988; Bel, 2006) and the non-existence (Hirsch, 1965;Callan and Thomas, 2001) of evidence.

58 P. Simões et al. / Resources, Conservation and Recycling 67 (2012) 56– 66

Table 1Data statistics.

Staff Vehicles OOPEX Refuse collected

Average 33.33 7.94 611,179.73 18,853.09Str. dev. 91.33 14.72 1,130,153.29 30,577.95Median 12.00 3.00 164,210.00 8507.66

i

lscic

4

4

Tuacmntotnpa2

ptpop

ido

∂

4

ffaapaHmemh

Table 2Efficiency scores per type of model management.

CRS VRS SE

Average 0.491 0.611 0.826

Mun. 0.486 0.651 0.762

Min. 3.00 1.00 3858.00 380.00Max. 976.00 153.00 8,789,904.00 275,469.00

The statistics for the variables adopted in the model are givenn Table 1.

The model orientation is quite consensual in the refuse col-ection service. Although we are dealing with a service that isupposed to be sustainable, public service principles and serviceost reduction should prevail. Hence, we considered the input min-misation orientation, which focuses on the reduction of the inputsonsumed, keeping the same level of outputs produced.

. The use of traditional non-parametric techniques

.1. Introduction

The interest in the performance of the operational activity.he literature provides evidence of this concern by means oftility efficiency analysis through parametric and non-parametricpproaches. Despite having similar purposes, there are severalharacteristics that distinguish these approaches. For instance, theost important advantage of non-parametric methods rely on the

on-requirement to adopt, a priori, a function to represent the fron-ier of the production process (cost or production function) andther demanding hypotheses, e.g. the distribution for the errorerm. In this study, besides the computation of the traditionalon-parametric technique of DEA, we also applied the new non-arametric estimator of bootstrap (primarily mentioned by Simarnd Wilson, 1998) and order-m (firstly introduced by Cazals et al.,002) which are robust approaches for efficiency estimation.

A great amount of literature on how to specify and to estimateroduction frontiers or cost functions, and on how to measure theechnical efficiency of observations (production units), has beenublished. Adopting a production technology where the activity ofbservations is characterised by a set of inputs x ∈ �p+ consumed toroduce a set of outputs y ∈ �q+, the production set is defined by:

= {(x, y) ∈ �p+q+ |x can produce y} (1)

Algebraically, this set can be described by its dimensions. Fornstance, in the input space we have the input requirement setefined for all y ∈ as C(y) = {x ∈ �p+|(x, y) ∈ }. The radial (input-riented) efficiency frontier (�) is given by:

C(y) = {x|x ∈ C(y), �x /∈ C(y) ∀ 0 < � < 1} (2)

.2. DEA methodology

In this research, the first technique used to evaluate the per-ormance of the refuse collection services was the non-parametricrontier method of DEA. DEA does not require the specification of

particular functional relationship between production of inputsnd outputs and has the capability to deal easily with multi-le inputs and outputs (Fried et al., 2008). DEA imposes fewssumptions on the shape of the production surface or technology.owever, applications often require very large data sets to obtain

eaningful efficiency estimates. Overall, this technique compares

ach service with the best practices (i.e. the efficient ones) thatake up the efficient frontier. This aspect turns it into a technique

ighly sensitive to extreme observations and noise in the data.

SaU 0.537 0.630 0.878MC 0.525 0.644 0.837

The process of finding the best practices might be formulatedas a problem of linear programming. The efficiency of n producerscorresponds to a set of n linear programming problems. In the fol-lowing formulation, � i is a vector describing the weighting of otherobservations used to construct the virtual observation. X and Y are,respectively, the inputs and outputs of each observation. The for-mulation of the input-oriented constant returns to scale (CRS) DEA(hereafter just CRS) model is shown below.

DEA-CRS =

⎧⎨⎩

(x, y) ∈ �p+q+ |y ≤n∑i=1

�iYi; x ≥n∑i=1

�iXi for (�1, ..., �n)

such that �i ≥ 0, i = 1, ..., n

⎫⎬⎭ (3)

To allow for the variable returns to scale (VRS) technology, theconvexity constraint of

∑ni=1�i = 1 (Banker et al., 1984) is added

to the previous formulation. In an empirical perspective, the differ-ence between CRS and VRS models is the fact that the VRS modeltakes into account the scale effect. So, from these two models, wecan measure the scale efficiency by the ratio between the CRS modelefficiency score and the VRS model efficiency score.

4.3. Results

The performance of the refuse collection services in Portugalwas firstly measured by the DEA technique application, assuminginput orientation, as previously justified. Thus, two models werecomputed, respectively CRS and VRS models, from which the scaleeffect is also captured. Table 2 provides the average efficiency esti-mates. The results show that only 9 services out of 196 are efficientunder the CRS model and 29 under the VRS model. Concerning theVRS model, the results present an average efficiency score of 0.611,whereas the CRS model results display an average score of 0.491.The latter encompasses the scale inefficiency contribution as well.The results obtained mean that on average the municipalities couldreduce their inputs in 39% in the VRS model (or about 51% in theCRS model), producing the same level of outputs (residential wastecollected).

The results also indicate a significant amount of inefficiencydue to scale diseconomies. This means that the Portuguese ser-vices could save on average about 17.4% of the inputs consumedif they operated at an optimal scale. The sample is dominated byincreasing returns to scale, since less than 25% of the refuse col-lection services have decreasing returns to scale. This means thatan increase of scale (collected waste) will induce efficiency lossesin about 25% of the refuse collection services analysed, that is, themajority of them operate at a reduced scale level.

In Portugal the refuse collection service is provided under dif-ferent types of management. The services directly provided bythe municipalities (hereafter Mun), the semi-autonomous utilities(SaU) and the municipal companies (MC) are the most representa-tive types of management in Mainland Portugal. Fig. 2 shows thedistribution of these types of management by population provided.

In addition to the different levels of autonomy that distinguish

the types of service provision, the levels of inefficiency also divergeamong them. From Table 2, it is possible to observe that with theCRS model, the SaU and the MC present better performance thanthe Mun and the opposite happens with the VRS model.

P. Simões et al. / Resources, Conservatio

Fs

gtact

5

5

tpiitrgttc

aSari

berEbc

B

�

5

5

tstoi

ig. 2. Institutional arrangements used for residential refuse collection by serviceize.

At first glance, the services with more autonomy from the localovernment, that is SaU and MC, seem to show better performancehan the municipalities on their own. However, one cannot takeny credible conclusions without an outlier analysis which will bearried out in the following section. As highlighted, outliers andheir influence are a major issue in DEA technique.

. Robust non-parametric performance analyses

.1. DEA bootstrap approach

In this study, in order to overcome the main drawbacks of theraditional non-parametric techniques, we applied the robust non-arametric approach of bootstrap. The bootstrap method consists

n a resampling methodology that is used to perform statisticalnference in complex problems. It approximates the sampling dis-ributions of the estimator by using the empirical distribution ofesampled estimates obtained from a resample simulation. Theood performance of the bootstrap method can be seen on howhe data generating process (DGP) characterises the true produc-ion process. One of the most noteworthy benefits is to determineonfidence intervals to the corrected efficiencies estimated.

The first bootstrap applications associated with DEA onlyppeared in the mid-1990s. Besides, it was just after the work ofimar and Wilson (1998) that the results attained started to havepplication in the real world by using a bootstrap smooth algo-ithm, based on a DGP. The bootstrap algorithm can be describedn five steps, according to Simar and Wilson (1998).

The bootstrap estimate bias of the DEA estimator is computedy Eq. (4) in which �i corresponds to the average of the bootstrapfficiency result and the other term to the DEA estimate originalesults. The estimation of �i after the bias correction is computed byq. (5). Acknowledging the bootstrap efficiencies empirical distri-ution function �∗

ibwith b = 1, . . ., B and after a new bias correction,

onfidence intervals can be obtained:

ˆ iasi =1B

B∑b=1

�∗ib − �i (4)

˜i = �i − Biasi (5)

.2. Order-m approach

.2.1. Order-m and outlier detectionBeyond the absence of statistical inference through the tradi-

ional DEA technique, the (possible) presence of outliers in the

ample is definitely one of its most relevant weaknesses. Evenhough the methodologies applied may advise about the presencef an atypical service in the sample, the benefit of the doubt aboutt being truly a best practice should also be given. Outliers can arise

n and Recycling 67 (2012) 56– 66 59

from (a) the exceptionally high or low variables compared withthe remaining services in the sample, (b) the measurement errors(especially due to different measurement procedures, e.g. capitaloutlays), (c) the different structure in ownership and (d) the ser-vice that shows the best or worst practices (De Witte and Marques,2010).

When there is a suspicion about the presence of a (possible) out-lier in the sample it is necessary to know deeply the service underdiscussion or to apply procedures accordingly. Although in the liter-ature there are several methods, the order-m procedure developedby Simar (2003), based on the work of Cazals et al. (2002), has anenormous relevance. While other methods draw special attentionto the influence of atypical observations in the sample (e.g. the peercount index of Charnes et al. (1985)) or to the importance of individ-ual variables (e.g. peer index of Torgersen et al., 1996), the order-mmethod, drawing a partial frontier (with m services), is particularlytailored to emphasise atypical efficiency scores. The observationswith efficiency scores far above the unit (in both directions) showsigns of (possible) outliers (Daraio and Simar, 2007). This ambigu-ous role of outliers underlines the importance of detecting theproper atypical observations, so that researchers can take a closelook at these observations.

The order-m methodology came up with the probabilistic for-mulation of the production process. In this context, according toCazals et al. (2002), the production process can be defined by thejoint distribution function of inputs and outputs, while efficienciescan be obtained from a conditional distribution function resultingfrom the decomposition of that joint distribution function. Later,based on the work of Cazals et al. (2002), Daraio and Simar (2005,2007) proposed an alternative probabilistic formulation to the pro-duction process, which can be described by the measure of jointprobability of (X,Y) in �p+ × �q+, being characterised by the proba-bility function HXY(x, y):

HXY (x, y) = Prob(X ≤ x, Y ≥ y) (6)

In a context of input orientation such probability function canbe decomposed in the following way:

HXY (x, y) = Prob(X ≤ x|Y ≥ y)Prob(Y ≥ y) = FX|Y (x|y)SY (y) (7)

where FX|Y(x|y) is a conditional distribution function of X and SY(y)a survivor function of Y.

The input efficiency scores for a given point (x,y) can, therefore,be defined according to these probabilities and estimated using thefollowing estimator:

�m,n(x, y) =∫ ∞

0

(1 − FX|Y,n(ux|y))m du (8)

where FX|Y,n(ux|y) = (∑n

i=1I(Xi ≤ ux, Yi ≥ y))/∑n

i=1I(Yi ≥ y) andI(k) is the indicator function that takes the value of I(k) = 1 if k istrue or I(k) = 0 otherwise.

5.3. Operational environment influence

5.3.1. Order-m analysisThe order-m approach easily allows for the incorporation of

exogenous variables in the efficiency analysis. The inclusion ofthese variables is extremely useful, since they have a strong influ-ence on the production process. According to Cazals et al. (2002),the consideration of exogenous variables in the efficiency analy-sis is performed by conditioning the production process to a givenvalue of the exogenous variable (here, and usually, referred to as

Z), that is:

HXY (x, y) = Prob(X ≤ x|Y ≥ y, Z = z)Prob(Y ≥ y|Z = z)

= FX|Y,Z (x|y, z)SY |Z (y|z) (9)

60 P. Simões et al. / Resources, Conservation and Recycling 67 (2012) 56– 66

g the d

(ednsofNioa

�

w∑tf

dDnc�

etvo

5

berpittem

rseav(ria

Fig. 3. Percentage of outliers amon

Subsequently, it is possible to obtain the conditionalDebreu–Farrell) efficiencies. The computation of these conditionalfficiencies involves the estimation of a non-standard conditionalistribution function which requires the use of smoothing tech-iques for the exogenous variables. Such smoothing techniquestill require the choice of a kernel function and the determinationf a bandwidth. In this research, we used the Epanechnikov kernelunction and the likelihood cross validation based on k-Nearesteighbor method to obtain the optimal bandwidths. Thus, follow-

ng Daraio and Simar (2005, 2007), the conditional efficiencies ofrder-m approach, for a given point (x, y) (input oriented) and for

given value of Z = z, can be determined as:

ˆm(x, y|z) =∫ ∞

0

(1 − FX|Y,Z,n(ux|y, z))mdu (10)

here FX|Y,Z,n(ux|y, z) = (∑n

i=1I(Xi ≤ ux, Yi ≥ y)K((Z − Zi)/h))/ni=1I(Yi ≥ y)K((Z − Zi)/h) and I(k) is the indicator function that

akes the value of I(k) = 1 if k is true or I(k) = 0 and K(.) is the kernelunction and h the appropriate bandwidth.

To analyse the influence of exogenous variables on the pro-uction process, we used again a methodology proposed byaraio and Simar (2005). This methodology adopts a smoothedon-parametric regression considering the ratio of the order-monditional efficiencies with the unconditional efficiencies, Qzm =

ˆm,n(x, y|z)/�m,n(x, y), on Z.In this approach, considering an input oriented model, the influ-

nce of Z is taken from the slope of the regression. In particular, ifhe regression has a growing tendency, it means that the exogenousariable has a negative effect on efficiency, and a positive effect iftherwise.

.3.2. Double-bootstrap analysisThe capacity to transform the resources into products depends

oth on the operators’ technical efficiency and on the operationalnvironment. Observations which are working in favourable envi-onments can use few inputs to generate the same amount ofroducts or vice versa in unfavourable environments. Therefore,

t is essential to explain the efficiency scores taking into accounthe efficiency determinants (Fried et al., 1999). In addition tohe order-m and conditional efficiency procedure, the operationalnvironment might be analysed by a two-stage double-bootstrapethodology proposed by Simar and Wilson (2007).In a first stage, this technique solves the bootstrap DEA algo-

ithm without considering the operational environment. In aecond stage, the influence of the operational environment on thefficiency estimates is examined. A (semi-parametric) regressionnalysis is performed to estimate the influence of environmentalariables on the bias-corrected efficiency scores. Simar and Wilson

2007) describe a statistical model (i.e. a DGP) that allows foregressing non-parametric DEA efficiency estimates consistentlyn a second stage regression on environmental variables (covari-tes) distinct from the inputs in the first stage. By considering the

ifferent input and output analysis.

separability conditions, the two-stage procedure is possible andunbiased. The second stage regression involves a generated depen-dent variable but, more importantly, the estimated efficiency scoresare serially correlated in an unknown fashion. Standard statisticalinference is therefore not appropriate (Alexander et al., 2007) andthe ordinary (naïve) bootstrap is inconsistent.

The influence of each explanatory factor on efficiency is given bythe sign of the regression coefficient. Considering that the depen-dent variable ıi (as presented in Eq. (11)) represents the level ofinefficiency, the variable’s influence on efficiency is showed (oncontrary) by the sign of the correspondent regression coefficient.

ıi = Zi + �i (11)

For instance, a negative regression coefficient means a positiveeffect of that independent variable (Zi) on the service performance,and vice versa. Moreover, its importance can be examined throughstatistical tests (t-value in this case).

5.4. Results

5.4.1. Outlier detectionIn the present study, first we analysed the efficiency of

Portuguese refuse collection services comprising 196 utilities.However, being aware of the sensitivity that the DEA techniquereveals in the presence of outliers and extreme points, we appliedthe methodology proposed by Simar (2003) of partial frontier mod-els, the order-m. This methodology intends to detect extreme dataand remove the outliers from the sample as well as some serviceswith “atypical” behaviour.

In this procedure, the order-m efficiency score of the service i ismeasured by excluding the corresponding observation i from thesample [�(xi, yi)

(i)m,n], for the diverse values of m. The outliers are

identified as the observations that show (input) efficiency scores[�(xi, yi)

(i)m,n] greater than the unit and (output) efficiency scores

[�(x, y)(i)m,n] smaller than one for the diverse values of m, and at the

same time, a reduced number of observations that produce morethan the corresponding observation i (y ≥ yi) in input orientationanalysis, and a reduced number of observations that consume lessthan the observation i (x ≤ xi) in output orientation.

In this case, we detected 40 extreme observations under inputorientation and 23 under output orientation. The merger of bothprocedures identified 10 extreme observations (the same obser-vation in input and output orientations). So, we came up with 10potential outliers in our sample, which correspond to about 5% ofthe refuse collection services considered. Even knowing that theclassification of utilities as outliers requires deeper investigation,we opted for excluding all probable candidates to be outliers from

further research.

Fig. 3 shows the percentage of observations of the sample withthe input efficiency score �(x, y)(i)

m,n greater than 1.1, 1.2, 1.3, 1.4and 1.5, and percentage of observations with the output efficiency

P. Simões et al. / Resources, Conservation and Recycling 67 (2012) 56– 66 61

Fig. 4. Comparison of DEA and bootstrap results.

Table 3Efficiency scores per utility and per population provided.

CRS VRS SE

25,000 50,000 100,000 >100,000 25,000 50,000 100,000 >100,000 25,000 50,000 100,000 >100,000

Mun.DEA 0.409 0.563 0.575 0.687 0.660 0.594 0.595 0.768 0.638 0.946 0.965 0.906Boot 0.367 0.511 0.518 0.608 0.599 0.541 0.534 0.636 0.633 0.941 0.965 0.958

SaUDEA 0.000 0.507 0.478 0.464 0.000 0.561 0.534 0.716 0.000 0.929 0.896 0.648Boot 0.000 0.560 0.528 0.529 0.000 0.605 0.577 0.890 0.000 0.942 0.917 0.595

6

9

sv

5

c(becFa

mitamttwa

TE

MCDEA 0.453 0.706 0.509 0.554 0.705 0.75Boot 0.417 0.621 0.459 0.470 0.656 0.67

core �(x, y)(i)m,n lower than 0.5, 0.6, 0.7, 0.8 and 0.9, respectively, for

arious values of m considered.

.4.2. DEA vs. bootstrap results in the sample without outliersThe DEA technique is recognised as deterministic. Hence, we

omputed the bootstrap approach, proposed by Simar and Wilson1998), not only to grant robustness to the performance resultsut also to support the results obtained from the DEA model (nowxcluding outliers). In global terms, the models displayed ineffi-iencies of about 55% in the refuse collection service in Portugal.ig. 4 shows the new efficiency scores obtained from the DEA models well as the bootstrap results and their comparison.

As previously observed, the services provided directly by theunicipality still depict the worst performance among the util-

ties analysed when the CRS model is applied. The reasons forhis might be related both to the inefficiency that has tradition-lly threatened the public sector and to the low investment thatost municipalities make in the waste sector. Even among other

ypes of management of the refuse collection service, in Portugal,he SaU stands out with the best performance (44% of inefficiency),hich might be linked to its higher financial and administrative

utonomy.

able 4fficiency scores per utility and per population density.

CRS VRS

100 250 400 >400 100 250

Mun.DEA 0.399 0.538 0.568 0.642 0.644 0.642

Boot 0.361 0.478 0.520 0.570 0.585 0.581

SaUDEA 0.457 0.627 0.494 0.550 0.476 0.685

Boot 0.417 0.565 0.448 0.489 0.437 0.635

MCDEA 0.446 0.602 0.728 0.554 0.648 0.659

Boot 0.405 0.549 0.638 0.470 0.598 0.609

0.535 0.711 0.704 0.930 0.944 0.8320.488 0.563 0.700 0.911 0.930 0.870

We also related the bootstrap results to the optimal scale andthe population density for the Portuguese refuse collection services.Tables 3 and 4 present the differences among the refuse collectionservices per population and per population density provided.

Considering the Table 3, the optimal scale diverges among theservices. For the service provided directly by the municipality,the most efficient are those that provide the refuse collection tomore than 100,000 citizens, while for the semi-autonomous util-ities and the municipal companies, the utilities that comprise apopulation between 25,000 and 50,000 are the most efficient. Thepossible reason behind this result can be related to the fact thatthe biggest municipalities may invest more on the waste sector, forinstance, in route optimisation, vehicles, etc. The remaining typesof management (more autonomous) are already more optimised(e.g. regarding the number of employees), therefore being moreefficient. These results also show that there are slight economies ofscale in the refuse collection sector. In fact, although there mightexist benefits from services with high levels of population density,the inherent problems of highly urbanised areas (e.g. congestion)

might affect the service (Zhang et al., 2010), overcoming the pre-conceived benefits.

Concerning the population density, the results are not so evi-dent. They revealed that the utilities with a population density of

SE

400 >400 100 250 400 >400

0.608 0.700 0.642 0.855 0.928 0.9260.556 0.595 0.640 0.842 0.927 0.957

0.560 0.763 0.965 0.920 0.883 0.7460.517 0.656 0.960 0.897 0.866 0.756

0.779 0.711 0.754 0.914 0.926 0.8320.700 0.563 0.745 0.902 0.904 0.870

62 P. Simões et al. / Resources, Conservati

Table 5Performance results with and without outsourcing.

CRS VRS SE

DEA Boot DEA Boot DEA Boot

Mun – In 0.425 0.381 0.616 0.555 0.713 0.712Mun – Out 0.571 0.513 0.712 0.635 0.816 0.820

SaU – In 0.540 0.485 0.642 0.572 0.873 0.872SaU – Out 0.532 0.481 0.598 0.556 0.890 0.865

otis

lalwltfi

Hwmpvt

55ovc

•

•

•

•

MC – In 0.500 0.444 0.608 0.542 0.858 0.859MC – Out 0.719 0.631 0.900 0.774 0.816 0.817

ver 400 (inh./km2), under 250 and under 400 were the best forhe Mun., SaU and MC, respectively, which makes it very difficult tonfer about an “optimal population density” in the refuse collectionervice in Portugal.

In Portugal, the presence of the private sector in the refuse col-ection service has been growing. Some doubts have been raisedbout its benefits to the sector, since these agreements are estab-ished in short-term and completely out of ERSAR’s (the Portuguese

aste regulator) scope of intervention which only occurs in theong-term contracts (e.g. concessions). In this regard, we analysehe different management models distinguishing the in-house (-In)rom the outsourced (-Out) service. Table 5 provides that compar-son.

The results still showed the MC as the most efficient utilities.owever, this is supported by the benefits of outsourcing. Whene isolate the in-house provision, the SaU presents better perfor-ance. To sum up, these results proved a priori the idea that the

rivate sector is, in general, beneficial for the refuse collection ser-ice. However, the absence of this evidence in the SaU can be relatedo the small size of the sample outsourced.

.4.3. Operational environment conditioning

.4.3.1. Variables definition. With the purpose of analysing theperational environment that involves the refuse collection ser-ice in Portugal, we encompassed diverse explanatory factors toharacterise and contextualise the services:

Management, Man – As already referred to, there are differenttypes of management models for the refuse collection service.In order to verify the results obtained from the DEA model, weattribute different values for the different management models(1 – SaU; 2 – MC; 3 – Mun), evaluating the slope between them.Distance covered, DC – One of the main variables that greatlyaffects the cost of the refuse collection is the distance covered bythe vehicles to assure the universality of the service. This variablewas evaluated through the number of km travelled. We expect anegative influence of the DC on the waste utilities performance(Antonioli and Filippini, 2002; Bel and Mur, 2009).Outsourcing, dOut – In spite of being a matter so widely discussedin the literature, the fact that the refuse collection services werenot regulated in Portugal increases the interest of evaluating theinfluence of outsourcing on these services. Thus, we incorporateda dummy variable where the value of 1 represents private sectorparticipation in their operation and the value of 0, if otherwise.The literature is somehow mixed as far as the outsourcing influ-ence is concerned. Despite some evidence related to the absenceof benefits (Pier et al., 1974), more recent studies have shown theopposite (Dijkgraaf and Gradus, 2003; Bel and Costas, 2006). So,in this line, we expected to find a positive influence of private

participation on the services efficiency.Gross Domestic Product (GDP) – GDP is a basic measure of themunicipality’s overall economic output. So, we considered theregional GDP for each municipality. The GDP is often positively

on and Recycling 67 (2012) 56– 66

correlated with the standard of living and wealth (Sullivan andSheffrin, 1996). At first, we may expect a positive influence onthe utilities performance, due to the higher purchasing power oftheir users; however, this result may be ambiguous (Simões et al.,2010) as areas of high waste production are also related to urbanareas with other correlated issues, e.g. congestion problems andlack of space for collection points. The GDP was adopted per capitaand per municipality. In this regard, we do not have a predefinedopinion.

• Purchasing Power, PP – PP is the number of goods/services thatcan be purchased with a unit of currency. It is considered permunicipality and it was collected from INE (Statistics Portugal).PP is measured as a percentage of the total purchasing power ofthe country. The prediction is the same as in the GDP. The resultsmay be mixed.

• Population density, PD – PD refers to the number of inhabitantsper square kilometre. Thus, this variable is defined by the ratiobetween population and area. In some way, PD will reflect thepresence (or not) of economies of scale in the refuse collectionservice. Such as the population density increases, so does theamount of waste at each container, thus reducing the collectioncosts (Domberger et al., 1986; Bello and Szymanski, 1996). How-ever, high-density areas also lead to some problems with directimplications on service inefficiency (Stevens, 1978; Callan andThomas, 2001), mainly due to (generally) smaller roads, whichimplies less space for storing waste, and to a great amount ofcars (e.g. per km2), with consequences on congestion. For thisreason, we also encompass the square population density, PD2,taking advantage of its U-shape, in order to evaluate the trade-off between the high concentration of waste and the inherenttraffic problems. Thus, we are not certain about the effect of thisvariable. PD was based on the 2001 Portuguese Census of inhab-itants.

• Geography – Considering the particular features of Portugal, wedecided to encompass two different variables, respectively thecollection services operating in the Portuguese islands (Islandsvs Mainland) and the location of these services in the Mainland,distinguishing the South, Centre and North regions which havedifferent topography and weather conditions. We are not sureabout the effect of these two variables.

In addition to the presence of economies of scale in the provisionof the refuse collection services, the economies of density are con-sidered even more relevant in the literature (Kemper and Quigley,1976; VanDoren, 1999). We included the number of inhabitants percontainer for the utilities analysed as a proxy of this variable. Thecontainers represent the number of collection points available forthe population to discard the waste produced. Moreover, we addedthe indicator number of km covered by trucks to provide the refusecollection service also per container to evaluate the relevance ofthis aspect. Regarding the outcomes, we expect to find economiesof density, that is, a positive effect of the indicator population percontainer on service performance (Szymanski and Wilkins, 1993),while for the km covered per container we expect the oppositeeffect (Tickner and McDavid, 1986).

The variables here adopted are the ones that the authors con-sider to better represent the provision of the refuse collectionservice; however, it should be noticed that other explanatory fac-tors could be considered as well, such as the frequency of thecollection service, the wage level, among others. Concerning thefrequency, in Portugal, only some exceptions (less than 5%) donot collect the residential waste 6 times per week. Despite the

predetermined idea that frequency does not benefit efficiency(Pommerehne and Frey, 1977; Reeves and Barrow, 2000), ourresults do not allow to obtain statistical significance to support anyconclusion. Regarding the wage level, its variation among regions

P. Simões et al. / Resources, Conservation and Recycling 67 (2012) 56– 66 63

Table 6Expectations of explanatory variables influence on efficiency.

Explanatory variables Brief description Expectation

Management Values to distinguish the different management models for refuse collection AmbiguousDistance covered km covered by trucks to collect residential waste −Outsourcing Dummy to represent utilities that look for better performance on outsourcing +Population Number of inhabitants per municipality AmbiguousArea Square km that comprise the boundaries of each municipality −Population density Index of population per square km AmbiguousGDP Gross Domestic Product per capita per municipality AmbiguousPP Purchasing Power per municipality as a % considering the country 100% AmbiguousKm/container Km per refuse container in order to evaluate the economies of density −Pop./container Number of inhabitants per refuse container in order to evaluate the economies of density +

e Porortugu

miCt

ae

5BomaaautorsaT

wpvt

obt

TS

K

Container Number of points to discard waste

Islands vs Mainland Dummy variable for the services in thRegion Different variables for the different P

ay influence the operational costs paid out by the utilities, andnevitably with negative implications on their efficiency (Bel andostas, 2006). However, it was not possible to use this variable dueo the difficulty to obtain reliable information.

Table 6 provides a snapshot of the variables adopted in the oper-tional environment analysis, encompassing a brief description andxpectations about their influence on the service performance.

.4.3.2. Results and analysis of the operational environment influence.earing in mind the relevance that the operational environment hasn the performance of refuse collection services, we used the recentethodology proposed by Daraio and Simar (2005, 2007) to evalu-

te the effect of the variables. As mentioned, this procedure relies on non-parametric regression analysis, where the dependent vari-ble is the ratio of the order-m conditional efficiency scores with thenconditional efficiency scores, Qzm = �m,n(x, y|z)/�m,n(x, y), andhe independent ones are the explanatory variables. The influencef the explanatory factors is dictated by the inverse trend of theegression line. This means that a regression positive slope repre-ents the negative effect of that variable on service performance. Inddition, its importance can be examined through statistical tests.he results are shown in Fig. 5.

In order to provide more reliability to the order-m conclusions,e also applied the recent methodology of double bootstrap pro-osed by Simar and Wilson (2007). For most of the explanatoryariables, the results obtained coincide and, generally, correspondo our expectations.

As expected, the variable distance covered has a negative effectn service efficiency. This indicates that the number of km coveredy trucks to collect the residential waste is, obviously, disadvan-ageous for the performance improvement of the refuse collection

able 7ummary and statistical analysis of the explanatory variables influence.

Explanatory variables Expected Nonparametric regression Double-b

Effect p-Value Regressio

Management Ambiguous MC + 0.049* 0.832

Distance covered − − 0.048* 2.8E−06

Outsourcing + + 0.000*** −0.279

Population Ambiguous Tend to − 0.075 −1.7E−05Area − − 0.052 2.4E−04

Population density Ambiguous Tend to − 0.063 0.001

Square population density Ambiguous Tend to − 0.073 −1.6E−07GDP Ambiguous Tend to − 0.135 0.020

PP Ambiguous − 0.036* 1.154

Km/container − − 0.007** 0.001

Pop./container + Tend to + 0.000*** −4.8E−05Container Ambiguous Tend to − 0.053 −2.0E−04Island vs Cont. Ambiguous 0 0.022* −0.570

Region Ambiguous 0 0.048* 0.815

ernel regression significance test: ***0.001; **0.01; *0.05; Confidence interval: ***** 99%

Ambiguoustuguese islands and in the Mainland Ambiguousese regions Ambiguous

service. Moreover, the negative slope of the regression line provedthe savings with outsourcing, supporting also the efficiency anal-ysis of DEA. This shows the benefits of private sector participationon the performance of refuse collection services.

Regarding the management model analysis, these results con-firm the previous ones obtained with DEA. The negative slopebetween “1” and “2” indicates a positive effect of MC over the SaU.The results show the opposite for the positive slope between “2”and “3”, corresponding to a negative effect when we compare theMun and the MC.

A relevant issue is to find out whether economies of density existin the refuse collection service, even overcoming the importance ofeconomies of scale. Accordingly, we included the variable popula-tion per container as a proxy of the waste production. The outcomesprove its benefits on utility performance, but even with betterresults for more than 30 inhabitants per container. For instance, thevariable population, on its own, revealed a negative effect, whichcan be a consequence of external and adverse effects of dealing withhigh volumes of waste (an aspect deeply analysed by Palmer et al.,1997).

In addition, regarding the indicator km per container, the resultsdepicted a negative influence. This means, as expected, that thelonger the distance between collection points, the higher the costsof service, and therefore, there is a negative effect on services per-formance.

Concerning the influence of population density, the results wereinconclusive. Due to the absence of statistical significance and the

constant regression trend, the conclusions about this variable areimpossible to take. Despite its negative general impact, in the lowdensity areas the tendency inflects in the 500 inhabitants per km2.The same is shown by square population density.

ootstrap

n coefficient Lower bound Upper bound t-Value Effect

0.705 1.000 173.5 −1.1E−06 2.9E−06 −50.71 −−0.236 0.183 80.88 +

−1.6E−05 −5.4E−06 68.99 +−1.6E−04 0.001 3.22***** −6.6E−05 0.001 −20.15 −

−2.6E−07 1.5E−09 10.28 +0.019 0.025 45.65 −0.214 1.183 −55.32 −−0.002 0.004 −2.33**** −

−4.5E−04 0.001 7.08 + −2.2E−04 −6.6E−05 46.44 +

−1,036 −0.178 −40.61 +0,771 0.868 440.54 −

, **** 90%.

64 P. Simões et al. / Resources, Conservation and Recycling 67 (2012) 56– 66

Fig. 5. Results of the influence of operational environment analysis on efficiency.

ervatio

dnfirw

tufPi

oo

datvlvood

6

saono

ovrwmcda

urirf

auotlfeoGditniP

P. Simões et al. / Resources, Cons

The variables population, area and containers showed thatespite the negative trend (against efficiency improvement), it isot possible to infer about their influence on efficiency for a con-dence interval of 95% or more. Regardless of their unequivocalelevance for the service performance, analysing them on their ownas proved to be pointless.

Considering the GDP and PP variables, the results confirmedhe expectations. They also showed their correlation with highlyrban areas, revealing a global negative trend for the service per-ormance improvement. Regarding the geographical analysis of theortuguese refuse collection services, it was observed a lack ofnfluence of both analyses between the services.

Table 7 summarises our expectations about the influencef exogenous variables on efficiency and the empirical resultsbtained from both methodologies.

The results provide evidence of slight discrepancies between theouble bootstrap methodology and the one proposed by Daraiond Simar (2005, 2007). This can be explained by the fact thathe double bootstrap method allows for the confidence inter-als determination, and in some particular points, it may have aower bound with a negative value and a positive upper boundalue. Furthermore, the sensitivity to the outliers and extremebservations is different in the two methods. Therefore, the effectf the variable might not be conclusive when the results areiverse.

. Concluding remarks

This paper evaluated the performance of 196 refuse collectionervices in Portugal by means of non-parametric techniques. Inddition to the DEA application, the non-parametric robust modelsf bootstrap and order-m were also computed to provide robust-ess to this evaluation. The latter was also used to detect atypicalbservations.

As far as the DEA model is concerned, the results pointut significant levels of inefficiency in the refuse collection ser-ices in Portugal. For instance, using CRS and VRS models,espectively, an average level of 51% and 39% of inefficiencyere estimated. Besides, if the services operated at an opti-al size, they would be able to save about 17% of their

osts (inputs consumed) for the same quantity of outputs pro-uced. The results were integrally supported by the bootstrappplication.

The analysis of DEA and bootstrap efficiency scores alloweds to infer that there are slight economies of scale among theefuse collection services (up to 50,000 inhabitants), but it wasmpossible to conclude about the optimal population density. Theesearch proved the benefits of outsourcing in this service, soar.

The empirical evidence also proved the relevance of the oper-tional environment on the performance of refuse collectiontilities. We encompassed diverse variables in this scope. Mostf the results met our expectations. Besides corroborating thathe MC have the most efficient management model of refuse col-ection service in Portugal, we found out that the most harmfulactors for the waste utilities performance were the distance cov-red by the vehicles, and the distance between containers, whileutsourcing proved to be beneficial. The results obtained from theDP, the purchasing power and the population density requireseeper analysis. This work reflected the importance of this aspect

n the waste utilities performance particularly due to the fact

hat almost all the explanatory variables were statistically sig-ificant. At last, the geographical variables revealed themselves

nconclusive about their effect on the refuse collection services inortugal.

n and Recycling 67 (2012) 56– 66 65

References

Alexander R, Haug A, Jaforullah M. A Two-stage double-bootstrap data envelopmentanalysis of efficiency differences of New Zealand secondary schools. EconomicsDiscussion Papers no. 0714, University of Otago; 2007.

Antonioli B, Filippini M. Optimal size in the waste collection sector. Review of Indus-trial Organization 2002;20:239–52.

Banker R, Charnes A, Cooper W. Some models for estimating technical andscale inefficiencies in data envelopment analysis. Management Science1984;30:1078–92.

Bel G. Gasto municipal por el servicio de residues solidos urbanos. Economia Apli-cada 2006;14:5–32.

Bel G, Costas A. Do public sector reforms get rusty? Local privatisation in Spain. TheJournal of Policy Reform 2006;9:1–24.

Bel G, Mur M. Intermunicipal cooperation privatization and waste managementcosts: evidence from rural municipalities. Waste Management 2009;29:2772–9.

Bello H, Szymanski S. Compulsory competitive tendering for public services in theUK: the case of refuse collection. Journal of Business Finance & Accounting1996;23:881–903.

Callan S, Thomas J. Economies of scale and scope: a cost analysis of municipal solidwaste services. Land Economy 2001;77:548–60.

Cazals C, Florens J, Simar L. Nonparametric frontier estimation: a robust approach.Journal of Econometrics 2002;106:1–15.

Charnes A, Cooper W, Lewin A, Morey R, Rousseau J. Sensitivity and stability analysisin DEA. Annals of Operations Research 1985;2:139–56.

Charnes A, Cooper W, Rhodes E. Measuring the efficiency of decision making units.European Journal of Operational Research 1978;2:429–44.

Daraio C, Simar L. Introducing environmental variables in nonparametric fron-tier models: a probabilistic approach. Journal of Productivity Analysis2005;24:93–121.

Daraio C, Simar L. Advanced robust and nonparametric methods in efficiency anal-ysis. Methodology and applications. New York: Springer; 2007.

De Witte K, Marques R. Influential observations in frontier models, a robustnon-oriented approach to the water sector. Annals of Operations Research2010;181:377–92.

Dijkgraaf E, Gradus R. Cost-savings of contracting out refuse collection. Empirica2003;30:149–61.

Dollery B, Brynes J, Crase L. Is bigger better? Local government amalgamation and theSouth Australian rising to the challenge inquiry. Journal of Economic Analysis &Policy 2007;37:1–14.

Domberger S, Meadowcroft S, Thompson D. Competitive tendering and efficiency:the case of refuse collection. Fiscal Studies 1986;7:69–87.

Dubin J, Navarro P. How markets for impure public goods organize: the caseof household refuse collection. Journal of Law, Economics, and Organization1988;4:217–41.

Fried H, Lovell K, Schmidt S. The measurement of productive efficiency and produc-tivity change. New York: Oxford University Press; 2008.

Fried H, Schmidt S, Yaisawarng S. Incorporating the operating environment into anonparametric measure of technical efficiency. Journal of Productivity Analysis1999;12:249–67.

Guimarães B, Simões P, Marques R. Does performance evaluation help publicmanagers? A Balanced Scorecard approach in urban waste service. Journal ofEnvironmental Management 2010;91:2632–8.

Hirsch W. Cost functions of an urban government service: refuse collection. TheReview of Economics and Statistics 1965;47:87–92.

Kemper P, Quigley J. The economics of refuse collection. Ballinger Publishing Com-pany; 1976.

Marques R, Simões P. Does the sunshine regulatory approach work? Governanceand regulation model o the urban waste services in Portugal. Resources, Con-servation and Recycling 2008;52:1040–9.

Palmer K, Sigman H, Walls M. The cost of reducing municipal solid waste. Journal ofEnvironmental Economics and Management 1997;33:128–50.

Pier W, Vernon R, Wicks J. An empirical comparison of government and privateproduction efficiency. National Tax Journal 1974;27:653–6.

Pommerehne W, Frey B. Public versus private production efficiency in Switzerland:a theoretical and empirical comparison. Urban Affairs Annual Review1977;12:221–41.

Reeves E, Barrow M. The impact of contracting-out on the costs of refuse collec-tion services. The case of Ireland. The Economic and Social Review 2000;31:129–50.

Ronchi E, Federico A, Musmeci F. A system oriented integrated indicator for sustain-able development in Italy. Ecological Indicators 2002;2:197–210.

Simar L. Detecting outliers in frontier models: a simple approach. Journal of Produc-tivity Analysis 2003;20:391–424.

Simar L, Wilson P. Sensitivity analysis of efficiency scores: how to bootstrap innonparametric models. Management Science 1998;44:46–61.

Simar L, Wilson P. Estimation and inference in two-stage, semi-parametric modelsof production processes. Journal of Econometrics 2007;136:31–64.

Simões P, De Witte K, Marques R. Regulatory structures and operational envi-ronment in the Portuguese waste sector. Waste Management 2010;30:

1130–7.

Sjöström M, Östblom G. Decoupling waste generation from economic growth — aCGE analysis of the Swedish case. Ecological Economics 2010;69:1545–52.

Stevens B. Scale market structure, and the cost of refuse collection. Review of Eco-nomics and Statistics 1978;60:438–48.

6 ervati

S

S

T

6 P. Simões et al. / Resources, Cons

ullivan A, Sheffrin S. Economics: principles in action. Upper Saddle River, NJ: Pear-son Prentice Hall; 1996.

zymanski S, Wilkins S. Cheap rubbish? Competitive tendering and contracting outin refuse collection. Fiscal Studies 1993;14:109–30.

ickner G, McDavid J. Effects of scale and market structure on the costs of res-idential solid waste collection in Canadian cities. Public Finance Quarterly1986;14:371–93.

on and Recycling 67 (2012) 56– 66

Torgersen A, Førsund F, Kittelsen S. Slack-adjusted efficiency measures and rankingof efficient units. Journal of Productivity Analysis 1996;7:379–98.

VanDoren P. Time to trash government intervention in garbage service. Policy Anal-ysis 1999;331:1–25.

Zhang D, Tan S, Gersberg R. Municipal solid waste management in China:status, problems and challenges. Journal of Environmental Management2010;91:1623–33.