Green Public Procurement in academic literature: a survey

Sandra Lange1, Jan Telgen2, Fredo Schotanus3

Abstract

The first author’s master thesis (published elsewhere) is a comprehensiveliterature review of 16 years of academic research on Public Procurement.Here we focus on the literature on Green Public Procurement. The originalliterature review concerns papers published in English in academic jour-nals covered by Scopus and Web of Science in the 1997-2012 timeframe.Through a rigorous review process we identified 378 publications on PublicProcurement in general, out of which 29 were in the area of Green PublicProcurement. A first observation is that less than 10% of academic paperson Public Procurement deal with Green Public procurement, so Green Pub-lic Procurement can hardly be considered a dominant topic in Public Pro-curement academic research. But there are more observations to be made.Even though 29 papers is only a limited number to base any comparisonson we find a number of striking differences with the general Public Procure-ment literature. To mention a few: relatively low percentage of academicpapers in typical purchasing and supply management journals, wide spreadof authorship and countries studied, relatively low number of citations, noquantitative modelling.

1. Introduction

Public procurement is a powerful tool to make governments moreefficient. It accounts for 13% to 20% of worldwide GDP [3], meaningthat a significant proportion of all produced products and services arebought by governments. Many researchers emphasized the powerfuleffects that public procurement can have on fostering green produc-tion [13] [18]. To achieve such socially desirable outcome, govern-ments operate as both regulators, passing laws and regulations, andmarket participants [10] [21]. And in times of global supply chains,

1This work was based upon Sandra Langeas master thesis at the University of Twente2Professor of Public Procurement, University of Twente, PO Box 217, 7500 AE Enschede, The

Netherlands, corresponding author j.telgen@utwente.nl3Assistant professor, University of Twente

1

public procurement is not only a means to improve upon social out-comes on domestic markets, but also internationally [21].

This paper is a detailed analysis of findings obtained on the sub-ject of Green Public Procurement (GPP) during a comprehensiveliterature review of 16 years of academic research on Public Pro-curement. While Public Procurement is a highly fragmented field, asystematic overview of research is still lacking. Crossan and Apaydin(2010) state that “fragmentation of the field prevents us from seeingthe relations between these facets and ultimately impedes consolida-tion of the field.” [11, p. 1154]. By showing which topics have beenaddressed by research, to what extent, detecting possibly understud-ied, as well as mature sub-fields, and by enabling researchers withthe tools to conduct syntheses on findings for sub-fields, the field isdeveloped to a new state of more clarity and unification. The prac-tical impact of this work is therefore indirect, by stimulating andenabling a research agenda to derive at generalizable findings.

This literature review provides an overview of the most influentialscientific literature published on the topic of Green Public Procure-ment. Moreover, the overall state of GPP research is assessed, pro-viding insights into the maturity of the field. According to Cheon,Grover and Sabherwal (1993) mature research fields are character-ized by studying a variety of different topics and applying variousresearch methods instead of narrowly focusing on few [9]. Therefore,this review focuses on addressed topics, as well as employed method-ologies and their development over time. The literature review isfocused on the past sixteen years (1997 to 2012).

2

2. Methodology

The methodology was informed by Wynstra (2010) [28]. For re-viewing the Journal of Purchasing and Supply Management ’s pub-lications of the years 1994 to 2009 he developed an extensive listof categories each article was classified into. The main categoriesemployed by Wynstra are: topic, research strategy, data collection,data analysis, type of product, type of purchase, as well industryand sector. With respect to the topics, the author classified eacharticle into a maximum of three subject categories, while another,similar review conducted by Carter and Ellram (2003) [7] on theJournal of Supply Chain Management categorized each article intoonly one subject category which summarized the article’s focus thebest. We believe that Wynstra’s approach yields a more accurate rep-resentation of research topics, since procurement subjects are oftenresearched against a clear background and therefore categorizationinto only one subject field will under-represent the other(s). More-over, this categorization enables us to look more specifically, whichtopics have been studied in the context of the green procurementstrategy.

Each of the main categories was further divided by Wynstra intosub-categories. He also gathered general article data including pub-lication year, contributing authors, institutions and citations. Allmain categories employed in his research were adopted with the ex-ception of “type of purchase”, as a scoping study of the PP fieldhad revealed that this category was irrelevant. While the main cat-egories were mostly adopted, the sub-categories were modified. Histopic sub-categories were to a great extent not applicable to thisreview, since his research mainly focused on private sector procure-ments. The research strategies were all adopted except for “expertinterviews/Focus group” and “laboratory experiment”, which wereinstead grouped as data collection methods. Wynstra’s subcategories

3

for data collection are very specific, such as distinguishing betweenfour kinds of questionnaires. As this level of specificity may falselycreate the impression of variety, the subcategories were simplified.The same applies to his thirty-nine items list of data analysis tech-niques, which was simplified to only distinguish between quantitativeand qualitative research. The product types were amended to ac-count for works while the product type combination “good/service”was omitted. Nine industry and sector categories were adopted fromthe Wynstra classification scheme yet further extended during thedata extraction stage of the review. Instead of collecting institutedata, the country of the institute was noted per article. While insti-tutional data would have provided interesting insights into institu-tional collaborations, as well as most active institutions on the field,this review adopted a global perspective limited to cross-country asopposed to cross-institutional differences. Finally, his categorizationswere extended with data on studied country/ies, publishing journaland the procuring government level. This latter study character-istic is specific to public procurement and inclusion in the reviewprovided valuable information on the context of conducted research.Specifying the research context is according to Denyer, Tranfield andvan Aken (2008) supportive to practical relevance [14]. The completeclassification scheme may be obtained from the authors upon request.The classification scheme was directly transcribed into an SPSS dataextraction form, which was later used to conduct the analyses.

This review’s objective was to assess the overall status developmentof Green Public Procurement research, its predominant study charac-teristics and research designs, as well as addressed topics. The statuswas operationalized as annual publications. Publication counts arean effective outcome measure to assess the scientific importance ofa research field [11] and their development is a reliable indicationwhether the relevance of the field changed. To further character-ize the time developments, publishing countries and journals were

4

assessed over time as well as authorships. It is believed that thesevariables provide a meaningful overview of the main stakeholders ofthe field.

Predominant study characteristics were assessed against the indus-tries and sectors from which the government procures, the procuringgovernment level, studied country/ies and types of products pro-cured. There was no limitation for those categories with regard tomaximum sub-categories per article.

Most frequently employed research strategies were assessed againstthe research methodology, time dimension, research strategy anddata collection methods. While there was no limit as to how manydata collection methods each article was grouped into, the other cat-egories held mutually exclusive sub-categories. This exclusivenesswas only breached when articles clearly articulated to have utilizedmore than one strategy.To detect developments over time, the sixteen years of research havebeen subdivided into equal time intervals of four years each, inspiredby Wynstra (2010) [28] and Carter and Ellram (2003) [7].

Searches were conducted by use of the databases Scopus and Webof Science. Both databases are well-established, multi-disciplinaryresearch platforms, holding a wide variety of peer-reviewed journals,and they are being kept up to date. We chose for two databases toensure all relevant papers are included, since it is possible that onedatabase omits relevant research [11].

To assess whether high impact papers differ from low and mediumimpact publications with regard to study characteristics and researchdesigns, we conducted citation analyses based on mean scores ofthe Scopus and Web of Science citation counts. We included bothdatabases’ citation counts as citations differ per database and there-

5

fore reliance on only one source may over- or undervalue individualpapers. A mean citation count is believed to provide a more realisticassessment of each paper’s scientific impact. Also, not every paperis enlisted in both databases, therefore considering only one of themcould mean that some papers could not be assigned an impact as-sessor although they may be of value to research. A shortcoming ofcitation analysis is that recent papers have had less time to accumu-late citations.

Searches were limited to English articles, published in peer-reviewedjournals. While some authors have critiqued peer-review to be anobscure process [4], potentially biased by knowing the identity ofthe author [23], the limitations of this research did not allow for aquality appraisal of each included article. Therefore, it is believedthat peer-review was the best available measure to have some qualityappraisal in place. It is assumed that high impact research on thesubject of Green Public Procurement will have been translated intoEnglish and that therefore no high impact papers have been disre-garded from the review based on the language restriction.

The timeframe chosen for this systematic review are the years be-tween January 1, 1997 and December 31, 2012. Around the millen-nium a number of initiatives have been launched by the National In-stitute of Governmental Purchasing, Inc. (NIGP) to foster academiato pay more attention to the until then largely neglected field ofpublic procurement [26] [8]. Those included, beyond others, undera partnership agreement with the Florida Atlantic University (FAU)the establishment of the Public Procurement Research Center, aswell as the launch of the first scholarly journal on the field, the Jour-nal of Public Procurement. We decided on setting the cut-off yearfor this review a few years before the launch of those initiatives to,among other things, be able to assess their impact on the field. Thefinal year of consideration, 2012, was the most current research year

6

when this systematic review was initiated in 2013 and was thus cho-sen to represent the most recent developments.

Search terms were developed by testing individual term’s effective-ness against batches of twenty search results. As the main literaturereview’s objective was to assess the status of public procurementat large and not green public procurement in specific, search termswere developed, which address the main research field. After the testbatches had revealed many terms to be ineffective in that they didnot add any new or relevant results, the following are the final searchterms utilized in combination.

• Variations of public: public; government

• Variations of procurement: procurement; purchasing; con-tracting; buying; commissioning

For the database Scopus, searches for the search terms were restrictedto title, abstract and keywords of the article. The proximity oper-ator of W/5 was included between two consecutive search terms toinclude results where the two search terms appear within five words.Scopus advises researchers to use a proximity operator of either 3, 4or 5, if they wished to find the search terms within phrases [2]. Tolower the threat of falsely omitting relevant literature, we utilized thewidest of the advised proximity operators. For the Web of Sciencedatabase searches were restricted to the topic subject and title. Inline with the Scopus searches, the proximity operator NEAR/5 wasused. For the Scopus database searches were restricted to the subjectarea of Social Sciences Humanities. For the Web of Science databasesearches were restricted to the subject areas Science Citation IndexExpanded and Social Sciences Citation Index.

Search results were assessed for relevance in a three-step processbased on Bettany-Saltikov (2010) by comparing title, abstract and

7

the full text against stipulated criteria for in- and exclusion [6]. Themain rationale was that we only wanted to include articles that werestrictly on the topic of public procurement and which provided ex-emplars of current practices, best or worst, as well as guidelines forpractice and research. The criteria may be requested from the au-thors. Relevance assessments were partly conducted in a team offour. The other three researchers could not finish all batches dueto personal agendas that no longer allowed for the time intensive ef-fort. As a result, the title assessment was fully conducted in a team,while the abstract assessment was only partly conducted in a team,and the final full text assessment was conducted by the first authoralone. The remaining 378 articles were then coded against elevenmain categories and subsequently anaylzed by means of descriptivestatistics using the software SPSS.

3. Findings and discussion

3.1. Status of Green Public Procurement Research

3.1.1. Annual Publications

Only 29 articles, translating to 7.7% of the papers included in thecomprehensive literature review, were on the topic of Green PublicProcurement. While the systematic review regarded the time frame1997 to 2012, the first relevant GPP article only got published by theyear 2003. As Figure 1 shows, publications remained low throughoutthe following years, yet with an increase by 2011, which continued in2012. While the annual publications on Green Public Procurementare still low, this increase indicates a raise in scientific awareness.Very likely the annual publications will continue to increase in theyears succeeding 2012.

8

Figure 1: Publication trend

3.1.2. Authorships

59 authors published relevant GPP articles over the years under re-view. Of those, only six authors published more than one article,the most active being Lutz Preuss who published four. Those fourpublications made Preuss the overall fourth most active publisherof Public Procurement research at large. The two authors Parikka-Alhola and Walker published three relevant articles on Green PublicProcurement and they are also amongst the most active authors onPP research overall with their amount of GPP publications. Thisindicates that while the PP research field at large is fragmented witha high amount of individual authors who each only published one ortwo articles, the sub-field of Green Public Procurement appears tohave been approached by researchers more specialized on that par-ticular sub-field. The otherwise high amount of authors with merelyone publication, 89.8%, poses to a scattered field, which is assessedby many researches unfamiliar with GPP. It is believed that researchquality enhances with specialized scientists who are knowledgeableand experienced on the field. In this respect the relatively youngfield of Green Public Procurement research shows some promisingindications of emerging to a well-grounded field.

9

Table 1: Authorships

Authorship Frequency Percent

Single author 6 20.7%Two authors 14 48.3%Three authors 5 17.2%Four authors 2 6.9%Five authors 1 3.4%Six authors 0 0.0%Seven authors 1 3.4%

Total 29 100.0%

48.3% of reviewed paperswere published inco-authorships between two.This is followed by 20.7%of single author papers and17.2% of articles published incollaboration between three.13.8% of articles were pub-lished in collaboration be-tween more than three au-thors, with one paper writtenby seven researchers, whichis the highest amount of au-thors observed for all 378 pa-pers on the PP field at large. With the exception of the very firstpublication in 2003, collaborations appear to have been most promi-nent from the start. This is very different to what we observed forthe PP field at large, where collaborations only became most promi-nent over time.

3.1.3. Publishing Journals

24 journals published GPP literature. The Journal of Cleaner Pro-duction is the most active publicist on the field with a total of 3relevant articles. 20 journals published only one article. This highfragmentation shows that there is no clear, most knowledgeable jour-nal on the field, which is in line with what we observed for the PPfield at large. Few journals have a clear (public) procurement or pub-lic administration background. Instead, the majority are specializedon environmental issues.

A limitation of this review, which was detected during the complete

10

review, is the search restriction to scientific databases which havedemanding requirements on journals to be listed. We found that dueto those requirements the well-established Journal of Public Pro-curement was not listed in the Web of Science database at all, whileScopus only includes articles from 2012 on. Journals specialized onnew, emergent fields such as GPP may not have had enough timeto accumulate those requirements. Therefore, it is likely that thisreview omitted journals with higher publications.

3.1.4. Publishing Countries

Each paper originated from at least one and a maximum of sevencountries, depending on authorship. Per reviewed article the countryof each author was listed, even if they came from the same country.12 countries published relevant articles. The UK were the most activewith 15 publications, closely followed by Finland with 11. With theexception of Japan and South Africa, which each have only published1 article, the GPP research field is not dominated by few highlypublishing countries. This is different to what we observed for thecomplete review of PP research, which showed a strong dominanceby publications from the UK and USA. Notably, 8 of the countries,which published relevant GPP articles, are Western European. Thus,we can still see a dominance, yet by a region instead of by individualcountries. Overall, more countries became active on the field of GPPover the course of time, which further indicates a raise in relevanceof the topic.

3.2. Predominant Study Characteristics

3.2.1. Procuring Government Levels

The majority of papers, 48.3%, did not specify the procuring gov-ernment level. This high proportion of research imprecision is com-parable to what we found during the complete review, where 56.1%

11

of articles did not specify the government level. Moreover, the timeseries analysis shows that the proportion of unspecified governmentlevels increased over time. Specifying the research context makesfindings of more direct practical applicability, as it also enables otherresearchers to assess reasons for contradictory findings.The local government level was studied most frequently, in 27.6% ofpapers, closely followed by the municipal level, which was researchedin 24.1% of papers. The proportion of papers studying the municipallevel increased over time, while the local level’s relevance proportion-ately decreased. Notably, merely one paper studied the federal level.While the overall dispersion as well as development over time is thesame as observed for the main PP research field, the federal level wasstudied close to equally often as the municipal one by articles on PPat large.

Table 2: Government levels

Government Frequency Percent

Governmental 1 3.4%Municipal 7 24.1%Local 8 27.6%Unspecified 14 48.3%

Total 30 103.4%

We further conducted analy-ses which juxtapose the vari-able of interest with the pub-lishing country. For theseanalyses it must be notedthat each contributing au-thor’s affiliation was consid-ered. For instance, whilethere were only two paperspublished by Norwegian au-thors, both were written inco-authorships and thus by in

total five Norwegian authors. Accordingly, the sum of the countries’publications is greater than the sum of reviewed GPP publications.As observed on the basis of a cross-analysis of publishing countryagainst studied government level, the one paper on the federal leveloriginated in the USA. All countries but the USA and France showhigh proportions of unspecified government levels. Especially the

12

Netherlands appear most imprecise in that all five authors, who con-tributed to a total of two articles, did not specify the procuringgovernment level. Further, all countries but the USA merely stud-ied one government level. The USA are the only publishing country,which addressed all three government levels.

A further cross-analysis assessed differences with respect to paperimpact. For this purpose, mean scores of the Scopus and Web ofScience citation counts were calculated. Since observations of singlepapers would have been impossible to generalize, citation counts weregrouped into decadic categories of 0 - 10, 10.5 - 20, and so on, up tothe highest citation of the complete review, 135. A first observationmade on the basis of this cross-analysis is that all twenty-nine articlesreceived low mean citations. The highest citation category is 20.5 to30, and the highest mean citation a GPP paper received was 23. Adetailed look at the data showed that the citation counts were com-parable across the two databases, thus the mean citation presentsa solid representation of the true impact of each paper. Overall,papers published in the final year under review, 2012, received lowcitations between 0 and 3, while three of them did not accumulateany citations in either database. This demonstrates that in fact ci-tation analysis discriminates against recent publications. However,the paper with the highest mean impact, 23, was published in 2009,whereas the first relevant publication of 2003 received a mean of 11citations. Thus, we can still see that time is not the only determinantof citations, but that scientific impact is most influential. While theoverall low citations are a testament to the field’s recency, they alsoindicate a low relevance of the GPP field to other fields.

The data show that papers with more citations are more contextspecific in that none of the highest category papers of 20.5 to 30 meancitations did not articulate the government level, while only 16.7% ofthe medium impact papers did. This sharply contrasts with 65% of

13

low impact papers not specifying the government level. This findingis in line with the previously made annotation that good research isspecific, since only specific research’s findings can be tested againstdifferent contextual backgrounds regarding generalizability.

3.2.2. Product Types

For product types we could also observe a high proportion of im-precision: 41.4% did not specify a product type. This proportion isnoticeably higher than what we found for the PP field at large, where28.6% of articles did not specify the product type. Furthermore, allthree of the highest impact papers did not specify the product types.Procurement requirements vary across different product types andtherefore this variable is key in putting research and its findings intoperspective and making it of practical relevance. Works were rarelystudied, by only two papers. In contrast, in the complete review ofthe PP field at large, works were found to increasingly have gained inrelevance, making up 25% of papers published in the final four years2009 to 2012. A further contrast is that goods were most frequentlystudied in GPP papers (41.4%) while for PP at large, goods wereleast prominent (18%). Over the course of time, goods’ relevanceto GPP research remained relatively stable, while services’ increasednotably. This may indicate an imminent change in rank order ofthose two product types.

The country analysis uncovered that while works have overall beenrarely studied, they were considered relevant by five of the twelvepublishing countries. Thus, there still is potential to expand researchactivity on this particular product class. Finland, Italy, Sweden andthe UK addressed all product types. Dutch and Norwegian pub-lications, on the contrary, did not address a particular product inany publication. According to the time series analysis of publishingcountries, both countries were active in either one (Norway) or both

14

Table 3: Product types per interval

Product type 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Goods 5 50.0% 7 38.9%Services 1 100.0% 1 10.0% 8 44.4%Works 1 10.0% 1 5.6%Unspecified 5 50.0% 7 38.9%

Total 1 100.0% 12 120.0% 23 127.8%

(Netherlands) of the two final four year intervals. Thus, while thisimprecision with respect to the important variable of product classmay indicate an exploratory state of research, published in the earlyyears, this is not confirmed by our data. Both countries are thereforeadvised to make their research more specific, especially the Nether-lands, which also never studied any particular government level.With respect to differences across varying paper impacts we findthat the degree of impreciseness increases with each higher citationcategory: while 30% of the low impact papers did not specify aproduct type, 50% of the medium impact papers did and 100% ofthe highest impact papers.

3.2.3. Industries and Sectors

7 different industries and sectors were studied, yet each with a lowrelevance of only 1 to 2 papers (Table 4). The majority of papersdid not specify the providing private industry or sector. Wheneverthose papers referred to services, they were grouped under a pooledcategory ”services, not specified”. Whenever articles referred to ei-ther goods or did not specify a product type while not specifyingan industry or sector, they were categorized under ”private sector,not specified”. 69% of GPP papers were categorized under ”privatesector, not specified”, which is higher than for the PP field at large,where 38.1% of articles fell into this category. This difference is

15

most likely attributable to the overall higher prominence of goods aswell as unspecified product types for GPP research. A further 20.7%were categorized under ”services, not specified”, which is in line with22.5% of the PP articles categorized accordingly. The proportionateshares of these two pooled categories increased over time, howeveraccomplished by an increase in specified industries and sectors.

Table 4: Industries and sectors

Industry / sector Frequency Percent

Private sector, not specified 20 69.0%Services, not specified 6 20.7%Construction 2 6.9%Catering 2 6.9%Defense sector 1 3.4%Professional services 1 3.4%Manufacturing 1 3.4%Transportation 1 3.4%SMEs 1 3.4%

Total 35 120.7%

Finland and Sweden were most versatile in that they addressed 3specific industries and sectors, while the myjority of countries onlyaddressed the pooled categories.Once again, lower impact papers show greater variation in that thetwenty articles falling into the first citation category of 0-10 meancitations addressed all industries and sectors but small and mediumenterprises (SMEs). The medium impact papers were all catego-rized into either the pooled services or pooled goods and unspecifiedprivate category. Highest impact paper with 20.5-30 citations onlyaddressed the pooled goods category as well as SMEs.

16

3.2.4. Studied Countries

While 12 countries published GPP research, 28 different countriesand regions were studied.

Table 5: Countries studied per paper

# Countries Frequency Percent

1 23 79.30%2 2 6.90%3 1 3.40%7 2 6.90%none 1 3.40%

Total 29 100.00%

We found a strong home biasof researchers in that themajority studied their homecountry or the country of theinstitution they were associ-ated with. The majority ofpapers, 79.3%, merely stud-ied one country. As publicprocurement practices varyacross countries, embeddedin specific institutional sys-tem contexts, valuable in-sights can be gained from

critically assessing the home country’s practices against those ofother countries.

The UK was studied most frequently, in 9 papers, whereas 19 coun-tries and regions were only studied once. Further, only 6 of the stud-ied countries are non-European, which indicates that applicability ofresearch findings on GPP is limited to Europe.

3.3. Predominant Research Designs

3.3.1. Methodologies

While it was aspired to only group each paper into either the qual-itative or quantitative category, 1 paper clearly followed a mixedapproach and was therefore categorized as applying both methods.The 29 reviewed GPP articles mainly applied qualitative data analy-sis methods, overall in 79.3% of papers. While during the first eight

17

years, 2001-2008, qualitative research was conducted in all papers,quantitative methods became more prominent in the final four years,2009-2012, when 38.9% of papers applied them. While still 66.7%of papers conducted qualitative research in this final interval, thisdevelopment towards more balance in research methodologies indi-cates maturation. This development also observed for the PP fieldat large, yet with a stronger balance of 48.9% versus 55% in the finalfour years for the qualitative and quantitative methods, respectively.

Table 6: Methodologies

Methodology Frequency Percent

Qualitative 23 79.3%Quantitative 7 24.1%

Total 30 103.4%

All countries, except for Fin-land, Italy and Norway, con-ducted qualitative analysesin the majority of their pa-pers. Finland and Italyapplied both the qualitativeand quantitive analyses closeto eqally often, whereas Nor-way conducted quantitativeanalyses in 100% of its publi-

cations, while qualtitative were stil conducted in 60% of Norwegianpapers.With respect to differences across paper impact categories we findthat lower impact papers show more balance in research method-ologies, while highest impact papers exclusively relied on qualitativeanalyses. This is most likely attributable to the fact that papers withhigher citations were published in the earlier years.

3.3.2. Time Dimension

82.8% of GPP papers designed their research cross-sectional and notlongitudinal. This preference for the snapshot perspective stayedhigh throughout all reviewed years, yet the longitudinal design gainedsome more relevance over time. This development is in line with what

18

we found during the complete review, where overall 79.6% of the 378reviewed articles applied a cross-sectional design.While the majority of countries relied exclusively on the cross-sectionaldeisgn, the 1 paper by Japan was designed longitudinal, whereas Fin-land and the USA still adopted this design in the majority of theirpublications, in 63.6% and 71.4% of papers, respectively.

Table 7: Time dimension per publishing country

Publishing country Cross-sectional Longitudinal

Finland (11) 36.4% 63.6%France (2) 100.0%Germany (4) 100.0%Italy (8) 100.0%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 100.0%South Africa (1) 100.0%Spain (4) 100.0%Sweden (6) 100.0%UK (15) 100.0%USA (7) 28.6% 71.4%

As with research methodology, lower impact papers were more bal-anced than higher impact papers. According to Babbie (2006), gen-eralizations should only be made with great caution from cross-sectional studies [5]. Researchers on the field of Green Public Pro-curement are advised to adopt the longitudinal design more fre-quently to make findings meaningful to other contexts.

3.3.3. Data Collection Methods

Interviews and reviews of non-academic literature were most promi-nent in GPP research, each applied by 44.8% of papers. Literaturewas characterized as non-academic when it pertained to non-scienticliterature, such as books, law texts, business reports, web-sites, man-

19

Figure 2: Data sources per interval

uals, contracts and similar written sources. Questionnaires were uti-lized by 20.7% of papers, followed by secondary analysis, 13.8% andreviews of scientific literature, 10.3%. Focus groups were only con-ducted by one paper, whereas no article was based on results fromexperiments. One paper did not specify its data sources. Specify-ing data sources makes research more transparent and enables otherresearchers to test the findings. Accordingly, all researchers are ad-vised to articulate their data sources clearly to develop the researchfield to a more systematic science. Notably, however, is that withregard to data collection imprecision, GPP appears much more spe-cific than the PP field at large, where every fifth article (20.4%) didnot articulate its sources. Besides this difference, the rank order ismostly the same as for PP at large.

Over the course of time, more data collection methods were utilized,

20

and in the final four years all methods except for experiments weredeployed. This is a positive development, as it enriches research withversatility and thus increases generalizability of findings. Moreover,both interviews as non-academic literature reviews were applied con-tinuously less often over time. As survey methods, such as interviewsand questionnaires, record self-reports, potentially biased by factorssuch as respondents’ experience level, their findings have limited va-lidity. Podsakoff and Organ (1986) describe the main problems ofself-reports to be the consistency motif, social desirability and non-verifiability [22, pp. 533 - 535]. Non-scientific literature, too, canhold whitewashed, non-verifiable information, such as corporate web-sites or business reports. Accordingly, the decrease in prominence ofboth these methods is a further positive trend. Secondary analysesshow a slight increase in utilization. By utilizing past researchers’efforts and findings, secondary analysis has the potential to developthe field of Green Public Procurement research to a state of provenfindings and generally accepted paradigms.

Most countries showed variety in data collection in that each applied3 to 4 methods with relatively balanced proportions. Exceptions areFrance, the Netherlands, South Africa and Spain, which each onlydeployed one data source. Japan’s one reviewed article did not spec-ify its sources.

While lower impact papers adopted all data collection methods andare thus the most diversified, utilization of scientific literature reviewsis higher per citation category: while 40% of the low impact papersreviewed scientific literature, 50% of te medium impact papers didand 66.7% of the highest impact papers. This indicates that higherimpact articles utilized past researchers efforts and findings more,which is important in developing the GPP field to a state of definitefindings.

21

3.3.4. Research Strategies

Single case studies were most prominent in Green Public Procure-ment research, conducted by 41.4% of papers, followed by surveyresearch, 27.6%, and multiple case studies, 17.2%. Case studiesare valuable in developing an understanding of one particular phe-nomenon or event holistically. A shortcoming of single case studiesis their generalizability to other units, as the information gatheredis strongly embedded in the constructs of the unit of analysis [20].

Figure 3: Data sources per interval

GPP research relied on this unit specific assessment noticeably moreoften than the complete PP field (28.3%). Eisenhardt (1989) statedthat case studies are particularly suited for new research areas [16],and thus their high prominence may be attributable to the recencyof the GPP field. Multiple case studies improve generalizability since

22

findings can be compared, analyzed and contrasted [17]. Survey re-search has the aforementioned shortcoming of limited verifiability.The overall low prominence of literature studies, conducted in 10.3%of papers, and most notably meta-studies, conducted in one paper,shows that research findings have rarely been synthesized in the past.As research on Green Public Procurement is still in its infancy, thisappears logical. Still, researchers should keep in mind that synthe-ses deliver generalizable findings with broader applicability than casestudies or survey research.None of the reviewed papers conducted quantitative modelling. Thedispersion is mostly the same for the complete public procurementresearch field with the sole difference that quantitative modelling hasoverall been conducted in 17.7% of PP articles. Over the course oftime, more research strategies were applied, which is a positive mat-uration indicator.

Finland is most diversified in that it applied all strategies but surveyresearch with close to equal proportions. This is in line with whatwe found during the complete review: more active publishers showgreater variety in their research designs and are thus most mature.Moreover, Finland is the only publishing country that utilized meta-studies, while it is also strong on literature studies.Papers with lower citations show greater variation in employed re-search strategies: while the first citation category employed all strate-gies, the second only utilized 3 and the third only 2 research strate-gies. Remarkably, the relevance of multiple case studies also increasesper citation category: where 66.7% of highest impact papers con-ducted multiple case studies, 33.3% of medium impact and merely5% of low impact papers did.

23

3.4. Topics

While for the complete review we only analysed how often each topicwas addressed, this focused analysis of Green Public Procurementpapers enabled us to look specifically which topics were addressedagainst the background of GPP. 11 papers addressed a second topic,which translates to 37.9%. The majority of papers thus exclusivelylooked at GPP as a whole concept without studying specific pro-curement decisions. Overall 5 topics were addressed. Legal aspectswere studied most often, by 4 papers in total. Legal aspects werean issue from the beginning on, in fact the very first, relevant articleon GPP was on this topic. Selection was studied by 3 papers andsupplier relations by a further 2. Contracting and procurement toolswere only studied once each. The tool studied was a priority rankingscheme for green procurement. Although this model was developedbased on qualitative attributes data (see [25]) this kind of tool in-dicates that GPP decisions can well be determined mathematicallyafter qualitative data informed a scheme. Accordingly, researchersmay start utilizing quantitative modelling, which we earlier found tohave been a never utilized research strategy.Over the three intervals in which GPP research was published, theamount of topics studied continuously increased. In the final fouryears all topics except for PP tools were assessed. This indicatesa maturation of GPP research from the state of exploration to onewhere specific research problems are assessed.

9 of the publishing countries studied specific topics. This shows thatGPP research is maturing on a global level. Again, Finnish researchis the most developed in that its papers addressed 2 topics, selec-tion and contracting, whereas the other countries studied 1 topic.Legal aspects have been most relevant globally in that they wereassessed by 4 countries, followed by selection and supplier relations,each assessed by 2 countries.

24

Table 8: GPP topics per interval

Topic 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Supplier rela-tions

2 25.0%

Selection 3 37.5%Contracting 1 12.5%Legal aspects 1 100.0% 1 50.0% 2 25.0%PP tool 1 50.0%

Total 1 100.0% 2 100.0% 8 100.0%

Figure 4: Topics per publishing country

3.4.1. Study Characteristics per Topic

With respect to addressed government levels per sub-topic of GPPresearch we find that the majority of papers did not specify theprocuring level. While this lack of context inhibits practical rele-vance the high proportion is mainly attributable to the topic of legalaspects of which all four papers did not articulate the level. Besidesthis imprecision, each government level was addressed by the sub-topics, the municipal slightly more often.Services were most frequently studied against each of the sub-topics

25

apart from the procurement tool, which was only directed at theprocurement of goods. Goods were the second most often studied,studied by 4 papers as opposed to 7 on services, whereas the pro-curement of works was never studied.3 industries and sectors were addressed by the 5 sub-topics exclud-ing the two pooled categories. Each of these, namely professionalservices, catering and transportation, were only assessed in 1 papereach, whereas the majority of papers were grouped into the pooledcategories of unspecified private providers of services, goods and un-specified products. The topic of selection was assessed most diverse,against 2 specific industries and sectors, professional services andtransportation. Only the topic of supplier relations was also stud-ied against a specific industry, catering, while all other sub-topics‘papers did not specify the providing industry or sector. This lackof context directly contradicts these papers interest to develop theGPP research further to a state of resolving more specific procure-ment decisions.All sub-topics with the exception of the procurement tool, have beenstudied against multiple countries, which increases generalizability.While 10 countries have been studied against the 5 sub-topics, 3of them are non-European countries, namely Brazil, South Africaand the USA. Where GPP research generally shows a clear tendencytowards Europe, this is a positive sign of research findings becomingmore relevant to countries outside Europe.

3.4.2. Research Strategies per Topic

Qualitative data analysis was most prominent per sub-topic, exceptfor the subject of selection of which all three papers conducted quan-titative analyses. According to Babbie (2006), quantification makesfindings more “explicit” [5, p. 23], while also fostering syntheses, thusthe comparison and pooling with other findings. For the topic of se-lection, decisions are often characterized by economic assessments

26

of the best supplier or partner. Accordingly, a high prominence ofquantitative research appears understandable. It is, however, be-lieved that research on the selection of green suppliers could alsogain from employing both methodologies more equally, since thiswill yield a more complete understanding of the phenomenon. Thiscall for a more balanced approach is applicable to all sub-topics.With respect to the applied time dimension the image across the5 sub-topics is close to balanced except for the subject of legal as-pects which was exclusively studied cross-sectional. An explanationfor this contrast to the overall predominantly applied snapshot per-spective is that articles which studied GPP in general possibly onlylooked at what this strategy entails and how it could be implemented.In contrast, articles with a more specific issue at stake conceivablywere more interested in assessing the long-term effects of followingthe GPP strategy. While an effectiveness assessment should followeach newly introduced procurement procedure, GPP in specific isconcerned about the long-term effects of taken measures.Regarding utilized data sources we find that papers on the sub-topicsmade use of 4 of the collections methods: questionnaires, interviews,secondary analyses and non-academic literature reviews. Articles onselection, the second most prominent subject, were most versatile inthat they deployed 3 of the methods and each with an equal pro-portion. The most often studied sub-topic of legal aspects utilized2 different collection methods, yet non-academic literature reviewswere conducted in all 4 papers on the subject. This high proportionis attributed to the study of law texts and directives, which informedthose researches. We believe that this sub-field could gain interest-ing insights from also consulting practitioners’ experiences with legaldirectives and frameworks. Neglecting personal experiences may oth-erwise result in research of little or detrimental practical relevance.Single case studies were the most prominently applied research strat-egy, conducted by all topics except for contracting, which one paperinstead conducted a literature study. As annotated earlier, single

27

case studies provide deep insights into single phenomena, yet lackgeneralizability to other contexts. Multiple case studies, which in-crease generalizability, have not been conducted by any of the sub-jects. Articles on supplier selection again appear most diversified,having utilized 3 of the strategies with equal shares.

4. Conclusion

This review analyzed the overall status of Green Public Procurementresearch published between 1997 and 2012. We found that the fieldis still largely neglected, with 29 total publications and a moderatepublication rate since the first paper of 2003. A slight upward trendwas observed, which started in 2011 and continued in 2012. This mayindicate that GPP will become of more relevance in the upcomingyears. This increase in scientific awareness is further substantiatedby the fact that an increasing amount of countries published relevantresearch during the later years.Green Public Procurement research is dominated by Europe. Aspublic procurement practices are country specific, embedded in dif-ferent institutional contexts, findings can thus only be applied tonon-European countries with great caution. This inhibition is cor-roborated by the finding that 79.3% of reviewed papers only stud-ied one country, of which European countries were most prominent.Contrasting different countries’ practices generates valuable insightsfor improving domestic procurements.Research collaborations are common and although a high amountof individual authors only published one article, specialized authorsemerged whose publications made them some of the most influentialpublishers on the field of Public Procurement research at large. Suchspecialization is important as research gains quality from researcherswho are more literate on the field.Overall, the field received low citations. The highest mean citationa paper received is 23 (as of March 2014), the majority of reviewed

28

papers received 10 mean citations or less. While this can partly beattributed to the recency of publications, it may also indicate a lowrelevance of green procurement practices to other fields.With respect to study characterisctis and research designs we foundthat although the field is still clearly in its infancy it developed toa diversified state. Even though only 29 relevant research paperswere published in the timeframe under consideration, all variablesof interest were studied. Diversity, as a indicator of maturation,is paramount for deriving at definite findings, tested against variedbackgrounds and by varied means.With the exception of experiments, all data collection methods havebeen utilized by research, as well as all research strategies but quan-titative modelling. Interviews and non-academic literature reviewswere most prominent data sources, while case studies and surveyresearches were most often utilized research strategies. Both theseresearch designs pose to a practice-oriented field. Shortcomings ofthem refer, paradoxically, to practical relevance as all those measureshave limited reliability and their findings are diffcult to generalize.The mostly neglected measures to pool findings (meta-analyses) orapply them to further analyses (scientific literature studies) disablethe field from deriving at definite findings, which can be applied bypractitioners. However, the field is still emerging and as we foundthat more diverse designs were utilized over time those obersvationsdo not pose an alarming research status.The federal government level was only studied once, which indi-cates that GPP is more relevant to lower government levels. Goodswere most often studied, whereas services’ relevance continuouslyincreased over time. Works were only researched in 2 papers. 7 dif-ferent industries and sectors were studied, yet each with a low shareof 1 to 2 papers.Observed shortcomings of published research mainly refer to a highlevel of context inspecificity: 48.3% of reviewed papers did not specifythe procuring government level, a further 41.4% did not refer to any

29

type of product. 69% of the articles were grouped into the pooledcategory of unspecified private industry or sector providing goods orunspecified products. 20.7% of papers were grouped into the pooledcategory of unspecified private industry or sector providing services.Imprecision inhibits both practical applicability of findings, as it alsodisables other researchers to directly assess validity, as well as reasonsfor contradictory findings.In the final years an increasing amount of papers were published witha specific Green Public Procurement problem at stake. This devel-opment shows that research on the field is surpassing its exploratorystatus.

5. Review Limitations

A limitation of this review is that the categorizations of all includedarticles were done by the main author alone. To make the findingsobjective, systematic literature reviews should be conducted withina team of researchers [27] [24]. As this limitation was known atthe beginning of the research project, this paper aimed at makingthe review process highly transparent to enable other researchers toreplicate the work and test the findings.A further limitation regards the exclusion of articles published inother languages than English, which may have yielded a languagebias: Egger et al. (1997) found that for medical sciences researcheswere more likely to be translated in English, if they found significantresults [15].Restricting the search for relevant papers to only two databases mayhave omitted relevant papers, since even the renowned databasesScopus and Web of Science do not hold all relevant articles. Thisapprehension was confirmed by the fact that ten years of publicationsfrom the Journal of Public Procurement were missed because thejournal was rejected by Thomson Reuters, and only accepted byElsevier in 2012.

30

It is generally considered important to include grey literature in asystematic literature review to develop a more complete overview[27] [24] [19]. Grey literature refers to “multiple document typesproduced on all levels of government, academics, business, and orga-nization in electronic and print formats not controlled by commercialpublishing i.e. where publishing is not the primary activity of theproducing body.” [1]. Due to its nature, grey literature is difficultto locate and can be abundant, which would have exceeded the timerange of this review. Moreover, this type of literature does not satisfythe research aim of presenting an overview of only the most influ-ential scientific literature. According to Davies (2000) publicationbias may adversely affect the validity of findings as journals tend tofavour publishing positive results [12].

6. Directions for Future Research

To address this review’s limitations, other researchers are encouragedto replicate the work to test the findings.While the review provides a detailed overview of researched topicsand the designs applied to study them, researchers should take thesefindings into account when designing upcoming studies. To increaseversatility and increase generalizability of findings, subjects shouldbe assessed against different backgrounds and by different means.

31

References

[1] URL http://www.greynet.org/greynethome.html.

[2] URL http://help.scopus.com/Content/h_

bscssrchFASTprox.htm.

[3] February 2013. URL http://www.oecd.org/gov/ethics/

meetingofleadingpractitionersonpublicprocurement.htm.

[4] Lawrence K. Altman. The ingelfinger rule, embargoes, and jour-nal peer review-part 1. The La, 347(9012):1382–1386, May 1996.doi:10.1016/S0140-6736(96)91016-8.

[5] Earl R. Babbie. The Practice Of Social Research. ThomsonWadsworth, 11th edition, January 2006.

[6] Josette Bettany-Saltikov. Learning how to undertake a system-atic review: part 2. Nursing Standard, 24(51):47–56, August2010. doi:10.7748/ns2010.08.24.51.47.c7943.

[7] Craig R. Carter and Lisa M. Ellram. Thirty-five years ofthe journal of supply chain management: Where have webeen and where are we going? Journal of Supply ChainManagement, 39(1):27–39, March 2003. doi:10.1111/j.1745-493X.2003.tb00152.x.

[8] Rosalyn Y. Carter and Rick Grimm. Journal of public procure-ment under the fau-nigp partnership. Journal Of Public Procure-ment, 1(1):3–8, 2001. URL http://ippa.org/jopp/download/

vol1/Carter%20&%20Grimm.pdf.

[9] Myun J. Cheon, Varun Grover, and Rajiv Sabherwal. The evo-lution of empirical research in is - a study in is maturity. Infor-mation & Management, 24(3):107–119, 1993. doi:10.1016/0378-7206(93)90060-7.

32

[10] Jeong-Wook Choi. A study of the role of public pro-curement can public procurement make society better?2010. URL http://www.ippa.org/IPPC4/Proceedings/

13ProcurementPreferences/Paper13-4.pdf.

[11] Mary M. Crossan and Marina Apaydin. A multi-dimensionalframework of organizational innovation: A systematic review ofthe literature. Journal of Management Studies, 47(6):1154–1191,September 2010. doi:10.1111/j.1467-6486.2009.00880.x.

[12] Philip Davies. The relevance of systematic reviews to educa-tional policy and practice. Oxford Review of Education, 26(3 &4):365–378, 2000. doi:10.1080/713688543.

[13] Catherine Day. Buying green: the crucial role of publicauthorities. Local Environment, 10(2):201–209, April 2005.doi:10.1080/1354983042000388214.

[14] David Denyer, David Tranfield, and Joan Ernst van Aken.Developing design propositions through research synthe-sis. Organization Studies, 29(3):393–413, March 2008.doi:10.1177/0170840607088020.

[15] Matthias Egger, Tanja Zellweger-Zahner, Martin Schneider,Christoph Junker, Christian Lengeler, and Gerd Antes. Lan-guage bias in randomised controlled trials published in englishand german. The Lancet, 350(9074):326–329, August 1997.doi:10.1016/S0140-6736(97)02419-7.

[16] Kathleen M. Eisenhardt. Building theories from case study re-search. The Academy of Management Review, 14(4):532–550,October 1989. doi:10.5465/AMR.1989.4308385.

[17] Kathleen M. Eisenhardt and Melissa E. Graebner. The-ory building from cases: Opportunities and challenges.Academy of Management Journal, 50(1):25–32, February 2007.doi:10.5465/AMJ.2007.24160888.

33

[18] Edeltraud Gunther and Lilly Scheibe. The hurdle analysis. aself-evaluation tool for municipalities to identify, analyse andovercome hurdles to green procurement. Corporate Social Re-sponsibility and Environmental Management, 13(2):61–77, 2006.doi:10.1002/csr.92.

[19] Sally Hopewell, Steve McDonald, Mike J. Clarke, and MatthiasEgger. Grey literature in meta-analyses of randomized trials ofhealth care interventions (review). The Cochrane Library, (2),2007. doi:10.1002/14651858.MR000010.pub3.

[20] Mary M. Kennedy. Generalizing from single case stud-ies. Evaluation Quarterly, 3(4):661–678, November 1979.doi:10.1177/0193841X7900300409.

[21] Christopher McCrudden. Using public procurement to achievesocial outcomes. Natural Resources Forum, 28(4):257–267,November 2004. doi:10.1111/j.1477-8947.2004.00099.x.

[22] Philip M. Podsakoff and Dennis W. Organ. Self-reports in organizational research: Problems andprospects. Journal of Management, 12(4):531–544, 1986.doi:10.1177/014920638601200408.

[23] Arnold S. Relman. Peer review in scientific journals-what goodis it? The Western Journal Of Medicine, 153(5):520–522,November 1990. URL http://www.ncbi.nlm.nih.gov/pmc/

articles/PMC1002603/.

[24] Denise M. Rousseau, Joshua Manning, and David Denyer. Ev-idence in management and organizational science: Assemblingthe field’s full weight of scientific knowledge through synthe-ses. The Academy of Management Annals, 2(1):475–515, 2008.doi:10.1080/19416520802211651.

[25] Mary Swanson, Arthur Weissman, Gary Davis, Maria LeetSocolof, and Kim Davis. Developing priorities for greener

34

state government purchasing: a california case study.Journal of Cleaner Production, 13(7):669–677, June 2005.doi:10.1016/j.jclepro.2003.12.011.

[26] Khi V. Thai. Public procurement re-examined. Journal OfPublic Procurement, 1(1):9–50, 2001. URL http://ippa.org/

jopp/download/vol1/Thai.pdf.

[27] David Tranfield, David Denyer, and Palminder Smart. To-wards a methodology for developing evidence-informed man-agement knowledge by means of systematic review. BritishJournal of Management, 14(3):207–222, September 2003.doi:10.1111/1467-8551.00375.

[28] Finn Wynstra. What did we do, who did it and did itmatter? a review of fifteen volumes of the (european) jour-nal of purchasing and supply management. Journal of Pur-chasing & Supply Management, 16(4):279–292, December 2010.doi:10.1016/j.pursup.2010.09.003.

35

A. Bibliography of Reviewed ArticlesT

itle

Yea

rJou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Ad

dre

ssin

gsu

stai

n-

able

dev

elop

men

tth

rou

ghp

ub

lic

pro

-cu

rem

ent:

Th

eca

seof

loca

lgo

vern

men

t

2009

Su

pp

lyC

hai

nM

anag

e-m

ent:

An

Inte

rnat

ion

alJou

rnal

2818

23P

reu

ssL

.

Th

ehu

rdle

anal

-ysi

s.A

self

-ev

alu

atio

nto

olfo

rm

un

icip

alit

ies

toid

enti

fy,

anal

-yse

and

over

com

ehu

rdle

sto

gree

np

rocu

rem

ent

2006

Cor

por

ate

Soci

alR

e-sp

onsi

bil

ity

and

Envi-

ron

men

tal

Man

agem

ent

2222

Gu

nth

erE

.,S

chei

be

L.

Fos

teri

ng

sus-

tain

abil

ity

thro

ugh

sou

rcin

gfr

omsm

all

bu

sin

esse

s:p

ub

lic

sect

orp

ersp

ecti

ves

2008

Jou

rnal

ofC

lean

erP

rod

uct

ion

2518

21.5

Wal

ker,

H,

Pre

uss

,L

Con

tinu

edon

nex

tp

age

36

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Su

stai

nab

lep

ro-

cure

men

tin

pra

c-ti

ce:

Les

son

sfr

omlo

cal

gove

rnm

ent

2007

Jou

rnal

Of

Envir

onm

en-

tal

Pla

nn

ing

An

dM

anag

e-m

ent

2115

18T

hom

son

,J,

Jac

kso

n,

T

Bu

yin

gin

toou

rfu

ture

:S

ust

ain

-ab

ilit

yin

itia

tive

sin

loca

lgo

vern

men

tp

rocu

rem

ent

2007

Bu

sin

ess

Str

ateg

yan

dth

eE

nvir

on-

men

t

1616

Pre

uss

L.

Dev

elop

ing

pri

or-

itie

sfo

rgr

een

erst

ate

gove

rnm

ent

pu

rch

asin

g:a

Cal

ifor

nia

case

stu

dy

2005

Jou

rnal

Of

Cle

aner

Pro

du

ctio

n

1913

16S

wan

son

,M

,W

eiss

man

,A

,D

avis

,G

,S

oco

lof,

ML

,D

avis

,K

Con

tinu

edon

nex

tp

age

37

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Exp

erie

nce

sw

ith

gree

nin

gsu

pp

lier

s.T

he

Un

iver

sita

tA

uto

nom

ad

eB

arce

lon

a

2008

Jou

rnal

ofC

lean

erP

rod

uct

ion

1712

14.5

Bal

aA

.,M

un

ozP

.,R

iera

dev

all

J.,

Yse

rnP

.

Gre

enp

rocu

rem

ent

inN

orw

ay;

asu

r-ve

yof

pra

ctic

esat

the

mu

nic

ipal

and

cou

nty

leve

l

2009

Jou

rnal

ofE

nvi-

ron

men

tal

Man

agem

ent

1311

12M

ich

else

n,

O,

de

Boer

,L

Mak

ing

the

mar

-ke

tw

ork

for

the

envir

onm

ent:

Ac-

cep

tan

ceof

(som

e)’g

reen

’co

ntr

act

awar

dcr

iter

iain

pu

bli

cp

rocu

rem

ent

2003

Jou

rnal

ofE

nvir

onm

en-

tal

Law

1111

Ku

nzl

ik,

P

Con

tinu

edon

nex

tp

age

38

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Gre

ener

pu

bli

cp

urc

has

ing:

op-

por

tun

itie

sfo

rcl

imat

e-fr

ien

dly

gove

rnm

ent

pro

-cu

rem

ent

un

der

WT

Oan

dE

Uru

les

2006

Cli

mat

eP

ol-

icy

119

10va

nA

ssel

t,H

,va

nd

erG

rijp

,N

,O

oste

rhu

is,

F

Envir

onm

enta

lcr

i-te

ria

inth

ep

ub

lic

pu

rch

ases

abov

eth

eE

Uth

resh

old

valu

esby

thre

eN

ord

icco

untr

ies:

2003

and

2005

2009

Eco

logi

cal

Eco

nom

ics

127

9.5

Nis

sin

enA

.,P

arik

ka-

Alh

ola

K.,

Rit

aH

.

Con

tinu

edon

nex

tp

age

39

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Pro

tect

ing

the

en-

vir

onm

ent

thro

ugh

pu

bli

cp

rocu

re-

men

t:T

he

case

ofS

outh

Afr

ica

2008

Nat

ura

lR

esou

rces

For

um

117

9B

olto

nP

.

Ali

fecy

cle

ap-

pro

ach

toG

reen

Pu

bli

cP

rocu

re-

men

tof

bu

ild

ing

mat

eria

lsan

del

-em

ents

:A

case

stu

dy

onw

ind

ows

2011

En

ergy

106

8T

aran

tin

i,M

,L

opri

eno,

AD

,P

orta

,P

L

Lif

ecy

cle

app

roac

hin

the

pro

cure

men

tp

roce

ss:

Th

eca

seof

def

ence

mat

erie

l

2006

Th

eIn

tern

a-ti

onal

Jou

rnal

ofL

ife

Cycl

eA

sses

smen

t

86

7H

och

sch

orn

er,

E,

Fin

nve

den

,G

Con

tinu

edon

nex

tp

age

40

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Rel

ease

the

pow

erof

the

pu

bli

cp

urs

e20

06E

ner

gyP

olic

y5

44.

5B

org

N.,

Blu

me

Y.,

Th

omas

S.,

Irre

kW

.,F

anin

ger-

Lu

nd

H.,

Lu

nd

P.,

Pin

dar

A.

Su

stai

nab

lep

art-

ner

ship

sfo

ra

gree

nec

onom

y:

Aca

sest

ud

yof

pu

bli

cp

rocu

rem

ent

for

hom

e-gr

own

sch

ool

feed

ing

2011

Nat

ura

lR

esou

rces

For

um

33

3O

tsu

ki

K.

Con

tinu

edon

nex

tp

age

41

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Gre

enp

ub

lic

pro

-cu

rem

ent:

An

aly-

sis

onth

eu

seof

envir

onm

enta

lcr

i-te

ria

inco

ntr

acts

2010

Rev

iew

ofE

uro

pea

nC

omm

un

ity

&In

tern

a-ti

onal

En

-vir

onm

enta

lL

aw

33

Pal

mu

joki,

A,

Par

ikka

-A

lhol

a,K

,E

kro

os,

A

Psy

chol

ogic

alb

ar-

rier

sin

the

road

tosu

stai

nab

led

evel

-op

men

t:E

vid

ence

from

pu

bli

cse

ctor

pro

cure

men

t

2011

Bu

sin

ess

Str

ateg

yan

dth

eE

nvir

on-

men

t

32

2.5

Pre

uss

L.,

Wal

ker

H.

Th

ere

lati

onsh

ipb

etw

een

sust

ain

-ab

lep

rocu

rem

ent

and

e-p

rocu

rem

ent

inth

ep

ub

lic

sect

or

2012

Inte

rnat

ion

alJou

rnal

ofP

rod

uct

ion

Eco

nom

ics

32

2.5

Wal

ker

H.,

Bra

mm

erS

.

Con

tinu

edon

nex

tp

age

42

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Bar

rier

sto

add

ress

-in

gsu

stai

nab

leco

nst

ruct

ion

inp

ub

lic

pro

cure

men

tst

rate

gies

2011

Pro

ceed

ings

ofth

eIn

stit

u-

tion

ofC

ivil

En

gin

eers

:E

ngi

nee

rin

gS

ust

ain

abil

ity

31

2S

oura

ni,

A,

Soh

ail,

M

Pu

bli

cp

rocu

re-

men

tin

centi

ves

for

sust

ain

able

des

ign

serv

ices

:S

wed

ish

exp

erie

nce

s

2009

Arc

hit

ectu

ral

engi

nee

rin

gan

dd

esig

nm

anag

emen

t

22

Sp

orro

ng

J.,

Bro

chn

erJ.

Wh

atfa

ctor

sin

flu

-en

ceth

eu

pta

keof

GP

P(g

reen

pu

bli

cp

rocu

rem

ent)

pra

c-ti

ces?

New

evi-

den

cefr

oman

Ital

-ia

nsu

rvey

2012

Eco

logi

cal

Eco

nom

ics

11

1T

esta

F.,

Iral

do

F.,

Fre

yM

.,D

add

iT

.

Con

tinu

edon

nex

tp

age

43

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Pro

cure

men

tfo

rsu

stai

nab

lelo

cal

econ

omic

dev

elop

-m

ent

2012

Inte

rnat

ion

alJou

rnal

ofP

ub

lic

Sec

tor

Man

agem

ent

11

Nij

aki

L.K

.,W

orre

lG

.

Su

stai

nab

ilit

yan

dlo

cal

food

pro

cure

-m

ent:

aca

sest

ud

yof

Fin

nis

hp

ub

lic

cate

rin

g

2012

Bri

tish

Food

Jou

rnal

20

1L

ehti

nen

,U

Gre

enp

ub

lic

pro

-cu

rem

ent

inp

rac-

tice

-T

he

case

ofN

orw

ay

2011

Soci

ety

and

Eco

nom

y1

1F

etA

.,M

ich

else

nO

.,B

oer

L.

Con

tinu

edon

nex

tp

age

44

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Th

eim

pac

tof

sus-

tain

able

pu

bli

cp

ro-

cure

men

ton

sup

-p

lier

man

agem

ent

-T

he

case

ofF

ren

chp

ub

lic

hos

pit

als

2012

Ind

ust

rial

Mar

keti

ng

Man

agem

ent

20

1O

ruez

abal

aG

.,R

ico

J.-

C.

Envir

onm

enta

lim

pac

tsan

dth

em

ost

econ

omi-

call

yad

vanta

geou

ste

nd

erin

pu

bli

cp

rocu

rem

ent

2012

Jou

rnal

ofP

ub

lic

Pro

-cu

rem

ent

00

Par

ikka

-A

lhol

aK

.,N

issi

nen

A.

Su

stai

nab

lep

ro-

cure

men

tin

hea

lth

and

soci

alca

rein

Nor

ther

nIr

elan

d

2012

Pu

bli

cM

oney

&M

anag

e-m

ent

00

0E

rrid

geA

.,H

enn

igan

S.

Con

tinu

edon

nex

tp

age

45

Bibliogra

phyofReviewed

Articles–co

ntinued

from

previouspage

Tit

leY

ear

Jou

rnal

Cit

atio

ns

Sco

pu

sC

itat

ion

sW

oSM

ean

cita

-ti

ons

Au

thor

s

Are

flec

tion

onth

eD

utc

hS

ust

ain

able

Pu

bli

cP

rocu

re-

men

tP

rogr

amm

e

2012

Jou

rnal

ofIn

tegr

ativ

eE

nvir

onm

en-

tal

Sci

ence

s

00

0M

elis

sen

,F

,R

ein

der

s,H

46

B. Authors

Author Publications

Preuss L. 4Parikka-Alhola K. 3Walker H. 3de Boer L. 2Michelsen, O 2Nissinen A. 2Bala A. 1Blume Y. 1Bolton P. 1Borg N. 1Brammer S. 1Brochner J. 1Daddi T. 1Davis, G 1Davis, K 1Ekroos, A 1Erridge A. 1Faninger-Lund H. 1Fet A. 1Finnveden, G 1Frey M. 1Gunther E. 1Hennigan S. 1Hochschorner, E 1Iraldo F. 1Irrek W. 1Jackson, T 1Kunzlik, P 1Lehtinen, U 1

Continued on next page

47

Table 10Authors – continued from previous page

Author Publications

Loprieno, AD 1Lund P. 1Melissen, F 1Munoz P. 1Nijaki L.K. 1Oosterhuis, F 1Oruezabala G. 1Otsuki K. 1Palmujoki, A 1Pindar A. 1Porta, PL 1Reinders, H 1Rico J.-C. 1Rieradevall J. 1Rita H. 1Scheibe L. 1Socolof, ML 1Sohail, M 1Sourani, A 1Sporrong J. 1Swanson, M 1Tarantini, M 1Testa F. 1Thomas S. 1Thomson, J 1van Asselt, H 1van der Grijp, N 1Weissman, A 1Worrel G. 1Ysern P. 1

Continued on next page

48

Table 10Authors – continued from previous page

Author Publications

B.1. Authorships per interval

Authorship 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Single author 1 100.0% 2 20.0% 3 16.7%Two authors 4 40.0% 10 55.6%Three authors 1 10.0% 4 22.2%Four authors 1 10.0% 1 5.6%Five authors 1 10.0%Seven authors 1 10.0%

Total 1 100% 10 100.0% 18 100.0%

C. Journals

Journal Publications Percent

Journal of Cleaner Production 3 10.3%Natural Resources Forum 2 6.9%Ecological Economics 2 6.9%Business Strategy and the Environment 2 6.9%The International Journal of Life Cycle Assess-ment

1 3.4%

Supply Chain Management: An InternationalJournal

1 3.4%

Society and Economy 1 3.4%Review of European Community & Interna-tional Environmental Law

1 3.4%

Public Money & Management 1 3.4%

Continued on next page

49

Table 11 – continued from previous page

Journal Publications Percent

Proceedings of the Institution of Civil Engi-neers: Engineering Sustainability

1 3.4%

Journal of Public Procurement 1 3.4%Journal of Integrative Environmental Sciences 1 3.4%Journal Of Environmental Planning And Man-agement

1 3.4%

Journal of Environmental Management 1 3.4%Journal of Environmental Law 1 3.4%International Journal of Public Sector Manage-ment

1 3.4%

International Journal of Production Economics 1 3.4%Industrial Marketing Management 1 3.4%Energy Policy 1 3.4%Energy 1 3.4%Corporate Social Responsibility and Environ-mental Management

1 3.4%

Climate Policy 1 3.4%British Food Journal 1 3.4%Architectural engineering and design manage-ment

1 3.4%

Total 29 100.0%

50

D. Publishing Countries

D.1. Publishing Countries per Interval

Publishing country 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Finland 2 20.0% 9 50.0%France 2 11.1%Germany 4 40.0%Italy 1 10.0% 7 38.9%Japan 1 5.6%Netherlands 3 30.0% 2 11.1%Norway 5 27.8%South Africa 1 10.0%Spain 4 40.0%Sweden 4 40.0% 2 11.1%UK 1 100.0% 5 50.0% 9 50.0%USA 5 50.0% 2 11.1%

Total 1 100.0% 29 290.0% 39 216.7%

E. Government Levels

E.1. Government Levels per Interval

2001 - 2004 2005 - 2008 2009 - 2012Government level Frequency Percent Frequency Percent Frequency Percent

Governmental 1 10.0%Municipal 1 10.0% 6 33.3%Local 4 40.0% 4 22.2%Unspecified 1 100.0% 4 40.0% 9 50.0%

Total 1 100.00% 10 100.0% 19 105.6%

51

E.2. Government Levels per Publishing Country

Publishing country Governmental Municipal Local Unspecified

Finland (11) 27.3% 72.7%France (2) 100.0%Germany (4) 50.0% 50.0%Italy (8) 50.0% 50.0%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 40.0% 60.0%South Africa (1) 100.0%Spain (4) 100.0%Sweden (6) 33.3% 66.7%UK (15) 53.3% 46.7%USA (7) 71.4% 28.6% 28.6%

E.3. Government Levels per Paper Impact

Citation category Governmental Municipal Local Unspecified

0 - 10 (20) 25.0% 15.0% 65.0%10.5 - 20 (6) 16.7% 16.7% 50.0% 16.7%20.5 - 30 (3) 33.3% 66.7%

52

F. Product Types

F.1. Studied Product Types

Product type Frequency Percent

Goods 12 41.4%Services 10 34.5%Works 2 6.9%Unspecified 12 41.4%

Total 36 124.1%

F.2. Product Types per Publishing Country

Publishing country Goods Services Works Unspecified

Finland (11) 54.5% 45.5% 18.2% 27.3%France (2) 100.0% 100.0%Germany (4) 50.0% 50.0% 50.0%Italy (8) 100.0% 50.0% 12.5%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 100.0%South Africa (1) 100.0% 100.0%Spain (4) 100.0%Sweden (6) 66.7% 33.3% 33.3%UK (15) 13.3% 20.0% 13.3% 66.7%USA (7) 100.0% 28.6%

53

F.3. Product Types per Paper Impact

Citation category Goods Services Works Unspecified

0 - 10 (20) 50.0% 45.0% 10.0% 30.0%10.5 - 20 (6) 33.3% 16.7% 50.0%20.5 - 30 (3) 100.0%

G. Industries and Sectors

G.1. Industries and Sectors per Interval

2001 - 2004 2005 - 2008 2009 - 2012Industry /sector

Frequency Percent Frequency Percent Frequency Percent

Construction 1 10.0% 1 5.6%Defensesector

1 10.0%

Professionalservices

1 5.6%

Services,not speci-fied

1 100.0% 1 10.0% 4 22.2%

Privatesector, notspecified

8 80.0% 12 66.7%

Manufacturing 1 5.6%Catering 2 11.1%Transportation 1 5.6%SMEs 1 10.0%

Total 1 100.0% 12 120.0% 22 122.2%

54

G.2.In

dustriesand

Sectors

perPublish

ingCountry

Pu

bli

shin

gco

untr

yC

onst

ruc-

tion

Def

ense

sect

orP

rofe

s-si

onal

ser-

vic

es

Ser

vic

es,

not

spec

ified

Pri

vate

sect

or,

not

spec

i-fi

ed

Man

ufa

c-tu

rin

gC

ater

ing

Tra

nsp

or-

tati

onS

ME

s

Fin

lan

d(1

1)18

.2%

27.3

%72

.7%

9.1%

18.2

%

Fra

nce

(2)

100.

0%10

0.0%

Ger

man

y(4

)50

.0%

100.

0%

Ital

y(8

)12

.5%

50.0

%62

.5%

37.5

%Jap

an(1

)10

0.0%

Net

her

-la

nd

s(5

)10

0.0%

Nor

way

(5)

100.

0%S

outh

Afr

ica

(1)

100.

0%10

0.0%

Sp

ain

(4)

100.

0%S

wed

en(6

)33

.3%

33.3

%33

.3%

33.3

%U

K(1

5)13

.3%

6.7%

66.7

%13

.3%

US

A(7

)28

.6%

100.

0%

55

G.3.In

dustriesand

Sectors

perPaperIm

pact

Cit

atio

nca

te-

gory

Con

stru

c-ti

onD

efen

sese

ctor

Pro

fes-

sion

alse

rvic

es

Ser

vic

es,

not

spec

i-fied

Pri

vate

sect

or,

not

spec

i-fied

Man

ufa

c-tu

ring

Cat

erin

gT

ransp

or-

tati

onSM

Es

0-

10(2

0)10

.0%

5.0%

5.0%

25.0

%65

.0%

5.0%

10.0

%5.

0%10

.5-

20(6

)16

.7%

83.3

%20

.5-

30(3

)66

.7%

33.3

%

56

H. Studied Countries

Studied country Frequency Percent

UK 9 32.1%Sweden 6 21.4%Finland 5 17.9%USA 4 14.3%Italy 3 10.7%France 2 7.1%Netherlands 2 7.1%Norway 2 7.1%EU 2 7.1%Austria 1 3.6%Brazil 1 3.6%Canada 1 3.6%Denmark 1 3.6%Estonia 1 3.6%Germany 1 3.6%Greece 1 3.6%Hungary 1 3.6%Ireland 1 3.6%Japan 1 3.6%Korea 1 3.6%Slovakia 1 3.6%South Africa 1 3.6%Spain 1 3.6%Poland 1 3.6%Western Europe 1 3.6%Eastern Europe 1 3.6%Scandinavia 1 3.6%Unspecified regions 1 3.6%

Total 54 192.9%

57

H.1. Home Bias

Publishing country Home bias

Finland 81.8%France 100.0%Germany 100.0%Italy 100.0%Japan 0.0%Netherlands 40.0%Norway 100.0%South Africa 100.0%Spain 100.0%Sweden 100.0%UK 93.3%USA 100.0%

I. Methodologies

I.1. Methodologies per Interval

Methodology 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Qualitative 1 100.0% 10 100.0% 12 66.7%

Quantitative 7 38.9%

Total 1 100.0% 10 100.0% 19 105.6%

58

I.2. Methodologies per Publishing Country

Publishing country Qualitative Quantitative

Finland (11) 54.5% 45.5%France (2) 100.0%Germany (4) 100.0%Italy (8) 50.0% 50.0%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 60.0% 100.0%South Africa (1) 100.0%Spain (4) 100.0%Sweden (6) 66.7% 33.3%UK (15) 86.7% 13.3%USA (7) 100.0%

I.3. Methodologies per Paper Impact

Citation category Qualitative Quantitative

0 - 10 (20) 75.0% 30.0%10.5 - 20 (6) 83.3% 16.7%20.5 - 30 (3) 100.0%

J. Time Dimension

Time dimension Frequency Percent

Cross-sectional 24 82.8%Longitudinal 5 17.2%

Total 29 100.0%

59

J.1. Time Dimension per Interval

Time dimension 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Cross-sectional 1 100.0% 9 90.0% 14 77.8%Longitudinal 1 10.0% 4 22.2%

Total 1 100.0% 10 100.0% 18 100.0%

J.2. Time Dimension per Paper Impact

Citation category Cross-sectional Longitudinal

0 - 10 (20) 80.0% 20.0%10.5 - 20 (6) 83.3% 16.7%20.5 - 30 (3) 100.0%

K. Data Collection Methods

Data collection method Frequency Percent

Interview 13 44.8%Literature review, non-academic 13 44.8%Questionnaire 6 20.7%Secondary analysis 4 13.8%Literature review 3 10.3%Observation 1 3.4%Focus group 1 3.4%Unspecified 1 3.4%

Total 42 144.8%

60

K.1. Data Collection Methods per Interval

Data collec-tion method

2001 - 2004 2005 - 2008 2009 - 2012

Frequency Percent Frequency Percent Frequency Percent

Questionnaire 2 20.0% 4 22.2%Interview 6 60.0% 7 38.9%Secondaryanalysis

1 10.0% 3 16.7%

Observation 1 5.6%Literature re-view

2 20.0% 1 5.6%

Literaturereview, non-academic

1 100.0% 6 60.0% 6 33.3%

Focus group 1 5.6%Unspecified 1 5.6%

Total 1 100.0% 10 170.0% 18 133.3%

61

K.2.Data

Collection

Meth

odsperPublish

ingCountry

Publish

ing

countr

yQ

ues

tion

-nai

reIn

terv

iew

Sec

ondar

yan

alysi

sO

bse

rvat

ion

Lit

erat

ure

revie

wL

iter

ature

revie

w,

non

-ac

adem

ic

Focu

sgr

oup

Unsp

ecifi

ed

Fin

land

(11)

27.3

%27

.3%

9.1%

63.6

%F

rance

(2)

100.

0%G

erm

any

(4)

50.0

%10

0.0%

50.0

%50

.0%

Ital

y(8

)62

.5%

37.5

%12

.5%

Jap

an(1

)10

0.0%

Net

her

lands

(5)

100.

0%

Nor

way

(5)

40.0

%40

.0%

60.0

%60

.0%

Sou

thA

fric

a(1

)10

0.0%

Spai

n(4

)10

0.0%

Sw

eden

(6)

33.3

%66

.7%

33.3

%33

.3%

UK

(15)

26.7

%66

.7%

46.7

%13

.3%

USA

(7)

71.4

%28

.6%

28.6

%

62

K.3.Data

Collection

Meth

odsperPaperIm

pact

Cit

atio

nca

te-

gory

Ques

tion

-nai

reIn

terv

iew

Sec

ondar

yan

alysi

sO

bse

rvat

ion

Lit

erat

ure

revie

wL

iter

ature

revie

w,

non

-ac

adem

ic

Focu

sgr

oup

Unsp

ecifi

ed

0-

10(2

0)15

.0%

35.0

%15

.0%

5.0%

10.0

%40

.0%

5.0%

5.0%

10.5

-20

(6)

33.3

%50

.0%

16.7

%50

.0%

20.5

-30

(3)

33.3

%10

0.0%

33.3

%66

.7%

63

L. Research Strategies

Research strategy Frequency Percent

Single case study 12 41.4%Survey research 8 27.6%Multiple case study 5 17.2%Literature study 3 10.3%Meta-study 1 3.4%

Total 29 100.0%

L.1. Research Strategies per Interval

Research strategy 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent

Survey research 1 10.0% 7 38.9%Literature study 1 10.0% 2 11.1%Meta-study 1 5.6%Single case study 1 100.0% 4 40.0% 7 38.9%Multiple case study 4 40.0% 1 5.6%

64

L.2. Research Strategies per Publishing Country

Publishingcountry

Surveyresearch

Literaturestudy

Meta-study

Singlecasestudy

Multiplecasestudy

Finland (11) 27.3% 27.3% 27.3% 18.2%France (2) 100.0%Germany (4) 50.0% 50.0%Italy (8) 50.0% 37.5% 12.5%Japan (1) 100.0%Netherlands(5)

60.0% 40.0%

Norway (5) 40.0% 60.0%South Africa(1)

100.0%

Spain (4) 100.0%Sweden (6) 33.3% 33.3% 33.3%UK (15) 40.0% 33.3% 26.7%USA (7) 28.6% 71.4%

L.3. Research Strategies per Paper Impact

Citation cate-gory

Surveyresearch

Literaturestudy

Meta-study

Singlecasestudy

Multiplecasestudy

0 - 10 (20) 30.0% 15.0% 5.0% 45.0% 5.0%10.5 - 20 (6) 16.7% 50.0% 33.3%20.5 - 30 (3) 33.3% 66.7%

65

M. Topics

M.1. Topics per Paper Impact

Citation cate-gory

Supplierrelations

Selection Contracting Legal as-pects

PP tool

0 - 10 (9) 22.2% 33.3% 11.1% 33.3%10.5 - 20 (2) 50.0% 50.0%

M.2. Government Levels per Topic

Topic Governmental Municipal Local Unspecified

Supplier relations (2) 50.0% 50.0%Selection (3) 66.7% 33.3%Contracting (1) 100.0%Legal aspects (4) 100.0%PP tool (1) 100.0%

M.3. Product Types per Topic

Topic Goods Services Unspecified

Supplier relations (2) 50.0% 100.0%Selection (3) 66.7% 33.3%Contracting (1) 100.0% 100.0%Legal aspects (4) 25.0% 50.0% 50.0%PP tool (1) 100.0%

66

M.4. Industries and Sectors per Topic

Topic Professionalservices

Services,not speci-fied

Privatesector, notspecified

Catering Transpor-tation

Supplier relations (2) 50.0% 50.0% 50.0%Selection (3) 33.3% 33.3% 33.3%Contracting (1) 100.0% 100.0%Legal aspects (4) 50.0% 75.0%PP tool (1) 100.0%

67

M.5.CountriesStu

died

perTopic

Top

icB

razi

lD

enm

ark

Fin

land

Fra

nce

Net

her

lands

Nor

way

Sou

thA

fric

aSw

eden

USA

EU

Supplier

rela

tion

s(2

)50

.0%

50.0

%Sel

ecti

on(3

)33

.3%

33.3

%10

0.0%

Con

trac

ting

(1)

100.

0%10

0.0%

Leg

alas

pec

ts(4

)25

.0%

25.0

%25

.0%

25.0

%25

.0%

PP

tool

(1)

100.

0%

68

M.6. Methodologies per Topic

Topic Qualitative Quantitative

Supplier relations (2) 100.0%Selection (3) 100.0%Contracting (1) 100.0%Legal aspects (4) 100.0% 25.0%PP tool (1) 100.0%

M.7. Time Dimension per Topic

Topic Cross-sectional Longitudinal

Supplier relations (2) 50.0% 50.0%Selection (3) 66.7% 33.3%Contracting (1) 100.0%Legal aspects (4) 100.0%PP tool (1) 100.0%

69

M.8.Data

Collection

Meth

odsperTopic

Top

icQ

ues

tion

nai

reIn

terv

iew

Sec

ondar

yan

alysi

sL

iter

ature

revie

w,

non

-aca

dem

icU

nsp

ecifi

ed

Supplier

rela

tion

s(2

)50

.0%

50.0

%Sel

ecti

on(3

)33

.3%

33.3

%33

.3%

Con

trac

ting

(1)

100.

0%L

egal

asp

ects

(4)

25.0

%10

0.0%

PP

tool

(1)

100.

0%

70

M.9. Research Strategies per Topic

Topic Surveyresearch

Literaturestudy

Meta-study

Singlecasestudy

Supplier rela-tions (2)

50.0% 50.0%

Selection (3) 33.3% 33.3% 33.3%Contracting(1)

100.0%

Legal aspects(4)

100.0%

PP tool (1) 100.0%

71