1

Common Approaches for Studying the Advocacy Coalition Framework: Review of Methods and Exemplary Practices

Jonathan J. Pierce*, Katherine C. Hicks*, Holly L. Peterson**, and Leanne Giordono**

*Seattle University and **Oregon State University

Version 8.31.17

European Consortium for Political Research General Conference 2017 in Oslo, Norway

Abstract

This paper strives to better understand the methodologies that scholars use in applying the

advocacy coalition framework (ACF) across purposes and contexts, as well as to address past

criticisms of ACF methodology. Content analysis is conducted of 161 English language peer

reviewed journal articles published between 2007 and 2014. Methods of data analysis and

collection for ACF applications are collected and categorized according to theory analyzed,

topical policy domain, and continent of application. Results indicate that the majority of ACF

articles use qualitative analysis and make use of interviews and/or documents. Frequency

varies depending on theory analyzed, policy domain, and continent of application. In addition,

methodologically exemplary articles for each of the ACF’s three theories are identified to help

guide future research. Overall, this paper contributes a snapshot of current and historical

methodological variation among ACF applications and identifies future opportunities for ACF

research methods.

Acknowledgements: This research was supported by a grant provided by Seattle University and a fellowship from the National Science Foundation. Also, the work of Kathleen Hannick in data collection is greatly appreciated.

2

Introduction

Jenkins-Smith et al. (2014) argue that to continue developing the advocacy coalition

framework (ACF), common methodological approaches should be established, and applications

should continue to expand to unique contexts (207). To these ends, Jenkins-Smith et al. (2014)

identify three key issues to be addressed in support of the ACF’s advancement. This paper

adopts those three issues as its research questions: (1) Which approaches to analysis and data

collection are currently being used to study the ACF, and do they vary by ACF theory? (2) To

what degree is the ACF applied to different contexts, both policy domains and governing

systems, and are these contexts associated with different methods of analysis and data

collection? (3) What exemplary methods exist for analyzing coalition structure, policy change,

and policy-oriented learning?

Advocacy Coalition Framework

The ACF comprises three major theories: advocacy coalitions, policy change, and policy-

oriented learning. A brief introduction to each follows.

Theory of Advocacy Coalitions

Advocacy coalitions are defined by their shared beliefs and coordinated actions (Jenkins-

Smith et al., 2014). Researchers studying coalitions often explore questions related to

identification and formation of coalitions, coalition stability over time, degree of shared beliefs,

coordination, and expression of actor viewpoint (Pierce et al., 2017A). The five hypotheses

identified by the ACF regarding coalitions can be found in Sabatier and Weible (2007, 220).

Theory of Policy Change

3

The ACF posits that policy change reflects a winning advocacy coalition’s policy beliefs

(Pierce & Weible, 2016). Accordingly, major policy change reflects alterations in policy core

beliefs, which concern policy problem definition and policy objectives. Minor policy change

reflects alterations in secondary or instrumental beliefs (Sabatier & Jenkins-Smith, 1999). Policy

change can be bottom-up or top-down. Four primary pathways are associated with bottom-up

policy change: coalitions taking advantage of perturbations or events external to the

subsystem; coalitions taking advantage events internal to the subsystem; policy-oriented

learning among or between coalitions; and negotiated agreements between coalitions (Sabatier

& Weible, 2007; Jenkins-Smith et al., 2014). Jenkins-Smith et al. (2014) hypothesize that these

pathways and combinations thereof are necessary, but not sufficient conditions for policy

change. A fifth, top-down pathway to policy change is imposed by a hierarchically superior

jurisdiction (Sabatier & Jenkins-Smith, 1993, 217). This pathway is identified in a second policy

change hypothesis by Jenkins-Smith et al. (2014).

Theory of Policy-Oriented Learning

Jenkins-Smith and Sabatier (1993A) define policy-oriented learning as “enduring

alterations of thought or behavioral intentions that result from experience and which are

concerned with the attainment or revision of the precepts of the belief system of individuals or

collectives” (42). Learning refers to lasting changes in beliefs and strategies that are rationally

instrumental. Changes in beliefs may concern problem perception and causality, identification

of viable alternatives, or strategies (Jenkins-Smith et al., 2014). Jenkins-Smith et al. (2014)

identify five hypotheses about the conditions that facilitate policy-oriented learning among and

between coalitions.

4

Limitations and Criticisms of the ACF

Fischer (2003) building on Hajer (1995) argues that the ACF is too empirical leading to

neglect of social and historical contexts, stating the framework,

neglects the social and historical context in which such [policy] change takes place … contextless statements are essentially a consequence of Sabatier’s empiricist desire to develop empirical hypotheses that are universally applicable to the widest range of social contexts. To engage in this kind of science, however, important explanatory factors have to be put in ‘black boxes’. In short, they have to be placed outside the analysis (101).

ACF scholars are thusly constricted in their understanding of coalitions, policy change, and

learning because they value empirical generalizability over specificity. This study considers the

critiques of Fischer (2003) and Hajer (1995) by exploring the degree to which ACF applications

use qualitative analysis as well as incorporate data collection methods that can provide context

from interviews and documents.

A second critique voiced by scholars (John, 1998; Parsons, 1995; Andersson, 1998) is

that because of its roots in American populism, the ACF may have limited applicability to a

European setting. In addition, Jenkins-Smith et al. (2014) argue that the ACF is not being applied

comparatively across different political systems. However, ACF applications outside North

America and by non-North American authors have burgeoned in recent years. In fact, there are

now more European ACF authors and applications to European policy process than any other

continent (Pierce et al., 2017A). Applications to other continents have also appeared; Henry et

al. (2014) identify and discuss 27 articles applying the ACF outside of North America and

Western Europe, and Jang et al. (2016) identify 67 articles applying the ACF to South Korea

(most of which are in Korean) between 2002 – 2014. Therefore, this study identifies articles

5

that compare across political systems, and compares the methods used to study policy

processes in North America, Europe, and other continents including Asia.

Weible et al. (2011) argue that there is a need to better understand how the ACF applies

to different policy domains. Jenkins-Smith et al. (2014) make a related argument about the

need to better understand what methods of analysis and data collection are being used to

study the ACF in different contexts. Most ACF applications have historically been in the

environment and energy policy domain, which constituted a majority of applications from 1987

to 2006 (Weible et al., 2009). But among more current applications, from 2007 to 2014,

environment and energy represented only a plurality of articles (Pierce et al., 2017A).

Therefore, we compare the methods used to study environment and energy with other policy

domains.

With this expansion to new contexts and policy domains comes a need to better

understand their methodological trends. Scholars have already begun this work; Henry et al.

(2014) and Jang et al. (2016) identify that applications of the ACF outside of North America and

Europe use informal methods depending on documents. Pierce et al. (2017A) discuss methods

in general, but not specifically related to different theories, political systems, or policy domains.

This paper seeks to continue this inquiry by systematically comparing the methods of data

analysis and collection used in ACF applications to North America, Europe, and Asia, Africa and

Australia.

Despite some skepticism and criticism, ACF applications continue to grow. We expect

that a systematic analysis will clarify the state of the ACF literature, address selected criticisms,

and contribute to the framework’s advancement.

6

Methods

This paper uses content analysis of 161 ACF articles published in English language peer-

review journal articles between 2007 – 2014. The unit of analysis is each article. This paper

builds on the data collection efforts associated with previous publications (Pierce et al., 2017A,

and Pierce et al., 2017B)1.

Articles are coded on six characteristics. Applications tend to focus on one of the ACF’s

three theories (advocacy coalitions, policy change, and policy-oriented learning). (1) Articles are

coded for theory of focus, with combinations of theories categorized as using multiple theories.

Articles are also coded for policy domain or topic (2). After environment and energy

applications, the next most frequent policy domain is public health, with only 15 articles. Thus,

articles are coded dichotomously as primarily analyzing the environment and energy (n=70)

versus other policy domains (n=91).2 Articles are also coded for continent of the policy process

in a manner similar to Weible et al. (2009) and Henry et al. (2014). Continent rather than

geographic location of first author is utilized because the former best captures political context.

Regardless, the continent of the policy process is 86% positively associated with the continent

of the first author, suggesting this grouping decision does not greatly vary results.

(3) Continent is coded as Europe, North America, and an “other” category including Asia, Africa

and Australia. There are no articles applied to South America in the dataset.

1 A list of the 161 articles and details of past data collection efforts available in the appendix. 2 While many articles compare across policy domains, only one article (Fischer, 2014) compares an environment and energy policy with other policy domains. In this case, it was coded as “other” policy domain because the vast majority of policy domains analyzed (10) are not environment and energy.

7

To evaluate methods of data analysis and collection, three additional codes are used for

analysis method, data collection, and whether or not number of observations (n) are reported.

Analysis method indicates if articles use quantitative and/or qualitative methods (4). In other

words, the 23 articles that use quantitative and qualitative analysis are coded as belonging to

both categories. Quantitative analysis indicates the use of inferential statistics (i.e. probability),

while qualitative analysis refers to the explicit or implicit use of logic, including interpretive

approaches (Goertz & Mahoney, 2012). Qualitative analysis can include numbers such as

descriptive statistics, but does not include inferential statistics. For example, Crow (2008) and

Fidelman et al. (2014) both use descriptive statistics and logic but not inferential statistics, so

they are coded as qualitative articles. Similar to Goertz and Mahoney (2012), we do not argue

that one form of analysis is superior to the other, but rather that both are appropriate for

different research tasks and goals.

Data collection methods (5) are also coded, with four possible options: documents,

interviews, surveys, and/or “other” forms. Data collection methods are only coded if they are

explicitly identified in the text of the article, footnotes, or endnotes. Documents refer to

primary sources such as government reports, media sources, etc. that are explicitly referenced

in the article. Articles do not need to identify interview subjects to meet this collection method.

Surveys also include questionnaires. “Other” forms of data collection include participant

observation such as attending public meetings or hearings and focus groups. Last, coders

indicated whether or not articles reported the total number of observations (6). In cases where

the number of observations is reported for one form of data collection and not another, the

article is coded as reporting the number of observations.

8

A discussion of how the population was identified, the sampling scheme, and inter-rater

reliability is in the appendix. At least 50% of the articles are randomly selected for inter-rater

reliability and the results are over 80% percentage agreement with a Cohen’s Kappa score of

0.4, indicating moderate agreement (Landis and Koch, 1977).

Exemplary articles are identified to demonstrate a range of clear and transparent

methods for operationalizing the main dependent variables associated with each ACF theory:

coalition structure, policy change, and policy-oriented learning. This list is not exhaustive or

systematic; its purpose is to model a range of high-quality methodological practices.

Results

The results are organized in three sections. The first section reports the frequency that

all six categories of inquiry (theory used, policy domain, continent of application, quantitative

vs. qualitative, data collection method, and reported observations) occur among the 161

articles. The second section organizes the results of the methods of analysis and data collection

into three categories of inquiry based on the research questions and criticisms: theory,

continent analyzed, and policy domain. The third section discusses exemplary methods for

operationalizing and analyzing coalition structure, policy change and policy-oriented learning.

This section is followed by a discussion of how these results address the present study’s

research questions as well as past criticisms and reviews of the ACF.

9

Overall Results

Table 1. Frequency of All Categories of Inquiry including Theory, Continent Analyzed, Policy Domain, Data Analysis, Data Collection, and Number of Observations Reported

Categories Frequency

Theory

Multiple 42% (67)

Only Coalitions 41% (66)

Only Policy Change 9% (14)

Only Learning 9% (14)

Continent Analyzed3

Europe 46% (71)

North America 36% (56)

Other 18% (28)

Policy Domain

Other 57% (91)

Environment/Energy 43% (70)

Data Analysis4

Qualitative 91% (147)

Quantitative 23% (37)

Data Collection5

Interviews 67% (108)

Document Analysis 60% (96)

Survey 18% (29)

Other 11% (17)

Number Observations Reported

69% (111)

Table 1 shows the frequency that all categories occur. Several trends can be observed.

Articles tend to either analyze multiple ACF theories at once (42%) or focus on coalitions (41%).

In contrast, few articles focus only on policy change (9%) or learning (9%). Environment and

3 This does not include six articles from the original 161 that are comparative across geographical categories (n=155). 4 This total is greater than 161 articles and 100% as the 23 articles that include qualitative and quantitative analysis are included in both categories. 5 This total is also greater than 161 and 100% because most articles include multiple forms of data collection.

10

energy (43%) is not studied by a majority of articles. The most frequent geographic location

studied is Europe (46%), followed by North America (36%) and Other (18%). Twenty articles

compare multiple countries, but most do so within a single continent. Six articles compare

across continents, such as the US with European countries (Montpetit, 2009), as well as

European countries with those in Africa and Asia (Huntjens et al., 2011). To limit confusion

about categorization, these six articles are not analyzed as part of the geographic application

section of the results. They are included in the subsequent analysis of theories and policy

domains.

Over 90% of the articles use some form of qualitative analysis, but less than 25% use

quantitative analysis. Twenty-three articles use both quantitative and qualitative analysis. In

total, 14 articles are quantitative-only and 124 articles are qualitative-only. Both interviews and

documents are used in at least 60% of articles. In contrast, surveys and other forms of data

collection (i.e. participant observation, focus groups, etc.) are used in less than 20% of articles.

Finally, 68% of all articles report the number of observations made, and 32% do not.

Table 2. Frequency by Data Analysis of Collection and Number of Observations Reported

Qualitative

(n=147) Only Qualitative

(n=124) Quantitative

(n=37) Only Quantitative

(n=13)

Collection

Interviews 72% (106) 71% (88) 54% (20) 8% (1)

Documents 63% (93) 66% (82) 38% (14) 23% (3)

Surveys 12% (17) 4% (5) 65% (24) 85% (11)

Other 12% (17) 13% (16) 3% (1) 0% (0)

Number Observations Reported 65% (96) 60% (74) 100% (37) 100% (13)

The associations between data analysis and collection methods are presented in Table

2. Articles that use qualitative analysis (n=147) depend mostly on interviews (72%) and

11

documents (63%). The use of surveys (12%) and other forms of data collection (12%) is

infrequent. Thirty-five percent of articles that include qualitative analysis do not report the

number of observations. Articles that use only qualitative analysis (n=124) have similar

patterns. Interviews (71%) and documents (66%) are the preferred forms of data collection.

Other forms of data collection occur infrequently (12%), and surveys are rarely used (4%). Also,

the number of articles that do not report the number of observations slightly increases from

35% for all qualitative articles to 40% for articles that only use qualitative analysis.

Articles that use quantitative analysis (n=37) depend mostly on surveys (65%) and

interviews (54%). Document analysis is also a common form of data collection (38%), but not

other forms of data collection (3%). In addition, in all cases of quantitative analysis the number

of observations are reported. Among the 13 articles that only use quantitative analysis, 85% use

surveys, 23% documents, and only 8% interviews. None use other forms of data collection or

fail to report the number of observations.

In comparing qualitative and quantitative analysis, there are a couple of clear

differences. Articles that use quantitative analysis (65%) are almost six times more likely to use

a survey than articles that use qualitative analysis (12%). In contrast, articles that use

qualitative analysis collect data using documents (+25%), interviews (+18%), and other forms of

data collection (+9%) more than articles that use quantitative analysis. Reporting of

observations always occurs among quantitative articles while only 65% of articles using

qualitative analysis do so. These trends comparing methods of data collection and reporting

observations between articles that use qualitative and quantitative analysis are even starker

when articles that use both forms of analysis are removed.

12

Description of Methods by Theory, Continent Analyzed, and Policy Domain

Table 3. Frequency by Theory of Methods of Data Analysis, Collection, and Number of Observations Reported

Multiple (n=67)

Only Coalitions (n=66)

Only Policy Change (n=14)

Only Learning (n=14)

Analysis

Qualitative 99% (66) 86% (57) 100% (14) 71% (10)

Quantitative 16% (11) 30% (20) 0% (0) 43% (6)

Collection

Interviews 73% (49) 64% (42) 57% (8) 64% (9)

Documents 73% (49) 44% (29) 71% (10) 57% (8)

Surveys 12% (8) 24% (16) 0% (0) 36% (5)

Other 13% (9) 8% (5) 7% (1) 14% (2)

Number Observations Reported 69% (46) 70% (46) 57% (8) 79% (11)

There are multiple associations between methods based on theory analyzed, in

particular when theories are analyzed in isolation. A majority of articles use qualitative analysis,

but this varies from a low of 71% among articles analyzing only learning to 99% for multiple

theories and 100% for only policy change. In contrast, quantitative analysis is used in almost

half of only learning articles (43%), while it is never used in only policy change articles. In

articles focusing on only coalitions, quantitative analysis is used in 30%, almost double the rate

it is used in multiple theory articles (16%). Among the multiple theory articles, there are eight

articles that include policy change that use quantitative analysis. All eight of these articles

quantitatively analyze policy actor beliefs focusing on coalitions rather than explicitly policy

change. Therefore, there are no articles that quantitively analyze policy change. Qualitative

analysis is the predominant form of analysis across all three theories, while quantitative

analysis tends to be only used to study learning and coalitions.

13

Most articles use interviews in frequencies ranging from a low of 57% in only policy

change articles to a high of 73% in multiple theory articles. Documents are used by a majority of

the articles that study multiple theories, only policy change, and only learning, while they are

used in 44% of only coalition articles. Surveys are utilized with great variation. While 36% of

only learning articles and 24% of only coalition articles use surveys, 12% of multiple theory

articles and none of the only policy change articles do so. Other forms of data collection are

infrequent, but tend to be used to study only learning (14%) and multiple theories (13%).

Overall, the form of analysis and data collection differ depending on the theory analyzed.

Articles that focus only on policy change (57%) are the least likely to report the number

of observations made. In comparison, articles that study only coalitions (70%), only learning

(79%), and multiple theories (69%), include the number of observations more frequently.

Regardless of theory analyzed, about 20 – 40% of articles do not report the number of

observations.

Table 4. Frequency by Continent Analyzed of Methods of Data Analysis, Collection, and Number of Observations Reported

Europe (n=71) North America (n=56) Other (n=28)

Analysis

Qualitative 96% (68) 88% (49) 96% (27)

Quantitative 18% (13) 29% (16) 14% (4)

Collection

Interviews 70% (50) 68% (38) 68% (19)

Documents 66% (47) 60% (34) 54% (15)

Surveys 8% (6) 30% (17) 11% (3)

Other 4% (3) 11% (6) 29% (8)

Number Observations Reported

65% (46) 80% (45) 57% (16)

14

A plurality of articles analyze policy processes in the continent of Europe (46%),

including comparative studies among members of the European Union. The next most frequent

continent is North America (36%), where studies are limited to the US and Canada. The other

category, which includes Asia, Africa, and Australia, amounts for 18% of the total. Six articles

were not included in the analysis because they compare across continents. Examples include

Montpetit (2009, 2011), who compares policy processes in North America to Europe, and

Huntjens et al. (2011), who compares policy processes in Europe, Asia, and Africa.

Qualitative analysis dominates all three continental categories. Articles analyzing Europe

(96%) and other continents (96%) use qualitative analysis only slightly more frequently

compared to North America (88%) articles. As was true for theory, a minority of articles among

all three continental categories use quantitative analysis but there is variation. Articles studying

North America (29%) use quantitative analysis two times more likely compared to Europe (18%)

and other continents (14%).

A majority of articles analyzing Europe, North America, and other continents use

interviews and documents to collect data. Interviews are used at about the same rate across all

three geographies. Documents are used at varied rates, most frequently in articles analyzing

Europe (66%), followed by North America (60%) and other continents (54%). While a minority

of articles across all three geographic locations use surveys and other forms of data collection,

they are used differentially. North America (30%) articles are three times more likely to use

surveys compared to those studying Europe (8%) or other continents (11%). Articles studying

other continents (29%) are about three times more likely to use other forms of data collection

compared to those studying North America (11%) or Europe (4%).

15

Articles studying all continents are likely to report the number of observations made,

but reporting occurs at different rates depending on geographic location. Articles studying

other continents (57%) are the least likely to report the number of observations, followed by

Europe (65%). The reporting of the number of observations occurs most frequently among

articles analyzing policy processes in North America (80%).

Table 5. Frequency by Policy Domain of Methods of Data Analysis, Collection, and Number of Observations Reported

Other (n=91) Environment/Energy (n=70)

Analysis

Qualitative 90% (82) 93% (65)

Quantitative 15% (14) 33% (23)

Collection

Interviews 66% (60) 69% (48)

Documents 59% (54) 60% (42)

Surveys 13% (12) 24% (17)

Other 13% (12) 7% (5)

Number Observations Reported

67% (61) 71% (50)

Table 5 presents the results of data analysis, collection and number of observations

reported categorized by policy domain. Other policy domain articles include public health

(n=15), education (n=14), social welfare (n=12), science and technology (n=12), and many other

domains. Qualitative analysis is utilized in almost all articles, occurring at about the same

frequency among other policy domain articles (90%) as environment and energy articles (93%).

Quantitative analysis is used in a minority of articles, and is used in environment and energy

articles (33%) at more than double the frequency of other policy domain articles (15%).

Interviews and documents are used by both policy domains a majority of the time and

at about the same rate. Meanwhile, surveys and other forms of data collection are used in a

16

minority of articles in both domains, but occur at differential rates. Articles studying

environment and energy (24%) are about twice as likely to use surveys to collect data compared

to other policy domains (13%). In contrast, articles studying other policy domains (13%) are

about twice as likely to use other forms of data collection compared to environment and energy

(7%). About one in three articles in both domains fail to report the number of observations

(33% of other domains and 29% of environment/energy domains).

Exemplary Applications

Jenkins-Smith et al. (2014) call for the development of best practices methodologically

for analyzing coalition structure, policy change, and learning. The articles identified and

discussed below are not necessarily best practices methodologically, but provide a range of

clear and transparent examples for how to operationalize and analyze the main dependent

variables associated with each ACF theory: coalition structure, policy change or policy-oriented

learning. The common adoption of these methods of operationalizing key dependent variables

by scholars will increase transparency, replicability, and comparability which are a first step

towards methodological best practices.

The study of advocacy coalition membership and structure can be considered the most

established part of the ACF, as suggested by the availability of detailed methodological

appendix instructing how to conduct content analysis of policy elite beliefs by Jenkins-Smith

and Sabatier (1993B), as well as the prevalence of articles focused on the ACF theory of

advocacy coalitions, identified in this research as well as by Weible et al. (2009). There are

several exemplary articles that operationalize and analyze coalition structure that represent a

range of policy domains and in both North America and Europe. Many researchers follow

17

Jenkins-Smith and Sabatier’s (1993B) prescription of using documents to gather data about

policy beliefs. Among these applications, the documents used to collect data include

government documents (Nohrstedt, 2008; Pierce, 2011), media sources (Leifeld, 2013), or

documents from the coalitions (Heikkila et al., 2014). Other applications use surveys with a

wide range of respondents, ranging from about 50 by Ansell et al. (2009) to hundreds of

respondents by Henry (2011). Others use interviews, such as Ingold (2011) and Ingold and

Fischer (2014). The content analysis as well as survey and interview questions from these

articles are examples of how to operationalize policy beliefs that are essential for studying

coalition membership and structure.

Most articles that offer an exemplary analysis of advocacy coalitions are focused on

coalition membership and structure, and typically use social network analysis (SNA) or cluster

analysis. Both depend on quantitative analysis, but data gathering can be done also using

qualitative approaches of content analysis and interviews. SNA can be used to identify

coalitions by mapping coordination (Ingold, 2011, DeBray et al., 2014), beliefs (Leifeld, 2013), or

both (Fischer, 2014; Ansell et al., 2009; Ingold and Gschwend, 2014). Another approach is

cluster analysis, which focuses on analyzing the beliefs of policy actors and clustering them into

coalitions based on shared beliefs. Exemplary articles include Weible (2007), Nohrstedt (2008),

and Weible and Sabatier (2009).

While exemplary articles that study advocacy coalitions often make use of quantitative

forms of analysis, exemplary articles studying policy change are qualitative. Two main

challenges of studying policy change include operationalization of policy change and

contextualization (see Howlett and Cashore, 2009). Three exemplary approaches to

18

operationalizing policy change are by Cairney (2007), Fischer (2014), and Nohrstedt (2010).

Cairney (2007) uses policy design to understand the degree of policy change over time by

analyzing eight policy instruments. Fischer (2014) distinguishes between minor and major policy

change by using closed-ended interview questions that ask respondents how much specific

policies have changed over time on a Lickert scale of 1 to 5. Nohrstedt (2010) identifies both

successful and failed policy changes by chronicling the outcomes of public referendum records

and parliamentary bills over a period of about 20 years. These articles are examples of how to

operationalize the degree of policy change as well as policy stasis and change.

Jenkins-Smith et al. (2014) echo Fischer (2003) and Hajer (1995) in their assertion that

ACF studies of policy change should include social and historical contexts. An example of

providing social context is Heikkila et al. (2014). The authors conduct interviews with a wide

range of stakeholders including government officials, members of interest groups, and media

about the policy process, as well as conduct content analysis of documents from competing

coalitions before, during, and after public hearings to best capture the overall social context of

a policy change. Historical context over a decade or more is provided by Lodge and Matus

(2014), using media documents, and by Nohrstedt (2010), using government hearings and

comments. In all of these articles the number of interviews and documents analyzed is

reported.

Jenkins-Smith et al. (2014) argue that exemplary applications of the ACF should identify

how to clearly conceptualize and measure policy-oriented learning. ACF scholars are typically

concerned with two main concepts related to policy-oriented learning: (1) information and (2)

beliefs and/or strategies. Articles tend to analyze the production of information that concerns

19

either the policy subsystem or problems and/or solutions. Information can be studied using

data collected from surveys (Leach et al., 2014), documents (Lodge & Matus, 2014), or a

combination of documents and interviews (Beem, 2012). These sources typically ask policy

actors directly about the production of such information (Leach et al., 2015; Beem, 2012) or

chronicle the existence of and references to such information over time in documents (Lodge &

Matus, 2014; Beem, 2012).

ACF scholars studying beliefs and/or strategies tend to study changes in opinions

towards information that may occur in professional forums (Leach et al., 2014; Lodge & Matus,

2014; Beem, 2012), and changes in beliefs or strategies that occur after acquiring information

(Leach et al., 2014; Lodge & Matus, 2014; Beem, 2012; Montpetit, 2009; Nedergaard, 2009).

Exemplary articles use surveys (Leach et al., 2014; Montpetit, 2009; Nedergaard, 2009),

documents (Lodge & Matus, 2014) or a combination of documents and interviews (Beem,

2012). These are operationalized either directly, through interviews or surveys asking policy

actors if learning and/or belief change has occurred (Leach at et al., 2015; Beem, 2012;

Montpetit, 2009; Nedergaard, 2009), or by analyzing belief changes using content analysis of

documents in association with acquiring information (Lodge & Matus, 2014).

Discussion

This paper explores three research questions around the methodological issues

identified by Jenkins-Smith et al. (2014), as well as past criticisms of the ACF. The first research

question asks what approaches to analysis and data collection are currently being applied by

ACF scholars, and if methods vary by the theory analyzed. The results show that the majority of

articles employ qualitative analysis with data collected using interviews and documents.

20

Different forms of data analysis are associated with different forms of data collection. While the

majority of articles using quantitative analysis collect data using surveys this approach is rarely

used in association with qualitative analysis. In contrast, articles using qualitative analysis are

more highly dependent on documents and interviews as well as other forms of data collection

than articles using quantitative analysis. Finally, the reporting of the number of observations is

a problem occurring in about 35% of articles using qualitative analysis.

We also find there are differences in analytic and data collection methods depending on

the theory analyzed. Articles that focus on only learning are more likely to use quantitative

analysis and less likely to use qualitative analysis than other articles. Articles that focus on only

policy change do not use quantitative analysis. In addition, the eight articles among the multiple

theories that analyze policy change and use quantitative analysis all only analyze coalition

membership. Thus, a future direction for ACF research is to quantitatively operationalize and

analyze policy change. Data collection methods also vary by theory. Compared to articles that

study policy change alone, the study of only coalitions and only learning rely more on

interviews and surveys. Documents are more commonly used among articles that study

multiple theories and policy change, compared to articles that only analyze coalitions. Articles

that only study policy change are the most likely to not report the number of observations.

Overall, methods should vary depending on different research goals (see Goertz & Mahoney,

2012), and this is reflected in the variation of analysis and data collection by theory.

The second research question asks to what degree the ACF is applied to different policy

domains and governing systems, and if these contexts in turn are associated with different

methods of analysis and data collection. While environment and energy (n=70) is the most

21

frequently studied policy domain (the next most frequent domain is public health with 15

articles), it does not represent a majority of articles. This differs from previous research on ACF

articles spanning 1987 to 2006 by Weible et al. (2009). More importantly, it shows that the

methods of analysis and data collection used for articles studying environment and energy are

similar to methods used for other policy domains. A majority of both categories of articles use

qualitative analysis with data collected using interviews and/or documents, and a minority of

articles use quantitative analysis along with collecting data using surveys and/or other

methods. Both fail to report the number of observations at about the same frequency. The

major difference between the two groups are that articles studying environment and energy

are about twice as likely to use quantitative analysis and surveys than those studying other

policy domains.

The ACF faces questions about its applicability outside of the US and pluralist governing

systems (Sabatier, 1998), and more recent questions have arisen about its applicability outside

of North America and Europe (Henry et al., 2014). This research shows that the most frequent

geographic location for applying the ACF is Europe. In addition, only about one third of articles

are applied to North America. While articles outside of Europe and North America are less

frequent, there is a substantial amount. This is supported by a recent review by Henry et al.

(2014), which identifies 27 articles applying the ACF between 1999 and 2013 to countries

outside of North America and Europe. Further, both Henry et al. (2014) and this research may

under-count the number of applications outside of North America and Europe due to language

constraints. For example, one recent review of the ACF identified 62 articles from 2002 to 2014

that were applied to South Korea and written in Korean (Jang et al., 2016). Finally, we identify

22

20 articles that use the ACF to compare policy processes in multiple countries, including six that

compared across continents. We can therefore conclude that the ACF is being applied outside

of the US and pluralistic political systems; it is being applied outside of North America and

Europe; and is being used comparatively across political systems.

The ACF is being applied in a similar manner in Europe, North America, and in other

continents, but there are some key differences in the frequency of methods used. Overall, a

majority of all articles use qualitative analysis, using data collected from interviews and/or

documents, and reporting the number of observations, while a minority use quantitative

analysis, using surveys or other methods. Articles studying North America are more likely to be

quantitative, use surveys for data collection, and report the number of observations as

compared to articles analyzing Europe and other continents.

One study (Henry et al., 2014) finds that articles studying continents other than North

America and Europe tend to rely on informal document analysis, while some studies use

interviews and even fewer use surveys, which seems to indicate a lack of quantitative analysis.

However, our analysis of 26 articles analyzing policy processes outside of North America and

Europe between 2007 and 2014 provide mixed support for Henry et al. (2014). While articles

applied outside of Europe and North America rarely use quantitative methods of analysis and

surveys for data collection, these methods are utilized at about the same frequency as articles

analyzing policy processes in Europe. Also, the use of documents occurs less frequently,

compared to articles analyzing Europe and North America. Articles studying Asia, Africa, and

Australia (the other category) tend to use other forms of data collection rather than surveys,

such as participant observation and focus groups.

23

Finally, Fischer (2003) and Hajer (1995) argue that ACF applications are overly empirical

and lack context, especially those studying policy change. While this may have been the case

prior to 2003, current articles are highly contextual. About 90% of all ACF articles use qualitative

analysis, and a majority collect data using interviews and/or documents. This is similar to other

policy theories as a recent review of 311 multiple streams approach articles from 2000 to 2013

finds that 88% rely on qualitative analysis (Jones et al., 2014). Fischer’s criticism is therefore not

supported.

A previous review of the ACF and its methods by Weible et al. (2009) has similar overall

findings with this paper, despite somewhat different classification systems. While Weible et al.

(2009) do not specify the analytic method (qualitative or quantitative), or frequencies of data

collection by theory, policy domain, or continent of application, they produce comparable

findings about data collection methods and reporting of observations. For example, Weible et

al. (2009) find that 40% of articles use interviews, 17% use surveys, and 3% other forms of

observation for data collection. In comparison, this research finds that 67% use interviews, 18%

use surveys, and 11% use other forms of data collection including participant observation. Thus,

recent articles are more likely to use interviews and other forms of data collection, while

surveys continue to be used at the same frequency. Weible et al. (2009) specify qualitative

content analysis, which occurs in 20% of articles, rather than the broader source of data

collection of documents that is used in this research. However, Weible et al. (2009) do report

that an additional 41% of articles use existing documents and reports in an underspecified

manner. Therefore, we can estimate that about 61% of ACF articles analyzed by Weible et al.

(2009) use documents, which is consistent with the 60% found here.

24

Weible et al. (2009) also find that among 80 articles from 1987 to 2006, 41% “used

methods that were underspecified and appeared to rely on unsystematic collection and analysis

of existing documents and reports” (p. 125). Similarly, this research finds that a substantial

minority (32%) do not report the number of observations made. However, our finding is likely

an underestimate, because unlike Weible et al. (2009), it only reports if the number of

observations are not reported.

Conclusion

The ACF has changed dramatically over the past couple of decades. It has transformed

from a framework that was criticized in the 1990s and early 2000s for being US-centric, mostly

applied to the environment and energy, and too dependent on empirical and quantitative

analysis; to one that is mostly applied to policy processes in Europe, to policy domains other

than environment and energy, and is mostly qualitative. It is being applied to political systems

around the world and in comparative contexts. Depending on the theory analyzed, researchers

differ in their purpose, leading to some differences in data analysis and collection. This is a

positive sign for the ACF as different research tasks and goals should have different forms of

data collection and analysis (Goertz & Mahoney, 2012).

This research has multiple limitations. It only includes English language articles,

therefore articles in other languages such as Spanish, Korean, etc. are not included in the

analysis. This research only compares policy domain in a binary fashion, rather than deeper

analysis of other topics such as education and social welfare. Similarly, it categorizes analysis by

continent applied for ease of comparison rather than by country, political system, or first

author. In addition, only a couple of exemplary articles are identified in relation to each theory.

25

There are many more articles that could be identified, in particular those articles published

prior to 2007. These are all limitations of this paper, but overall the paper achieves its purposes

of describing the current states of methods used to study the ACF by various categories as well

as addressing past criticisms.

If the observed trend continues, the ACF will probably continue to grow more diverse in

terms of policy domains and political systems applied. Scholars will continue to study the ACF to

identify and analyze coalitions in the policy process, and should further expand their research

on policy change and policy-oriented learning, as also argued by Jenkins-Smith et al. (2014).

Methodologically, the future of the ACF is uncertain. Similar to the first two decades of articles

(Weible et al., 2009) this past decade has seen a consistently large minority of articles lacking

transparency. Despite that observation, this paper identifies multiple articles within each

theory that use various forms of data collection and analysis to operationalize and measure the

central variables of concern for ACF scholars: coalition membership and structure, policy

change, and policy-oriented learning. In order for the ACF to balance theoretical generalization

with unique contexts, scholars should continue to use diverse methods of analysis and data

collection, but should seek common conceptual operationalization.

26

References

Andersson, Magnus. 1998. Change and Continuity in Poland’s Environmental Policy. Dordrecht,

Netherlands: Kluwer Academic Publishers.

Ansell, Chris, Sarah Reckhow, and Andrew Kelly, A. 2009. “How to Reform a Reform Coalition:

Outreach, Agenda Expansion, and Brokerage in Urban School Reform.” Policy Studies

Journal 37(4): 717-743.

Beem, B. (2012). Learning the wrong lessons? Science and fisheries management in the

Chesapeake Bay blue crab fishery. Public Understanding of Science, 21(4), 401-417.

Cairney, Paul. 2007. “A ‘Multiple Lenses’ Approach to Policy Change: The Case of Tobacco Policy

in the UK.” British Politics 2 (1): 45-68.

Crow, Deserai. A. 2008. “Stakeholder Behavior and Legislative Influence: A Case Study of

Recreational Water Rights in Colorado.” Social Science Journal 45 (4): 646-658.

DeBray, Elizabeth, Janelle T. Scott, Christopher A. Lubienski, and Huriya Jabbar. 2014.

“Intermediary Organizations in Charter School Policy Coalitions Evidence from New

Orleans.” Educational Policy 28 (2): 175-206.

Fidelman, Pedro, Louisa S. Evans, Simon Foale, Christopher Weible, Franciska Von Heland, and

Dallas Elgin.2014. “Coalition Cohesion for Regional Marine Governance: A Stakeholder

Analysis of the Coral Triangle Initiative.” Ocean & Coastal Management 95: 117-128.

Fischer, Frank. 2003. Reframing Public Policy: Discursive Politics and Deliberative Practices. New

York: Oxford University Press.

Fischer, Manuel. 2014. “Coalition Structures and Policy Change in a Consensus Democracy.”

Policy Studies Journal 42(3): 344-366.

Goertz, Gary, and James Mahoney. 2012. A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences. Princeton: Princeton University Press.

Howlett, Michael, and Benjamin Cashore. 2009. “The Dependent Variable Problem in the Study of Policy Change: Understanding Policy Change as a Methodological Problem.” Journal of Comparative Policy Analysis: Research and Practice 11 (1): 33–46.

Hajer, Maarten A. 1995. Interpreting policy change: Discourse coalitions versus advocacy

coalitions. Paper presented at the Annual Meeting of the American Political Science Association, Chicago, IL (USA), August 31.

27

Heikkila, Tanya, Jonathan J. Pierce, Samuel Gallaher, Jennifer Kagan, Deserai A. Crow, and

Christopher M. Weible. 2014. “Understanding a Period of Policy Change: The Case of

Hydraulic Fracturing Disclosure Policy in Colorado.” Review of Policy Research 31 (2): 65-

87.

Henry, Adam Douglas. 2011. “Ideology, Power, and the Structure of Policy Networks.” Policy

Studies Journal 39(3): 361-383.

Henry, Adam D., Karin Ingold, Daniel Nohrstedt, and Christopher M. Weible. 2014. Policy

change in comparative contexts: Applying the advocacy coalition framework outside of

Western Europe and North America. Journal of Comparative Policy Analysis, 16 (4), 299-

312.

Huntjens, Patrick, Claudia Pahl‐Wostl, Benoit Rihoux, Maja Schlüter, Zsuzsanna Flachner,

Susana Neto, Romana Koskova, Chris Dickens, and Isah Nabide Kiti. 2011. “Adaptive

Water Management and Policy Learning in a Changing Climate: A Formal Comparative

Analysis of Eight Water Management Regimes in Europe, Africa and Asia.”

Environmental Policy and Governance 21 (3): 145-163.

Ingold, Karin. 2011. “Network Structures within Policy Processes: Coalitions, Power, and

Brokerage in Swiss Climate Policy.” Policy Studies Journal 39 (3): 435-459.

Ingold, Karin, and Manuel Fischer. 2014. “Drivers of Collaboration to Mitigate Climate Change:

An Illustration of Swiss Climate Policy over 15 Years.” Global Environmental Change-

Human and Policy Dimensions 24: 88-98.

Ingold, Karin, and Muriel Gschwend. 2014. “Science in Policy-Making: Neutral Experts or

Strategic Policy-Makers?” West European Politics 37(5): 993-1018.

Jang, Sojin, Christopher M. Weible, and Kyudong Park. 2016. “Policy Processes in South Korea through the Lens of the Advocacy Coalition Framework.” Journal of Asian Public Policy, 9 (3), 274-290.

Jenkins-Smith, Hank C., Daniel Nohrstedt, Christopher M. Weible, and Paul A. Sabatier. 2014.

“The Advocacy Coalition Framework: Foundations, Evolution, and Ongoing Research.” In

Theories of the Policy Process, eds. Paul A. Sabatier and Christopher M. Weible. Boulder,

CO: Westview Press, 183-223.

Jenkins-Smith, Hank C., and Paul A. Sabatier. 1993A. The Dynamics of Policy-Oriented Learning.

In Policy Change and Learning, eds. Paul A. Sabatier and Hank Jenkins-Smith. Boulder,

CO: Westview Press, 41-58.

28

Jenkins-Smith, Hank C., and Paul A. Sabatier. 1993B. Methodological Appendix: Measuring

Longitudinal Change in Elite Beliefs Using Content Analysis of Public Documents. In

Policy Change and Learning, eds. Paul A. Sabatier and Hank Jenkins-Smith. Boulder, CO:

Westview Press, 237-256.

Jones, Michael D., Holly L. Peterson, Jonathan J. Pierce, Nicole Herweg, Amiel Bernal, Holly Lamberta Raney, and Nikolaos Zahariadis. 2016. “A River Runs Through It: A Multiple Streams Meta-Review.” Policy Studies Journal 44 (1): 13–36.

John, Peter. 1998. Analysing Public Policy. London: Pinter.

Landis, J. Richard, and Gary G. Koch. 1977. “The Measurement of Observer Agreement for Categorical Data.” Biometrics 33 (1): 159–74.

Leach, William D., Christopher M. Weible, Scott R. Vince, Saba N. Siddiki, and John C. Calanni.

2014. “Fostering Learning through Collaboration: Knowledge Acquisition and Belief

Change in Marine Aquaculture Partnerships.” Journal of Public Administration Research

and Theory 24 (3): 591-622.

Leifeld, Philip. 2013. “Reconceptualizing Major Policy Change in the Advocacy Coalition

Framework: A Discourse Network Analysis of German Pension Politics.” Policy Studies

Journal 41 (1): 169-198.

Lodge, Martin, and Kira Matus. 2014. “Science, Badgers, Politics: Advocacy Coalitions and Policy

Change in Bovine Tuberculosis Policy in Britain.” Policy Studies Journal 42 (3): 367-390.

Lombard, Matthew, Jennifer Snyder-Duch, and Cheryl Campanella Bracken. 2002. “Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability.” Human Communication Research 28 (4): 587–604.

Montpetit, Éric. 2009. “Governance and Policy Learning in the European Union: A Comparison

with North America.” Journal of European Public Policy 16 (8): 1185–1203.

Montpetit, Éric. 2011. “Scientific Credibility, Disagreement, and Error Costs in 17 Biotechnology

Policy Subsystems.” Policy Studies Journal 39 (3): 513-533.

Nedergaard, Peter. 2009. “Policy Learning Processes in International Committees.” Public

Management Review 11 (1): 23-37.

Nohrstedt, Daniel. 2008. “The Politics of Crisis Policymaking: Chernobyl and Swedish Nuclear

Energy Policy.” Policy Studies Journal 36 (2): 257-278.

Nohrstedt, Daniel. 2010. “Do Advocacy Coalitions Matter? Crisis and Change in Swedish Nuclear

Energy Policy.” Journal of Public Administration Research and Theory 20(2): 309-333.

Parsons, Dwayne W. 1995. Public Policy: An Introduction to the Theory and Practice of Policy Analysis. Aldershot, UK: Edward Elgar.

29

Pierce, Jonathan. J. 2011. “Coalition Stability and Belief Change: Advocacy Coalitions in US

Foreign Policy and the Creation of Israel, 1922-44.” Policy Studies Journal 39 (3), 411-

434.

Pierce, Jonathan J., Holly L. Peterson, Michael D. Jones, Samantha P. Garrard, and Theresa Vu. 2017A. “There and Back Again: A Tale of the Advocacy Coalition Framework.” Policy Studies Journal 45 (S1): S13–46.

Pierce, Jonathan J., Holly L. Peterson, and Katherine C. Hicks. 2017B. Policy Change: An Advocacy Coalition Framework Perspective, Policy Studies Journal, forthcoming.

Pierce, Jonathan J., and Christopher M. Weible. 2016. “Advocacy Coalition Framework.” In

American Governance eds. Stephen Schechter, Thomas S. Vontz, Thomas A. Birkland,

Mark A. Graber, and John J. Patrick. Farmington Hills, MIL Gale, Cengage, Learning, 22-

23.

Riffe, Daniel, Stephen Lacy, and Frederick Fico. 2005. Analyzing Media Messages: Using Quantitative Content Analysis in Research. New York: Routledge.

Sabatier, Paul A. 1986. “Top-down and Bottom-up Approaches to Implementation Research: A

Critical Analysis and Suggested Synthesis.” Journal of Public Policy 6 (1): 21–48. Sabatier, Paul A. 1988. “An Advocacy Coalition Framework of Policy Change and the Role of

Policy-Oriented Learning Therein.” Policy Sciences 21 (2-3): 129–68. Sabatier, Paul A. 1998. “The Advocacy Coalition Framework: Revisions and Relevance for

Europe.” Journal of European Public Policy 5 (1): 98–130. Sabatier, Paul A., and Hank C. Jenkins-Smith (Eds). 1993. Policy Change and Learning: An

Advocacy Coalition Approach. Boulder, CO: Westview Press. Sabatier, Paul A., and Hank Jenkins-Smith. 1993. The Advocacy Coalition Framework:

Assessment, Revisions, and Implications for Scholars and Practitioners. In Policy Change

and Learning, eds. Paul A. Sabatier and Hank Jenkins-Smith. Boulder, CO: Westview

Press, 211-235.

Sabatier, Paul A. and Hank C. Jenkins-Smith. 1999. "The Advocacy Coalition Framework: An

Assessment." In Theories of the Policy Process, ed. Paul A. Sabatier. Boulder, CO:

Westview Press, 117-166.

Sabatier, Paul A., and Christopher M. Weible. 2007. “The Advocacy Coalition Framework:

Innovations and Clarifications.” In Theories of the Policy Process, ed. Paul A. Sabatier.

Boulder, CO: Westview Press, 189–222.

30

Van Gossum, Peter, Liselot Ledene, Bas Arts, Rik De Vreese, and Kris Verheyen. 2008. “Implementation Failure of the Forest Expansion Policy in Flanders (Northern Belgium) and the Policy Learning Potential.” Forest Policy and Economics 10 (7): 515–522.

Weible, Christopher M. 2007. “An Advocacy Coalition Framework Approach to Stakeholder

Analysis: Understanding the Political Context of California Marine Protected Area Policy.” Journal of Public Administration Research & Theory 17 (1): 95–117.

Weible, Christopher M., and Paul A. Sabatier. 2009. “Coalitions, Science, and Belief Change:

Comparing Adversarial and Collaborative Policy Subsystems.” Policy Studies Journal, 37

(2): 195-212.

Weible, Christopher M., Paul A. Sabatier, and Kelly McQueen. 2009. “Themes and Variations: Taking Stock of the Advocacy Coalition Framework.” Policy Studies Journal, 37 (1): 121- 140.

Weible, Christopher W., Paul A. Sabatier, Hank C. Jenkins-Smith, Daniel Nohrstedt, Adam Douglas Henry, and Peter deLeon. 2011. “A Quarter Century of the Advocacy Coalition Framework: An Introduction to the Special Issue.” The Policy Studies Journal, 39 (3): 349-360.

31

Appendix.

Coding Methods

This review uses methods of article selection based on recent reviews of the Multiple

Streams Approach (Jones et al., 2016) and the ACF (Pierce et al. 2017). A list of peer-reviewed

journal articles citing at least one of the six foundational documents developing the ACF

(Sabatier 1986, 1988, 1998; Sabatier and Jenkins-Smith 1993, 1999; Sabatier & Weible 2007)

was generated using the Web of Science database. Due to resource constraints, as well as the

existence of the previous systematic review of the entire ACF from 1987 to 2006 by Weible et

al. (2009), the sampling frame was limited to English peer-reviewed journal articles published

between 2007-2014, producing an initial data set of 1,067 peer-reviewed articles.

Content analysis of these articles proceeded in three rounds. First, five coders recorded

the bibliographic information of each article. This included 10 identification codes such as title,

author, and journal name. Four codes were utilized to determine whether an article was an

application of the ACF. These codes identified keywords (coalition, learning, or advocacy) in the

title, abstract, and keywords sections, and whether or not theoretical foundation citations were

used at least twice. This practice led to the identification of 512 articles. However, relying only

on the frequency of keywords and citations may lead to Type I errors. In addition to keyword

and reference searches, coders examined articles to determine if they include data and/or a

case study about a topic, and identify and analyze at least one theory of the ACF (coalitions,

policy change, and/or learning). To mitigate subjectivity, inter-coder reliability assessments for

this coding were conducted on a random sample of 256 articles being reviewed by an inter-

coder. This process narrowed the pool to 161 articles identified as ACF applications.

32

In the second round of coding, seven coders analyzed the remaining 161 articles to

identify the theory, methods of data analysis and collection, as well as country of application. In

this round 87 articles or 54% were subject to inter-rater reliability. The results of this coding are

in Pierce et al. (2017A). For A third round of coding was conducted again on these 161 articles

by four coders to identify the theory (single or multiple), continent of application, policy

domain, and greater emphasis on methods of data analysis and collection as discussed in the

methods section. This third round of coding was completed for this paper because of the shift

in focus on methods in comparative contexts and purposes in comparison to Pierce et al.

(2017A) which focused more on a general description of how the ACF is being applied.

For each round of coding, at least 50% of all articles were randomly selected for inter-

coder reliability. Round one included 256/512 articles, round two 87/161, and round three

86/161 articles, all of which are appropriate sample sizes for determining inter-rater reliability

(Lombard et al., 2002). Percent agreement was calculated at or above 80% for all coders during

the first round of coding which is sufficient (Lombard et al., 2002; Riffe et al., 2005). The second

and third rounds of coding achieved both at or above 80% agreement and a Cohen’s kappa

score of 0.4, indicating moderate agreement (Landis and Koch 1977). Cohen’s kappa is used to

mitigate the effect of chance agreement as all of the codes are binary indicating presence.

33

List of All Articles (n=161)

Adams, Neil, Giancarlo Cotella, and Richard Nunes. 2014. “The Engagement of Territorial

Knowledge Communities with European Spatial Planning and the Territorial Cohesion Debate: A

Baltic Perspective.” European Planning Studies 22 (4): 712-734.

Adshead, Maura. 2011. “An Advocacy Coalition Framework Approach to the Rise and Fall of

Social Partnership.” Irish Political Studies 26 (1): 73-93.

Afonso, Afonso. 2014. “How to Please Voters without Alienating Friends? Parties, Organised

Interests and Advocacy Coalitions in Swiss Immigration Policy.” Comparative European Politics

12 (6): 568-583.

Airey, David, and King Chong. 2010. “National Policy-Makers for Tourism in China.” Annals of

Tourism Research 37 (2): 295-314.

Albright, Elizabeth A. 2011. “Policy Change and Learning in Response to Extreme Flood Events in Hungary: An Advocacy Coalition Approach.” Policy Studies Journal, 39(3): 485-511. Amougou, Jules, and James S. Larson. 2008. “Comparing Implementation of Internet Diffusion

in the United States and France: Policies, Beliefs, and Institutions.” Review of Policy Research

25(6): 563-578.

Ansell, Chris, Sarah Reckhow, and Andrew Kelly, A. 2009. “How to Reform a Reform Coalition:

Outreach, Agenda Expansion, and Brokerage in Urban School Reform.” Policy Studies Journal

37(4): 717-743.

Babon, Andrea, Daniel McIntyre, Gae Y. Gowae, Caleb Gallemore, Rachel Carmenta, Monica Di

Gregorio, and Maria Brockhaus. 2014. “Advocacy Coalitions, REDD+, and Forest Governance in

Papua New Guinea: How likely is Transformational Change?” Ecology and Society 19 (3): 13.

Bandelow Nils C. and Stefan Kundolf. 2011. "Belief Systems and the Emergence of Advocacy

Coalitions in Nascent Subsystems: A Case Study of the European GNSS Program Galileo.”

German Policy Studies 7 (2): 113-139.

Battams, Samantha, & Fran Baum. 2010. “What Policies and Policy Processes Are Needed to

Ensure that People with Psychiatric Disabilities have Access to Appropriate Housing?” Social

Science & Medicine 70 (7): 1026-1034.

Baumann, Christiane, and Stuart White. 2014. “Collaborative Stakeholder Dialogue: A Catalyst

for Better Transport Policy Choices.” International Journal of Sustainable Transportation 9 (1):

30-38.

Beard, Virginia. 2013. "A Theoretical Understanding of Housing and Homelessness: Federal

Homelessness and Housing Policy through the lenses of Punctuated Equilibrium Theory and

Advocacy Coalition Framework." Poverty and Public Policy 5 (1): 67-87.

34

Beem, Betsi. 2012. “Learning the Wrong Lessons? Science and Fisheries Management in the

Chesapeake Bay Blue Crab Fishery.” Public Understanding of Science 21 (4): 401-417.

Beverwijk, Jasmine, Leo Goedegebuure, and Jeroen Huisman, J. 2008. “Policy Change in Nascent

Subsystems: Mozambican Higher Education Policy 1993-2003.” Policy Sciences 41 (4): 357-377.

Blatter, Joachim. 2009. “Performing Symbolic Politics and International Environmental Regulation: Tracing and Theorizing a Causal Mechanism beyond Regime Theory.” Global Environmental Politics 9 (4): 81-110. Brecher, Charles. Caitlyn Brazill, Beth C. Weitzman, and Diana Silver. 2010. “Understanding the

Political Context of "New" Policy Issues: The Use of the Advocacy Coalition Framework in the

Case of Expanded After-School Programs.” Journal of Public Administration Research and

Theory 20 (2): 335-355.

Breton, Eric, Lucie Richard, France Gagnon, Marie Jacques, and Pierre Bergeron. 2008. “Health

Promotion Research and Practice Require Sound Policy Analysis Models: The case of Quebec's

Tobacco Act.” Social Science & Medicine 67 (11): 1679-1689.

Bromfield, Nicole Footen. 2012. "Underlying Motives, Moral Agendas and Unlikely Partnerships:

The Formulation of the U.S. Trafficking in Victims Protection Act through the Data and Voices of

Key Policy Players". Advances in Social Work 13 (2): 243-261.

Brusis, Martin. 2010. “European Union Incentives and Regional Interest Representation in

Central and East European Countries.” Acta Politica, 45(1-2), 70-89.

Bukowski, Jeanie. 2007. “Spanish Water Policy and the National Hydrological Plan: An Advocacy

Coalition Approach to Policy Change.” South European Society and Politics 12(1): 39-57.

Bukowski, J. (2007). Spanish water policy and the national hydrological plan: An advocacy

coalition approach to policy change. South European Society and Politics, 12(1), 39-57.

Buller, Jim, and Nicole Lindstrom. 2013. “Hedging its Bets: The UK and the Politics of European

Financial Services Regulation.” New Political Economy 18 (3): 391-409.

Cairney, Paul. 2007. “A ‘Multiple Lenses’ Approach to Policy Change: The Case of Tobacco Policy in the UK.” British Politics 2 (1): 45-68. Caveen, Alex. J., Tim S. Gray, Selina M. Stead, and Nicolas V.C. Polunin. 2013. “MPA Policy:

What Lies Behind the Science?” Marine Policy 37: 3-10.

Cent, Joanna, Cordula Mertens, and Krzysztof Niedzialkowski. 2013. “Roles and Impacts of Non-

Governmental Organizations in Natura 2000 Implementation in Hungary and Poland.”

Environmental Conservation 40(2): 119-128.

35

Cheng, Abtony S., Cecilia Danks, and Shoma R. 2011. The Role of Social and Policy Learning in

Changing Forest Governance: An Examination of Community-Based Forestry Initiatives in the

US. Forest Policy and Economics 13 (2): 89-96.

Cherlet, Jan, and Jean-Philippe Venot. 2013. Structure and Agency: Understanding Water Policy

Changes in West Africa. Water Policy 15 (3): 479-495.

Cibulka, James. G., and Nathan Myers. 2008. “Fearful Reformers - The Institutionalization of the

Christian Right in American Politics. Educational Policy 22 (1): 155-180.

Crow, Deserai. A. 2008. “Stakeholder Behavior and Legislative Influence: A Case Study of

Recreational Water Rights in Colorado.” Social Science Journal 45 (4): 646-658.

DeBray, Elizabeth, Janelle T. Scott, Christopher A. Lubienski, and Huriya Jabbar. 2014.

“Intermediary Organizations in Charter School Policy Coalitions Evidence from New Orleans.”

Educational Policy 28 (2): 175-206.

DeBray-Pelot, Elizabeth. Christopher A., Lubienski, and Janelle T. Scott. 2007. The Institutional

Landscape of Interest Group Politics and School Choice. Peabody Journal of Education 82 (2):

405-428.

Dela Santa, Edieser. 2013. “The Politics of Implementing Philippine Tourism Policy: A Policy

Network and Advocacy Coalition Framework Approach.” Asia Pacific Journal of Tourism

Research 18 (8): 913-933.

Diaz-Kope, Luisa M., John R. Lombard, Katrina Miller-Stevens. 2013. "A Shift in Federal Policy

Regulation of the Automobile Industry: Policy Brokers and the ACF." Politics & Policy 41 (4):

563-587.

Dougherty, Kevin J., Rebecca S. Natow, Rachel Hare Bork, Sosanya M. Jones, and Blanca E. Vega.

2013. “Accounting for Higher Education Accountability: Political Origins of State Performance

Funding for Higher Education.” Teachers College Record 115 (1): 50.

Dougherty, Kevin J., H. Kenny Nienhusser, and Blanca E. Vega. 2010. “Undocumented

Immigrants and State Higher Education Policy: The Politics of In-State Tuition Eligibility in Texas

and Arizona.” Review of Higher Education 34 (1): 123-173.

Dressel, Björn. 2012. “Targeting the Public Purse: Advocacy Coalitions and Public Finance in the

Philippines.” Administration & Society 44 (6): 65S-84S.

Dudley, Geoffrey. 2007. “Individuals and the Dynamics of Policy Learning: The Case of the Third

Battle of Newbury.” Public Administration 85 (2): 405-428.

Dziengel, Lake. 2010. "Advocacy Coalitions and Punctuated Equilibrium in the Same-Sex

Marriage Debate: Learning from Pro-LGBT Policy Changes in Minneapolis and Minnesota.”

Journal of Gay & Lesbian Social Services 22 (1-2): 165 – 182.

36

Elgin, Dallas J., and Christopher M. Weible. 2013. “A Stakeholder Analysis of Colorado Climate

and Energy Issues Using Policy Analytical Capacity and the Advocacy Coalition Framework.”

Review of Policy Research 30 (1): 114-133.

Ellison, Brian A., and Adam J. Newmark. 2010. “Building the Reservoir to Nowhere: The Role of

Agencies in Advocacy Coalitions.” Policy Studies Journal 38 (4): 653-678.

Eriksson, Johan, Mikael Karlsson, M., and Marta Reuter. 2010. “Technocracy, Politicization, and

Noninvolvement: Politics of Expertise in the European Regulation of Chemicals.” Review of

Policy Research 27 (2): 167-185.

Feindt, Peter H. (2010). “Policy-Learning and Environmental Policy Integration in the Common Agricultural Policy 1973-2003.” Public Administration, 88 (2): 296-314.

Fidelman, Pedro, Louisa S. Evans, Simon Foale, Christopher Weible, Franciska Von Heland, and

Dallas Elgin.2014. “Coalition Cohesion for Regional Marine Governance: A Stakeholder Analysis

of the Coral Triangle Initiative.” Ocean & Coastal Management 95: 117-128.

Fischer, Manuel. 2014. “Coalition Structures and Policy Change in a Consensus Democracy.”

Policy Studies Journal 42(3): 344-366.

Fisher, Dana R., Philip Leifeld, and Yoko Iwaki. 2013. “Mapping the Ideological Networks of

American Climate Politics.” Climatic Change 116 (3-4): 523-545.

Fitzpatrick, Patricia, Alberto Fonseca, and Mary Louise McAllister. 2011. “From the Whitehorse

Mining Initiative Towards Sustainable Mining: Lessons Learned.” Journal of Cleaner Production

19(4): 376-384.

Fleury, Marie-Josée, Guy Grenier, Catherine Vallée, Roch Hurtubise, and Paul-André Lévesque.

2014. “The Role of Advocacy Coalitions in a Project Implementation Process: The Example of

the Planning Phase of the at Home/Chez Soi Project Dealing with Homelessness in Montreal.”

Evaluation and Program Planning 45: 42-49.

Frahsa, Annika, Alfred Rütten, Ulrike Roeger, Karim Abu-Omar, and Diana Schow. 2014.

“Enabling the Powerful? Participatory Action Research with Local Policymakers and

Professionals for Physical Activity Promotion with Women in Difficult Life Situations.” Health

Promotion International 29 (1): 171-184.

Francesch-Huidobro, Maria, and Qianqing Mai. 2012. “Climate Advocacy Coalitions in

Guangdong, China.” Administration & Society 44 (6): 43S-64S.

Gasteyer, Stephen P. 2008. “Agricultural Transitions in the Context of Growing Environmental

Pressure over Water.” Agriculture and Human Values 25 (4): 469-486.

37

Gupta, Kuhika. 2014. “A Comparative Policy Analysis of Coalition Strategies: Case Studies of

Nuclear Energy and Forest Management in India.” Journal of Comparative Policy Analysis 16 (4):

356-372.

Haar, Roberta. 2010. "Explaining George W. Bush's Adoption of the Neoconservative Agenda

after 9/11." Politics and Policy 38 (5): 965-990.

Han, Heejin, Brendon Swedlow, and Danny Unger. 2014. “Policy Advocacy Coalitions as Causes

of Policy Change in China? Analyzing Evidence from Contemporary Environmental Politics.”

Journal of Comparative Policy Analysis 16(4): 313-334.

Hansen, Janus. 2013. "The Danish Biofuel Debate: Coupling Scientific and Politico-Economic

Claims" Science as Culture 23 (1): 73-97.

Heikkila, Tanya, Jonathan J. Pierce, Samuel Gallaher, Jennifer Kagan, Deserai A. Crow, and

Christopher M. Weible. 2014. “Understanding a Period of Policy Change: The Case of Hydraulic

Fracturing Disclosure Policy in Colorado.” Review of Policy Research 31 (2): 65-87.

Heinmiller, B. Timothy. 2013 "Advocacy Coalitions and the Alberta Water Act." Canadian

Journal of Political Science 46 (3): 525-547.

Henry, Adam Douglas. 2011. “Ideology, Power, and the Structure of Policy Networks.” Policy

Studies Journal 39(3): 361-383.

Henry, Adam Douglas, Mark Lubell, and Michael McCoy. 2011. “Belief Systems and Social

Capital as Drivers of Policy Network Structure: The Case of California Regional Planning.”

Journal of Public Administration Research and Theory 21 (3): 419-444.

Hersperger, Anna M., Maria-Pia Gennaio Franscini, and Daniel Kübler. 2014. “Actors, Decisions

and Policy Changes in Local Urbanization.” European Planning Studies 22 (6): 1301-1319.

Hirsch, Rachel, Jamie Baxter, and C. Brown. 2010. The Importance of Skillful Community

Leaders: Understanding Municipal Pesticide Policy Change in Calgary and Halifax.” Journal of

Environmental Planning and Management 53 (6): 743 – 757.

Hirschi, Christian, and Thomas Widmer. 2010. “Policy Change and Policy Stasis: Comparing

Swiss Foreign Policy towards South Africa (1968-94) and Iraq (1990-91).” Policy Studies Journal

38 (3): 537-563.

Howarth, Anita. 2013. “The Weakest Link in Existing Studies: Media-Government Risk

Interactions.” Journal of Risk Research 16 (1): 1-18.

38

Huntjens, Patrick, Claudia Pahl‐Wostl, Benoit Rihoux, Maja Schlüter, Zsuzsanna Flachner,

Susana Neto, Romana Koskova, Chris Dickens, and Isah Nabide Kiti. 2011. “Adaptive Water

Management and Policy Learning in a Changing Climate: A Formal Comparative Analysis of Eight

Water Management Regimes in Europe, Africa and Asia.” Environmental Policy and Governance

21 (3): 145-163.

Hysing, Erik, and Jan Olsson. 2008. “Contextualising the Advocacy Coalition Framework:

Theorising Change in Swedish Forest Policy.” Environmental Politics 17 (5): 730-748.

Ingold, Karin. 2011. “Network Structures within Policy Processes: Coalitions, Power, and

Brokerage in Swiss Climate Policy.” Policy Studies Journal 39 (3): 435-459.

Ingold, Karin, and Manuel Fischer. 2014. “Drivers of Collaboration to Mitigate Climate Change:

An Illustration of Swiss Climate Policy over 15 Years.” Global Environmental Change-Human and

Policy Dimensions 24: 88-98.

Ingold, Karin, and Muriel Gschwend. 2014. “Science in Policy-Making: Neutral Experts or

Strategic Policy-Makers?” West European Politics 37(5): 993-1018.

Ingold, Karin, and Frédéric Varone. 2012. “Treating Policy Brokers Seriously: Evidence from the

Climate Policy.” Journal of Public Administration Research and Theory 22 (2): 319-346.

Jang, Jiho, Sunhyuk Kim, and Chonghee Han. 2010. “Advocacy Coalitions in Regulating Big

Business in South Korea: Change of Chaebol's Holding Company Policy.” Korea Observer 41 (2):

161-188.

Jegen, Maya, and Gabriel Audet. 2011. “Advocacy Coalitions and Wind Power Development:

Insights from Quebec.” Energy Policy 39 (11): 7439-7447.

Johnson, Donna B., Elizabeth C. Payne, Molly A. McNeese, and Deborah Allen. 2012. “Menu-

Labeling Policy in King County, Washington.” American Journal of Preventive Medicine 43 (3):

S130-S135.

Karapin, R. 2012. Explaining Success and Failure in Climate Policies Developing Theory through

German Case Studies. Comparative Politics, 45(1), 46-68.

Kettell, Steven, and Paul Cairney. 2010. “Taking the Power of Ideas Seriously - the Case of the

United Kingdom's 2008 Human Fertilisation and Embryology Bill.” Policy Studies 31 (3): 301-317.

Kim, Doo-Rae. 2011. “Do Local Policy Networks Deter the Race to the Bottom in Environmental

Regulation? The Case of South Korea.” Environment and Planning C-Government and Policy 29

(6): 1037-1053.

Kim, Pan Suk. 2012. “Advocacy Coalitions and Policy Change: The Case of South Korea's

Saemangeum Project.” Administration & Society 44 (6): 85S-103S.

39

Kingiri, Ann Njoki. 2011. “Conflicting Advocacy Coalitions in an Evolving Modern Biotechnology

Regulatory Subsystem: Policy Learning and Influencing Kenya's Regulatory Policy Process.”

Science and Public Policy 38(3): 199-211.\

Kingiri, Ann Njoki. 2014. “Comparative Strategic Behavior of Advocacy Coalitions and Policy

Brokers: The Case of Kenya's Biosafety Regulatory Policy.” Journal of Comparative Policy

Analysis 16 (4): 373-395.

Klindt, Mads Peter. 2011. “From Rhetorical Action to Policy Learning: Understanding the

European Commission's Elaboration of the Flexicurity Concept.” Journal of Common Market

Studies 49 (5): 971-994.

Knox-Hayes, Janelle. 2012. "Negotiating Climate Legislation: Policy Path Dependence and

Coalition Stabilization." Regulation & Governance 6 (4): 545-567.

Kuebler, Daniel. 2007. “Understanding the Recent Expansion of Swiss Family Policy: An Idea-

Centred Approach.” Journal of Social Policy 36: 217-237.

Kwon, Huck-ju. 2007. “Advocacy Coalitions and Health Politics in Korea.” Social Policy &

Administration 41(2): 148-161.

Lahat, Lihi. 2011. “How Can Leaders' Perceptions Guide Policy Analysis in an Era of

Governance?” Policy Sciences 44(2): 135-155.

Lansang, Liza G.F. 2011. “NGOs, Coalition Building and the Campaign for a Minerals

Management Policy in the Philippines.” Philippine Political Science Journal 32 (55): 127-166.

Leach, William D., Christopher M. Weible, Scott R. Vince, Saba N. Siddiki, and John C. Calanni.

2014. “Fostering Learning through Collaboration: Knowledge Acquisition and Belief Change in

Marine Aquaculture Partnerships.” Journal of Public Administration Research and Theory 24 (3):

591-622.

Leifeld, Philip. 2013. “Reconceptualizing Major Policy Change in the Advocacy Coalition

Framework: A Discourse Network Analysis of German Pension Politics.” Policy Studies Journal

41 (1): 169-198.

Leifeld, Philip, and Volker Schneider. 2012. “Information Exchange in Policy Networks.”

American Journal of Political Science 56(3), 731-744.

Ley, Aaron J., and Edward Weber. 2014. “Policy Change and Venue Choices: Field Burning in

Idaho and Washington.” Society & Natural Resources 27 (6): 645-655.

Li, W. X. 2012. Advocating Environmental Interests in China. Administration & Society, 44(6),

26S-42S.

40

Lipsky, Rachel S., and Clare M. Ryan. 2011. “Nearshore Restoration in Puget Sound:

Understanding Stakeholder Values and Potential Coalitions.” Coastal Management, 39 (6): 577-

597.

Lodge, Martin, and Kira Matus. 2014. “Science, Badgers, Politics: Advocacy Coalitions and Policy

Change in Bovine Tuberculosis Policy in Britain.” Policy Studies Journal 42 (3): 367-390.

Lubell, Mark. 2007. Familiarity Breeds Trust: Collective Action in a Policy Domain. Journal of

Politics 69 (1): 237-250.

Lugg, C. A., & Robinson, M. N. 2009. Religion, Advocacy Coalitions, and the Politics of US Public

Schooling. Educational Policy, 23(1), 242-266.

Mailand, Mikkel. 2010. “The Common European Flexicurity Principles: How a Fragile Consensus

was Reached.” European Journal of Industrial Relations 16 (3): 241-257.

Mandelkern, Ronen, and Michael Shalev. 2010. “Power and the Ascendance of New Economic

Policy Ideas Lessons from the 1980s Crisis in Israel.” World Politics, 62 (3): 459-495.

Mann, Stefan, and Maria-Pia Gennaio. 2010. “The Central Role of Centralisation in

Environmental Policy Initialisation.” Journal of Environmental Planning and Management 53 (3):

283-295.

Marfo, Emmanuel, and James P. Mckeown. 2013. “Negotiating the Supply of Legal Timber to

the Domestic Market in Ghana: Explaining Policy Change Intent Using the Advocacy Coalition

Framework.” Forest Policy and Economics 32: 23-31.

Marichal, Jose. 2009. “Frame Evolution: A New Approach to Understanding Changes in Diversity

Reforms at Public Universities in the United States.” Social Science Journal 46 (1): 171-191.

Matti, Simon, and Annica Sandström. 2011. “The Rationale Determining Advocacy Coalitions:

Examining Coordination Networks and Corresponding Beliefs.” Policy Studies Journal 39 (3):

385-410.

Matti, Simon, and Annica Sandström. 2013. “The Defining Elements of Advocacy Coalitions:

Continuing the Search for Explanations for Coordination and Coalition Structures.” Review of

Policy Research 30 (2): 240-257.

Mavrot, Céline. 2012. "The Status of Ideas in Controversies on Public Policy. Analyzing Beliefs as

Dependent Variables: A Case Study on Harm Reduction Policies in Switzerland" German Policy

Studies 8 (1): 113-156.

Meijerink, Sander. 2008. “Explaining Continuity and Change in International Policies: Issue

Linkage, Venue Change, and Learning on Policies for the River Scheldt Estuary 1967-2005.”

Environment and Planning A 40 (4): 848-866.

41

Michalowitz, Irina. 2007. “What Determines Influence? Assessing Conditions for Decision-

Making Influence of Interest Groups in the EU. Journal of European Public Policy 14 (1): 132-

151.

Miller, Edward. Alan. 2011. “Repealing Federal Oversight of State Health Policy: Lessons from

the Boren Amendment.” Review of Policy Research 28 (1): 5-23.

Montefrio, Marvin Joseph F. 2014. “State versus Indigenous Peoples' Rights: Comparative

Analysis of Stable System Parameters, Policy Constraints and the Process of Delegitimation.”

Journal of Comparative Policy Analysis 16 (4) 335-355.

Montefrio, Marvin Joseph F., and David A. Sonnenfeld. 2011. “Forests, Fuel, or Food?

Competing Coalitions and Biofuels Policy Making in the Philippines.” Journal of Environment &

Development 20 (1): 27-49.

Montpetit, Éric. 2011. “Scientific Credibility, Disagreement, and Error Costs in 17 Biotechnology

Policy Subsystems.” Policy Studies Journal 39 (3): 513-533.

Montpetit, E. 2012. Does Holding Beliefs with Conviction Prevent Policy Actors from Adopting a

Compromising Attitude? Political Studies, 60(3), 621-642.

Nedergaard, Peter. 2007. “Maximizing Policy Learning in International Committees: An Analysis

of the European Open Method of Coordination (OMC) Committees.” Scandinavian Political

Studies 30 (4): 521-546.

Nedergaard, Peter. 2008. “The Reform of the 2004 Common Agricultural Policy: An Advocacy

Coalition Explanation.” Policy Studies 29 (2): 179-195.

Nedergaard, Peter. 2009. “Policy Learning Processes in International Committees.” Public

Management Review 11 (1): 23-37.

Ness, Erik C. 2010. “The Politics of Determining Merit Aid Eligibility Criteria: An Analysis of the

Policy Process.” Journal of Higher Education 81 (1): 33-60.

Neville, J. 2012. Explaining Local Authority Choices on Public Hospital Provision in the 1930s: A

Public Policy Hypothesis. Medical History, 56(1), 48-71.

Nohrstedt, Daniel. 2008. “The Politics of Crisis Policymaking: Chernobyl and Swedish Nuclear

Energy Policy.” Policy Studies Journal 36 (2): 257-278.

Nohrstedt, Daniel. 2010. “Do Advocacy Coalitions Matter? Crisis and Change in Swedish Nuclear

Energy Policy.” Journal of Public Administration Research and Theory 20(2): 309-333.

Nohrstedt, Daniel. 2011. “Shifting Resources and Venues Producing Policy Change in Contested

Subsystems: A Case Study of Swedish Signals Intelligence Policy.” Policy Studies Journal 39 (3):

461-484.

42

Nohrstedt, Daniel. 2013. “Advocacy Coalitions in Crisis Resolution: Understanding Policy

Dispute in the European Volcanic Ash Cloud Crisis.” Public Administration 91 (4): 964-979.

Olsson, Jan. 2009. “The Power of the Inside Activist: Understanding Policy Change by

Empowering the Advocacy Coalition Framework (ACF).” Planning Theory and Practice 10 (2):

167-187.

Parrish, Richard. 2008. "Access to Major Events on Television under European Law." Journal of

Consumer Policy 31(1): 79-98.

Parrish, Richard. 2011. "Social Dialogue in European Professional Football." European Law

Journal 17(2): 213-229.

Parsell, Cameron, Suzanne Fitzpatrick, and Volker Busch-Geertsema. 2014. “Common Ground

in Australia: An Object Lesson in Evidence Hierarchies and Policy Transfer.” Housing Studies 29

(1): 69-87.

Patel, Kiran Klaus. 2013. “Integration by Interpellation: The European Capitals of Culture and

the Role of Experts in European Union Cultural Policies.” Journal of Common Market Studies 5

1(3): 538-554.

Penning-Rowsell, Edmund C., Sally Priest, and Clare Johnson. 2014. “The Evolution of UK Flood

Insurance: Incremental Change over Six Decades.” International Journal of Water Resources

Development 30 (4): 694-713.

Pierce, Jonathan. J. 2011. “Coalition Stability and Belief Change: Advocacy Coalitions in US

Foreign Policy and the Creation of Israel, 1922-44.” Policy Studies Journal 39 (3), 411-434.

Pollak, Melisa, Sarah Johnson Phillips, and Shalini Vajjhala. 2011. “Carbon Capture and Storage

Policy in the United States: A New Coalition Endeavors to Change Existing Policy.” Global

Environmental Change-Human and Policy Dimensions 21 (2): 313-323.

Poulsen, Camilla Aavang. 2014. “Introducing Out-of-Pocket Payment for General Practice in

Denmark: Feasibility and Support.” Health Policy 117 (1): 64-71.

Princen, Sebastiaan. 2007. “Advocacy Coalitions and the Internationalization of Public Health

Policies.” Journal of Public Policy 27 (1): 13-33.

Quaglia, Lucia. 2010. “Completing the Single Market in Financial Services: The Politics of

Competing Advocacy Coalitions. Journal of European Public Policy 17 (7): 1007-1023.

Quaglia, Lucia. 2012. “The 'Old' and 'New' Politics of Financial Services Regulation in the

European Union.” New Political Economy 17 (4): 515-535.

Rastogi, Archi, Gordon M. Hickey, Ruchi Badola, and Syed Ainul Hussain. 2013. Diverging

Viewpoints on Tiger Conservation: A Q-Method Study and Survey of Conservation Professionals

in India.” Biological Conservation 161: 182-192.

43

Ripberger, Joseph T., Kuhika Gupta, Carol L. Silva, and Hank C. Jenkins‐Smith. 2014. “Cultural

Theory and the Measurement of Deep Core Beliefs Within the Advocacy Coalition Framework.”

Policy Studies Journal 42 (4): 509-527.

Roßegger, Ulf, and Ralf Ramin. 2013. “Explaining the Ending of Sweden's Nuclear Phase-Out

Policy: A New Approach by Referring to the Advocacy Coalition Framework Theory.” Innovation-

the European Journal of Social Science Research 26 (4): 323-343.

Runkle Ken, Sharron LaFollette, Josiah Alamu. 2013. "Public Health Policy Options for Improving

Well-Water Quality in West Point, Liberia." World Medical & Health Policy 5 (4): 304-323.

Sandström, Annica. 2010. “Institutional and Substantial Uncertainty-Explaining the Lack of

Adaptability in Fish Stocking Policy.” Marine Policy 34 (6): 1357-1365.

Sarvašová, Zuzana, Jaroslav Šálka, and Zuzana Dobšinská. 2013. “Mechanism of Cross-sectoral

Coordination between Nature Protection and Forestry in the Natura 2000 Formulation Process

in Slovakia.” Journal of Environmental Management 127: S65-S72.

Schilling, Joseph, and Sheila D. Keyes. 2008. “The Promise of Wisconsin's 1999 Comprehensive Planning Law: Land-Use Policy Reforms to Support Active Living.” Journal Politics, Policy and Law. 33(3): 455-496.

Schröer, Arne. 2014. “Lessons Learned? German Security Policy and the War in Afghanistan.”

German Politics 23 (1-2): 78-102.

Shakespeare, Christine. 2008. “Uncovering Information's Role in the State Higher Education

Policy-Making Process.” Educational Policy 22 (6); 875-899.

Shanahan, Elizabeth A., Mark K. McBeth, Paul L. Hathaway, and Ruth J. Arnell. 2008. Conduit or

Contributor? The Role of Media in Policy Change Theory. Policy Sciences 41(2): 115-138.

Sistrom, Maria Gilson. 2010. "Oregon's Senate Bill 560: Practical Policy Lessons for Nurse

Advocates." Policy Politics Nursing practice 11: 29-35.

Sloboda, Marián, Eszter Szabó-Gilinger, Dick Vigers, and Lucija Šimičić. 2010."Carrying out A

Language Policy Change: Advocacy Coalitions and the Management of the Linguistic

Landscape." Current Issues in Language Planning 11 (2): 95-113.

Smith, Katherine E. 2013. “Understanding the Influence of Evidence in Public Health Policy:

What Can We Learn from the "Tobacco Wars'?” Social Policy & Administration, 47 (4): 382-398.

Smith, Mark. P. (2009). “Finding Common Ground: How Advocacy Coalitions Succeed in

Protecting Environmental Flows.” Journal of the American Water Resources Association 45 (5):

1100-1115.

44

Sotirov, Metodi, and Michael Memmler. 2012. “The Advocacy Coalition Framework in Natural

Resource Policy Studies - Recent Experiences and Further Prospects.” Forest Policy and

Economics 16: 51-64.

Stamelos, George and Aggelos Kavasakalis. 2013. "European Higher Education Area and the

Introduction of a Quality Assurance Program in Greek Universities: Is Policy-Oriented Learning

Present?" CEPS Journal 3(3): 105-124.

Stensdal, Iselin. 2014. “Chinese Climate-Change Policy, 1988-2013: Moving On Up.” Asian

Perspective 38(1): 111-135.

Stich, Bethany, and Chad R. Miller. 2008. “Using the Advocacy Coalition Framework to

Understand Freight Transportation Policy Change.” Public Works Management & Policy 13(1):

62-74.

Svihula, Judie, and Carroll L. Estes. 2007. “Social Security Politics: Ideology and Reform.”

Journals of Gerontology Series B-Psychological Sciences and Social Sciences 62 (2): S79-S89.

Szarka, Joseph. 2010. “Bringing Interests Back In: Using Coalition Theories to Explain European

Wind Power Policies.” Journal of European Public Policy 17 (6): 836-853.

Van den Bulck, Hilde and Karen Donders. 2014. “Of Discourses, Stakeholders and Advocacy

Coalitions in Media Policy: Tracing Negotiations Towards the New Management Contract of

Flemish Public Broadcaster VRT.” European Journal of Communication, 29 (1): 83-99.

Van Gossum, Peter, Liselot Ledene, Bas Arts, Rik De Vreese, and Kris Verheyen. 2008.

“Implementation Failure of the Forest Expansion Policy in Flanders (Northern Belgium) and the

Policy Learning Potential.” Forest Policy and Economics 10 (7-8): 515-522.

Van Overveld, Perry JM, Leon M. Hermans, and Arne RD Verliefde. 2010. “The Use of Technical

Knowledge in European Water Policy-Making.” Environmental Policy and Governance 20(5):

322-335.

Vergari, Sandra. 2007. “The Politics of Charter Schools.” Educational Policy 21(1): 15-39.

Weber, Miriam, Peter PJ Driessen, Ben J. Schueler, and Hens AC Runhaar. 2013. “Variation and

Stability in Dutch Noise Policy: An Analysis of Dominant Advocacy Coalitions”. Journal of

Environmental Planning and Management 56 (7): 953-981.

Weible, Christopher M. 2007. “An Advocacy Coalition Framework Approach to Stakeholder

Analysis: Understanding the Political Context of California Marine Protected Area Policy.”

Journal of Public Administration Research and Theory 17:95-117.

Weible, Christopher. 2008. “Caught in a Maelstrom: Implementing California Marine Protected

Areas.” Coastal Management Journal 36: 350-373.

45

Weible, Christopher M., and Paul A. Sabatier. 2009. “Coalitions, Science, and Belief Change:

Comparing Adversarial and Collaborative Policy Subsystems.” Policy Studies Journal 37 (2): 195-

212.

Weible, Christopher M., Andrew Pattison, and Paul A. Sabatier. 2010. “Harnessing Expert-based

Information for Learning and the Sustainable Management of Complex Socio-ecological

Systems.” Environmental Science & Policy 13 (6): 522-534.

Weible, Christopher M., Saba N. Siddiki, and Jonathan J. Pierce. 2011. “Foes to Friends:

Changing Contexts and Changing Intergroup Perceptions.” Journal of Comparative Policy

Analysis 13 (5): 499-525.

Wellstead, Adam M., and Richard C. Stedman. 2007. “Coordinating Future Adaptation Policies

Across Canadian Natural Resources.” Climate Policy 7 (1): 29-45.

Wilson, Kumanan, Meredith Barakat, SunitaVohra, Paul Ritvo, and Heather Boon. 2008.

“Parental Views on Pediatric Vaccination: The Impact of Competing Advocacy Coalitions.” Public

Understanding of Science.17(2): 231-243.

Winkel, Georg, and Metodi Sotirov. 2011. “An Obituary for National Forest Programmes?

Analyzing and Learning from the Strategic use of ‘New Modes of Governance’ in Germany and

Bulgaria.” Forest Policy and Economics 13(2): 143-154.