Review Essay: Beyond Correlational Analysis: Recent Innovations in Theory and Method

P1: GFU

Sociological Forum [sofo] PP225-343076 August 24, 2001 8:15 Style file version Nov. 19th, 1999

Sociological Forum, Vol. 16, No. 3, September 2001 ( c© 2001)

Review Essay

Beyond Correlational Analysis: RecentInnovations in Theory and Method

James Mahoney1

Social Mechanisms: An Analytical Approach to Social Theory. PeterHedstrom and Richard Swedberg, eds. Cambridge: CambridgeUniversity Press, 1998.

Fuzzy-Set Social Science. Charles C. Ragin. Chicago: University of ChicagoPress, 2000.

INTRODUCTION

Correlational analysis is the basis for much explanation in contemporarysociology. Mainstream sociologists often infer causation through the use ofquantitative techniques that depend, in one way or another, on the existenceof bivariate correlations. Even sophisticated multivariate statistical methodsthat allow for the parceling of variables and the estimation of average netcausal effects ultimately rely on such correlations. Correlational analysis sois central to explanation that many sociologists simply take it for granted,assuming that the existence of a correlation is a basic component of causality.

The two books reviewed in this essay, however, argue that correlationalanalysis is by itself an inadequate mode of causal assessment. The prob-lems identified are not simply familiar themes such as a correlation failsto represent causation because it might be spurious or because the timeorder of correlated variables may not be clear. Rather, these works sug-gest that even nonspurious correlations in which the time order of vari-ables is well-established may be inherently limited representations of causalprocesses.

1Department of Sociology, Box 1916, Maxcy Hall, Brown University, Providence, Rhode Island02912; e-mail: james [email protected].

575

0884-8971/01/0900-0575$19.50/0 C© 2001 Plenum Publishing Corporation

P1: GFU


576 Mahoney

In Social Mechanisms: An Analytical Approach to Social Theory, editedby Peter Hedstrom and Richard Swedberg, contributors argue that correla-tional analysis fails to identify the mechanisms that produce observed associ-ations and that explain why empirical associations exist in the first place. Theauthors of this volume—many of them statistical researchers who employthe conventional methods—call for a new “tool kit” of mechanisms that willenable sociologists to make sense of their empirical correlations. In Fuzzy-Set Social Science, Charles C. Ragin suggests that correlational analysis isbased on strong assumptions about unit homogeneity and linear causationthat often do not hold in comparative research. He calls on researchers whostudy complex patterns of causation to go beyond statistical correlationsby employing fuzzy-set methods that assume causal heterogeneity and thatallow for the analysis of necessary and sufficient conditions.

These books reflect a broader trend in the literature toward questioningthe utility of conventional techniques for analyzing causation (e.g., Abbott,1988). However, the works do not reject the enterprise of causal analysisitself. Both studies are intended for mainstream sociologists who accept thatexplanation and causal analysis are basic tools of the trade, but who mayhave some healthy skepticism about the utility of conventional techniquesfor achieving these goals.

SOCIAL THEORY AND THE IMPROVEMENTOF EMPIRICAL RESEARCH

The central goal of the volume edited by Hedstrom and Swedberg is“to explicate the social mechanisms that generate and explain observed as-sociations between events” (p. 1). In this section, I explore how and whysuch an enterprise can offer important advances for social analysis. I be-gin by considering the limitations of correlational analysis from the stand-point of a social mechanisms perspective. I then examine how a focus onmechanisms—conceptualized here as unobserved relations or processes thatgenerate outcomes—can allow researchers to go beyond correlations. I con-clude the section by considering how a mechanisms perspective might helpsociologists overcome the gap that currently exists between social theoryand quantitative research.

Correlational Analysis and Its Limitations

Three basic limitations of correlational analysis can be identified fromthe articles in the Social Mechanisms volume and related writings in the

P1: GFU


Beyond Correlational Analysis: Recent Innovations in Theory and Method 577

philosophy of science. First, on a substantive level, analysts have expressedconcerns that correlational analysis provides a weak foundation for integrat-ing diverse quantitative findings into a single coherent framework that cansustain knowledge accumulation. With conventional methods, researcherstypically discover that a highly heterogeneous group of variables are as-sociated with an outcome, and these kinds of methods offer no basis forunderstanding why such diverse factors are related to the phenomenon ofinterest. The result is a lack of theoretical integration, which in turn con-tributes to the fragmentation of knowledge that currently characterizes thesocial sciences (Dessler, 1991; Sørensen, 1998).

Second, according to critics, correlational analysis relies on the ques-tionable assumption that causation exists only to the degree that empiricalregularities also exist. In other words, correlational analysts implicitly as-sume that if there is no empirical regularity, there is no causation, or at leastcausal forces cannot be identified without such regularities. Given that corre-lational analysts usually “explain” only a portion of the variation of interest,however, they must also assume that much or most of the world is governedby stochastic forces or unknown variables.2 This conclusion is somewhat dis-illusioning in light of the substantial research that has been focused on manyoutcomes of interest (see Sørensen, 1998; Stinchcombe, 1998).

Finally, various analysts argue that correlational analysis is limited be-cause its fails to explore the so-called black box that connects independentand dependent variables in a causal relationship (e.g., Boudon, 1998; Cowen,1998; Hedstrom and Swedberg, 1998; Hernes, 1998). On this view, “we arenot satisfied with merely establishing systematic covariation between vari-ables or events; a satisfactory explanation requires that we are also ableto specify the social ‘cogs and wheels’ that have brought the relationshipinto existence” (p. 7; see also Elster, 1989:3). Knowledge of this black boxis needed to develop “deeper, more direct, and more fine grained expla-nations . . . [and] to distinguish between genuine causality and coincidentalassociation” (pp. 8–9).

Conceptualizing Causal Mechanisms

The proposed solution to the problems of correlational analysis offeredin the Hedstrom–Swedberg volume is the development of theories of socialmechanisms, or what I prefer to call “causal mechanisms.” This concern withcausal mechanisms grows out of a distinction sometimes made in the phi-losophy of science between correlational analysis and causal analysis. Whilecorrelational analysis involves identifying antecedents regularly conjoined

2Measurement error may also account for some of the unexplained variance.

P1: GFU


578 Mahoney

with outcomes, causal analysis consists of identifying the mechanism thatunderlies and generates empirical regularities and outcomes. With causalanalysis, the challenge of explanation involves postulating “entities, proper-ties, processes, relations . . . that are held to be causally responsible for theempirical regularities to be explained” (McMullin, 1984:210). These mech-anisms or causal agents explain why correlations exist in the first place;knowledge of their operation allows researchers to go beyond correlations(Harre, 1970:230, 1972:170).

Unfortunately, a good deal of confusion currently surrounds the precisemeaning of causal mechanism, including in the volume under review (seeTable I). To sort out this confusion, I separate different definitions found inthe literature, beginning with two common definitions that I do not believetake us beyond correlational analysis. I then discuss a third definition that Ido believe offers a real alternative to correlational research.

First, a causal mechanism is often understood as an intervening vari-able or set of intervening variables that explain why a correlation existsbetween an independent and dependent variable (e.g., Bennett and George,1997; Glennan, 1996; Goldthorpe, 1998; Hedstrom and Swedberg, 1998; Kinget al., 1994; Kiser and Hechter, 1991; Somers, 1998). In this formulation, thespecification of a mechanism helps researchers make an observed associa-tion more intelligible by filling in the black box between independent anddependent variables. Yet, while the notion of mechanism as intervening pro-cess is useful, this definition unfortunately does not go beyond correlationalassumptions. Causal mechanisms as intervening variables must be identi-fied and analyzed with correlational tools. Indeed, a variable’s status as a“mechanism” as opposed to an “independent variable” is arbitrary. Withthis definition, then, a correlation is “explained” simply by appealing to an-other correlation of observed variables (see Goldthorpe, 1998:19; King et al.,1994:86; McMullin, 1984:206).

A second definition views causal mechanisms as mid-level theories orvariables that can be used to explain a fairly wide range of outcomes. InStinchcombe’s vivid formulation (Stinchcombe, 1998:267) causal mecha-nisms are “bits of sometimes true theory”; or, as Elster (1998:45) puts it,“mechanisms are frequently occurring and easily recognizable causal pat-terns that are triggered under generally unknown conditions or with inde-terminate consequences.” Although we need better mid-level theories, thisdefinition has the disadvantage of not clearly differentiating the concept ofmechanism from the idea of a probabilistically formulated hypothesis – thevery essence of most correlational research. In other words, in asserting thata causal mechanism entails a probabilistic hypothesis, scholars who employthis definition in no way break with any of the correlational assumptions thatguide standard sociological analyses.

P1: GFU



Table I. Glossary of Definitions of “Mechanism”

Bennett and George (1997, p. 1): “the processes and intervening variables through whichcausal or explanatory variables produce causal effects.”

Bhaskar (1979, p. 15): “the construction of an explanation for . . . some identified phenomenonwill involve the building of a model, utilizing such cognitive materials and operating underthe control of something like a logic of analogy and metaphor, of a mechanism, whichif it were to exist and act in the postulated way would account for the phenomenon inquestion.”

Boudon (1998, p. 172): “A SM [social mechanism] is, in other words, the well-articulated setof causes responsible for a given social phenomenon. With the exception of typical simpleones, SMs tend to be idiosyncratic and singular.”

Cowen (1998, p. 125): “I interpret social mechanisms (defined in greater detail below) asrational-choice accounts of how a specified combination of preferences and constraints cangive rise to more complex social outcomes.”

Elster (1989, p. 3): “nots and bolts, cogs and wheels– that can be used to explain quite complexsocial phenomena.”

Elster (1998, 45, his emphasis): “Roughly speaking, mechanisms are frequently occurring andeasily recognizable causal patterns that are triggered under generally unknown conditions orwith indeterminate consequences.”

Gambetta (1998, p. 102): “I take ‘mechanisms’ to be hypothetical causal models that makesense of individual behavior. They have the form, ‘Given certain conditions K, an agentwill do x because of M with probability p.’ M refers either to forms of reasoning govern-ing decision making (of which rational choice models are a subset) or to subintentionalprocesses that affect action both directly (as impulsiveness) or by shaping preferences orbeliefs.”

Goldthorpe (1998, p. 18): “some process existing in time and space, even if not perhaps directlyobservable, that actually generates the causal effect of X on Y and, in doing so, producesthe statistical relationship that is empirically in evidence.”

Harre (1970, pp. 101, 102, 104): “the structures, states and inner constitutions from which thephenomena of nature flow. . . . the permanent or enduring conditions under which a certainkind of phenomenon will occur.” “The inner constitutions, structures, powers, encompassingsystems, and so on, of which natural generative mechanisms are constituted, and of whichthe connection between cause and effect usually consists.”

Hedstrom and Swedberg (1998, p. 11): “Mechanism-based explanations usually invoke someform of ‘causal agent’ that is assumed to have generated the relationship between theentities observed.”

Hedstrom and Swedberg (1998, p. 13): “Mechanisms . . . are analytical constructs that providehypothetical links between observable events.”

Hernes (1998, p. 74): “A mechanism is a set of interacting parts – an assembly of elementsproducing an effect not inherent in any one of them. A mechanism is not so much about‘nuts and bolts’ as about ‘cogs and wheels’ – the wheelwork or agency by which an effect isproduced. But a mechanism or inner workings is an abstract, dynamic logic by which socialscientists render understandable the reality they depict.”

Keat and Urry (1982, p. 30): “causal explanations require the discovery both of regular rela-tions between phenomena, and of some kind of mechanism that links them. . . . In describ-ing these mechanisms and structures we will often, in effect, be characterizing the ‘nature,’‘essense,’ or ‘inner constitution’ of various types of entity.”

King et al. (1994, p. 85): “Some scholars argue the central idea of causality is that of aset of ‘causal mechanisms’ posited to exist between cause and effect. This view makesintuitive sense: any coherent account of causality needs to specify how the effects areexerted.”

Kiser and Hechter (1991, p. 5): “A complete explanation also must specify a mechanism thatdescribes the process by which one variable influences the other, in other words, how it isthat X produces Y.”

(Continued )

P1: GFU


580 Mahoney

Table I. (Continued )

Koslowski (1996, p. 6): “A causal mechanism is the process by which a cause brings about aneffect. A mechanism is a theory or an explanation, and what it explains is how one eventcauses another.”

Little (1991, p. 15): “A causal mechanism, then, is a series of events governed by lawlikeregularities that lead from the explanans to the explanandum.”

Schelling (1998, pp. 32–33): “I propose . . . that a social mechanism is a plausible hypothesis,or set of plausible hypotheses, that could be the explanation of some social phenomenon,the explanation being in terms of interactions between individuals and other individuals,or between individuals and some social aggregate.”

Somers (1998, p. 726, citing Coleman, 1986:1328): “meaningful connection between events asthe basic tool of description and analysis.”

Sørensen (1998, p. 240): “My definition of ‘mechanism’ is simple: It is an account of howchange in some variable is brought about – a conceptualization of what ‘goes into’ aprocess.”

Steinmetz (1998, pp. 177–178): “Within critical realism, a law is not a constant conjunctionof events but the characteristic pattern of activity, or tendency, of a mechanism. Morespecifically, real structures possess causal powers which, when triggered or realized, act withnatural necessity and universality as generative mechanisms.” [Accompanying footnote:“Generative mechanisms are ‘tendencies’ rather than ‘powers’ because they are not justpotentialities but potentialities that may be exercised without being manifested.”]

Stinchcombe (1993, pp. 24–25): “As I use the word, mechanism means (1) a piece of scientificreasoning which is independently verifiable and independently gives rise to theoreticalreasoning, which (2) gives knowledge about a component process (generally one withunits of analysis at a ‘lower level’) of another theory (ordinarily a theory with units ata different ‘higher’ level), thereby (3) increasing the suppleness, precision, complexity,elegance, or believability of the higher-level theory with excessive ‘multiplication of entities’in it, (4) without doing too much violence (in the necessary simplification at the lowerlevel to the make the higher-level theory go) to what we know as the facts at the lowerlevel.”

Stinchcombe (1998, p. 267): “I have defined mechanisms before as bits of ‘sometimes truetheory’ or ‘model’ that represent a causal process, that have some actual or possible empir-ical support separate from the larger theory in which it is a mechanism, and that generateincreased precision, power, or elegance in the large-scale theories.”

Tilly (in press): “mechanism-based explanations aim at modest ends: selective explanation ofsalient features by means of partial analogy. . . .Mechanisms are events that alter relationsamong some specified set of elements.”

I believe a third definition—derived from the realist school in the philos-ophy of science—does take us beyond a correlational approach. Accordingto this definition, a causal mechanism is an unobserved entity that—whenactivated—generates an outcome of interest. This understanding of mecha-nism goes beyond correlational analysis in at least two essential ways. First, itassumes that the activation of a mechanism actually generates an outcome inthe sense that the mechanism is sufficient to produce the outcome of interest;that is, if the mechanism actually operates, it will always produce the out-come of interest. This notion differs from most conventional analyses, whichtypically assume that a causal variable increases or decreases the probabilityof having higher or lower values on an outcome, net of the effects of all othervariables.

P1: GFU



The second distinctive feature of casual mechanisms concerns theirstatus as unobserved entities. Causal mechanisms are posited relations orprocesses that the researcher imagines to exist; they do not refer to anyparticular set of empirical conditions. As a consequence, they cannotenter into a correlational analysis as an empirical state to be measuredacross cases for its covariation with some outcome. Social analysts who positunobservables—analogous to theorists in the natural sciences3—try to spec-ify the shape or content of things they cannot directly see. Not every positedcausal mechanism is useful. Useful causal mechanisms are those that helpanalysts make sense of diverse observable phenomena and integrate ex-isting correlational knowledge into a coherent framework. For example, auseful causal mechanism should be able to explain why diverse kinds of vari-ables are empirically correlated with a phenomenon of interest. Likewise,a useful causal mechanism should suggest new correlations not previouslydiscovered.

Most contributors to the Social Mechanisms volume assume that ratio-nal choice models developed at an individual level provide the best causalmechanisms (e.g., Cowen, 1998; Goldthorpe, 1998; Hedstrom and Swedberg,1998; Schelling, 1998). These models are indeed potentially useful mecha-nisms, given that they explain outcomes through unobserved entities (e.g.,utility functions) that may directly generate behavior. However, causalmechanisms need not be developed using rational actor assumptions at anindividual level of analysis (Harre, 1970:35). For example, structural func-tionalists treat social systems with unobserved needs such as integrationor survival as causal mechanisms that can explain various observable phe-nomena. Likewise, and more compellingly, many historical sociologists treatpower relations within societies as unobserved forces that generate out-comes in the social world. A nice example is the work of Rueschemeyeret al. (1992), which posits a hypothetical set of power relations that, whenequitably distributed, will generate democratic outcomes. By employing thismechanism, the authors are able to explain certain longstanding correlations(e.g., economic development is positively associated with democracy) andidentify many new correlations (e.g., the observable size and organizationof the working class is positively correlated with democracy) in terms ofthe ways in which independent variables trigger shifts in the distribution ofpower relations within society.

3Natural scientists posit and use unobservables such as strings in superstring theory, spring-likeballs in elasticity theory, waves in Boyle’s theory of light, molecules in theories of Brownianmotion, and quarks in particle theory (Hacking, 1983; Musgrave, 1985). The utility of theseentities in scientific explanation is not as independent variables that can explain variationon an outcome; rather, these entities are useful because they are believed to be the realmechanisms that physically generate outcomes in the natural world.

P1: GFU


582 Mahoney

Unifying Sociology

Few would deny that there is a division within sociology between socialtheorists and quantitative researchers. The division derives as much frommutual neglect as from any open hostility; thoughtful analysts recognizethat the cleavage serves neither side well. In one of the best chapters in theSocial Mechanisms volume, for example, the quantitative sociologist AageSørensen (1998, pp. 238–239) writes that

Quantitative sociology remains very theory-poor. In fact, the mainstream has re-gressed rather than progressed. Quantitative sociology is now less theoretically in-formed and less relevant for theoretical progress than it was three decades ago . . . theenormous progress in methodological power has turned quantitative methodologyinto a branch of statistics. This has led to a fascination, if not obsession, with statis-tical models and concerns, and a neglect of the need to develop sociological modelsmirroring conceptions of mechanisms of social processes.

Similar claims can be made for social theory and its relationship to quanti-tative sociology. Social theorists are often so concerned with their conceptsand frameworks that they pay little attention to the findings of quantitativesociology; as a result, social theorists forfeit powerful evidence that could beused to adjudicate among rival theoretical frameworks.

Clearly, neither statistical researchers nor social theorists can afford tolive without one another. Correlational findings are incomplete and not fullyintelligible without an understanding of the mechanisms that generate thosefindings; by contrast, theories of causal mechanisms are entirely specula-tive until their power is revealed through empirical correlations. Hence, acomplete science must strive to identify both correlations and causal mech-anisms; as such, quantitative research and social theory are complimentaryand should be united.

A METHODOLOGICAL ALTERNATIVE TOCORRELATIONAL ANALYSIS

Charles Ragin’s book presents a methodological critique of standardcorrelational analyses and introduces “fuzzy-set” methods as an alternativeapproach. Among other things, fuzzy-set methods provide tools for ana-lyzing necessary and sufficient causes, and thus address the concerns of re-searchers who believe that these causes are little more than nonfalsifiabletautologies. Furthermore, by offering concrete procedures for identifyingsufficient causes, Ragin’s work can be linked to the study of causal mecha-nisms, although he himself does not do so.

P1: GFU



Methodological Limitations of Correlational Analysis

In the first half of Fuzzy-Set Social Science, Ragin outlines his pre-ferred methodology for social analysis—what he calls “diversity-orientedresearch”—through a contrast with correlational methods. Although thisdiscussion repeats certain themes offered in Ragin’s earlier work (Ragin,1987), it also updates his critique of conventional tools. Here I review threeaspects of this critique.

First, Ragin argues that scholars who employ conventional methodstypically treat their population of cases as a “given,” offering little reflec-tion on why cases are included and rarely revising their understanding of thepopulation in the course of research. By using unproblematized populations,Ragin argues, conventional researchers frequently and inappropriately in-clude heterogeneous cases in their studies. The consequence is that they“may relegate unrecognized heterogeneity to the error vector of probabilis-tic models when it should be conceived, if properly recognized, as multiplepopulations” (p. 50). To avoid treating heterogeneity as unexplained vari-ance, Ragin argues that scholars who study complex social phenomena mustassume causal heterogeneity is the norm and treat populations as evolvingconstructs to be modified in light of theoretical and substantive knowledge.

Second, Ragin is critical of the conventional practice of treating casesas collections of analytically separate variables that exert causal effects in-dependent of one another. When analyzing complex social phenomena, hesuggests that the effects of any given variable will depend on the broadercontext within which that variable is situated. As a result, researchers musttreat their cases as configurations of aspects that are examined together,as overall “packages.” Statistical researchers could try to study cases inthis fashion by applying saturated interaction models to all variables, butthese models would generally be difficult to interpret due to collinearityamong interaction terms and the tendency of multiple models to fit thedata (p. 72).

Third, Ragin argues that correlational methods are insensitive to nec-essary and sufficient conditions. The conventional approach of treating vari-ables as open-ended categories and exploring their covariation with one an-other is not designed to locate necessary and sufficient conditions, especiallycombinations of variables that join together to produce a sufficient cause.To analyze these conditions, Ragin points out, one must treat variables asqualitative states in which cases are either “in” or “out” of the category, orare partially in or partially out of the category. For example, to argue thatsome condition X is a sufficient cause of some outcome Y, one must be ableto distinguish cases that are full or partial members of condition X from

P1: GFU


584 Mahoney

those that completely lack condition X. From the perspective of correla-tional research, however, the idea that some cases may be a “full” memberof a category or that some cases may be “completely outside” a category areforeign notions.

The first half of Fuzzy-Set Social Science goes beyond Ragin’s previouswork by elaborating a partial and probabilistic approach to the analysis ofnecessary and sufficient conditions. There are two central innovations here.First, Ragin notes that random error, measurement error, and human errorwill often make it difficult to locate any cause or combination of causes thatare necessary and/or sufficient. As a result, he develops terminology andbenchmark proportions for analyzing causes at different degrees of necessityor sufficiency. For instance, a benchmark of 0.5 is used for causes that are“more often than not” necessary or sufficient (i.e., causes that are necessaryor sufficient at least half of the time), while a benchmark of 0.8 is usedfor causes that are “almost always” necessary or sufficient. Second, using asimple binominal test, he calculates the minimal number of consistent casesat each benchmark needed to reach statistical significance across variouslevels (e.g., at the levels of 0.10, 0.05, and 0.01 significance). For instance,when analyzing a “usually” necessary cause (i.e., a cause that is necessaryat a 0.65 benchmark), seven or more consistent cases are needed to reach a0.05 level of significance.

Quantitative macrosociologists must seriously consider Ragin’s criti-cisms and his proposal for analyzing necessary and sufficient conditions.While statistical solutions may exist for some of the methodological problemsidentified by Ragin, these solutions are rarely applied in practice. Moreover,the point of Ragin’s discussion is not simply to illustrate the shortcomings ofstatistical research, but to show how qualitative research overcomes theselimitations. In Ragin’s view, the challenge is for statistical researchers toadapt their research to the more demanding standards of qualitative analy-sis rather than the reverse.

Ragin’s innovative discussion is not without limitations. For example, heoffers no sustained discussion of how qualitative and quantitative methodscan be combined in research. Although he believes there is a place for bothkinds of research (p. xv), the main thrust of his argument is to show howquantitative analysis embodies major deficiencies that qualitative researchcan overcome. In making this argument, he overlooks the ways in whichanalysts frequently combine the two approaches in their empirical research(see Mahoney, 2000).

There are also limitations that stem from Ragin’s decision not to addressissues related to temporality and sequencing. This omission is important notonly because temporality is fundamental to any understanding of causation,

P1: GFU



but also because qualitative methods are often designed to deal with tempo-ral issues (see Aminzade, 1992; Pierson, 2000; Rueschemeyer and Stephens,1997). Furthermore, the omission leads Ragin not to discuss how qualitativemethods may be especially appropriate for the study of causal mechanisms.While causal mechanisms do not refer to any observable set of conditions,the ability of researchers to find combinations of observable variables thatare usually or almost always sufficient can provide supporting evidence infavor of the existence of a given mechanism. In this sense, Ragin’s agendamight be more closely linked to the developments in social theory consideredin the Hedstrom-Swedberg volume.

Measuring Fuzzy Membership

The second half of Ragin’s book introduces fuzzy-set methods as ameans to overcome the limitations of correlational analyses. These methodswill seem so foreign to some quantitative researchers that their initial reac-tion may be to assume that Ragin fails to understand conventional statistics.In fact, however, Ragin presents a sophisticated argument that anticipatesmany criticisms that these researchers would raise.

Fuzzy-set measurement differs from both the binary categorization ad-vocated by Ragin in his earlier work and the continuous scales usually soughtby conventional researchers. With fuzzy sets, cases are scored from 0 to 1based on their degree of membership in a category. Some cases will be “full”instances of the category and thus receive a score of “1,” even if these caseswould have different scores on an interval scale. For example, with respect tothe category of “rich country,” both the United States and England are fullmembers, and thus both receive the same score of “1,” despite the fact thatthese countries have different levels of wealth on an interval scale. Othercountries receive scores based on the extent to which they overlap withthe category “rich country.” For example, a country that is “mostly in” thecategory will receive a score only slightly less than one (e.g., 0.83), while acountry that is “mostly out” of the category will receive a score closer to zero(e.g., 0.17). The decision to assign fuzzy-set membership scores to cases isbased on the analyst’s substantive and theoretical knowledge. In this sense,according to Ragin, fuzzy-set categories can be more empirically groundedthan conventional variables.

How one feels about this approach to measurement will depend in parton whether one believes qualitative categories should be employed in socialscience analysis. From the perspective of quantitative research, the deci-sion to ignore variation is viewed as a potential source of systematic error.

P1: GFU


586 Mahoney

However, when assessing necessary and sufficient conditions, this problem ofbias does not arise (in fact, error could be introduced if the additional vari-ation were included). Moreover, recent work in cognitive science suggeststhat the human mind processes the world in a manner that closely parallelsfuzzy-set categories (e.g., Lakoff, 1987). Given these considerations, I seelittle compelling reason why fuzzy-set measurement should be dismissed ei-ther on the grounds that it introduces bias into research or that it fails tomap the way in which ordinary language-users employ categories.

Nevertheless, I do believe there is a real potential for “overfitting” thedata when using fuzzy-set measures. Precisely because researchers mustrevise their measures in light of new evidence, they may be tempted torescale or recode data until they reveal a desired or otherwise interestingfinding. Although Ragin is aware of this problem (pp. 309–311; 316–321),straightforward solutions are not identified in the book.

Assessing Necessary and Sufficient Causes With Fuzzy-Set Methods

To see how fuzzy-set methods assess necessary and sufficient causes,it is helpful to imagine a scattergram in which cases are plotted based ontheir fuzzy membership in an hypothesized causal factor and an outcome.With a necessary cause, fuzzy membership scores on the outcome are lessthan or equal to fuzzy membership scores on the cause. The resulting scat-tergram thus has a triangular shape, with all points on or beneath the maindiagonal. For example, the hypothetical data in Fig. 1 plot countries basedon their fuzzy-set membership in “social revolution” (outcome) and “statebreakdown” (cause). These data support the notion that state breakdownis a necessary condition for social revolution, given that all scores on theoutcome are less than or equal to scores on the cause. By contrast, with asufficient cause, fuzzy membership scores on the cause are less than or equalto fuzzy membership scores on the outcome. Hence, all points will be locatedon or above the main diagonal. Ragin formally elaborates the logic behindthese procedures, including the important issues that arise when consideringcombinations of variables that are necessary or sufficient.

Several techniques are discussed to deal with the partial and probabilis-tic nature of necessary and sufficient causation. For example, the researchermight argue that if no case’s score on the outcome (or cause) exceeds itsscore on the cause (or outcome) by more than a small portion of a fuzzy-membership unit (e.g., 0.10 unit), then the pattern is still consistent with theinterpretation of causal necessity (or sufficiency). Likewise, the probabilisticbenchmarks and significance tests mentioned above can be applied whenusing fuzzy measures of variables. For example, Fig. 2 presents a simple

P1: GFU



Fig. 1. Example of necessary condition: Plot of “social revolution” against “statebreakdown.” Source: Ragin (2000, pp. 216–217).

illustration of a probabilistic approach to analyzing causal sufficiency. Inthis hypothetical example, 18 countries are plotted based on their fuzzymembership in “strong labor movements” (cause) and “generous welfarestates” (outcome). The plot has the characteristic shape of causal sufficiency,but two of the countries are below the main diagonal. In this sense, the causewould fail a nonprobabilistic test of sufficiency. However, using probabilis-tic criteria, this cause passes the test of “usually sufficient” at a 0.05 levelof significance. Although the mathematics involved becomes complicatedwhen combinations of variables are considered using probabilistic criteria,a free software package that performs the statistical operations is alreadyavailable (Drass and Ragin, 1999).

Fuzzy-set methods are a major breakthrough for the study of necessaryand sufficient conditions. They avoid the four most common failures of thiskind of analysis: (1) the failure to recognize that many variables cannot beadequately measured using dichotomous indicators; (2) the failure to assesscombinations of causal factors; (3) the failure to include a partial and proba-bilistic understanding of causation; and (4) the failure to consider statistical

P1: GFU


588 Mahoney

Fig. 2. Example of probabilistic sufficient condition: Plot of “generous welfare state”against “strong labor movement.” Source: Adapted from Ragin (2000, p. 251).

significance. Although other scholars may have previously considered theapplication of fuzzy-set methods to social science, Ragin systematically ad-dresses all of the methodological issues involved in employing this kind ofanalysis. As a result, he advances the discussion of qualitative methods to anentirely new level.

Having acknowledged this fundamental contribution, I recognize thereal trade-offs that face the researcher who chooses to use fuzzy-set meth-ods. For one thing, these methods require that one construct a populationwith enough cases to pass statistical significance tests. Typically, this will in-volve selecting more than just a handful of cases (though not a “large-N”).Selecting a “medium-N” is a problem, however, because fuzzy-set methodsalso demand the researcher has an excellent knowledge of all cases in orderto make sound judgements about measurement. When the analyst selectsmore than a handful of cases, achieving this knowledge may be difficult. In

P1: GFU



short, there is a tension between selecting enough cases to obtain statisti-cal significance and selecting a small enough number of cases to maintainconceptual validity.

Equally important, these methods cannot be used to detect standardlinear correlations when such associations are in fact present. Despite thelimitations of correlational techniques as a basis for representing causation,we know that correlational associations frequently are discovered by re-searchers. It is in fact an open question whether relationships based on thelogic of necessary and sufficient conditions or relationships derived from lin-ear correlations will be found more frequently by researchers. At this point,it is still too early to assume that social scientists will have more success find-ing convincing examples of necessary and sufficient conditions than theyhave had at locating strong correlational associations.

Ragin himself notes that fuzzy-set methods are most powerful as a tech-nique of “discovery,” not a technique for testing theory (p. 310). This observa-tion points to the value of these methods for uncovering causal mechanisms.Specifically, the discovery of combinations of factors that are sufficient (oralmost sufficient) for an outcome can help researchers develop theories ofcausal mechanisms.

To elaborate this idea a little further, it is useful to consider Ragin’sdiscussion of the origins of IMF protest in 35 Third World countries (i.e.,developing countries that experienced protest against austerity measuresmandated by the IMF). Ragin explores several possible causes of this out-come, including urbanization, economic hardship, investment dependence,political liberalization, and government activism. These causal factors andthe outcome are conceptualized as fuzzy sets and measured accordingly.Using the procedures discussed above, Ragin tests all possible combinationsof factors for sufficiency, applying logical rules for simplifying combinationswhen appropriate. He ultimately identifies three combinations of factors thatare “almost always” sufficient at a 0.01 level of significance. For example, thecombination of political liberalization, economic hardship, and investmentdependence is almost always sufficient to produce IMF protest. That is, whena country facing IMF pressure experiences political liberalization (measuredin terms of civil and political rights), economic hardship (measured in termsof inflation and consumer prices), and dependence on foreign investment(measured in terms of foreign capital distribution), protests against IMFausterity plans will almost always take place.

A finding such as this one can help social theorists conceptualize thecausal mechanism that generates IMF protest. For instance, one possibil-ity would be to view societal groups as a transportable entity that can bepushed from a state of nonprotest to a state of IMF protest by prevailing

P1: GFU


590 Mahoney

Fig. 3. Hypothetical causal mechanism generating IMF protest.

conditions in society (see Fig. 3). In this example, the presence of economichardship is important because it provides the kind of impetus that wouldmove groups toward protest. However, in the example, the ability of soci-etal groups to move from nonprotest to protest also depends on the absenceof any obstructions or blocking forces. The existence of political liberaliza-tion might therefore be important because it removes potential barriers toprotest. Finally, once the entity moves to a state of protest, it must be di-rected specifically toward anti-IMF protest. The presence of dependence onforeign investment might serve the role of directing societal groups towardprecisely this kind of protest activity.

The elements in Fig. 3 are postulated entities and relations that mayor may not adequately capture the causal mechanism that generates IMFprotest. The utility of this causal mechanism would have to be evaluated

P1: GFU



based on its ability to explain other empirical findings and to suggest newempirical associations. Nevertheless, the example does show how Ragin’smethods can be closely linked to theory building through the specificationof the causal mechanisms that generate outcomes in the social world.

CONCLUSION: THE IMPORTANCE OFCASE-ORIENTED RESEARCH

The two books under review suggest that case-oriented research hasa crucial role to play in social explanation. The importance of this kindof research is illuminated through recent innovations in both theory andmethod. First, with respect to theory, we know from the natural sciencesthat discovering causal mechanisms requires taking a close look at actualinstances of the phenomenon under investigation and working backwardfrom outcomes to causal mechanisms (e.g., see Hanson, 1958:85–86, andMcMullin, 1984, on “retroductive” explanation). If investigators lack knowl-edge of actual instances of the phenomena of interest, they are unlikely tomake good decisions about how to conceptualize the mechanisms that gen-erate these phenomena. Likewise, contrary to what is sometimes asserted,scientists do not formulate mechanisms purely or primarily through deduc-tive analysis; rather, they employ what sociologists call “analytic induction”to accomplish this task. Thus, should social scientists seek to replicate thekind of theory-building employed in the natural sciences, they must postu-late causal mechanisms based on intensive examinations of particular cases.Furthermore, they must in part work backward from the observed outcometo the theoretical mechanism in question.

Second, with respect to methodology, the examination of necessary andsufficient conditions requires much substantive and theoretical knowledge ofparticular cases. This is true because fuzzy-set measurement—which is nowthe best tool available for assessing these conditions—cannot be adequatelycalibrated if the analyst has a poor understanding of the cases that definea population. Furthermore, as Ragin suggests, many of the limitations ofconventional techniques grow out of the researcher’s inadequate knowledgeof the cases included in the analysis. In this sense, the improvement of bothmethodologies designed to identify necessary and sufficient conditions andmethodologies designed to locate correlational associations depends on theavailability of good case-study research.

ACKNOWLEDGMENTS

I thank Jeff Goodwin and Gerardo L. Munck for helpful comments ona previous draft.

P1: GFU


592 Mahoney

REFERENCES

Abbott, Andrew1988 “Transcending general linear reality.”

Sociological Theory 6:169–186.Aminzade, Ronald1992 “Historical sociology and time.” Socio-

logical Methods and Research 20:456–480.

Bennett, Andrew, and Alexander L. George1997 “Process tracing and case study

research.” Paper presented at theMacArthur Foundation Workshopon Case Study Methods, HarvardUniversity, Oct. 17–19, 1997.

Bhaskar, Roy1979 The Possibility of Naturalism: A Philo-

sophical Critique of the ContemporaryHuman Sciences. Atlantic Highlands,NJ: Humanities Press.

Boudon, Raymond1998 “Social mechanisms without black

boxes.” In Peter Hedstrom andRichard Swedberg, (eds.), SocialMechanisms: An Analytical Approachto Social Theory: 172–203. New York:Cambridge University Press.

Coleman, James S.1986 “Social theory, social research, and a

theory of action.” American Journal ofSociology 91:1309–1335.

Cowen, Tyler1998 “Do economists use social mecha-

nisms to explain?” In Peter Hedstromand Richard Swedberg, (eds.), SocialMechanisms: An Analytical Approachto Social Theory: 125–146. New York:Cambridge University Press.

Dessler, David1991 “Beyond correlations: Toward a causal

theory of war.” International StudiesQuarterly 35:337–355.

Drass, Kriss, and Charles C. Ragin1998 QC/FSA: Qualitative Compara-

tive/ Fuzzy-Set Analysis. Evanston:Institute for Policy Research,Northwestern University. Availableat: (http://www.nwu.edu/sociology).

Elster, Jon1989 Nuts and Bolts for the Social Sciences.

Cambridge: Cambridge UniversityPress.

1998 “A plea for mechanisms.” In PeterHedstrom and Richard Swedberg,(eds.), Social Mechanisms: An

Analytical Approach to Social Theory:45–73. New York: Cambridge Univer-sity Press.

Gambetta, Gudmund1998 “Concatenations of mechanisms.”

In Peter Hedstrom and RichardSwedberg, (eds.), Social Mechanisms:An Analytical Approach to So-cial Theory: 102–124. New York:Cambridge University Press.

Glennan, Stuart S.1996 “Mechanisms and the nature of causa-

tion.” Erkenntnis 44:49–71.Goldthorpe, John H.1998 “Causation, statistics and sociology.”

Unpublished Manuscript, NuffieldCollege, Oxford University.

Hacking, Ian1983 Representing and Intervening.

Cambridge: Cambridge UniversityPress.

Hanson, Norwood Russell1958 Patterns of Discovery: An Inquiry

into the Conceptual Foundationsof Science. Cambridge: CambridgeUniversity Press.

Harre, Rom1970 The Principles of Scientific Thinking.

Chicago: University of Chicago Press.1972 The Philosophies of Science. London:

Oxford University Press.Hedstrom, Peter, and Richard Swedberg1998 “Social Mechanisms: An introductory

essay.” In Peter Hedstrom and RichardSwedberg, (eds.), Social Mechanisms:An Analytical Approach to SocialTheory: 1–31. New York: CambridgeUniversity Press.

Hernes, Gudmund1998 “Real virtuality.” In Peter Hedstrom

and Richard Swedberg, (eds.), SocialMechanisms: An Analytical Approachto Social Theory: 74–103. New York:Cambridge University Press.

Keat, Russell, and John Urry1982 Social Theory as Science. London:

Routledge.King, Gary, Robert O. Keohane, and SidneyVerba1994 Designing Social Inquiry: Scientific

Inference in Qualitative Research.Princeton, NJ: Princeton UniversityPress.

P1: GFU



Kiser, Edgar, and Michael Hechter1991 “The role of general theory in com-

parative-historical sociology.” Ameri-can Journal of Sociology 97:1–30.

Koslowski, Barbara1996 Theory and Evidence: The Devel-

opment of Scientific Reasoning.Cambridge, MA: MIT Press.

Lakoff, George1987 Women, Fire, and Dangerous Things:

What Categories Reveal Aboutthe Mind. Englewood Cliffs, NJ:Prentice-Hall.

Little, Daniel1991 Varieties of Social Explanation: An In-

troduction to the Philosophy of SocialScience. Boulder Co: Westview Press.

Mahoney, James2000 “Strategies of causal inference

in small-N analysis.” SociologicalMethods and Research 28:387–424.

McMullin, Ernan1984 “Two ideals of explanation in natural

science.” Midwest Studies in Philoso-phy 9:205–220.

Musgrave, Alan1985 “Realism versus constructive em-

piricism.” In Paul Churchland andClifford Hooker, (eds.), Images of Sci-ence: Essays on Realism and Empiri-cism. Chicago: University of ChicagoPress.

Pierson, Paul2000 “Not just what, but when: Issues of

timing and sequence in political pro-cesses.” Studies in American PoliticalDevelopment.

Ragin, Charles C.1987 The Comparative Method: Moving

Beyond Qualitative and QuantitativeStrategies. Berkeley: University ofCalifornia Press.

Rueschemeyer, Dietrich, Evelyne HuberStephens, and John D. Stephens1992 Capitalist Development and Demo-

cracy. Chicago: University of Chicago.Rueschemeyer, Dietrich, and John D.Stephens1997 “Comparing historical sequences – a

powerful tool for causal analysis.”

Comparative Social Research 17:55–72.

Schelling, Thomas C.1998 “Social mechanisms and social dyna-

mics.” In Peter Hedstrom and RichardSwedberg, (eds.), Social Mechanisms:An Analytical Approach to SocialTheory: 32–44. New York: CambridgeUniversity Press.

Somers, Margaret R.1998 “We’re no angels: Realism, rational

choice, and relationality in social sci-ence.” American Journal of Sociology104:722–784.

Sørensen, Aage B.1998 “Theoretical mechanisms and the

empirical study of social pro-cesses.” In Peter Hedstrom andRichard Swedberg, (eds.), SocialMechanisms: An Analytical Ap-proach to Social Theory: 238–266.New York: Cambridge UniversityPress.

Steinmetz, George1998 “Critical realism and historical sociol-

ogy.” Comparative Studies in Societyand History 40:170–186.

Stinchcombe, Arthur L.1993 “The conditions of fruitfulness of the-

orizing about mechanisms in socialscience.” In Aage B. Sørensen andSeymour Spilerman, (eds.), Social The-ory and Social Policy: Essays in Honorof James S. Coleman: 23–41. Westport,CT: Praeger.

Stinchcombe, Arthur L.1998 “Monopolistic competition as a

mechanism: Corporations, universi-ties, and nation-states in competi-tive fields.” In Peter Hedstrom andRichard Swedberg, (eds.), SocialMechanisms: An Analytical Ap-proach to Social Theory: 267–305.New York: Cambridge UniversityPress.

Tilly, CharlesIn press “Historical analysis of politicalprocesses.” In Handbook of SociologicalTheory, edited by Jonathan H. Turner. NewYork: Plenum.

Review Essay: Beyond Correlational Analysis: Recent Innovations in Theory and Method

Documents

Transcript of Review Essay: Beyond Correlational Analysis: Recent Innovations in Theory and Method