The “Third Way?” How Computer Simulation Could Unite Qualitative and Quantitative Research to...

http://www.simian.ac.uk

Edmund Chattoe-Brown ([email protected])Department of Sociology, University of Leicester

The ‘Third Way?’ How Computer Simulation Could

Unite Qualitative and Quantitative Research to

Produce ‘Better’ Sociology

http://www.simian.ac.uk/

2 http://www.simian.ac.uk

Thanks

• This research funded by the Economic and Social Research Council as part of the National Centre for Research Methods (http://www.ncrm.ac.uk).

• The usual disclaimer applies regarding Nigel Gilbert (co-PI SIMIAN, Sociology, Surrey), Neil Hunt (independent researcher/University of Kent and Andrew Preston (Exchange Supplies).



Plan of the talk

• Will sociology now just carry on the same forever?

• Understanding behaviour, “methods lenses” and the micro-macro link.

• Ramifications of simulation as a radical innovation (if it turns out to be).

• A simple example: Schelling and segregation.• A substantive example: BBVsim.



Will sociology just carry on?

• Sociologists give quite a lot of attention to the public role of sociology. (What is the public role of plumbing?)

• They don’t seem to give much attention to the role of radical innovation (or incorporation) in sustaining the discipline.

• What was the last sociological innovation producing a major empirical research programme?

• Is sociology “exhausted” at present?• You can’t generalise without simplifying! (But data

can be got.)



The struggle for solid ground• It isn’t even clear that all sociologists agree that

the task of social science is to discover empirical regularities.

• Certainly they don’t agree about how such regularities should be discovered and what they are (attributes versus practices, statistics versus ethnography).

• Can these debates be resolved “from within” their respective positions?

• If not, is there a “from without” we can take advantage of? Maybe …



The Schelling segregation model• I’m not presenting this because it represents social reality but because it

is easy to explain and gives a good point of access to important issues.

• “Agents” live on a square grid so each has maximum 8 neighbours.

• There are two “types” of agents (red and green) and some grid spaces are vacant. Initially agents/vacancies distributed randomly.

• All agents decide what to do in the same very simple way.

• Each agent has a preferred proportion (PP) of neighbours of its own kind (0.5 PP means you want at least half your neighbours to be your own kind - but you would accept all of them i. e. PP is minimum.) Vacant grid spaces “don’t count” which is why the PP is a fraction not a number.

• If an agent is in a position that satisfies its PP then it does nothing otherwise it moves to a vacancy chosen at random.



Disclaimer

• Simulation is distinctive in representing social theories as computer programmes rather than narratives or equation systems.

• However I don’t need to explain to you exactly how these programmes work or how one writes them any more than a statistician has to know how to derive the formulae for regression or understand how a PC runs SPSS.

• But if you do want to know …


“Real” Quants (Equations)

• “The most important empirical findings of this study can be summarized as follows:

• Contrary to Hypothesis 1, there is a moderate tendency for individuals with higher service class origins to be more likely than others to enrol in PhD programmes.

• …

• The estimated effect of class drops to zero when controlling for parents’ education and employment in research or higher education.

• The overall implication of these findings is that the transition from graduate to doctoral studies is influenced by social origins to a considerable degree. Thus, the notion that such effects disappear at transitions at higher educational levels - due either to changes over the life course or to differential social selection - is not supported.” (Mastekaasa, Acta Sociologica, 2006, 49(4), pp. 448-449.)


“Real” Quals (Narrative)

• “Turkish interviewees do not include themselves when they are evaluating the status of ‘Turkish women’ in general. While referring to ‘Turkish women’, most Turkish interviewees use the pronoun ‘they’:

• Turkish women are more home-oriented. I think that they are left in the backstage because they do not have education, because they are not given equal opportunities with men. (T3)

• One of the Turkish interviewees stated that it was difficult for her to answer the questions related to her status ‘as a woman’, because:

• I don’t think of myself as a Turkish women, but as a Turkish person. I mean I never think about what kind of role I have in the society as a woman. (T1)

• Most Norwegian interviewees, on the other hand, identify with ‘Norwegian women’ in general, and they refer to ‘Norwegian women’ as ‘we’:

• I think that in a way Norwegian women, that is we, at least have our rights on paper. We have equal rights for education and we have good welfare arrangements … (N1)” (Sümer, Acta Sociologica, 1998, 41(1), p. 122)


The “Third Way”


Initial random state



Clustering

Schelling (early seventies) was interested in

urban residential

segregation of ethnic groups.



Two questions

• What is the smallest PP (i. e. a number between 0 and 1) that will produce clusters?

• What happens when the PP is 1?



Answers

• About 0.3.• No clusters form.• Challenge: Had you “seen” the cluster data

generated by PP=0.3, might you (if you were of a particular political or sociological persuasion) have attributed xenophobia/racism to the system? (Or preference?)



Why? So what?

• Because PP is a minimum, people are always happy “inside” a cluster of their own kind.

• If a cluster is “full” (no internal vacancies) then it cannot be “disrupted” except at the edges.

• Whether clusters form thus depends on whether their shape is compatible with the PP for “edge agents”. (No “sharp corners” possible: Minimum cluster size?)

• When PP is 1, no shape of the cluster edge is compatible with the satisfaction of edge agents so the cluster cannot form.

• An aggregate entity (the cluster) thus becomes a structuring principle for individual behaviour. Sound familiar?



Simple individuals/complex systemIndividual Desires and Collective Outcomes

-20

0

20

40

60

80

100

120

0 50 100 150

% Similar Wanted (Individual)

% Similar Achieved (Social)

% similar% unhappy

Counter-intuitive macro (social) results from simple micro interactions. A non-linear (and complex) system.



The punch line

• Simulation is a “macroscope” because it allows us to “see” complexity somewhat like a microscope allows us to see very small things.

• Even in this trivially simple (and behaviourally suspect) system, the link between micro and macro is not at all intuitive.

• We can now see why trying to “gross up” interactions and practices or “drill down” from aggregate regularities might be problematic.

• The solution offered by simulation is an explicit process specification. Compare this with the causal status of a regression model.


You said “uniting” methods …• In this framework, research methods can do almost (but not quite) what they always did. Simulation won’t “take over” and the worst work in simulation comes from avoiding data.

• Ethnography becomes more purposive because it really builds theory (rather than typology or campfire tales).

• Statistics does not compare models with data but real data with “simulated data”: How often people move house, how big clusters are, what shape and so on.

• What we want to know: How people see neighbourhoods, how (and where) they decide to move, what opportunities and constraints exist …

• Simulation combines (a good measure of) the rigour of statistics with (a good measure of) the richness of qualitative data. However, it can explore generalisability through experiments and largely does not have “built in” assumptions.



Aside 1

• The Schelling model isn’t just too basic or “wrong”, it isn’t even the right place to start because it was “dreamed up in an arm chair”.

• The “first” version of any simulation has to reproduce the stylised facts regarding whatever it is supposed to be “about” (social mobility, rioting, attitudes) and organise itself around the major (grounded) knowledge in that area.

• This is interesting because existing research methods don’t need to “digest” the past history of whole research areas systematically. How should this be done? A blessing and a curse of novelty.



Aside 2

• Notice that with the possible exception of PP (which is a reasonable candidate for measurement by a psychological scale) the Schelling model contains no arbitrary parameters and no “theory constructs” (like alienation or cognitive dissonance).

• Instead, it describes sequences of behaviours (including cognitive behaviours) in naturalistic language.

• There are no “technical” limits yet discovered on the richness of these sequences.



Methodology 1 (G+T)



Methodology 2

• “Version 0” model tries to synthesise what is known and “output” what is quantified (for example attitude shifts over time, the size distribution of strikes or the pattern of residential clustering).

• We must attend to the richness of the domain behaviour as a rationale for simulation. If all that needs to be explained is a straight line then nothing more complex than a regression model with a couple of variables can be justified.


Methodology Aside

• Another problem with Schelling: How many discernible states does the system have?

• What is similarity here? Mirror image, same cluster size distribution, same “topology” (nested or separated clusters for three types), just “there are clusters”.

• Last is already too weak: Significantly different models can already generate it (Mark Knopfler: “Two men say they’re Jesus, one of them must be wrong.”)


Methodology 3

• Systematic review and identification of research question.

• Construction of version 0 model and exploration of properties.

• Identification of missing knowledge in clear (but not necessarily definitive) form.

• Collection of data for version 1 (and iterate).• Significant current problems are cultural and

methodological: Very few sympathetic ethnographers, systematic reviewing techniques still in development, simulation itself still evolving fast.


Substantive example: BBVsim• Still not the “gold standard”: Expert knowledge (Neil Hunt and Andrew Preston) used as a proxy for systematic review. Aim of simulation to “teach” complexity and not (yet) to predict the future. (But it “got” a version 0 built.)

• Research question: What effects does needle sharing behaviour have on the prevalence of BBVs like hepatitis and HIV?

• Interesting mixture of social practices, bureaucracy (needle distribution versus exchange) and technology (coloured needles).

• Harm reduction paradigm.


BBVsim view


Examples 1: Heuristic use

• After the fact, this is a model about needle flow. The experts were very surprised to be asked about how many needles and what kind were “held” by typical users but this data isn’t “rocket science”.

• Even using ethnographic data, building a process model made it clear that there were significant details we didn’t know: Is the selection of needles for sharing “strategic?” What is the full rank ordering of sharing and needle status?

• Incoherent “NICE” definition of coverage.


Examples 2: “Qualitative Policy”• Mickawber Effect: As long as you have “enough” needles, there’s nothing to see. If you don’t, at some uncertain point in the future, things will start to go astray. (“For want of a needle …”)

• Is HEPC infection a leading indicator for HIV?• Models themselves may serve as knowledge

elicitation tools. It is possible that a “version 0.5” may arise from this being web available to practitioners and policy makers.

• Structured motivation to get more data (and through sensitivity analysis) to establish the “most important” data.


“Playing with” policy


Lessons learnt

• There’s an awful lot we don’t know.

• With no disrespect, expertise has to be treated with some caution!

• There are some interesting ethical issues about the lessons people should draw from “teaching” models. Things can be too easy to use and too definite.

• There’s no free lunch. Interviewing experts is quicker than a systematic literature review but they can’t decide if N=1 or 10 and need to be forced to declare their beliefs about outcomes “up front”.

• Always more you can add but does it make a difference and is there (or likely to be) data?


Conclusions

• This is something significantly different.

• It isn’t going to “take over” sociology or supersede existing research methods but gives us quite a new place to stand and, as evidence, quite a different take on important issues like mixing methods (and developing new ones like systematic reviewing), structuration, potential sources of social regularity and so on.

• Now I have to find a way of doing the “whole methodology” for a substantive area.

• If simulation is not the next big thing, do we need one rather badly anyway? Anyone for ologyology?


Now what?• Simulation Innovation, A Node (Part of NCRM: research,

training and advice): <http://www.simian.ac.uk>.• NetLogo (software used here, free, works on Mac/PC/Unix,

with a nice library of examples): <http://ccl.northwestern.edu/netlogo/>.

• Simulation for the Social Scientist, 2nd edition, 2005, Gilbert/Troitzsch. [Don’t get first edition, not in NL!]

• Agent-Based Models, 2007, Gilbert.• Journal of Artificial Societies and Social Simulation

(JASSS): <http://jasss.soc.surrey.ac.uk/JASSS.html>. [Free online and peer reviewed.]

• simsoc (email discussion group for the social simulation community): <https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=SIMSOC>.


The “Third Way?” How Computer Simulation Could Unite Qualitative and Quantitative Research to...

Education

Transcript of The “Third Way?” How Computer Simulation Could Unite Qualitative and Quantitative Research to...