And now for something completely different, or is it? Modelling contextuality and heterogeneity Or...
-
Upload
egbert-fowler -
Category
Documents
-
view
219 -
download
2
Transcript of And now for something completely different, or is it? Modelling contextuality and heterogeneity Or...
And now for something completely different, or is it?
• Modelling contextuality and heterogeneity
• Or Realistically complex modelling
• Lemma@bristol (http://www.ncrm.ac.uk/nodes/lemma/about.php)
Realistically complex modelling• ‘Statistical models as a formal framework of analysis with
a complexity of structure that matches the system being studied’
OUTLINE- Quantitative social science: much misunderstood?- What sort of questions does modelling answer- Complexity?- Diagrams representing complexity - What sort of questions does multilevel modelling answer?- Workshop: further set of graphical tools for thinking about
relationships- Thursday morning MLWIN and WINBUGS
Quantitative social science: much misunderstood?
• Quantification taught/developed/defended within particular version of “how to do research” which was oversold
• Quantification = testing/finding regularity = positivism = denigrated
• Quantification = technique = testing + correlation (not model-based) = superficial description not understanding
• Quantification = reductionist & over-simplifying & de-humanizing
• Many never get past entry level + bogged down in technique and/or getting very out-of-date view of capabilities of what (now) can be done
What some people think• According to Hamnett (2003), since the 1980s“ there has been a radical shift in the dominant methodology.
Quantitative techniques and aggregate social research have been largely abandoned, in favour of small scale, interpretative, qualitative, in-depth methodologies. Analysis of large data sets has become totally passé, the object of suspicion or even derision as ‘empiricist”
• Oakley (1989) aware critique of quantitative methods which are
'are cited as instituting the hegemony of the researcher over the researched , and as reducing personal experience to the anonymity of mere numbers” (p170).
And more…..• Practising human geography (Cloke et al, 2004 “enumeration in human geography is best seen as a form
of thin description, capable of identifying certain characteristics and patterns of data, but incapable of describing or explicating the meaningful nature of life” (p.283)
.• On quantification
“we do not think that such developments are central to contemporary human geography. They undoubtedly generate useful ‘tools’ to be deployed on occasion but we do not see how what are basically technical exercises can be more than a small part of the larger whole”
• An endangered species!
But an another view…
• Critical realism (state- of the-art philosophy of social science (see After Postmodernism: An Introduction to Critical Realism Lopez and Potter)
• Reality is stratified: “ multilevel ontology”Outcomes = mechanisms + contextsso that there are no ‘universal’ laws in social science that are independent of the context in which they are embedded
• encourages both intensive (qualitative) and extensive (quantitative) empirical work, but rejects the positivist position that causation equates with regularity
Moreover ……• Evidence–based approach
Quantification = “degree of evidential support in conditions of considerable uncertainty” (eg Campbell collaboration (“what harms, what helps, based on what evidence?” intervention in the social, behavioural and educational arenas)
• Open to scrutiny “science = method with use of explicit, codified and public methods to generate and analyze data whose reliability can be assessed” (transparency and replicability of protocols)
• Huge UK investment in data collection eg (BHPS; cohort studies) and analysis
Why do we need to quantify?
• Dealing with a world where cause and effect relationships that are neither:
NECESSARY: the outcome occurs only if the causal factor has operated NOR
SUFFICIENT: the action of a causal factor always produces the outcome
• Results in non-deterministic relations, so we need:
Quantitative evidence of the size of effect;
Quantitative evidence of the size of the effect taken account of other possible causal factors
A quantitative assessment of uncertainty about size of effect
Exemplar for observational design: smoking & lung cancer
Lifestyle/ genetic/environmental?
Lung cancer cases who have never smoked (NN)
Lifetime heavy smokers without illness (NS)
YET men who smoke increase risk of death from lung cancer by 22fold (2,200 percent higher); 1 = 11mins
Evidence beyond reasonable doubt, led to public policy
Cumulative evidence and passive smoking…..
Example: of statistical modelling; discrimination in the workplace
1. What sort of data can be analysed?
2. What results can be obtained from statistical models?
3. What sort of relations can we find?
1 What sort of data can be analysed?
Responses Predictors Respondent Number
Salary (£k)
Promotion (2 category)
Promotion (3 category)
Number of rejections
Time to promotion (yrs)
Gender Ethnic Years of educat
Years of Service
1 2 3 4 5 6 7 8 9 10 11 500
32.4 40.1 65.2 32.1 21.6 25.4 32.7 51.7 44.0 32.6 41.7 39.7
No Yes Yes No No No No Yes Yes No Yes No
No Yes Yes No Not Not No Yes Yes No Yes No
1 0 0 2 4 3 1 0 0 1 0 2
6.2+ 3.2 2.9 8.2+ 6.7+ 4.2+ 5.1+ 3.9 4.2 3.9+ 4.9 5.2
Female Male Male Female Female Male Female Male Female Female Male Male
White Whites Asian Black UnknowBlack White White Asian Black White Unknow
<11 11-13 14-16 >16 11-13 <11 14-16 <11 14-16 14-16 11-13 14-16
9.1 6.2 4.9 8.2 6.7 4.2 5.1 4.8 7.2 3.9 9.7 8.1
2 What do we get from statistical models?• a quantitative assessment of the size of the effect, eg
difference in salary between Blacks and Whites is £5k per annum;
• a quantitative assessment after taking account of other variables; eg a Black worker earns £6.5k less after taking account of years of experience (conditioning)
• a measure of uncertainty for the size of effect eg 95% confident Black-White difference be found generally in the population from which our sample is drawn, lies between £4.4k to £5.5k
• Use modelling in a number of modes: as description: average salary for different ethnic groups?
as part of causal inferences: does being Black result in a lower salary?in predictive mode :‘what happens if” questions
3: What sort of relations can we find?
MULTILEVEL MODELS
AKA
• random-effects models,
• hierarchical models,
• variance-components models,
• random-coefficient models,
• mixed models
Three KEY Notions • Modelling contextuality: firms as contexts
– eg discrimination varies from firm to firm– eg discrimination varies differentially for employees of different
ages from firm to firm
• Modelling heterogeneity– standard regression models ‘averages’, ie the general
relationship– ML models variances– Eg between-firm AND between-employee, within-firm variation
• Modelling data with complex structure - series of structures that ML can handle routinely
Structures: UNIT DIAGRAMS
• 1: Hierarchical structures
a) Pupils nested within schools: modelling progress
More examples follow…...
Examples of strict hierarchy • Education• pupils (1) in schools (2)• pupils (1) in classes( 2) in schools (3)
• Surveys: 3 stage sampling• respondents (1) in neighbourhoods(2) in regions(3)
• Business• individuals(1) within teams(2) within organizations(3)
• Psychology• individuals(1) within family(1)• individuals(1) within twin sibling pair(2)
• Economics• employees(1) within firms(2)
• NB all are structures in the POPULATION (ie exist in reality)
1: Hierarchical structures (continued)
b) Repeated measures of voting behaviour at the UK general election
1: Hierarchical structures (continued)
c) Multivariate design for health-related behaviours
Extreme case of rotational designs
2: Non- Hierarchical structures
• Can represent reality by COMBINATIONS of different types of structures
• But can get complex so….
a) cross-classified structure
b) multiple membership with weights
CLASSIFICATION DIAGRAMS
a) 3-level hierarchical structure
b) cross-classified structure
CLASSIFICATION DIAGRAMS(cont)c) multiple membership structure
d) spatial structure
ALSPAC • All children born in Avon in 1990 followed longitudinally• Multiple attainment measures on a pupil • Pupils span 3 school-year cohorts (say 1996,1997,1998)• Pupils move between teachers,schools,neighbourhoods• Pupils progress potentially affected by their own
changing characteristics, the pupils around them, their current and past teachers, schools and neighbourhoods
occasions
Pupil TeacherSchool Cohort
Primary school
Area
IS SUCH COMPLEXITY NEEDED?• Complex models are reducible to
simpler models • Confounding of variation across
levels (eg primary and secondary school variation)
M. occasions
Pupil TeacherSchool Cohort
Primary school
Area
Summary• Multilevel models can handle social science research problems with
“realistic complexity”
• Complexity takes on two forms and two types• As ‘Structure’ ie dependencies
- naturally occurring dependenciesEg: pupils in schools ; measurements over time
- ‘imposed-by-design’ dependencies
Eg: multistage sample
• As ‘Missingness’ ie imbalance- naturally occurring imbalances
Eg: not answering in a panel study- ‘imposed-by-design’ imbalances
Eg: rotational questions
• Most (all?) social science research problems and designs are a combination of strict hierarchies, cross-classifications and multiple memberships
So what? • Substantive reasons: richer set of research questions
– To what extent are pupils affected by school context in addition to or in interaction with their individual characteristics?
– What proportion of the variability in achievement at aged 16 can be accounted for by primary school, secondary school and neighbourhood characteristics?
• Technical reasons:
– Individuals drawn from a particular ‘groupings’ can be expected to be more alike than a random sample
– Incorrect estimates of precision, standard errors, confidence limits and tests; increased risk of finding relationships and differences where none exists
Conclusions3 Substantive advantages
1 Modelling contextuality and heterogeneity
2 Micro AND macro models analysed simultaneously
-avoids ecological fallacy and atomistic fallacy
3 Social contexts maintained in the analysis; permits intensive, qualitative research on ‘interesting’ cases
“The complexity of the world is not ignored in the pursuit of a single universal equation, but the specific of people and places are retained in a model which still has a
capacity for generalisation”