Post on 05-Feb-2016
description
DISMANTLING THE QUANTITATIVE – QUALITATIVE DIVIDE
Comments On Hypothesis Testing, Induction, Statistics, Fiction And Epistemological Anarchy
Presentation at the 3rd International Conference on Interdisciplinary Social Sciences, Prato, Italy,
July 2008
Michael Wood (michael.wood@port.ac.uk) and Christine WelchPortsmouth University Business School, UK
There is a revised draft paper at http://userweb.port.ac.uk/~woodm/QualQuant.pdf
and this presentation is at http://userweb.port.ac.uk/~woodm/QualQuant.ppt
My background …
We suspect this presentation may be a bit of a mess because
it’s largely about things which don’t make too much sense. So
we’ll try and impose a clear framework …
What conceptual framework is best for understanding research?
• Perspective 1 or
• Perspective 2or
• Perspective 3or
• Perspective 4
Which do you think will be the winner?
Perspective 1: Two kinds of research
• Positivism– Generalisation through
statistics– Research progresses
through hypotheses and deductions
– Observer independent– Large random samples– Etc, etc
• Social Constructionism– Generalisation through
theoretical abstraction– Research progresses
through rich data and induction
– Observer part of study– Small purposive samples– Etc, etc
Based loosely on Easterby-Smith et al (2002: 30). They list 8 dimensions. Other authors may use different labels for the two types: commonest probably Quantitative vs Qualitative.
The two kinds of research?
• Quantitative / positivist / deductive– Hard and spiky
• Qualitative / phenomenological / social constructivist / interpretivist / etc – Soft and cuddly
Often suggested that researchers need to choose one or the other.
Which side are you on?
• To hard and spiky people, soft and cuddly research is lacking in rigour
• To soft and cuddly people, hard and spiky research is superficial and lacking in richness and relevance
… but is this a genuine dichotomy?
No …
Hypothesis testing Inductive
Statistical E.g. 1: Glebbeek & Bax (2004)
E.g. 4: Moutafi et al (2007) http://tinyurl.com/2qb6p9
“Qualitative” / Non-statistical
E.g. 3: Meyer & Altenborg (2007)
E.g. 2: Britten et al (2000)
Perspective 2: More than two types of research: e.g. ….
Easterby-Smith et al (2002) list 8 bipolar dimensions: this leads to the possibility of 28 or 64 types of research.
Glebbeek and Bax (2004)
To test the hypothesis “that employee turnover and firm performance have an inverted U-shaped relationship: overly high or low turnover is harmful.”
They analysed the performance of “110 offices of a temporary employment agency” by the statistical technique of regression, and did in fact find the hypothesized curvilinear relationship.
Britten et al (2000)
To identify categories of “misunderstandings in prescribing decisions in general practice”.
They used a series of interviews with patients and doctors to identify fourteen categories of “misunderstandings”. These categories emerged from the data and were viewed as illustrating important possibilities with implications for the training of doctors. However, no attempt was made to estimate how common each category was – just the fact that they had occurred, and so it could reasonably be assumed that they might occur again, was sufficient
Meyer and Altenborg (2007)
To investigate the impact that a “spirit of equality or balance” has on the merger process.
The authors used a case study of a failed international merger. They found that “the principle of equality had the reverse effect on social integration to that predicted in the literature … negatively influencing social integration.”
Moutafi et al (2007)
To investigate the relationship between personality and managerial level.
They got 900 participants to complete two personality tests and looked at the relationship between the results and their managerial level. There were four hypotheses – the first of which was that “conscientiousness should be positively correlated with level of management” – but these hypotheses were not mentioned in the abstract, which simply listed personality traits which were found to correlate – positively or negatively – with managerial level.
What does qualitative mean?
• General term from right hand side of Perspective 1 ………?
• Opposite of statistical ……… ?
• Focussed on qualities not quantities … ?
• Detailed information / in depth analysis (eg not superficial questionnaire) …… OK
But …
• Why not use different approaches in one project?– E.g. Glebbeek and Bax (2004) found statistical
evidence for the hypothesised inverted U shaped relationship between staff turnover and firm performance, but why not back this up with case studies to look at possible reasons for this effect?
– Britten et al (2000) identified categories of misunderstanding between doctors and patients, but why not do a statistical survey to see how common each category is?
• So …
Perspective 3: Multimethodology or Mixed methods
Use different approaches in one project …
But …
• There are problems with many of the bipolar dimensions used to pigeonhole research– Concepts used may be vague, ambiguous or
prone to misinterpretation– May not be bipolar: reasonable approaches
may be omitted
Two bipolar dimensions?
Statistical “Qualitative”
Hypothetico-deductive “Inductive”
“ …” indicates vague terms we don’t like
Or … do these fit on the same dimension?
Statistical “Qualitative”X
Empirical possibilities
Deterministic laws
Fictional possibilities
??
Is this one dimension?
Extended statistical – non-statistical dimension
• Deterministic laws – what always happens
• Statistics – what sometimes happens
• Illustrative inference – what has happened at least once (some “qualitative research”)
• Fiction – what is possible / imaginable
And what about this … ?
Hypothetico-deductive Inductive
Deductive (e.g. applying a model)
Using a framework or paradigmto define questions
Much research is neither hypothetico-deductive nor inductive. There is no obvious linear dimension here, which is why we’ve made the layout of this slide a bit of a mess.So …
X
Fourth perspective
• Avoid unhelpful concepts. If in doubt, shut up, or use more straightforward language!– Induction?– Qualitative?– etc
• No useful general categorisation schemes for research (=only useful grand narrative)– If you stick to such a scheme you risk ignoring useful
possibilities– Epistemological anarchy. (= Postmodernism?)
• Feyerabend: “anything goes”
• But some concepts are worth careful thought …
Statistical methods
• Important approach– Over-estimated by proponents, underestimated by
opponents• Focus on null hypothesis tests is usually stupid
– Better to measure size of effect– E.g. Glebbeek and Bax (2002)– Statistics = formal methods for doing induction
• Sampling / context needs care– E.g. Glebbeek and Bax (2002)
• “Qualitative” data often analysed statistically– Should be done properly
Following a framework or paradigm
• Neither hypothesis testing nor (pure) induction
• Kuhn’s normal science
• Obviously a good idea but not in the standard menu of approaches
• More helpful concept than induction because focuses attention on the framework and presuppositions
Hypotheses
• Guiding vs formal• Formal hypotheses are tested. They may be
– Null• Sadly, statistical null Hs tend to dominate idea of
hypotheses. – Non-null. Popper’s bold conjectures.
• Require imagination. Not boring!– Rigour is in the testing process
• Guiding hypotheses are explored • Not restricted to statistical approaches. In fact
hypotheses are usually best avoided with statistics
Fictional “data”
• Made up data may be more convenient– E.g. confidentiality problems– Thought experiments
• Fictions, fables, utopias, dystopias to explore …• Widely used in mathematical modelling
– Interesting possibilities fabricated to play out what-ifs• Sometimes what is possible may be more
interesting than what has actually happened– E.g. if we are interested in improving things– Dogmatic empiricism may be unreasonable? If we
want to change things why focus exclusively on facts?
Does all this matter?We think so because …
• Avoids impoverishing research by adhering to very restricted perspectives
• Suggests new possibilities• Avoids wasting time talking rubbish• Fitting methods to the enquiry is important (e.g. check
the two CRITIC acronyms in http://userweb.port.ac.uk/~woodm/rm/rm.ppt)
• Fitting them to your favourite paradigm is not!• All comments and suggestions welcome. These slides
and a revised draft paper are at– http://userweb.port.ac.uk/~woodm/QualQuant.ppt – http://userweb.port.ac.uk/~woodm/QualQuant.pdf