DISMANTLING THE QUANTITATIVE – QUALITATIVE DIVIDE

Comments On Hypothesis Testing, Induction, Statistics, Fiction And Epistemological Anarchy

Presentation at the 3rd International Conference on Interdisciplinary Social Sciences, Prato, Italy,

July 2008

Michael Wood (michael.wood@port.ac.uk) and Christine WelchPortsmouth University Business School, UK

There is a revised draft paper at http://userweb.port.ac.uk/~woodm/QualQuant.pdf

and this presentation is at http://userweb.port.ac.uk/~woodm/QualQuant.ppt

My background …

We suspect this presentation may be a bit of a mess because

it’s largely about things which don’t make too much sense. So

we’ll try and impose a clear framework …

What conceptual framework is best for understanding research?

• Perspective 1 or

• Perspective 2or

• Perspective 3or

• Perspective 4

Which do you think will be the winner?

Perspective 1: Two kinds of research

• Positivism– Generalisation through

statistics– Research progresses

through hypotheses and deductions

– Observer independent– Large random samples– Etc, etc

• Social Constructionism– Generalisation through

theoretical abstraction– Research progresses

through rich data and induction

– Observer part of study– Small purposive samples– Etc, etc

Based loosely on Easterby-Smith et al (2002: 30). They list 8 dimensions. Other authors may use different labels for the two types: commonest probably Quantitative vs Qualitative.

The two kinds of research?

• Quantitative / positivist / deductive– Hard and spiky

• Qualitative / phenomenological / social constructivist / interpretivist / etc – Soft and cuddly

Often suggested that researchers need to choose one or the other.

Which side are you on?

• To hard and spiky people, soft and cuddly research is lacking in rigour

• To soft and cuddly people, hard and spiky research is superficial and lacking in richness and relevance

… but is this a genuine dichotomy?

No …

Hypothesis testing Inductive

Statistical E.g. 1: Glebbeek & Bax (2004)

E.g. 4: Moutafi et al (2007) http://tinyurl.com/2qb6p9

“Qualitative” / Non-statistical

E.g. 3: Meyer & Altenborg (2007)

E.g. 2: Britten et al (2000)

Perspective 2: More than two types of research: e.g. ….

Easterby-Smith et al (2002) list 8 bipolar dimensions: this leads to the possibility of 28 or 64 types of research.

Glebbeek and Bax (2004)

To test the hypothesis “that employee turnover and firm performance have an inverted U-shaped relationship: overly high or low turnover is harmful.”

They analysed the performance of “110 offices of a temporary employment agency” by the statistical technique of regression, and did in fact find the hypothesized curvilinear relationship.

Britten et al (2000)

To identify categories of “misunderstandings in prescribing decisions in general practice”.

They used a series of interviews with patients and doctors to identify fourteen categories of “misunderstandings”. These categories emerged from the data and were viewed as illustrating important possibilities with implications for the training of doctors. However, no attempt was made to estimate how common each category was – just the fact that they had occurred, and so it could reasonably be assumed that they might occur again, was sufficient

Meyer and Altenborg (2007)

To investigate the impact that a “spirit of equality or balance” has on the merger process.

The authors used a case study of a failed international merger. They found that “the principle of equality had the reverse effect on social integration to that predicted in the literature … negatively influencing social integration.”

Moutafi et al (2007)

To investigate the relationship between personality and managerial level.

They got 900 participants to complete two personality tests and looked at the relationship between the results and their managerial level. There were four hypotheses – the first of which was that “conscientiousness should be positively correlated with level of management” – but these hypotheses were not mentioned in the abstract, which simply listed personality traits which were found to correlate – positively or negatively – with managerial level.

What does qualitative mean?

• General term from right hand side of Perspective 1 ………?

• Opposite of statistical ……… ?

• Focussed on qualities not quantities … ?

• Detailed information / in depth analysis (eg not superficial questionnaire) …… OK

But …

• Why not use different approaches in one project?– E.g. Glebbeek and Bax (2004) found statistical

evidence for the hypothesised inverted U shaped relationship between staff turnover and firm performance, but why not back this up with case studies to look at possible reasons for this effect?

– Britten et al (2000) identified categories of misunderstanding between doctors and patients, but why not do a statistical survey to see how common each category is?

• So …

Perspective 3: Multimethodology or Mixed methods

Use different approaches in one project …

But …

• There are problems with many of the bipolar dimensions used to pigeonhole research– Concepts used may be vague, ambiguous or

prone to misinterpretation– May not be bipolar: reasonable approaches

may be omitted

Two bipolar dimensions?

Statistical “Qualitative”

Hypothetico-deductive “Inductive”

“ …” indicates vague terms we don’t like

Or … do these fit on the same dimension?

Statistical “Qualitative”X

Empirical possibilities

Deterministic laws

Fictional possibilities

??

Is this one dimension?

Extended statistical – non-statistical dimension

• Deterministic laws – what always happens

• Statistics – what sometimes happens

• Illustrative inference – what has happened at least once (some “qualitative research”)

• Fiction – what is possible / imaginable

And what about this … ?

Hypothetico-deductive Inductive

Deductive (e.g. applying a model)

Using a framework or paradigmto define questions

Much research is neither hypothetico-deductive nor inductive. There is no obvious linear dimension here, which is why we’ve made the layout of this slide a bit of a mess.So …

X

Fourth perspective

• Avoid unhelpful concepts. If in doubt, shut up, or use more straightforward language!– Induction?– Qualitative?– etc

• No useful general categorisation schemes for research (=only useful grand narrative)– If you stick to such a scheme you risk ignoring useful

possibilities– Epistemological anarchy. (= Postmodernism?)

• Feyerabend: “anything goes”

• But some concepts are worth careful thought …

Statistical methods

• Important approach– Over-estimated by proponents, underestimated by

opponents• Focus on null hypothesis tests is usually stupid

– Better to measure size of effect– E.g. Glebbeek and Bax (2002)– Statistics = formal methods for doing induction

• Sampling / context needs care– E.g. Glebbeek and Bax (2002)

• “Qualitative” data often analysed statistically– Should be done properly

Following a framework or paradigm

• Neither hypothesis testing nor (pure) induction

• Kuhn’s normal science

• Obviously a good idea but not in the standard menu of approaches

• More helpful concept than induction because focuses attention on the framework and presuppositions

Hypotheses

• Guiding vs formal• Formal hypotheses are tested. They may be

– Null• Sadly, statistical null Hs tend to dominate idea of

hypotheses. – Non-null. Popper’s bold conjectures.

• Require imagination. Not boring!– Rigour is in the testing process

• Guiding hypotheses are explored • Not restricted to statistical approaches. In fact

hypotheses are usually best avoided with statistics

Fictional “data”

• Made up data may be more convenient– E.g. confidentiality problems– Thought experiments

• Fictions, fables, utopias, dystopias to explore …• Widely used in mathematical modelling

– Interesting possibilities fabricated to play out what-ifs• Sometimes what is possible may be more

interesting than what has actually happened– E.g. if we are interested in improving things– Dogmatic empiricism may be unreasonable? If we

want to change things why focus exclusively on facts?

Does all this matter?We think so because …

• Avoids impoverishing research by adhering to very restricted perspectives

• Suggests new possibilities• Avoids wasting time talking rubbish• Fitting methods to the enquiry is important (e.g. check

the two CRITIC acronyms in http://userweb.port.ac.uk/~woodm/rm/rm.ppt)

• Fitting them to your favourite paradigm is not!• All comments and suggestions welcome. These slides

and a revised draft paper are at– http://userweb.port.ac.uk/~woodm/QualQuant.ppt – http://userweb.port.ac.uk/~woodm/QualQuant.pdf