Why research (and researchers) won’t always tell you the truth

• They make mistakes

• They talk to the wrong people (e.g. they can’t talk to people without landlines on CATI surveys)

• They confuse cause with effect (A and B regularly happen together – therefore A causes B)

• They are sure of the conclusion before the research – & design the research (or fudge the data) to prove it

• They are frightened of the wrong conclusions

• They have reasons to lie (e.g. ties to funding organisations)

Why researchers sometimes go wrong

3

Questionable research practices are common

John et al. (2011) Prevalence of Questionable Research Practices. Psychological Science 23(5) 524–532

Why don’t people always tell the truth?

• They want to be liked

• They are boasting (especially on social media!)

• They don’t want to feel stupid

• They hope to win something

• They get bored

• They can’t be bothered to think

• Or perhaps they do tell the truth but they are not a representative sample

• Social media participants and survey respondents are often “people with something to say”

How not to tell the truth in surveys

Sir Humphrey Appleby knew all about leading questions! http://www.youtube.com/watch?v=G0ZZJXw4MTA

Selecting the data

• What is the “best” set of data to fit my thesis

• Are there “outliers” I can ignore – or include

7

Selecting the axes

Data visualisation – the wrong chart

Pie charts can easily cause problems

And infographics can be very difficult to get right (area of B on the left is 9x that of B on the right)

Selecting what you report

mean = £150,000; median = £35,000; mode = £10,000

One celeb earning £4,464,000

Five professionals making £150,000

14 managers making £50,000

1 teacher earning £35,000

20 other people only making £10,000

What is the average income in my street?

10

Conclusions

Handle data with careBusiness is an art, not a science!

jeremy@mosoco.co.uk