20 January 2010

Internet polling: a more hesitant approach

Andrew Cooper,Founder, Populus

|

Choosing not to be an online pollster (2003)

NCRM seminar 2

|NCRM seminar 3

ALL MEN WOMEN AB C1 C2 DE 18-24

25-34

35-44

45-54

55-64

65+

59% 67% 52% 70% 72% 57% 31% 75% 80% 70% 71% 41% 22%

1. Low (and unrepresentative) internet penetration

Source: Ofcom, 2003

|

2. Panel issues• PROFESSIONAL PANELLISTS

Panels dominated by small minority of all internet-users, most of whom are members of several panels, raising further concerns about data quality and representativeness (research suggests that people who are members of several panels are relatively unrepresentative of all internet users – more likely to be female and lower socio-economic status).

• ‘HAPPY CLICKING’Absence of a human interviewer means there is no sanction for giving random responses to complete surveys as quickly as possible – and the primary motivation for respondents is to get the financial reward.

• PANEL CONDITIONING Respondents are polled multiple times – becoming more and more atypical.

4NCRM seminar

|NCRM seminar 5

EDD Group in the EP& U.K. Independence PartyPress DigestWednesday 10th July 2002UKIP & YouGov - Urgent action required by members/supportersDear Member/Supporter,I would recommend that all members visit the www.yougov.com website, and register to participate in polls. YouGov suffer from a shortage of persons in the older age groups (55+) on low incomes, and consequently are forced to weight the polling data from those groups accordingly. I would urge all members/supporters to register as retired persons on low incomes, former Labour voters who are now undecided. Do not mention UKIP! The result will be a disproportionate response in favour of UKIP (or 'Other Party' if they continue to miss us out).

3. Panel infiltration

|

4. Political sample bias

6NCRM seminar

PAST VOTE YouGov Populus 2001 resultLabour 58% 47% 42%

Conservative 28% 30% 33%Lib Dem 11% 16% 19%

Others 4% 7% 6%DNV 17% 31% 41%

Source: Populus & YouGov data tables, November 2004

|

5. Under-representing the don’t knows & the disaffected (and over-representing the informed & interested)

7NCRM seminar

Source: Populus & YouGov data tables, December 2007 & January 2010

YouGov Populus Dec-07 Jan-10 Dec-07 Jan-10WOULD NOT VOTE 7% 7% 11% 17%

DON'T KNOW 16% 13% 18% 10%REFUSE 0% 0% 7% 5%TOTAL 23% 20% 36% 32%

|

6. The theological questions

What exactly can internet poll data be said to be representative of?If you get the answer right, does it matter how you got to it?

“Respondents for this survey were selected from among those who have agreed to participate in Harris Interactive surveys. The data have been weighted to reflect the composition of the adult population. Because the sample is based on those who agreed to participate in the Harris

Interactive panel, no estimates of theoretical sampling error can be calculated.”

8NCRM seminar

|

Looking again at online polling (2010)

NCRM seminar 9

|

1. Internet penetration has improved.Nielsen estimates 80% of UK adults now have internet access (up from 64% in 2007 and 60% in 2005)

2. Best practice deals with ‘happy clickers’ & professional panellists.Consistency checks, time stamping, digital fingerprinting & data cleaning

3. Larger panels reduce over-sampling, panel-conditioning & onerous weighting factors.Many panels now have hundreds of thousands of members, rather than tens of thousands

4. ‘River sampling’ offers the prospect of moving away from panels altogether, enabling quasi-random online polls and reducing the infrastructural cost of panel maintenance.

10NCRM seminar

|

5. Fixed line telephony is in decline.87% of households have a fixed-line telephone, down from 90% in 2007 & 93% in 2005.In the US, in-home internet penetration has just overtaken fixed-line penetration.

6. 13% now have only a mobile phone, not a landline.26% of 15-24 year-olds, 21% of 25-34 year-olds and 22% of DEs

7. Market forces, technological change and cultural trends are making telephone polling more expensive.

8. YouGov has shown that the challenges of gauging voting intention by internet polling can be overcome!

11NCRM seminar

|

But... challenges remain

12NCRM seminar

Recalled past vote Telephone Online Actual resultLabour 44.7% 42.6% 36.2%

Conservative 31.8% 31.6% 33.2%Liberal Democrat 16.4% 18.8% 22.7%

Others 7.1% 7.0% 7.9%

Vote intention Telephone Online v.1 Online v.2Labour 28% 23% 25%

Conservative 40% 46% 45%Liberal Democrat 18% 18% 16%

Others 13% 13% 14%Net lead 12% 23% 20%

Source: Populus data tables, October 2009

... as do the theological questions