Can online deliberation transform citizens?

Preliminary findings from an internet field experiment in the UK

Informing Public Policy

Friday 24th April 2009

Hisako Nomura, Research Associate

Institute for Political and Economic Governance

University of Manchester

The project

• One element of the Rediscovering the Civic research programme

• Funded by ESRC/CLG/NWIN• Joint project of University of Manchester (Institute for

Political and Economic Governance) and University of Southampton (Centre for Citizenship and Democracy

• Variety of experiments (RCTs, design experiments) on aspects of civil engagement (voting, recycling, lobbying, pledging etc)

http://www.civicbehaviour.org.uk/research/

Background on online deliberation• Attempts to encourage deliberation on public policy are often

local and small scale (e.g. citizen’s juries) • Information and communication technology (ICT) potentially

offers new possibilities for public engagement– Opening a new “public sphere” of the sort envisioned by Habermas

(1962, 1989)• For the public, a new way to engage with others to discuss

– Online discussion • For the public sector, a new way to extend their reach

– Online questionnaires– Discussion forums– Webchats, texting, e-mailing responses, ePetitions, Interactive voting,

opinion meters etc.

Online deliberation

• The internet allows for an opportunity to ‘scale-up’ - to involve large numbers without the costs of physically bringing people together

• It is often claimed to promote honesty, directness in political exchanges and contact with people from different backgrounds which is appropriate for deliberation exercise (See Dahlberg 2001, Witschge 2004)

• On the other hand, anonymous encounters online can exert normative group influence (Postmes, Spears, & Lea, 1998)

• In that case, it runs the risk that the online deliberation experience may be diluted or even counterproductive

Information only vs Deliberation

• There is an ongoing debate in the literature on deliberative democracy as to whether it is information alone or information plus deliberation that has an effect on judgements

• Goodin (2003) argues that internal reflection (or ‘reflective deliberation’) is enough –interaction between citizen is not the important factor

• Others from public opinion theory argue that exposure to disagreement (i.e. interaction with others) does indeed contribute to more deliberative opinion formation (Price, Cappella and Nir, 2002)

RCT on deliberation

• Social scientist have evaluated deliberative forums to see if participants change their views, knowledge and opinions (Drysek 2002, Fishkin 1997, Delli Carpini et al 2004, Price, Nir and Cappela 2006)

• Few randomised control trials testing for the effect of deliberation (exceptions are Farrar et al 2003, Iyengar 2005)

• Moreover, existing studies have been confined to small numbers –mini publics – of participants who are able to participate in a face-to-face deliberation.

• No experiments testing the deliberative potential of online discussion forums

Research Questions

• To what extent does online engagement amongst large numbers lead to shifts in policy preferences and knowledge?

• To what extent can online engagement amongst large numbers be termed deliberative?

• What is the value of citizens’ interaction compared to the provision of information alone?

The RCT design

• 6,000 participants from Ipsos-MORI survey panel using random quota sampling.

• Randomly allocated to 4 treatment and 2 control groups

• 2 x deliberation groups– Goup 1: youth anti-social behaviour (ASB) followed by community

cohesion (CC)

– Group2: community cohesion (CC) followed by youth anti-social behaviour (ASB)

– Access to discussion boards and background information

• 2x information-only groups (video clips and documents)

• 2 x control groups

• All participants completed three surveys– T1 (before experiment), T2 (after first issue), T3 (after second issue)

Control 1ASB-CC

N=825 /1001Attrition=18%

Control 1CC-ASBN=825

Attrition=18%

Survey 1

Full delib 2 CC-ASB

N=742 /1002Attrition=26%

Full delib 2 ASB-CCN=695

Attrition=31%

Info. 1CC-ASBN=663

Attrition=34%

Info. 2 ASB-CCN=642

Attrition=36%

Full delib 1CC-ASBN=714

Attrition=29%

Control 2 ASB-CCN=797

Attrition=21%

Survey 3 Anti-Social Behaviour (ASB) or Community Cohesion (CC) issues

Total sample size

N=6000

Info. 1ASB-CC

N=718/1002Attrition=28%

Info. 1CC-ASB

N=731 /1001Attrition=27%

Full delib 1ASB-CC

N=786 /1002Attrition=22%

Control 2CC-ASB

N=825/1001Attrition=18%

Survey 2 Anti-Social Behaviour (ASB) or Community Cohesion (CC) issues

Deliberation

Deliberation

Information Only

Information Only

CONSORT Flow Diagram

Hypotheses

• H1: An internet deliberative forum has an impact on participants’ views, knowledge and opinions.

• H2: Information alone has an impact on opinion change • H3: The order of the topic can make an impact on

opinion change. • H4: The impact of deliberation is greater than information

alone • H5: Deliberation results in more opinion shift in youth

anti-social behaviour than in community cohesion

The initial findings

• Just over 50% of the treatment groups took part in the online activities

• Completion of survey was much higher

– 77.5% who had anti-social behaviour topic completed T2 survey (73.2% of them completed community cohesion questionnaire in T3)

– 76.5% who had community cohesion topic completed T2 survey (71% of them completed anti-social behaviour questionnaire in T3)

• No obvious large-scale shifts in opinions.• Many of the threads read like ‘speak your brains’ rather than

exchange of opinions and considered reflection. – There was lack of responsiveness

Further analysis

• Application of propensity score matching of those who actually took part in online deliberation in the treated group with similar members of the control group for further analysis

• Application of adapted Discourse Quality Index for qualitative analysis