CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

19

description

Paper presented at the 2009 Canadian Political Science Association Conference

Transcript of CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Page 1: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…
Page 2: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Structure of the Presentation

1. State of Internet and Social Media Use;

2. Parameters and Objectives;

3. Contextualization;

4. Defining the Blogging Population;

5. Web-based Sampling Strategies;

6. Discussion.

Page 3: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

State of Social Media Use

• Evolution and intensifying use of Web 2.0 tools;• Reconfiguration of the political communication,

mobilization and persuasion environment in the last twenty (20) years:

Sources; Xenos and Foot, 2008; Kushin and Yamamoto, 2009; Foot and Schneider, 2006; Kushin, Kitchner et al., 2009; Bode, 2008.

Information: Production; “Coproduction”; Dissemination; Acquisition.

Social relations: Temporality; Directionality.

Public deliberation processes

Page 4: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

2004: 88% of Web users believed the Web was part of their “daily media routine”;

December 2008: 74% of citizens went online for various reasons55% for political

information and commentary;

38% for information dissemination and discussion

2007: 78% of citizens are active Internet users;

2008: “Important source for current affairs and political information for Canadians”.

December 2007: 74% of citizens are Internet users;

Low levels of Internet penetration:Underdeveloped

digital infrastructure;

Web content consumption behaviour.

2007: 28% used the Web for political information;

State of Social Media Use

• Rising importance of the Internet:

Sources; Small, 2008; Fallows, 2004; Madden, 2006; Jones and Fox, 2009; Smith, 2009; CEFRIO, 2008; ISQ, 2009; Clavet 2002;.

Page 5: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

State of Social Media Use

• Enduring presence of a digital divide in many Western countries;

• Impact of the upcoming “generational succession”: Growth of connectivity levels and Internet-

related knowledge; Diversification and potential intensification of

online politicking activities:

Sources; Schlozman, Verba et al., 2009; Delli Carpini, 2000; Hargittai and Hinnant, 2008; Bennett, Wells et al., 2008.

Technologies provide “attractive ways” to be politically-engaged;

Rising importance of informal political communication, mobilization and persuasion ways.

Page 6: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

State of Social Media Use

• “Meteoric” growth of social media platforms in the last five (5) years:

Blogs; Social networking “services” (SNS); Status updating (microblogging) tools.

• Production of an increasingly large body of scientific work:

Social relations and identity formation; Education. Politically-oriented activities

• Social media platforms still an “understudied” phenomenon.

Sources; Gaines and Mondak, 2008; Ellison, Steinfeld et al., 2006; Kampitaki, Tambouris et al., 2009; Caverlee and Webb, 2008.

Page 7: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Parameters and Objectives

• Few studies on methodological challenges of studying Web-based social media platforms;

• “Slow pace of academic publishing” not keeping up with constantly-evolving objects of study.

Sources; Foot and Schneider, 2004; Kankowski and Van Selm, 2008; Karfp, 2009; Boyd and Ellison, 2007; Ahn, Han et al., 2007.

Objectives of the presentation

1. Overview of the challenges of studying Web-based social media platforms;

2. In-depth characterization of several concerns linked to the constitution of representative research samples through non-probabilistic, or decentralized, strategies;

3. Advantages and pitfalls of viral sampling.

Page 8: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Parameters and Objectives

• Presentation based on previous work on Quebec-based political bloggers;

• Politics-oriented blogs: Increasing importance since 2001:

Dislike and discontent of conventional media; Decline of “institutionalized forms” of

political participation. Four (4) distinct functions on the political scene:

Potential influence on agenda-setting processes;

Source of original reporting; Arena of political discussion; Communication between citizens and

government.

Sources; Johnson and Kaye, 2004; Mariën, Hooghe et al., 2008, Sweetser and Kaid, 2008; Siapera,2008; Wallsten, 2007.

Page 9: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Contextualization

• Social media research situated within the broader Internet-research framework;

• Development of new investigation techniques adapted to the nature of the Web:

Multidimensional; Deterritorialized; Constantly-evolving.

• Internet research “located in the interplays of online, onground and technical research spheres”;

• Eclosion of an aphysical space for scholars to “rethink assumptions and categories”

Sources; Dahlberg, 2004; Foot and Schneider, 2004; Maczewski, Storey et al., 2004; Baka and Scott, 2008; Morris and Ogan, 1996.

Deep structural transformation

s

Page 10: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Contextualization

• Influences to conduct Internet research:

Sources; Nancarrow, Pallister et al., 2001, Livingstone, 2004; Walther, Gay et al., 2005; Jankowski and Van Selm, 2008.

Researcher

“Pull factors”: Novelty; Facility; Flexibility; Accessibility; Etc.

“Push factors”: Constant

need to redefine methodological approaches

Challenges: Development of mixed, or hybrid, approaches

(triangulation); Visualization of findings; Sampling processes.

Page 11: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Defining the Blogging Population

• Structural characteristics of the blogosphere: Identification of political blogs:

Sources; Karpf, 2008; Sweetser, Golan et al., 2008, Trammell, Williams et al., 2006; Hookway, 2008; Sweetser and Kaid, 2006.

Personal blogs

“Blawgs”

Celebrity blogsBlogosphere

Political blogs

Different formats;Different objectives.

Blogging refers “[...] to a wide range of disparate activities that

are problematically grouped together”

Page 12: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Defining the Blogging Population

• Structural characteristics of the blogosphere: Identification of political blogs:

Sources; Wallsten, 2005; 2007; Giasson, Raynauld et al., 2009; Park, 2009; Hargittai, Gallo et al., 2009; Gruszczynski, 2009.

Quantitative keyword analysis of their

content

Bloggers’ assessment of their publications

Other techniques: Consideration of formal and informal public

directories; Exploration of political blogs’ blogrolls; Content analysis of conventional media’s

coverage.

Fatally flawed?

Page 13: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Defining the Blogging Population

• Structural characteristics of the blogosphere: Constant restructuration of the political

blogosphere: Rapid expansion since 2004; Blogs’ level of activity;

Potentially geographically-specific nature of projects.

Sources; Gruszczynski, 2009, Schmidt, 2007; Martin, 2009; Wallsten, 2005; Perseus, 2004; Halavais, 2002; Hargittai, Gallo et al., 2008.

Certain social media platforms are plagued by poor levels of retention.

Difficulty to produce comprehensive and up-to-date repertoire of the political blogosphere

Page 14: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Defining the Blogging Population

• Quebec-based French-speaking political bloggers:• Consideration of two (2) unscientific indexes of

political blogs;• Approximately 125 Quebec-based political

bloggers in April 2008.

Sources; Giasson, Raynauld et al., 2008; 2009.

Page 15: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Web-Based Sampling Strategies

• Methodologically- questionable sampling techniques used by researchers:

“Tentative”; “Not lacking creativity”.

• Probability sampling highly problematic (“if not impossible”):

Sources; Bar-Ilan, 2005; Calvert and Huffaker, 2005; Wallsten, 2005; Gruszczynski, 2009; McKenna and Pole, 2008; Munger, 2008.

Random identification and selection processes:“After browsing

blogspace for a while”;“randomly [...] selected

weblogs”.

Problem with the identification of the population.

Page 16: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Web-Based Sampling Strategies

• Non-probabilistic sampling techniques are the only “feasible” or “appropriate”:

Avoid “small and isolated clusters”; Other strategies do not adequately portray the

decentralized and unpredictable nature of Web-based social flows;

Extensively used for weblog research. Difficult to produce a truly representative

research sample of bloggers;• Impact of data-gathering techniques (Web-based

surveys):• Easy to circulate;• Low costs (printing, shipping, etc.);• Data collected can be easily managed.

Sources; Johnson, Kaye et al., 2008; Porter, Sweetser et al., 2008; Ahn, Han et al., 2007; Herring and Paolillo, 2006; Wallsten, 2005.

Page 17: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Web-Based Sampling Strategies

• Quebec-based French-speaking political bloggers: Online survey available from April 15th 2008 to

May 1st 2008 inclusively; 58 questions unevenly distributed in seven

sections: Constitution of the research sample was twofold:

Selection of 22 A-list political bloggers;

Non-probabilistic viral dissemination approach for the circulation of the survey:

Through A-list bloggers; Through conventional

media representatives (journalists).

Sources; Vergeer and Hermanns, 2008; Jankowski and Van Selm, 2008; Scheidt, 2008; Wallsten, 2008; Sweetser, 2008.

Research sample

56 respondents; 73% of A-list

bloggers responded;

71% recruited through the viral procedure.

Page 18: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

Web-Based Sampling Strategies

• Possible detrimental impact of viral sampling strategies:

Fragmentation of the online political audience; Politically-homogenous and highly partisan

nature of online socio-political networks; Potential research agenda of respondents;

• Sampling as possible research results:• Indicator of the socio-political behavioural

profile of certain Web-based political communities.

Sources; Johnson, Kaye et al., 2008; Porter, Sweetser et al., 2008; Ahn, Han et al., 2007; Herring and Paolillo, 2006; Wallsten, 2005.

Page 19: CPSA 2009 - Constitution of Representative and Reliable Web-based Research Samples: Th…

QUESTIONS OR COMMENTS