USING AUTOMATED CODING & SEMANTIC NETWORK ANALYSIS TO INVESTIGATE MEANING IN LARGE SCALE DISCOURSE TEXTSA METHODOLOGICAL WORKSHOP

Christian Baden • LMU Munich • GermanyWouter van Atteveldt • VU Amsterdam • Netherlands

&

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

WELCOME TO THE WORKSHOP!

Dr. Wouter van Atteveldt Dr. Christian BadenVU University Amsterdam LMU Munichwouter@vanatteveldt.com c.baden@lmu.de

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

AGENDA for TODAY

09:30 – 09:45 Introduction09:45 – 10:30 Quantitative analysis of discourse10:30 – 11:30 Automatic & manual coding using AmCAT

Coffee Break

12:00 – 13:00 Analytic opportunities within the AmCAT framework13:00 – 13:30 Examples & Applications

Lunch Break

14:30 – 15:30 Hands-on session (Computer Lab, optional)

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

AGENDA for WEDNESDAY

09:30 – 11:00 Vocabulary, Grammar, and Semantic Networks11:00 – 11:30 Discourse Analysis as Semantic Network Analysis

Coffee Break

12:00 – 12:45 Context, Patterns, and Associative Coherence12:45 – 13:15 Analytic opportunities beyond the AmCAT framework13:15 – 13:30 Questions & Answers

Lunch Break

14:30 – 15:30 Hands-on session (Computer Lab, optional)

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

LOGIC OF AUTOMATED CONTENT ANALYSIS

Content Analysis as rule-bound categorization of semantic contents in text

Codebooks contain all categories identify the rules that decide when a category must be coded,

using…

IF – THEN statements lists of indicators disambiguation criteria (typical examples) (logics) (etc.)

A perfect codebook achieves that even the most ignorant coder arrives at precisely the same coding decisions by following the rules.

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

LOGIC OF AUTOMATED CONTENT ANALYSIS

Content Analysis as rule-bound categorization of semantic contents in text

Ontologies contain all categories identify the rules that decide when a category must be coded,

using…

IF – THEN statements lists of indicators disambiguation criteria

An ontology necessarily assumes that it actually has to work with this most ignorant coder (a computer).

So in essence, an ontology is a codebook that has such precise rules that it leaves no interpretative decision to the coder

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

LOGIC OF AUTOMATED CONTENT ANALYSIS

Codebooks can be used to code very different things: Presence of specific actors/objects Presence of specific issues/topics Presence of evaluative statements/expressions

Association of specific attributes with actors/objects or issues/topics

Association of specific evaluations with actors/objects or issues/topics

Expression of specific actions Expression of specific kinds of relations between actors/objects Expression of specific kinds of relations between issues/topics

Qualification of specific actions and kinds of relations

descriptors

level of abstraction

denotationconnotation

ironic/figurative

use

explicit/ Implicit

associationexplicit/ implicit/

pragmatic actions

relation types

intensity, qualifiers

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

USING LANGUAGE AS AN INDICATOR

Lexical Indicators TRANSLATION Semantic Contents

Many existing tools… operate on lexical indicators but do not consider semantics “distill semantics” algorithmically use thesauri to detect semantics A proper content analysis needs to map semantic meaning carefully.

lexical indicators semantic contents

words concepts

grammatical units propositions

texts discourses

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

USING LANGUAGE AS AN INDICATOR

Lexical Indicators TRANSLATION Semantic Contents

Many semantic contents – specifically, concepts – have a relatively direct relation to the lexical indicators used to express them.

BUT

some are easier described than listed some are more complex than others some cannot be uniquely reduced some are latent

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

USING LANGUAGE AS AN INDICATOR

Access to language depends on kind of question:How would I see in the use of language whether X is the case? "usual" contents are conceptual: objects/actors, attributes,

actions etc. but not all contents are conceptual: think also of "unusual"

markers: tense (e.g., “we will”) conjunctions (e.g., “without”) negations/qualifications (e.g., “not a good idea”) grammatical functions (e.g., objects, subjects) nearby markers (e.g., “however”) text structures (e.g., interview turns, subheadings) ...

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

USING LANGUAGE AS AN INDICATOR

What exactly in language use is indicative? Presence of X: frequencies, or positions, or qualifiers, or… Co-occurrences: within what, how far, how qualified, etc… Semantic relations: Explicit, implicit, implicated, or… Evaluations: where, by whom, etc.?; list, balance, order, or…

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

USING LANGUAGE AS AN INDICATOR

What role does context play for my question? irrelevant (e.g., pure visibility over time) instrumental (e.g., for disambiguation/identification) information (e.g., focus on association patterns)

Think of different kinds of contexts: immediate contexts (e.g., same sentence, word distance) formal contexts (e.g., same text, same issue) temporal context (e.g., same day, same phase) topical contexts (e.g., other statements on same topic) actor contexts (e.g., other statements by same actor) genetic contexts (e.g., knowledge available when something

was said) etc.

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

SUMMARY: AUTOMATED CONTENT ANALYSIS…

…is a form of text analysis that requires a lot of precision in preparing the coding instructions, and therefore depends crucially on: a question that determines the kinds of contents needed to

answer it knowledge about the use of language as indicator

dictionary knowledge often insufficient qualitative pilot studies often useful

a lot of diligence to make sure all of the right indicators, and only the right indicators are considered in the ontology

Furthermore, it is helpful to have: order/conventionality in language use good familiarity with the language coded whenever post-coding computations are needed: scale

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

…any questions so far?Wouter takes it from here…

!?

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Van Atteveldt, 2008

Document frequency: How much publication is there about…? Simple salience analysis/Issue

careers Identifying events driving

news coverage Comparing different

outlets/subdiscourses

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Van Atteveldt, Ruigrok, Schlobach, van Harmelen, 2008

Document selection: Which discourse texts do I want to look at? validating qualitative text analysis identifying texts that differ systematically with regard to the use

of certain terms identifying prototypical texts for some kind of repertoires identifying the first texts when something came up in a debate

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Document scaling/grouping/categorization: What kinds of documents are there? Based on meta-data Based on content data

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Balmas & Sheafer, 2013Kleinnijenhuis, Schultz, Oegema, & van Atteveldt, 2013

Concept frequencies: How much talk is there about…? Identifying issue careers, new

topics, etc. Detecting biases in using concepts,

quoting actors/sources, etc. Assessing the relative importance of

concepts, actors, etc. in the news

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Van Nooije, 2010Van Atteveldt, 2008

Bag-of-words frequencies Comparing the prevalence of

certain kinds of words/language styles/types of actors/thematic domains/etc.

Wider notion of issue careers

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Cooccurrences & Associations Attribution, 2nd level agenda

setting, etc.: How frequently & how strongly are concepts associated?

Association measures: Uni-/bidirectional associations

Van Atteveldt, Ruigrok, Schlobach, van Harmelen, 2008

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Listing Concordances: What are the contexts wherein a specific concept or word occurs? Useful for discourse/framing analysis and the construction of

disambiguation rules

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Sheafer, Shenhav, Takens, & van Atteveldt, 2013

Explaining Association strengths Comparing association strengths

across discourses/corpora Correlating association strengths

with third variables

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Schultz, Kleinnijenhuis, Oegema, Utz, & van Atteveldt, 2012Van Atteveldt, 2008

Semantic Networks Detecting associations between

larger sets of concepts (more tomorrow)

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

APPLICATIONS & EXAMPLES

Van Atteveldt, Ruigrok, Schlobach, van Harmelen, 2008

Evaluative Statements & Communicative relations Rating the evaluative tendency of

texts and statements Reconstructing Actor relations

USING AUTOMATED CODING ANDSEMANTIC NETWORK ANALYSIS TO INVESTIGATEMEANING IN LARGE SCALE DISCOURSE TEXT

CHRISTIAN BADENWOUTER VAN ATTEVELDTHebrew University Jerusalem15 & 17 July 2013

that‘s all for today,see you on wednesday!

c.baden@lmu.de & w.h.van.atteveldt@vu.nl