Content Analysis

Content AnalysisAn Approach to Documentary Research (Holsti, 1969 p. vii)Documentary Research: Refers to research that is based on documentary sources or factual records and reports.1Content Analysis Multiple Definitions Background Applications and ProcedureAn Illustration Consistency within DiversityGrowthReliabilityValidity

General Definitions...a research technique that uses a set of procedures to make valid inferences from text (Weber, 1990, p. 9)

A summarizing, quantitative analysis of messages relying on scientific method and not limited to types of variables measured or context of messages (Neuendorf, 2002)

"Any technique for making inferences by objectively and systematically identifying specified characteristics of messages (Holsti, 1969 p. 14)

Text, in this sense, refers to all forms of recorded material.As a research technique, it is required that all studies be replicable. In order to meet this requirement, all forms of communication need to be stable and durable. (Holsti, 1969 p. 2). Documents include any text, audio recordings, music, videotapes, drawings, television, or images. In the digital age, content analysis includes the analysis of stable email, posts, blogs, podcasts, webinars, and all forms of digital communication. Lasswell, a communications scholar, used content analysis to study propaganda. .This well-known question combines quantitative procedures or enumeration and qualitative procedures or context and proximity.

3Definitions (continued) ...technique for objective, systematic, & quantitative description of manifest content communication ( Berelson, 1952, p. 18) systematic, replicable data reduction technique, compressing many words of text into content categories based on explicit rules of coding (Stemler, 2001) Close analysis of explicit and implicit messages of a text through classification and evaluation of key concepts, symbols, and themes to determine meaning and explain its effect it on the audience (Reitz, 2004) ...systematic reading of texts and symbolic matter not necessarily from an authors or user's perspective (Krippendorf, 2004, p. 3) Essential PurposeWho (says) What (to) Whom (in) what Channel (with) What Effect? (Lasswell, 1948) Goal of Content Analysis To draw inferences about: A text To obtain documentary evidence about: A sender A message A receiver (audience)ProceduresCommunication transmission model (manifest content) Semiotic model (latent content) of the signifier and the signifiedHistorical AntecedentsCategories were formed that encompassed a concept that many words could convey. The notion of the connotation and ambiguity of words and their meanings were recognized.8Analysis of TextsResearchers have analyzed texts since the start of writingContent analysis draws on Aristotelian principles of textual analysis as described in Poetics Subject of text Creator Audience Purpose Context Ethos Pathos

Analysis of TextsMedieval Scholastics recognized latent meanings of words, leading to categoriesFirst documented content analysis: 18th century Sweden Collection of religious hymns blamed for subverting orthodox state church and inciting dissent Words and symbols in songs were same as orthodox hymnal, but occurred in different contexts (Dovring, 1951; Krippendorf, 2009)

Analysis of TextsDuring early 20th century Weber studied content of newspapers raising fundamental questions:

How do newspapers affect readers?

How do the contents of newspapers compare quantitatively over a generation?

What is carried as newsWhat is carried as editorial What is omittedWeber is regarded as a pioneer in the sociology of the press

Psychological AnalysisGordon AllportPioneer in personality theoryAnalyzed Letters to Jenny (Diary)Aimed to understand her experienced world by identifying psychological traits as expressed in her letters and diaries (Krippendorf & Bock, 2009)

Content AnalysisTopicsContent Analysis and Data The Effect of Semantics on Public Opinion and Propaganda

Program Effectiveness

Readability of Text

Television Violence

Library Transactions

Characteristics of Reference Questions

Personality Traits

Determination of Authorship

The process of determining authorship involves compiling a list of probable authors and comparing their, language, use of language, themes, and all stylistic features that characterize each author. That author whose textual content is most similar in all defined categories to the text in question would be designed the rightful author. One of the most recent, successful examples of this use was in identifying the anonymous author of the 1990s book, Primary Colors. The book dealt with Clintons 1992 Presidential campaign, but the authorship had not been disclosed. Using the method of content analysis, the literary techniques of rhetorical analysis and narrative analysis, Donald Foster, a Vassar professor, identified Joe Klein, a Times magazine writer, as the author of the book.

14Questions for Content Analysis How are minority ethnic groups treated in short stories in popular magazines? (Berelson, 1952) How can communications be tested for propaganda elements?What ways do motion pictures reflect popular feelings and desires? What are the dominant images in Shakespeares plays?What happens to a good book when it becomes a movie? How do criteria used for program effectiveness reflect program objectives? How do attitudes reflect personality structure?What domains are currently being studied in library and information science?

Program effectiveness in education would look at mission statements and program objectives, then evaluate whether the criteria used to measure program effectiveness (measures of achievementtest scores, performance, behavior) reflects the objectives. Or, how do criteria used to assess learning reflect objectives established for a course. Objectives and criteria should be established at the start of a program or coursenot during.15A Quantitative IllustrationResearch Problem:

What role does popular magazine fiction play in affecting attitudes toward ethnic minorities?(Berelson and Salter, 1946)Hypotheses AssertedSympathetic characters are from the majorityUnsympathetic characters are from minorities and usually unsympathetic and secondarySocial interaction between majority and minority groups seldom occurs. When interaction occurs, minority group in a subordinate position.The problems of majority and minority groups differ The majority has individual (personal) problems The minority has social and economic problems The goals of the majority (love, comfort) more approved than goals of the minority (material gain).

Hypothesis and CategoriesHypotheses were translated into categories a priori assignationThe first hypothesis yielded 3 categories for analysis: Ethnic identification of character Role in the story Level of approval (sympathy for character) Other hypotheses provide more categories: Socio-economic status of characters Kind of interaction Problems GoalsAfter categories for the hypotheses are exhausted, they can be addressed by potential data

Final Step in AnalysisEstablish CategoriesDetermine Indicators (a priori)First general category was ethnicity Indicators of ethnicity rarely explicit (i.e. He was Italian-American)Indirect indicators of ethnicity conceived indicators involved names (i.e. Antonio), language, appearance, memberships, etc. Second category was characters role in story Was the role major or minor? Indicator was the space allotted to specific characterThird category was socio-economic level Indicator included characteristics as occupation, education, income, possessions, standard of living, etc.After examining the stories other categories and indicators were noted. These emergent categories extended the hypothesis (a posteriori)Strictly speaking, qualitative studies require a priori categories; qualitative studies are characterized by their emergent and inductive coding of categories.19Krippendorfs Six Procedural Questions(Krippendorf, 2003)Stemple (2001) cites three problems associated with compiling documents for content analysis 1. a significant number of documents from the population are missing 2. Records or Documents that do not meet the definition of the document should be discarded and a notes citing the reason kept. 3. Documents meet the definition, but can not be coded because they are missing passages or have ambiguous content.20The Basic StepsWhich data are analyzed? (Unit of Analysis)How are data defined? (Definition of Terms--these are the variables to measure. Both internal and external validity come into play here.)What is the population or unit from which the sampling data are drawn? (Sampling Frame or Unit of Sampling) What is the context relative to which the data are analyzed? What are the boundaries of the analysis?What is the target of inference?Code Book - Coding FormComparison: Berelsons Study and Krippendorfs 6 QuestionsKrippendorf

What data are analyzed? How are the data defined?What is the population or unit from which the sampling data are drawn? What is the context relative to which the data are analyzed?What are the boundaries of the analysis?What is the target of inference?

BerelsonUnits of analysis are words.Data defined by categories and their indicators.Frame is fictional story in popular magazine.Context involves attitudes toward ethnicities.Boundary is single story in selection of magazines. Do indicators support hypothesis?

Two MethodsQualitative and Quantitative ProceduresContent, System, Goals The objective of content analysis is determining answers to questions that the text implies explicitly and implicitly

It is a systematic technique of reading between the lines

Can this goal be attained most effectively with quantitative or qualitative techniques?

Distinguishing feature between 2 approaches is starting point:hypothesis or grounded inductionmethods of deriving categorieswhat depth of communication accepted as an indicator

Quantitative ProceduresContent analysis starts with the hypothesisCategories derived from possibilities of hypothesisCategories established a priori and indicators based on manifest (surface) meaning

Qualitative ProceduresQualitative approach derives categories after examining document emergent coding Factors in latent meanings of words or messages Context and proximity suggest the intended meaning of a message

Media: Qualitative AnalysisNarrative Analysis (focus on literary composition and structure)Rhetorical Analysis examines properties of text; emphasis not on meaning,but how meaning is conveyed)Discourse Analysis examines manifest language in text language and wordusageSemiotic Analysis focus is on latent meanings and deep structuresInterpretive Analysis formation of theory from observationConversation Analysis used by social scientists in psychology,communications and sociologyCritical Analysis analyzes media messages (how are groupsrepresented in media of all forms)

Consistency within Diversity The definitions, uses, and techniques vary according to whether the researcher conducts a quantitative study or a qualitative study Content analysis stipulates three core conditions for qualitative or quantitative character of studyThese central conditions include:ObjectivitySystem Generality (Holsti, 1969)

Objectivity (Holsti, 1969)The test of objectivity is whether another researcher, following identical procedures with the same data, can arrive at a similar conclusionObjectivity, in content analysis, means that explicit rules and procedures guide the research processFor example, what criteria will be applied in formulating a category?What criteria determines content unit (word, theme, webpage, animation, imagery, kinds of imagery, narrative, dialogue, space, advertisements, maps, transactions)?Objectivity requires researcher explicitly state ground a category is selectedand what rationale prompts one inference rather than an alternative

Paraphrased from Holsti (1969, p. 4).29SystemSystematic procedures facilitate objective, unbiased selection of content and categories (Holsti, 1969)Assigning firm rules to regulate inclusion & exclusion of content and categories prevents researcher from biased, self-serving selection Rules preclude an analyst from intentional or unintentional bias in selecting only content or category supporting hypothesis, proposition, or self-interest.

GeneralitySelect content based on clear definitions for determining content and categories fosters objectivityEstablish procedures for inclusion or exclusion of data -- contributes towards consistency and systematic, impartial selectionPromote principled, legitimate descriptive analysis with systematic procedures provides a basis for generalizing about the data collectionRelate data to an established theory or reasoned proposition for data to be conclusive or meaningfulObjectivity, System, and Generality are not exclusive to content analysis. They are required conditions of all scientific or reasoned inquiry.31Thorny IssuesQuality & Quantity ~ Manifest Content and Latent ContentArguments RefinedQuantitative and qualitative methods are different approaches to investigating same problem Must content analysis be defined by quantitative or qualitative methodologies?Holsti (1969) holds that other measures besides quantitative frequency counts & enumeration validAlternative method of quantifying data with greater refinement is contingency analysisContingency analysis codes presence or absence of a trait within document or unit ofsampling. Inferences derived from proximity of two or more units of analysis in sampling unitThe use of quantitative procedures or qualitative is not an either/or issue -- they can interactRestriction to quantitative analysis has been criticized on several groundsSuch restriction leads to bias in the problems to considerIt is also stated that qualitative methods can result in more meaningful or descriptive inferences(Holsti, 1969 p. 10)By interact, I mean that each methodology can inform the other. 33Manifest and Latent Content Considerations: Must content analysis be restricted to manifest content or surface meanings; elements physically present & countable (Neuendorf, 2002 p. 23) Holsti offers compelling example of limits of manifest content. Limit to manifest content would not yield this notion Describes patient in mental hospital who states he is in Switzerland. In this case, the patients meaning of Switzerland is freedom. By stating he is in Switzerland, he is stating that he wants freedom. Latent content & meaning would provide the context whereby the receiver of the message would understand the idiosyncratic meaning. Latent content need not be dismissed. However, as Holsti emphasizes, it should be verified by independent evidence.A Growing TrendDuring early 20th century Max Weber applied techniques of content analysis to journalism when he introduced the notion of studying the sociology of the press (Weber, 1924)Over 24 years -- 1971-1995, Riffe & Freitag (1997) noted six-fold increase in number of content analyses appearing in journal, J&MC QuarterlySoftware applications and the WWW enable text & data analysis Neuendorf (2002) reported more than 24 text content analysis programs are available (THAT WAS 9 YEARS AGO)Many electronic content analysis applications now available text, audio, video

Max Weber formally emphasized the importance of applying quantitative and qualitative approaches in analyzing how the contents of newspapers effect public opinion. Weber referred to this study as the sociology of the press.35ReliabilityReliability measured basically in two ways (Stemler, 2001)Stability (intra-rater reliability Does same coder get same results again and again?Reproducibility (inter-rater reliability) Do different coders, rating the same text, get the same results? This is a seriously rough means of testing reliability. However, it may reveal some of the difficulties involved with determining features and consistently coding them.36Internal ValidityInternal validity is degree a procedure of measurement represents the defined conceptDoes the procedure measure what one intends to measure?Research question determines what one intends to measure In previous example, the concern was how individuals in majority population and minority population were characterized in fictional stories carried by popular magazinesStudy valid if results of inquiry yield data related to this concern and research question

External ValidityExternal validity or the ability to generalize to the wider population, depends largely on the soundness of ones sampling technique and sampleNeed experimental controls or validity becomes serious concernTriangulation or independent corroboration is one means of establishing validityAnother involves conducting same inquiry using another method of inquiry formal experimental design employing surveys or qualitative procedure with focus group/interviews

Flowchart: Qualitative

BibliographyAllport. G. (1965). Letters from Jenny. In K. Krippendorff & M. Bock (Eds.), The content analysis reader (p. 28-37). London: Sage. Print.Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addision-Wesley Pub. Co. Retrieved from http:// www.questia.comKrippendorf, K. (2003). Introduction to Content Analysis. CA: Sage. PrintKrippendorff, K. & Bock, M. A. (2009). The Content Analysis Reader. CA: Sage. Print.Neuendorf, K. A. (2002). The content analysis guidebook. London: Sage. Online resource: http://academic.csuohio.edu/kneuendorf/content

Reitz, J. M. (2004). Dictionary for library and information science. CN: Libraries Unlimited. (Print).

Riffe, D. & Freitag, A. (1997). A content analysis of content analyses: Twenty-five years of Journalism Quarterly. J&MC, 74, 4, 873-882.

40Bibliography (continued)Stemler, S. (2001). An overview of content analysis. Practical Assessment, Research and Evaluation, 7 (17). Retrieved from http://PAREonline.netWeber, M. (1924). Toward a sociology of the press: An early proposal for content analysis. In K. Krippendorff & M. Bock (Eds.), The content analysis reader (p. 9-11). London: Sage. Print.

Weber, R. (1990). Basic Content Analysis. London: Sage. Print.

Thanks very much!Florence M. Paisey, April 2011