Feldman Zaller Article

8/8/2019 Feldman Zaller Article

1/39

A Simple Theory of the Survey Response: Answering Questions versus Revealing PreferencesAuthor(s): John Zaller and Stanley FeldmanSource: American Journal of Political Science, Vol. 36, No. 3 (Aug., 1992), pp. 579-616Published by: Midwest Political Science AssociationStable URL: http://www.jstor.org/stable/2111583

Accessed: 27/10/2010 03:23

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at

http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless

you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at

http://www.jstor.org/action/showPublisher?publisherCode=mpsa.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

Midwest Political Science Association is collaborating with JSTOR to digitize, preserve and extend access to

American Journal of Political Science.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=mpsahttp://www.jstor.org/stable/2111583?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=mpsahttp://www.jstor.org/action/showPublisher?publisherCode=mpsahttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/2111583?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=mpsa


2/39

A SimpleTheory fthe urvey esponse:AnsweringQuestions ersus evealing references*John aller,UniversityfCalifornia,osAngelesStanley eldman, tateUniversityfNew York tStony rook

Opinion esearchs besetbytwomajor ypes f "artifactual"ariance: uge mounts f over-time esponse nstabilitynd thecommon endencyor eeminglyrivialhangesnquestionnaireform o affecthe xpressionf attitudes.Wepropose simplemodel hat onvertshis nomalous;error ariance" nto ources f substantivensightnto henature fpublic pinion.The modelabandons he onventionalut mplausibleotion hatmost eoplepossessopinionst the evelofspecificityftypical urveytems-and instead ssumes hatmost eople re nternallyonflictedovermost oliticalssues-and thatmost espondosurvey uestionsn thebasisof whateverdeasareat the opoftheir eads tthemomentfanswering. umerousmpirical egularitiesreshowntobe consistent ith hese ssumptions.

Virtuallyll publicopinion esearch roceeds n the ssumptionhat iti-zenspossessreasonably ell formedttitudesnmajor oliticalssues nd thatsurveys re passivemeasuresfthese ttitudes. he standardiew s thatwhensurvey espondentsaythey avor theyre imply escribingpreexistingtateoffeeling avorablyoward .Accumulatingvidence n thevagaries f masspolitical ttitudes,owever,hasmade his iew ncreasinglyubious. f,as is wellknown, eople reaskedthesamequestion n a seriesof interviews,heir ttitudeeportsrehighlychangeable.Many, s much vidence lsoshows, eact tronglyo the ontextnwhich uestions re asked,to theordern which ptions represented,nd towholly onsubstantivehangesnquestionwording.hesephenomenaremorethanmethodologicaluriosities; heyraise seriousdoubts bout whatpublicopinion urveysmeasure.In view ofthis,we propose newunderstandingf the masssurvey e-

sponse.Mostcitizens,we argue, imply o notpossesspreformedttitudestthe evelofspecificityemandednsurveys. ather, hey arry roundntheirheads a mixofonlypartiallyonsistentdeasandconsiderations. hen ues-*Originallyreparedor eliverytthefifthnnualmeetingf thePoliticalMethodologyo-ciety. he data used nthepaperwere ollected ytheBoardofOverseers ftheNational lectionStudies nder grant rom heNational cienceFoundation. ollectinghese atamadeunusuallyheavy emands n the tafff the nstituteor ocial Research t theUniversityfMichigan,wherethe tudywas conducted.n this onnection e aregratefuloZoAnneBlackburn, tevePinney,andSantaTraugott.We also thank arbaraGeddes,Shantoyengar,onKinder,MattLyons, ndDouglas Rivers or heir elpful ommentsn various raftsfthepaper. inally, e would iketothank iroakiMinato ornvaluablessistancenpreparinghedatafor nalysis.Neitherhese ndi-vidualsnor nstitutionsear nyresponsibilityor ny rrors f udgmentr fact hatmay ppearnthis aper.

American ournal fPolitical cience, Vol. 36, No. 3, August 992,Pp. 579-616? 1992bytheUniversityf TexasPress,P.O. Box 7819, Austin, X 78713


3/39

580 John allerand Stanley eldmantioned, hey all to mind sampleof these deas, ncludingn oversample fideasmade alient y the uestionnairend other ecent vents,nduse them ochoose mong he ptions ffered. ut heir hoicesdo not, n most ases, reflectanythinghat an be describeds true ttitudes;ather,hey eflecthe houghtsthat remost ccessiblenmemoryt themomentfresponse.Amodel asedon these laims an,as we show, rovide far etterccountof the xistingvidence npolitical ttitudes,ncludinguchrelatedmatterssattitudeonsistencynd theeffectsfpolitical wareness, han an currentlydominant odels fthe urvey esponse.Webeginwith review fexistingmodels fpublic pinion ndthen ut-line nalternativeodel nddrawupon range f new ndexistingvidence odemonstratetsvalue. LimitsofExisting heoriesResponsenstability

Oneofthemost nsettlingindingsfopinion esearch as been hediscov-ery fa large omponentfrandomnessnmost eople's nswersosurvey ues-tions.fthe amepeople re sked he ame uestionnrepeatednterviews,nlyabouthalf ive he ame nswers. hedata nTable1,basedon nterviewsfthesamepersons ixmonthspart, llustrateheproblem. s canbe seenfrom heentries n themaindiagonals, nly45% to 55% gave the sameanswer othtimes,ven houghbout 0% couldhavedone o bychance lone. The amountof responsenstabilityiffersrom ne ssuetoanotherseeFeldman 989),butthe asesshownnTable1arefairlyypical.In hisfamous aper The Nature fBelief ystemsn Mass Publics," on-verse 1964) argued hat esponsenstabilitys duemainly o individuals holack meaningfulttitudes ut neverthelessndulge nterviewersy politelychoosing etweenheresponse ptions ut nfrontfthem-but hoosingnanalmost andomashion.Largeportionsfan electorate,"esuggested,simplydo not havemeaningful eliefs, venon issues thathave formed he basisfor ntense olitical ontroversymong litesfor ubstantialeriods f time"(1964, 245).Thisconclusion as been tronglyhallenged y cholarswho ontendhat,although eople's"survey esponses" luctuatereatly,itizens aveunderlying"true ttitudes"hat reoverwhelminglytable Achen 1975, 1983;Dean andMoran1977; Erikson1979; Feldman1989). The fluctuationshat ppear npeople'sovert urvey esponses re attributedo "measurementrror," here

'Given datafrom nly wopointsntime, t s impossibleodistinguishystematicttitudechange rom andom luctuation.owever,nalysis fdatafrom hree-ndfive-waveanels tronglysuggests hat almostall response nstabilityepresentsandom ather hansystematichange(Feldman 989).


4/39

A THEORY OF SURVEY RESPONSE 581Table 1. ResponseStability ver Repeated nterviews:TWoExamples

AmericanRelations withRussia(CornerPercentaging)Attitudes n January 980Cooperate Middle Tougher Unsure

June1980:Cooperate 25% 8 8 2Middle 7 4 5 1Tougher 5 5 17 2Unsure 4 2 3 4N = (338) (153) (266) (74)

Level of Government ervices(CornerPercentaging)AttitudesnJanuary 980Cut Middle Keep Same Unsure

June1980:Cut 24% 6 5 6Middle 8 4 2 2Keep Same 5 4 15 3Unsure 8 2 3 6N = (362) (122) (208) (138)

Note:The exactquestionswere: Somepeoplefeel t s importantorus to tryvery ard o get longwithRussia.Others eel t s a bigmistakeotry oohardtoget longwith ussia.Wherewouldyouplace yourselfn this cale,or haven'tyou thoughtbout his?"Respondents ere hen skedtoplacethemselvesn aseven-pointcale. In this able,points , 2, and 3 havebeencounted s "cooper-ate";4 iscounteds middle; , 6, and7 havebeen ounteds "tougher."The second uestionwas: "Somepeople hinkhe overnmenthould rovidefewer ervices, ven n areas suchas health ndeducation,norder o reducespending. ther eoplefeel t is importantor hegovernmento continue heservices t nowprovidesven f tmeansno reductionnspending.Wherewouldyou place yourselfn this cale, orhaven'tyou thoughtbout his?" temwasrecoded s above.Source:National lection tudies, 980PanelSurvey.

such rrors saidtostem rom he nherentifficultyfmappingne'sattitudesonto heunavoidablyague anguage fsurvey uestions.Bothapproaches o response nstabilityave critical eficiencies. on-verse's hesis,which akes ny nstabilitys evidence fa "nonattitude,"asanextreme laim ntendedo characterizettitudesnly nhighlybstractssues.


5/39


6/39

A THEORY OF SURVEY RESPONSE 583(1988),for xample, rgue hat esponseso attitudeuestionsan be understoodas the utcome f a question-answeringrocessn which eople 1) decidewhatthe ssue s; (2) canvas heirminds or elevanthoughts;3) combinedeas ntoa coherentttitude;nd (4) maptheresultingttitudentoavailable esponseoptions. ecause,as theymaintain,eaturesfthe nterviewrocess an affecteach of these teps, hequestionnairean also readilyffect hatgetsreportedaspublic pinion.Morerecently, ilson nd Hodges 1991) haveproposed model nwhichattitudesre"temporaryonstructs"hat re madeupat themomentfresponseon thebasis of ideas na largebut nternallyonflicteddatabase." nperhapsthe most mbitiousttempto deal with hequestion-answeringrocess andmuch lse), Wyer ndSrull 1989)offern nformation-processingodelhavingmore han hree ozen lements.WhatNeedstoBe Done

Despite he vidence rom sychologistsndsurveymethodologists,ublicopinion esearchersargely gnore oth he ongstandingroblem fmassiveover-timeesponse nstabilitynd the newer indingsn questionnaireffects.Moreover, any f thosewhorecognizeheproblemsmakewhat mountssen-tially o patch-upsf the traditionaliew. n thecase ofresponseffects,hepatch-uponsists ftryingopreventheproblemrom ecoming onspicuous;this s doneby, or xample,makingure okeepquestion rderonstantcrosstime eries urveys r, n some omputer-assistedurveys,andomizinguestionorder cross espondents.n the ase ofresponsenstability,he atch-uponsistsof statisticalorrectionsormeasurementrror,orrectionshat reate he om-fortablellusion hat ixed true ttitudes"xistbeneathheenormous urfacenoise.Thechallenge, hen,s to devise theoryhat ccommodatesoth esponseinstabilityndresponseffectsnd that s craftedo thekinds fproblemsnddatafacing nalysts fpublicopinion.This is whatwe attempto do in thispaper. hetheory epropose s,weadmit, implerhanwouldbe necessaryoexplain ll of the indingshat sychologistsavenowdocumented.ut theorysufficientlyomplex o do thiswouldhave ittle alue o most oliticalcientists,and our aimis, aboveall, to reach hisgroup-convincingtthat heconven-tional nderstandingfpublic pinions unworkablendthat practicallter-natives available.

AnAlternativeModel oftheSurveyResponseAccordingo conventionalttitudeheory,ndividualshoosewhicheverprespecifiedption omesclosest o their wnposition. ut f,as we contend,peopletypicallyo nothave fixed ositionsn issues,howdo theymake heirchoices?Since most urvey esearchakes he true ttitude"s itsprimitivenit f


7/39

584 John allerandStanley eldmananalysis, ittlettentionas beendevoted othis uestion.f,however,ne turnstostudieshatmploy epthnterviews,ne findsmuch seful vidence.Amongthe est fthese s Hochschild's1981) study fattitudesoward quality,What'sFair?From er nterviews ith 8 persons, ochschildound hat eoplewould,ifaskedto do so, readily nswer ixed-choiceuestions, ut hat iven heop-portunityo talk, peopledo notmake imple tatements;hey hade,modulate,deny, etract,r ustgrind o a halt nfrustration.hesemanifestationsfuncer-taintyre ustas meaningfulnd nterestings thedefinitivetatementsfa be-lief ystem"238).Hochschild articularlymphasizes he mbivalencef many fher espon-dents. hisambivalencerequentlyeadsthem o contradicthemselves-whichis tosay, o give emporallynstable esponsesnthe ourse f single onver-sation.Consider his ccount fthe ttitudesfoneof her ubjectsoward ov-ernmentncome uarantees:Vincent artori annot ecidewhetherrnot hegovernmenthouldguaranteencomes,because he cannotdecide howmuchweight ogive to thevalueofproductivity.e believes hat herich re mostlyundeservingnd. . . yethe sangryt welfareheats'whorefuse owork. ..Caught etween is desire or qualityndhisknowledgefexistingnjustice,onthe nehand, ndhisfear hat guaranteedncomewillbenefitven hirkers,on the ther, e remainsmbivalentbout olicies oward hepoor" 252).

Currentttitude odels eem uite rrelevantothese bservations.herea-sonfor artori's acillationsnot, s studentsfConversemightay, hat ehasnoopinion n this uestion, or s itthat, s users f measurementrrormodelsmightay, artori as a "true ttitude"hatHochschildsunable o measure eli-ablyIt is ratherhat artori asconflictingpinions,rat east onflictingon-siderations,hat eadhim ogivedifferentesponsestdifferentimes, ependingonhowhe thinksbout he ssue.It is easy to object o the imitationsn rigornherentndepthnterviewssuch s Hochschild's.Nonetheless, e arepersuadedhat hebasicpoint boutambivalence-thatndividualsossessmultiplend often onflictingpinionstowardmportantssues-representsn importantnsight.Muchpsychologicalresearch einforceshis iew.Memory esearchers,or xample, ave hown hatpeople tore uge mountsf nformationntheirong-termemories,ut anretrievenduseonly fractionf tat one time.Theparticular aterialhey orecalldepends n a combinationfchance ndrecency factivation. ence,peoplemakequitedifferentudgmentsnd belief tatements,ependingntheinformationheyhappen o recallfromong-term emoryRaaijmakersndShiffren981;Wyer ndHartwick984).Anotheresearchradition, ainlyoncerned ith ocial ognition,ocusesontheorganizationf deas nthemind.A centralonceptnmuch f this e-searchsthe schema," termhat as been dapted romognitive sychology.A schema sa cognitivetructurehat rganizes riornformationnd xperience


8/39

A THEORY OF SURVEY RESPONSE 585around central alueor dea and that uides he nterpretationfnew nforma-tion ndexperience.

A critical oint bout chemass that eople ypicallyave everal f themavailable orunderstandingny givenphenomena.or example, n individualbeing ntroducedo a "professor" ould eact uitedifferentlyfthenewpersonwere nsteaddescribed s "a motherf four." hat s, differentssociationswould ometo mind, ifferentualities fthepersonwouldbe noticed, ifferentconclusionswouldbe drawn rom he person'smannerisms,nd so forth.nshort,heperceiver'sttitudeoward hepersonwouldbe different.hus,Tesser(1978), instatementshat epresenthedominanthrustfmuch ognitivesy-chologyndthat icely apture central eaturefthemodelwe propose,writes:"An attitudet a particularoint n time s the result f a constructivero-cess. . . . And,theres not single ttitudeoward n objectbut,rather,nynumberf attitudesependingn thenumberfschemas vailable or hinkingabout he bjects" 297-98). And persons o nothave single eelingrevalu-ation fanobject.Feelings ary epending pon he articularognitivechemawe tunen'" (307).These studies endbothto corroborate ochschild'snsightsoncerningambivalencend to underminehe conventionalolitical cienceassumption(which s at theheart fbothConverse's lack-and-whiteodel ndAchen's"true ttitude" easurementrrormodel) hat t snormal orndividualso havea single,coherentttituden issues. In viewof this,ourmodel will followHochschildnassuminghat eople arryround n their eads mix fmore rless consistentconsiderations,"here considerations defineds a reason orfavoringne side ofan issue ratherhan nother.E.g., a personwho thinksabout Pentagonwaste"whiledeciding question boutdefense pending asraised considerationhatmaywellcontrol er ecision n thatssue.)Thefirstxiom fourmodelmaynowbe stated s:

AXIoM1: The ambivalencexiom.Mostpeoplepossessopposing onsid-erationsn most ssues, hats, considerationshatmightead them ode-cidethe ssue ither ay.Weemphasize hat heconcept fconsideration,s used n this xiom, snotustanother ordfor chema.First,t is cast nthe anguage feverydaypolitical iscourseseeKelley1983),as befits termntendedor olitical atherthan sychologicalnalysis. econd, t makesno referenceomental tructuresoroperations,uch s the nterpretationfraw ensorynput,hat recentralothe oncept f chema.Ournext roblems todecidehow ndividualsransformhe iverse onsid-erationsn their eads nto losed-endedesponses. ne possibility,s TaylorandFiske 1978) suggest,s that ndividualsmakechoices off hetopof the


9/39

586 John aller andStanley eldmanhead"on thebasis ofthefirstdea that omes omind.Thus,peoplemaymakesocial udgmentsy seizing n "a single, ufficientnd salient xplanation . .often he firstatisfactoryne that omes along. . . . [I]nsteadofemploying aserate r consensusnformationogically, eoplearemore ftennfluencedyasingle, colorfulpiece of case history vidence. . . . Insteadofreviewing ll theevidence hat earsupon particularroblem, eoplefrequentlysethe nfor-mationwhich s most alient ravailable o them, hat s, thatwhich s mosteasilybroughto mind" 251). Tverskynd Kahneman's1982) well-knownwork nframingffectseinforcesheviewthat ndividualsftenre overly n-fluencedy single, ominantonsideration.Atthe ame ime,much ata nboth oliticalcienceCampbell t al. 1960)andcognitivesychologyAnderson 974) ndicate hat n otherccasions, n-dividuals each ecisions yaveragingcross range fcompetingdeas.Thus,Kelley 1983) shows hat oters eemtodecidewhich residentialandidate osupport ysumming p all oftheir likes" nd "dislikes" bout achpartyndpresidentialandidate ndchoosinghe ne with hehighestet otal.Theaxiomswepropose llow ndividualsorespondosurveyuestionsnthebasis of eitherne ormany onsiderations,ependingnhowmany appentobereadilyccessiblenmemorytthemomenthe uestionsposed:

AxIoM2: Theresponsexiom. ndividualsnswerurveyuestions y av-eragingcross he onsiderationshat appen o be salient tthemomentfresponse, here aliencysdeterminedythe ccessibilityxiom.AxIoM3: Theaccessibilityxiom.Theaccessibilityfanygiven onsider-ationdepends n a stochasticampling rocess,where onsiderationshathavebeenrecentlyhoughtbout re omewhat oreikely obesampled.2

For thecase in which person evotes reat houghtndattentiono an issue,Axiom implies hat heremaybemultipleonsiderationsalient nmemorytthemoment fansweringuestionsbout he ssueand hencemany onsidera-tions o be averaged cross.But a personwhorarelyhinks bout n issue andwho s confrontedy an interviewituationhat equires succession fquickanswersFeldman 990) mayhave nly neconsiderationmmediatelyvailableinmemory,nwhich ase the veragingule educes oansweringnthebasisofa single top-of-the-head"onsideration,s suggestedyTaylorndFiske.Thesethreexioms, lthoughpare nd nformal,an beusedboth oorga-nize much xisting esearchndtogenerateestable ewhypothesesbout henaturefthemass urveyesponse,swe shallnowbegin o show.2The mpirical arrantor his xiom sextremelytrongsee Bargh t al. 1986;Higgins ndKing 1981;BodenhausenndWyer 987).


10/39

A THEORY OF SURVEY RESPONSE 587Data

Sincewe base much four nalysis ndatafromhe1987Pilot tudy ftheNationalElectionStudies NES), it is worth ausingbrieflyo describe hisstudy. he survey ttemptedomeasure,nterlia,the considerations"hat n-derlie eople'sresponseso standardlosed-endedurveytems. hestudywasconductedn twowaves monthpart; 57personswere nterviewedn theMaywaveand 360 in theJunewave.All hadpreviouslyarticipatedn the1986Na-tional lection tudy. ther echnical etails fthe tudyre available hroughtheNES at theUniversityfMichigan.The basic method as to askpeople closed-ended olicytem ndthen oask them o talk ntheirwnwords bout he ssues t raised.The closed-endeditemswere elephone ersionsf the tandard ES items n ob guarantees,idto blacks, nd governmentervicesnd spending.n form , respondents ereasked theopen-ended robes mmediatelyfter nswering he givenclosed-ended olicy tem. he exact orm fthe retrospective"pen-endedrobeswas:

Stillthinkingbout he uestion ou ust answered,'d likeyoutotellmewhat deascametomind s youwere nsweringhat uestion. xactlywhatthings ent hroughourmind.Any thers?)Thisprobewas designed o elicit "memory ump" f theconsiderationsm-mediatelyalientnpeople'sminds.PriorworkbyEricsson nd Simon 1984)shows hat uchprobes an work ffectivelyfasked mmediatelyfter giventaskhas been arried ut.In form , interviewersead he tems nthe sualway, ut,without aitingfor nanswer, hey skedrespondentsogive heir eactionso theprincipaldeaelementsnthequestion. or the ob guaranteesuestion, heprobeswereasfollows:

Before ellingmehowyoufeel bout his, ouldyoutellme what inds fthingsome tomindwhenyouthink boutgovernmentakingurethatevery ersonhas a goodstandardf iving?Any thers?)Now,what omesto mindwhenyouthink bout ettingachpersongetahead ontheir wn? Any thers?)Immediatelyollowingheseprobes, he nterviewereread heoriginalclosed-endeduestionndtook heperson's eplyo t. Fullquestionwordingsinthe ppendix.)The twotypes f probes re clearlynotequivalent. he "retrospective"probes,whichwereposedafter eoplehad answered he uestionnthenormalway,were esignedofind utwhat xactlywas onpeople'sminds tthemoment


11/39

588 John allerandStanley eldmanofresponse. he "prospective"r "stop-and-think"3robes, n the ther and,weredesigned o inducepeople to search heirmemoriesmore arefullyhanthey rdinarilyouldforpertinentonsiderations.ote hat he top-and-thinkprobes o notraisenew deas or push herespondentna particularirection;they imply equireherespondentosay explicitly hatmeaning e orshe at-taches o thedefininghrases fthe uestion.Respondents ererandomlyssigned o question orm nd answeredhesame type f thequestionneach wave of the tudy. he three est tems ndassociated pen-ended robes ppeared ear he ndofeachwaveof the urvey.Interviewersrote own s faithfullys possible ll responseso the pen-endedprobes, ncludingncidentalide commentse.g., "This s a tough ne"). Thetranscribedomments ere ubjectedo anelaborate lassificationcheme,withas many s four ommentsodedfor achprobe.4 espondentsnthe top-and-thinkideaveragedbout .7 codable ommentserpolicy tem,with lmost llrespondentsfferingt east necodable omment.heaverage n the etrospec-tive idewas 2.9.All comments,ncludingidecomments, ererated n severalvariablesbystaff oders t the nstituteor ocial Research t theUniversityfMichi-gan.Becausethe oding rojectwas considered difficultne,only xperiencedcoderswereused.The most mportantariablewas "directionalhrustf com-ment," hich ndicated hich ideof the ssue, fany, heremark avored. l-thoughhis ariable oted mbivalence,onfusion,ndnonissueoncerns,5%ofcomments ad a cleardirectionalhrust. he other eycoding lassificationwas "frame freference,"variablehatncludedmore han 40categoriesndtriedocapturehe ubstantiveontentfeachremark.heframe odesreferredtogeneral rinciplese.g., equality,heroleofgovernment),ndividualismndthe work thic, hefairness f theeconomic ystem, articular roupse.g.,blacks, he lderly), ersonal xperience,ndparticularovernmentrograms.(Furthernformationbout hese odesmaybe foundnTable5 and associatedtext; llcodes arefully escribedn the tudy odebook.)Thesedata renotwithoutimitations.he most bvious scoder eliability.About tenth fall interviews ere ouble-coded,nd, lthoughxact eliabilitydata renot vailable, he oding upervisoreported"difference"etweenod-erson 10% to 15% of all cases. This differenceatewasregardeds normal ormaterialf this ype,5ut t shigherhan newouldhopefor.naddition, 0%ofremarks ere o unclear hat hey ouldnotbe assigned directionalhrust.A final imitations thedifficultynconfidentlyistinguishingne"considera-

3This ptdesignationsthe nventionf Kathleen night.4For he id toblacks tem, herewereupto sixprobes: hree uestions, ach followed yaquery or anyothers?" s many s four emarks ere oded n connection ith ach of these ixprobes.On the thertems,herewere wo nitial robes, achwith ollow-uprobes.5Personalommunicationromteve inney, ho upervisedurproject t SR.


12/39

A THEORY OF SURVEY RESPONSE 589tion" rom nother.When woremarks aveclearly pposing hrusts,his s noproblem. ut people ometimesffer series fremarksn the ame ide of anissue. Do suchremarksepresenteparateonsiderations,r ust laborationsna single dea? Evena personisteningothe nterviewss they ccurred,s wedid,would ometimeseuncertain;odersworkingrom n mperfectranscriptwould xperience reaterncertainty.Even inretrospect,e are not surehow theseproblemsould have beensignificantlymelioratedn the ontext f a masssurvey.We thus eelthePilotStudy ata rethebest hat rereadilyttainableor irectlyxamininghe dea-tional nderpinningsf massattitudes,utwe admit hat hey re,nonetheless,far rom erfect.

Tests ftheModelPreliminaryheck ftheAmbivalencexiom

Webegin ssessmentf themodelbymaking plausibilityheckof theaxiom hat laims hatmost ndividualsoldmultiple,onflictingdeas on mostissues.Our datagiveus threeways omeasure he xtent f ambivalencen thepublic, s follows:1. A count fthenumberfopposing emarksyeachperson hat an bepaired gainst neanother f,for xample, respondent akes wocommentswith liberal hrustndtwowith conservativehrust,is score nthe onflictscale s two. f he makes hreeor more) n one sideofthe ssue ndonly woonthe ther, he onflictcore s still wo, ince henumberfopposing ommentsthat anbepaired emains wo.2. A count fthe imes eople spontaneouslyxpressmbivalencerdiffi-cultynmaking ptheirminds.A specialcodewascreated ocaptureuchre-marks;treads s follows: Mentionndicatesmbivalence,onflicte.g., 'I seemeritn bothsides'; 'That's a toughquestion'; Depends'; 'Both are validpoints')."3. A count fthenumber ftimes eoplemake"two-sidedomments."n-cluded ntheframe f referenceodesarespecial"star odes" thatndicatedirectionalhrustothe ommentut lsoambivalence ith espectothat irec-tion.Star odesapply o cases inwhich espondentsada preferenceutwereclearly ayingome ttentionothe ther ideofthe ssue. nstructionsocodersfor seofstar odesread s follows:A starcode is used onlyfor ases in which heres a single houghtrcommenthat ncompasseswo pposinglementse.g., "AlthoughthinkX, I neverthelessavor " Star odesareusedfor ommentsnwhich seestwo ides oan ssue.


13/39

590 John aller and Stanley eldmanExamples f star odes are"peopleshould ry ogetaheadon their wn,butgovernmenthouldhelpwhennecessary"ndrespondentadmits roblem(s)with ny programr type fprogram,ut nsists t is worthwhilenyway."count fthe tar-codedemarksmaythusbe considered measure f ambiva-lence.From hese hreemeasures e created fourth:summaryfthe ndices nwhich person cored + 1 or higher. ecause conflictnd ambivalencereequally onsequential hetherhey ccurwithinhe ourse f one interviewracrossseparatenterviews,ll indices re calculated crossbothwavesofthesurvey.Frequency istributionsn thesefourmeasures reshown n Table2. Ascanbe seen, ach measureapturesubstantialmountsfambivalence,resultthat s consistentith hefirstxiom.Evenonthemore onservativevidence ftheretrospectiverobes,which nvolve nly nequeryneachwave,6 he um-marymeasurendicates hat 6% to 48% of respondentsretosomedegree m-bivalentn these hreessues.And this s surely n understatement.hat heretrospectiverobes apture,s explained,sthe eason heperson as answeredthe tem s he usthas; they annot apturenythingike hefull ange f deasin theperson'shead.However,heprospectiverobesweredesigned otapawider ange fthe deas npeople'sminds; n evidence romhem, oughly5%ofrespondentsreat east omewhatonflictedn the hreessues.Theseresults rovide lear nitial upportor hemodel'sfirstxiom, heambivalencexiom.Sincethe therwo xioms annot etested irectly,eturnnow to an examinationf thedeductivemplicationsfthe hree xioms akenalltogether.First eductionsromheModel

Webeginwith eductionsromhemodel hat reentirelytraightforwardandperhaps ninterestingndproceed omore seful nd mportantnes. f,asthe ccessibilityxiom laims, he ccessibilityfa given onsiderationependson the mount fthoughtevoted o an issue,we should indhat eoplewhoare, ngeneral,more oliticallynvolved avemore onsiderationstthe opoftheir eads nd available or se inansweringurvey uestions. his s thefirstof 18deductionsrom hemodel hatwe make nd testDeduction ). (Tokeeptrackf thedeductions,achwillbe numberednparentheses,s here.)Despitesome ndicationfnonmonotonicitynthedata,Table3 essentiallyonfirmshisexpectation.7Similarly,ewould xpect ersonswhohavegreaternterestnan issueto

6More robesweremade,but ince heywere irective, e makenouse ofthem.7Thenonmonotonicitys probably eal, s a more ophisticatedersionfourmodel eadsonetoexpect seeZaller,npress, hap.8).


14/39

Table2. Expressions fConflictnd Ambivalence n Political ssuesRetrospectiverobes Stop-and-ThinkrobesConflictingonsiderations Conflictingonsiderations

Jobs Services Aid to Blacks Jobs Services Aid toBlacksCount0 739%a 57.8 73.4 36.9% 30.7 29.01 22.6 33.6 22.6 27.3 29.0 21.62 3.5 5.2 4.0 22.2 21.6 25.03+ 0.0 3.4 0.0 13.1 18.7 24.4

Expressionsf Ambivalence Expressionsf Ambivalence0 76.9 83.5 78.8 63.2 72.2 71.11+ 23.1 15.5 27.8 36.8 27.8 28.9

Two-SidedRemarksStarCodes) Two-SidedRemarksStarCodes)0 75.0 91.7 81.4 64.9 85.2 72.31+ 25.0 8.3 18.6 35.1 14.8 27.7

Total ndications Total ndications0 60.2 51.4 63.6 25.9 24.3 24.71 15.7 37.6 16.9 39.1 50.9 44.02 18.5 7.3 12.7 9.8 14.8 12.03 5.6 3.7 6.8 25.3 10.1 19.3Note: Measuresre describedn text.Source:1987 NES Pilot tudy.

Table 3. Effect fPoliticalAwareness nVolume fOpen-EndedCommentsLevelofPoliticalAwareness

Low Medium HighStandardf iving 2.3 2.9 3.5 3.5 3.2Governmentervices 2.3 3.2 4.3 4.6 4.7Aid toblacks 3.3 3.9 4.4 5.2 4.4N = 44 53 54 38 46Note: Cell entriesre the veragenumberf discrete ubstantiveemarksn thegiven ell. Thesedataare derived romtop-and-thinkrobes, utresults rom etrospectiverobes howthe amepattern.he awarenessmeasures describedn the ppendix.Source: 1987NES Pilot tudy.


15/39

592 John allerandStanley eldmanhave, llelse equal, more houghtsbout hatssuereadily ccessible nmemorythan ther ersonsDeduction ). Since thePilotStudy id notdirectlysk re-spondents ow mportantr interestingach of thepolicy ssues was to them,ourabilityo test his xpectations limited.Wedid,however, ind hat lacksraisedmore onsiderationshanwhites nconnection ith he id tominoritiesitem p < .01). Governmentmployees lso hadslightly ore osaythan therpersons boutthegovernmentervices temp = .07). Unemployedersons,however,were notmore ikelyto raise considerationsertinento the obguaranteestem.Axioms and3 claimpeople nswer urvey uestions yaveragingcrosswhateveronsiderationsre alient nmemoryfthis s so,weshould ind trongcorrelationsetweenmeasuresfpeople's houghtss theynswer surveytemand thedirection f decision n the tem tselfDeduction ). Thus f,for x-ample, personmakes woremarkshat avor he iberal ide of the ssueandone that avors he onservativeide,wewould xpect hat hepersonwould, naverage, ake the iberal ide of the ssue. Althoughhis nference ay seemhardly orthesting,t s byno means bvious hattcan be confirmed.ocialpsychologists,orkingn thedomain fsocialcognition,aveturnedpcasesinwhich hedirectionfpeople'sopen-endedhoughtss uncorrelated,revennegativelyorrelated, ith valuationsfthegivenssue.

As Hastie nd Park 1986) havecontendedn an influentialssay, he ur-prising oncorrelationsccurbecausepeopletypicallyo not onstructttitudestatementsrom deas they an retrieve rommemorys they requestioned.Instead, eoplerecall attitudesormed t anearlier ime.Thus,theymaintain,theres nonecessaryorrelationetweenop-of-the-headdeas nd ttitudetate-mentsseeLodge,McGraw,ndStroh 989).Notwithstandinghis, urdata ndicateubstantialorrelationsetween heideasmost ccessible o ndividualst themomentfresponsendthe esponsegiven.To show his,we created dditivendices fpeople's pen-endedemarks,codedfordirectionalhrust.Wethen orrelatedhese ndiceswith esponsesothe losed-endedtems, s shown n Table4. Onthe top-and-thinkide,corre-lations etween he ndices nd theirssociated ichotomoustemn eachwaveofthe urveyveragedbout 40. When n index f allremarksverbothwavesof the urveys correlated ith scale that onsists fresponsesothe losed-ended tems rom he wowaves, he orrelationsverage bout 50. Inthe therhalf fthe tudy,he orrelationsetweenndividuals'etrospectiveemarksndtheirlosed-endedesponsesnthe amewaveof the urveyveraged70. Whenretrospectiveemarksnd temswere ummed ndcorrelatedcrosswaves fthesurvey,hecorrelationsveraged80. (Muchof the differencen correlationsbetween he top-and-thinkide and theretrospectiveideappears obe due toerrornthecodingoftheclosed-endedesponses,which, s discussed elow,seemstohavebeenhigher n thestop-and-thinkide.) Giventhat heclosed-


16/39

A THEORY OF SURVEY RESPONSE 593Table 4. Relationship etweenAvailableThoughts nd Closed-Ended tems

Wave1 Wave2 CombinedCorrelationsithRemarksMade Just eforeAnsweringlosed-Ended uestionJobguarantees .39 .39 .50(212) (161) (173)Governmentervices .31 .36 .41(187) (153) (165)Aid to blacks .57 .48 .63(220) (165) (166)

Correlations ithRemarksMade JustfterAnsweringlosed-Ended uestionJobguarantees .79 .70 .79(126) (123) (105)Governmentervices .79 .70 .78(137) (105) (106)Aid to blacks .67 .83 .83(144) (114) (112)Note:Cell entrys Pearson orrelationetweenpen-endedemarks adeustbeforeor ustafter)answeringlosed tem nd cores n the losed tem. irstolumn hows orrelationsromirst aveofsurvey; econdcolumn howscorrelationsrom econdwave;third olumn hows orrelationsbetween emarksrom othwaves nd combinedtem cores nbothwaves.Source: 1987 NES Pilot tudy.

ended temsnthese ests reessentiallyichotomies,8ndthefragile aturefouropen-endedata, hese resizable orrelations.9We should dd that hefindingsf correlationsetween op-of-the-headthoughtsnd attitudeeportsrenot,taken lone, clinchingvidence hat heformer ave ausedthe atter. ather,he orrelations-which,s wehave ndi-cated, ouldnotbe taken or rantedn ight fpastresearch-simply epresentone ofmore han dozencasesinwhichwehavebeen ble todevelop videncethatsconsistent ith he hree-axiom odel.8A few espondentsolunteereditdepends"middle esponses, hichwere ccepted.9The substantial agnitudesf these orrelationso not,however,howthat losed-ended

survey esponsesre,afterll, perfectlyalid ndicatorsfthe onsiderationsnpeople'sheads.Aswe note mmediatelyelow, ifferentonsiderationsresalient tdifferentimes,eading eople omakedifferentttitudeeportst differentimes.What able4 shows s only hat losed-endedtti-tude eportsre reliable ndicatorsfthe onsiderationshat re salient t themomentfmakingresponse.


17/39

594 John aller andStanley eldmanExplaining esponsenstability

Response nstabilityverrepeatednterviewss, as we have ndicated,neof the most mportantnd disturbingmpirical egularitiesssociatedwith hemass survey esponse.n this ection,we attempto use ourmodel o explainthis nstability.e beginwith imple llustrationsf our pproachnd then ro-ceed tomore ystematicnalysis.When askedin the May interviewboutthe proper evelof governmentservices, ne respondent,dentifieds a teacher, mphaticallyavored igherlevelsof services nd spending. he country as facing n educational risis,the eacheraid,and more xpendituresor ducationweredrasticallyeeded.Any cuts n federal ervices r spendingwould nevitablyeduce he alreadyinadequate unds vailable or ducation. ust monthater, owever,he ameindividual avoreduts ngovernmentpending. overnmentas too big andhadto be cut back.Therewas no referenceo theeducationalrisis hathadpreoccupiedhisndividualust fewweeks arlier.0Researchers ave long known hatdifferenteoplecan answer denticalquestionss ifthey oncernedifferentopics.What hevignettef thevacillat-ing eacher hows sthat he ameperson an answer he amequestion t differ-ent times s if t involved ifferentopics.This can happen, ccording o themodel,becausetheconsiderationshat etermineeople's urveynswers aryacross nterviews.hus,peoplecangivestronglyelt, ontradictoryurveye-sponseswithoutitherhangingheirmix ffeelingsn the ssueorconsciouslyexperiencingnyambivalencer conflict-if heparticularonsiderationshatdetermineheirurveyesponses ave hifted.Our data werecollectedwith specific iewtodetectingndmeasuringsuchshifts. able5 presentslosed-endedummariesfthesedata for our e-spondents.Note hat hesedata arefromhe top-and-thinkide, nwhich e-spondents ere ncouragedothinkbout ssues omewhatmore ullyhan heyordinarilyould.Consider espondent. His firsteactiono a guaranteedtandardf ivingwas thatt was inconsistentithAmericandeals;hewas also botheredytheunfairnessfsupportinghosewho refuseowork.Yethe worried bout ettingindividualsetaheadon their wn, saying hat omepeopleneedspecialhelpandthat ocietyhas an obligationohelptheneedy.n thesecond nterview,however,herewasnosignofthis mbivalence. espondent gavesixreasonswhy ndividualsught oget headontheirwn, ncluding restatementfhis

'?Wewould iketopresent erbatimranscriptionsfwhat his ndother espondentsaid.However,heHuman ubjects ommitteet theUniversityfMichigan asdeterminedhatuchuseofthe awprotocols ould e an invasionfthe espondents'ightoprivacy."Theselections ere ubject othe onstrainthatwe wanted worespondentshowere tableon the losed-endedtems ndtwowhowere nstable.


18/39

Table5. Content fOpen-EndedResponses n Job GuaranteesQuestionCouldyoutellmewhat inds fthings ome o mindwhen outhinkbout ..FirstWave SecondWave

... governmentak- ... lettingachper- ... governmentak- ... lettingachper-ing ure hat very songet head on ing ure hat ach son get head onpersonhasa good their wn? person asa good their wn?standard f iving? standardf iving?Respondent : (StableConservativen FixedResponse)110. Idea is un- 180. Somepeople 158. Taxburdenoo 140. Individualism/American needhelp Lib.) greatCon.) work thic(Con.) (Con.)161. Duty ohelp he 110. Idea is un- 133. Equal opportu-

needy Lib.) American nity xists or(Con.) all (Con.)136. Unfairf ome 346. Program/food 344. Program/educa-don'twork stampsCon.) tion Con.)(Con.)145. All shouldmakeit alonebutsomeneedhelp(ETU)Respondent: (StableConservativenFixedResponse)158. Tax burden oo 344. Program/educa- 344. Program/educa- 140. Individualism/greatCon.) tionLib.) tion Lib.) work thic(Con.)156. Gov't.red ape 344. Program/educa- 219. Group ef. o 142. Shiftless eople(Con.) tionLib.) middle lass deserve ate(Con.) (Con.)144. Valueofcom- 110. Idea is un- 152. Limited ov't. 147. Motivationopetition American (Con.) workCon.)(Con.) 140. Individualism/work thic

(Con.)Respondent: (Unstable, onservativeoLiberal)140. Individualism/ 140. Individualism/ 151. Gov't. must n- 140. Individualismwork thic work thic sure qual opp. work thic(Con.) (Con.) (Lib.) (Con.)161. Duty ohelp heneedyLib.)Respondent: (Unstable, iberal o Conservative)344. Program/educa- 140. Individualism/ 150. Idea of welfare 111. Fairnessf

tionLib.) work thic stateLib.) Amer. ystem(ETU) (Con.)


19/39

596 John allerandStanley eldmanTable (continued)

Couldyou ellme whatkinds f hings ome o mindwhen ou hinkboutFirstWave SecondWave... governmentak- . . . lettingachper- . . . governmentak- . . . lettingachper-ing ure hat very songet headon ing ure hat ach songet head onpersonhas a good their wn? personhas a good their wn?standardf iving? standardf iving?357. Program/hous- 348. Program/healthing Lib.) (Lib.)356. Program/wel- 143. Work welfarefareLib.) (Lib.)Note:A full escriptionfthese odescan be foundnthe CPSR codebook or he1987Pilot tudy.Each remarks identifiednparenthesess having liberal, onservative,r uncertainirectionalthrustETU indicates hat he valuationalhrustfthe emark asunclear). valuationalhrust ascoded ndependentlyfsubstantiveontentf remark.feelinghatob guaranteesreun-American,ithoutaisingnyopposing on-siderations.Respondent , who opposedgovernmentob guaranteesn bothof hisclosed-endedttitudeeports,s thusess stablenhisreactionotheguaranteedstandard f living uestion hanhisstable losed-endedesponseswould ug-gest.He wentfrom eing n ambivalentonservativen this ssue tobeingconfidentonservative.iven he mbivalencefthe nitialnterview,newouldnot esurprisedf,overmanyncounters ith his uestionrslightlyephrasedversions fthe uestion, eoccasionallyawthe entralssue s aid totheneedyratherhan n-Americandealsand,onthis asis,expressedupport or he o-cialwelfareption. ertainlye hasreal mpulsesnthat irection.Now consider espondent. Althoughhe s ambivalenttbothnterviewson the basis of similar onsiderations,hechangesherclosed-endedesponsefromonservativeo iberal.One can maginehathemayhave, neffect,osseda mental oin indecidinghowto answer hefixed uestion-notbecause,asConverse's onattitudeshesis uggests,hehad nomeaningfuleactionotheissue clearly,hedid),and notbecause, s themeasurementrrormodels ug-gest, hewasnot uite urewhat he uestionsked shesaw itthe ameway tbothnterviews),utbecause he wasundecidedetweenargelytable ut on-flictingmpulses.These cases showthat ven f,as we contend, eople base their ttitudereportsnthe deasthat remostmmediatelyalient othem, t sbynomeanseasytoexplain ver-timeesponsenstability.nthemost traightforwardase,instabilityrisesfromhangesnthe onsiderationshat re mmediatelyalient


20/39

A THEORY OF SURVEY RESPONSE 597at the ime fmakingn attitudeeport,s in the ase ofthevacillatingeacher.Butother,uitedifferentcenariosre lsopossible, swehave een. Withhesecomplexitiesn mind, et us proceed o a more ystematicxaminationf re-sponse nstability.If, as the modelclaims, ndividuals ossesscompetingonsiderationsnmost ssues, nd f heynswer n thebasisofwhateverdeashappenobeatthetopoftheirminds t themomentfresponse,ne would xpect fair mountfover-timenstabilitynpeople's ttitudeeportsDeduction). The reasons thatthe onsideration(s)hat re stochasticallyccessible tone nterview ight otbe so prominentt thenext.This nferences stronglyupportedya massofexistingvidence e.g., Table1).

Themodelnotonly nticipatesesponsenstability,ut lso expects t tohave definitetructure.uppose hat 0% ofthe onsiderationsn oneperson'shead nduce er oward liberal esponsen a givenssue,while 0% induce ertoward conservativeesponse;ndsuppose hat or second erson, hese ro-portionsre reversed.f each based hersurvey esponses n a one-elementsamplefrom hedistributionf considerationsnherhead,each would xhibitresponsenstabilityver ime, utover he ongrun, hefirstersonwouldbeliberal 0% of the ime nd the econdwouldbe conservative0% ofthe ime.Thus,citizenswouldhavecentralendencieshat re stable ver ime, ut heirattitudetatementsouldfluctuatereatlyround hese entral endenciesDe-duction ). This s, infact, xactlyhepatternhat as beenobtainedepeatedlybyresearchersnthe measurementrror"raditionAchen1975;Erikson 979;Judd nd Milburn 980; Judd,Milburn,nd Krosnick 981; Feldman1989;Zaller1990; ee,however,rosnick 988).If, s shownnTable3,more oliticallyware ersons ave larger umberof considerationst thetopoftheir ead and accessibleforuse in answeringquestions, hey hould, ll else beingequal, exhibit reatertabilityn theirclosed-endedesponses. hereasonsthat ttitudeeportsormed rom n aver-age ofmany onsiderationsill be a more eliablendicatorftheunderlyingpopulationfconsiderationshan naverage asedon ustone ortwo onsidera-tionsDeduction ).Althoughnitial esearchailed oconfirmhis xpectation,eldman1989)and Zaller 1990) havemore ecentlyhownnseparate ata setsthat oliticalawarenesss, infact, ssociatedwith reductionnthechancevariationsso-ciatedwith eople's ttitudeeports.Thedifferenceetween he nitial nd atertests f this xpectations that he ecent ork ses tests fpoliticalnformationas themeasurefpoliticalwareness.)Byparallel ogic,people hould e more table ntheir esponsesoclose-ended olicy tems oncerningoorstepssues-that s, issues oclose toevery-day oncernshatmost eople outinelyive ome houghtothemDeduction).


21/39

598 John allerandStanley eldmanEvidence endingo supporteduction may e foundn thediscussionfracialissues in Converse 1964), thediscussion f moral ssues in Converse ndMarkus1979),andgenerallyn Feldman 1989).Anothermplicationf ourmodel s that reatermbivalenceught o beassociatedwith igherevels fresponsenstabilityDeduction ). Since, s wejustsaw, ome ndividuals hoexhibit oapparentmbivalence ithin singleinterview aynonethelesse quite onflicted,t s essential hat,n testinghisimplicationfthemodel,weemploy measure fambivalencehat pansbothinterviews.ccordingly,ehavebuilt measurefthe xtento which n ndi-vidual's onsiderationsonsistentlyavor ne or theother ideof a given ssueacrossbothwavesof the urvey.Weconstructedhismeasure y means f thefollowingormula:I (liberal emarks) X (conservativeemarks)

X (liberal)+ E (conservative) X (ambivalent)A scoreof one on thismeasurewould ndicate hat heperson's emarks ereeitherll liberal ntheirhrustr all conservative,hile scoreof zero wouldindicate hat hepersonhadmadean equalnumberf liberal nd conservativeremarks.We expect ighertabilityn the losed-endedtems or ases n whichall oftheperson's emarksun nthe amedirection.he data n Table6 supportthis xpectation.n five f sixtrials,hismeasurewas associatedwith statisti-cally ignificantncreasenresponsetability;n the ixth ase,the elationshipachievesmarginaltatisticalignificancep = .07).Theresultsre not,however,s strongs theymight e. If peopleformedresponses y averagingcross accessible onsiderations,tabilityates houldvary etween 0% (forpeopleevenly ivided ntheironsiderations)o 100%(forpeoplewithperfectlyonsistentonsiderations).articularlyn thestop-and-thinkide,thedatasignificantlyepart rom his xpectation.ow can thisbe explained?The most ikely eason or he hortfallromxpectationss coding rror.As reportedarlier,andomheck-codingndicatedhat oders isagreedn thecoding f10%to15% of allopen-endedemarks,nd uchmiscodingbviouslyimpairsur bilityodeterminehich espondentshould eperfectlytable ndwhich erfectlyandom.To test his xplanationor he mperfectesultsnTable6, wedevelopednitem-level easure fcoding rrornd correlatedt with hemagnitudef thestability-consistencyelationshipshownn Table6. Thecorrelationas,as ex-pected, uitehigh.2Moreover,oding rror asconsistentlyighernthe top-

'2To stimatetem-levelrror ates,we used tem-levelercentagesf remarkshat he odersfound o beuncodable. he rate funcodable emarksaried rom low of bout % for ne tem oa high fnearly0% for nother.Weexpectedhathemagnitudefthe tability-consistencyelation-


22/39

A THEORY OF SURVEY RESPONSE 599Table6. ResponseStabilitynd ConsistencyfConsiderations

Job Government Aid toGuarantees Services BlacksRetrospective onsiderationsConsistencyof onsiderations: N N N.00 .50 (7) .59 (11) .57 (7).01 to 50 .80 (20) .70 (25) .71 (19).51 to .99 .77 (15) .78 (16) .80 (15)1.00 .91 (63) .87 (54) .96 (71)

(p < .01) (p < .02) (p < .01)Stop-and-Think onsiderations

Consistencyof onsiderations: N N N.00 .63a (16) .54 (14) .57 (14).01 to 50 .68 (74) .77 (63) .83 (66).51 to 99 .73 (37) .80 (50) .84 (44)1.00 .88 (45) .73 (37) .88 (41)(p < .02) (p < .07) (p < .01)Note:Cell entriesreproportiontablentheiresponseso closed-endedtems rom ave 1 towave2. Measure f consistencys describedntext. -values re basedonuncollapsedmeasure.Source:1987NES Pilot tudy.and-thinkide,which xplainswhy he hortfallromxpectationss highernthat ormfthe urvey.3Withinhe imits f thedata, hen, he esultsn Table6 areprobablyboutas strongs couldbe expected.We obtainedizablestabilityffectstconven-tionalevels fstatisticalignificancen five fsixtrialsndwere loseto signif-icanceonthe ixth, espite ignificanteasurementrror, small ample, ndabnormallyow base rates f instabilitywing o therelativelyhort imebe-tween einterviews.One additional ointneedsto be made.The measure f consistencyn

ships n Table6, as summarizedyunstandardizedegressionlopes,wouldbe greatestor hoseitemswithhe owest ates funcodableemarks.hiswasstronglyhe ase r2= 0.93ona logafith-micfit; f= 5; p < .05). Furtheretails fthis est rereportedn Zaller inpress, hap.4).'3It s, wemightdd,not urprisinghat rror ates rehighern the top-and-thinkide. Itwould eemeasier, priori,o determinehedirectionalhrustf a remarkf theremarkas beenmade n ustificationf anopinionustrendered,s on the etrospectiveide, hanf thasbeengivenaspart fa discussionfwhat given hrasemeans o the espondent,s on the top-and-thinkide.


23/39

6oo John aller and Stanley eldmanTable 6 was calculated verbothwaves fthe urveynorder o capture onsist-ency f considerationsothwithinnterviewsnd across hem. he consistencymeasure an, however, e calculatedwithin single nterview. henthis sdone,we comeupwith strong,learfinding:he ne-wavemeasure fconsist-ency f considerationsas almostnocapacity o predict ver-timeesponse n-stabilityineitherorm).This findingndicates hat he onflictmost esponsibleor esponsensta-bilitysconflicthat ccurs cross atherhanwithinnterviewsndthat espon-dents re often naware f their onflicts they nswer uestions. hus,thevacillatingeacher xhibited oconflictvergovernmenterviceswithin itherinterview,ut ubstantialonflictcross nterviews. ost ikely,ncethe eacherbegan o viewthegovernmentervicestem hroughheprismf either bloatedgovernment"r "education risis,"he or she fell nto mindset hatblockedthinkingbout theother oint f view.'4This suggestionitsnicelywith urmodel,which olds hat eople nswerurvey uestions astilyndonthebasisof ncomplete emoryearches.Explaining Response ffects"

The model an also explain everalmportantesponseffectsn mass ur-veys, hat s, cases in which eeminglyrrelevanteaturesfquestion rder rdesign ffectsheresponses iven. n this ectionwe survey varietyftheseresponse ffects.Consider irstorder ffects,"uch s the ffectfa prior uestionnsup-port or herightsf communisteporters,s discussed arlier.f, as themodelclaims,people are normallymbivalentn issuesbutanswer n the basis ofwhateverdeas aremost ccessible t themomentf answering, aising ewconsiderationsn mmediateroximityoa questionhould e ableto affectheanswers iven ymaking ifferentonsiderationsalientDeduction ).The intrusionf unexpectedr novel considerationsnto thequestion-answering rocesswouldnot, however, e expected o affect ll respondentsequally. omepeople maypossessconsiderationshat re so consistentnsup-port f one side of an issuethat he dmission fonecompetingonsiderationshould aveno effect.thers, owever, ay edeeplymbivalentnthe ssue-thats, maypossess roughlyvenbalance fpro ndconconsiderations.heseare thepersonswho shouldbe most tronglyffectedyartificialhanges nquestionrderDeduction 0).Tourangeaut al. (1989) havereportedupportor his xpectation.eoplewhoreportedhat hey ad mixed eelingsbout n issue andwho lso said the

'4Tourangeaund Rasinski 1988) distinguishetweeninterpretation"f a questionnd "re-trieval" f nformationelevantoansweringt. n termsf this istinction,urresultsuggest hatinterpretationsthemoremportantource fresponsenstability.


24/39

A THEORY OF SURVEY RESPONSE 6oiissue was very mportantothem)werequite usceptibleoquestion rder f-fects-or what heauthors all "carryoverffects" while ther ersonswerenot usceptibletall. The carryoverffectsn thevulnerable roups where hegroups aried romssueto ssue)ranged romowsof4 and 8 percentageointson abortion ndwelfare, o highs f 34 and 36 percentageoints n aid to theContra ebels nd defensepending, ith naverage arryoverf19 percentagepoints.Anothermportantype fresponseffects whatmaybe called the en-dorsementffect."n this ase, referenceo a political igurergroup ystem-atically lters hepublic'sresponseso a givenpolicydilemma. orexample,Mueller 1973) foundhat uestionshatndicatedhat residentohnsonavoreda particularolicyoption ed togreaterupportor t. Similarly,eferencesocommunismystematicallyncreasedupportorU.S. involvementnKorea ndVietnam. hisanticommunismndorsementffect,ccordingoMueller, ug-gested somewhatonflictingbservations.n the nehand, upportor hewarwas clearly ied othe nti-CommunistpiritnAmerica t the ime.To generatea kind f warfever,nemerely adto tossthewords,Communistnvasion,'into hediscussion. n the ther and, heCommunistlement as not ntirelybuilt nto heresponse othewarbecause Americans ad to be remindedf itbefore heir nti-Communismasfully ctivated"1973, 48). Thetendencyfpeople obaseattitudeeportsnthe deasthat re mostmmediatelyalient othem,s specifiednAxioms and3,well xplainsuch ffectsDeduction 1).Althoughmost esponse ffectsreconsideredmethodologicalrtifacts,"theyre ometimesiven ubstantiveames r nterpretations.onsiderhese:

1. Race ofinterviewerffects.hortlyfter 1986NewYork imes ollfoundhat residenteagan's pproval ate mong lackswas37%, a Washing-ton ostpollestimatedhat lack pprovalfReaganwasonly 3%. Thediffer-ence wastraced o thefact hat,while heTimes ollowed ormalnterviewro-cedures, hePostusedblack nterviewersho nformedheir lackrespondentsthat heywouldbe participatingna study f the ttitudesfblackAmericans.As Sussman 1986) maintains,he ikely ffect f thiswas to induceblackre-spondentso "think lack" ntheir valuationsfReagan's erformancethefirstitem n the urvey).5

'5This ndother ace-of-interviewerffects ighteinterpreteds "socialdesirability"ffects.Ifbysocialdesirabilityffectsne means ases in which eopleconsciouslymisreportttitudesnorder oavoidembarrassment,socialdesirabilitynterpretations at variancewith urargument.But f, s seems qually onsistent ith vailable vidence,ocialdesirabilitys taken omean asesin which eople reunsure hat heir ttitudesrebut re nfluencedy hemmediateontextogivegreater eighto a particularonsideration,hen ocialdesirabilityffectsepresentimplynothertype fsituationn which eoplemake ttitudeeportsnthebasis of the deasmost mmediatelysalient o them.


25/39

602 John allerand Stanley eldman2. Reference roupeffects.n a classic social psychologyxperiment,ChartersndNewcomb1958) found hatRomanCatholicsweremore ikely o

state ttitudeshatwere onsistent ith hurch octrinee.g., on birthontrol)if, ust prior oquestioning,heir eligion as made alient o them. his effectwas taken s evidence fthe mportancef"referenceroups"n attitudes.3. "Primingffects"ftelevision ews.AccordingoIyengarndKinder(1987) and Iyengar1991), television ewsoften unctionso "prime" ertainideas,therebymaking hemmore ccessibleforuse inevaluatingresidentialperformance,eciding etween andidatesn elections, ndassessing hena-tion'smost mportantroblems.n thisway,TV news s saidtoaffectttitudereports ithoutermanentlyhangingndividuals'nderlyingttitudes.4. "Framingffects"fquestionwordingndquestionrder f the onsid-erationseopleuseinansweringuestionsanbeprimedyTV news, hey analsobe primed yhowquestionsre worded r framed. or example, au andSears 1983) andLau, Sears,andJessor1990) have xperimentallyhown hatquestionsboutpersonal inancialtatus re more tronglyorrelated ith val-uations f ncumbentoliticians hen he uestionsre sked n close proximity.Thereason, resumably,s that considerationsed nansweringnequestionremains vailable or nsweringubsequentuestions,herebynducing corre-

lation.6Themechanismesponsibleor achof these ffectsppears obe a ten-dency or eople oanswer uestionst eastpartlyn the asisof deas hat avebeenmade momentarilyalient o them.As such,theymaybe counted s re-sponse ffectsorwhich urmodel f the urveyesponse ives nexplanation(Deductions 2,13, 14, 15).Milbum1987) hasdocumentedet nother ay n which uestion rdereffectsanoccur.He found hat sking espondentso"tellmeeverythinghatcomestomindwhenyouthinkf a Liberal Conservative)"aused subsequentattitudeeportsobe more deologicallyonsistent ith ne anotherhanweretheattitudeeportsf a control roup.Milbum btained his ffect, owever,onlyforpersonswho hadeitheriberal r conservativeeanings obeginwith.Price 1991) hasrecentlyeplicatedhesefindingsn an experimentonductedoncollege tudentsnd nnonexperimentalatafrom ationalamples.nbothcases,Pricefound hat implysking ersons oplacethemselvesn a liberal-conservativeatingcale wassufficiento ncrease he deological onsistencyfresponses o subsequent olicy tems-but,again,onlyforpersonswhopos-sessed a clear deological eaning i.e., werenotcentristsr unableto placethemselvesnthe elf-ratingcale).'6Theres disagreementnwhetherhese ffects ave rtificiallynflatedstimatesf"pocket-book voting"n NES surveyssee Lewis-Beck 985;Lau, Sears, ndJessor 990),butnodisagree-ment hat ramingffectsould, nprinciple,o so.


26/39

A THEORY OF SURVEY RESPONSE 603Thistype fquestion rder ffects alsoexplainable romhemodel.Hav-inghadtheirdeological rientationsade aliento themustprioroanswering

policytems, hose espondents hopossess uchorientationsre more ikely orely n them s a considerationnformulatingesponses osubsequent olicyquestions, herebymakinghoseresponsesmore tronglyorrelated ith heirideological ositionsndhence lsomore deologicallyonsistent ith ne an-otherDeduction 6).Thissection as examined varietyfempiricalegularities-questionr-der ffects,ndorsementffects,ace-of-interviewerffects,eferenceroup f-fects, rimingffects,uestion rameffects,nd the ffectfmakingdeologysalient.Withinonventionalttitudeheory,omeofthese mpiricalegularitiesaretaken s substantiveindingsnd some s methodologicalrtifacts. ehaveshown, owever,hat llmaybeexplaineds manifestationsf a commonheo-reticalmechanism, amely,henormalendencyfpeople orespondosurveyquestions n thebasis of the deas that appen obe, forwhatevereason,m-mediatelyalient o them.Explaininghe ffectsfExtraThought

Survey esponses, s conceivedhere, renot attitudes"er se; they reunreliablendicatorsf themixofconsiderationsn theperson'smind-unreli-able because, amongother hings, eople normallynswerwithoutetrievingfrommemoryllrelevantonsiderations.f,however,eople ouldbeartificiallyinduced oretrievelargerhannormal umberfconsiderations,tshould m-prove he eliabilityf theiresponsesoclosed-endedtems.Our ntentndesigninghe top-and-thinkrobeswas tocreate uch n in-ducement. y requiringndividualsodiscuss he lements f a question eforeansweringt,wewere nducinghemo call tomind ndtake ccount fa widerrange f deas than hey ormally ould.Wethereforexpected hat esponsesfollowinghestop-and-thinkreatmentouldbe, all else equal,more eliableindicatorsf the setofunderlyingonsiderationshanresponsesmade n thestandard ay, hats,inthe etrospectiveonditionDeduction 7).Unfortunately,urabilityo test his xpectations compromisedyan ar-tifact. ecauseoftheuse of anexplicitnointerest"ptionntheretrospectiveconditionutnot nthe top-and-thinkondition, ore espondentsailed ntheretrospectiveonditiono respond o the ssue item.Low-awarenessersonsweremost ffectedythisquestion ifference;heir o opinion ate veraged38% inthe etrospectiveondition ut nly % inthe top-and-thinkondition.'7Thismeans hat etrospectiveespondents,speciallyess-awarenes, re moreselected roupndwould, or his eason lone,beexpectedo bemoredeolog-ically onsistenthan heirtop-and-thinkounterparts.his rtifactuns gainst

'7By ontrast,hedifferencennoopinion ates crossformsveraged 3% (16% vs. 3%) inthehighestnformationuintile.


27/39

604 John aller andStanley eldmanthegrain fthe nticipatedtop-and-thinkffect, akingtmore ifficultodem-onstratehe ffect,articularlyor ess-aware espondents.

We developed wotests f the xpectationf ncreasedesponse eliabilityin the top-and-thinkondition.n thefirstest,we expected measure f socialwelfare deology see appendix) o be more tronglyorrelated ith hetargetitems jobs, governmentervices,ndaid toblacks) nthe top-and-thinkondi-tion han n the etrospectiveondition.We usedthe ollowingnteractiveegres-sionmodel o test his xpectation,here orm efersoquestion orm:Item= bo + b, x Form+ b2 x Ideo. + b3 x Form x Ideo.Whenwe estimatedhismodel or espondentsho cored nthe pper 0%

of ourmeasure fpolitical wareness, e found hat he riticaloefficient,3,ran nthe xpected irectionor ll threetems, ut chieved tatisticalignifi-cance nonly ne case. To increasehe tatisticalower f the nteractionest nour smallsample-the numberfcases in each test veraged bout 140-wereestimatedhemodelunder heconstrainthat ll coefficientse equal acrossthe hreetems. heresults,hown nthe oppanel fTable7, confirmedxpec-tations:he ffectf deologys twice s large nthe top-and-thinkondition,differencehats statisticallyignificant.We also estimatedhemodel,under he ame constraints,or espondentsscoringn the ottom0% ofthe warenessmeasure. erewe foundhat,s also

Table 7. The Effect fStop-and-ThinknIdeologicalConsistencyLow Awareness HighAwareness

Intercept 0.01 - .20Ideology 0.89 0.62(.21) (.14)Form -0.36 0.00(.21) (0.20)Ideologyx Form -0.24 0.62(.27) (.26)N= 434 437

Note:Model s shownntext;modelwas estimatedimultaneouslycross hejobs, services, ndminorityid items.Data are taken romwave 1 of thesurvey,xceptncasesin which espondentsnswereddon'tknow" nwave1, n which ase their ave2 responses,f ny,wereused.Testwas conductedfor espondentscoringnthebottom 0% andtop40% ofawareness cale.Measures redescribednthe ppendix.Source: 1987NES Pilot tudy.


28/39

A THEORY OF SURVEY RESPONSE 605shownnTable7, the top-and-thinkestnot nly ailed o ncreaseonsistency,butmight ctually avereduced t.We shallreturnothis pparent eversaln amoment. utfirst, e havea second est fDeduction 7. Ifextra houghtn-ducesmore eliable ttitudeeports,t should nhance ot nly orrelations ithideology utalso theover-timetabilityfthese esponses.As can be seen nTable 8, however,hedatacompletelyailto supporthis xpectation.n fact,less-aware eople xhibitess consistencyn the top-and-thinkondition, hilemore-awarenes showno effect.It is essential o evaluate heseresults n light fthe election rtifact ehavedescribed. hegain nreliabilitymong ighlynformedersonsnTable7runs gainst hegrain fthe rtifactnd o isespeciallyikely obereal.The null

Table 8. The Effect fStop-and-ThinkonTest-Retest orrelationsRetrospective Stop-and-Think

LowAwarenessJobguarantees .68 .45(40) (62)Governmentervices .56 .43(41) (58)Minorityid .79 .53(53) (57)

MiddleAwarenessJob uarantees .64 .41(29) (60)Governmentervices .48 .38(31) (51)Minorityid .81 .51(33) (58)

HighAwarenessJob uarantees .70 .55(39) (51)Governmentervices .61 .58(37) (48)Minorityid .70 .86(32) (45)

Note: Cell entriesretest-retestearson orrelations. umbers re nparen-theses.Source:1987NES Pilot tudy.


29/39

6o6 John aller and Stanley eldmanfindingsnd apparent eversals, ecause theymight ave beencaused by theartifact,re more uspect.Butmight ot he eversalslso representealeffectsof the top-and-thinkreatment?ndeed, heymight.n an impressiveeries fexperiments,im Wilson and colleagues Wilson et al. 1989; Wilson andHodges 1991)haveshown hat, ontraryo ourmodel, sking eople o articu-late thereasons or heir ttitudesonsistentlyeduces hepredictiveeliabilityof attitudeeports, specially orpersonsess knowledgeablebout hegivenattitudebject.Theexplanationor hedisruptiveffectsf thought,s Wilson t al. main-tain, s that sking eopleto thinkbout hereasonsfor heirttitudesausesthem osample deas that re tooheavilyweightednthedirectionfcognitivereactionsothe ttitudebject atherhan ffectivenes. Attitudeeportshat rebased on thisunrepresentativeample re,as they onclude,ess reliable hanreports asedonthe deasthat reotherwise ost ccessiblenmemoryNote hat his rgumentccepts he entral ssertionfourmodel,which sthat eople's ttitudeeportsandalso,as theWilson tal. studieshow, ehav-iors)reflecthe deasthat re at thetopof theheadat themoment f decisionratherhan ny deeper ype f "true ttitude."n fact, t is precisely ecauseattitudeeports epend nimmediatelyalient deas that xtra hought,nbring-ing biased ample f deasto the opofthehead,proves isruptive.

TheargumentfWilson t al. about versamplingognitionst the xpenseoffeelingsouldexplain heunexpectedesultsweobtainednTables7 and 8.Sniderman, rody,nd Tetlock1991) have massed onsiderablevidence hatless-aware ersonsremore ikely o base attitudeeportsnfeelings, hereasmore-awarenes tend orespondn thebasis of deological rinciple.t wouldfollow rom his hat he top-and-thinkreatment,ith tsrelativelyognitiveflavor,houldprovemostdisruptiveo the ttitudeeportsf the ess awarewhich s whatwe havefound.Sincethis osthocexplanationunsnthe irectionfthe electionrtifact,we cannot e surethat t is correct. tnonetheless akes heuseful oint hatfailure o confirmDeduction17) maybe due to a thought anipulationhatfailed o mimic eople'snaturalhoughtrocesses.fso,a more arefullyraftedmanipulation ight et roducehe eliabilityain hat urmodel nticipates.'8

SummaryThe modelwe haveproposeds, like all models, simplificationfwhatactually ccurs. In addition, t has less formal recision hanconventional'8There re, n fact, sychologicaltudieshowinghathought anipulationshattress itherthe ffectivercognitivespects f ituationsroduce redictablyifferentffects,isruptingttitudereportsn some ituationsutnot therssee Millar ndTesser 986). Forfurtheriscussion f theseissues s they elate o themodel, ee Zaller forthcoming,hap.5).


30/39

A THEORY OF SURVEY RESPONSE 607measurementrrormodels see Achen1975)andpays ess attentionomentalprocesses hando theattitudemodelsofpsychologists.ut,despite ts imi-tations, he simple three-axiommodel is, we believe, uniquelysensitiveto thewiderange f empirical egularitiesssociatedwith hemass survey e-sponse.Ourhope,therefore,s that s others riticize urmodel ndproposealternatives,heykeeptheirheorizingn a sufficientlyroadplanethattcanaccommodatehe ange fempiricalegularitiesehave xamined.The empiricalhenomenaorwhich urmodel ffersn explanation,hichwe summarizenTable9,maybegroupednderhree eneral eadings:

1. Dependence f ttitude eportsnprobabilistic emoryearch.Becauseattitudeeportsre basedon memoryearches hat rebothprobabilisticndincomplete,ttitudeeportsend o be (1) unstablever ime;2) centerednthemeanof theunderlyingonsiderations;nd 3) correlated ith heoutcomes fmemoryearchesDeductions -5). This s alsowhy eoplewhoare more on-flictedntheir nderlyingonsiderationsremore nstablentheirlosed-endedsurveyesponsesDeduction ).

2. Effectsf deasrecently ade alient. henotionhatndividuals'urveyresponsesanbe deflectednthe irectionf deasmade ecentlyalient asbeenusedtoexplain uestion rder ffects,ndorsementffects,ace-of-interviewereffects,eferenceroup ffects,uestion ramingffects,ndTV newsprimingeffectsDeductions-16).

3. Effectsfthoughtnattitudeeports. he notionhat hinkingbout nissue, s gaugedbygeneralevels fpoliticalwareness,nables eople o recalla larger umberf considerationsndhence omakemore eliable esponses asbeenusedtoexplainwhymore oliticallyware ersonsxhibitreateresponsestabilitynd why hepublic s a whole s more table n"doorstep"ssues De-ductions, 7). It alsoexplainswhymore oliticallywarepersons,ndpersonsespeciallyoncernedbout n issue, reableto recallmore houghtselevantoit Deductions , 2). Finally,henotion hat reaterhought akes ttitudee-portsmore eliable as been nvoked,with nly imiteduccess, oexplain heeffectsfextra houghttthemomentfrespondingo an ssue Deduction 7).Althought s easyto maginelternativexplanationsormany fthepar-ticular henomena e have xplainedwith urmodel, twouldnot,wethink,eeasytodevelop nalternativehat as therangendsimplicityfourmodel.Yetourmodeldoes,as we have dmitted,epresentsimplificationfa process hatmust e muchmore omplicated.What orts fsimplificationsavewemade?

One of themost mportantnvolves he ssueof"on-line" rocessing. sindicatedarlier, astie ndPark 1986), among thers,rgue hat eopleoftenusea "judgmentperator"oupdate ontinuouslyheir ttitudeson-line" s theyacquirenew information.eople are said to store heir pdated ttitudesn


31/39

6o8 John allerandStanley eldmanTable9. Listof EmpiricalPhenomenaThat theModelClaims toExplain

1 Peoplewho re, ngeneral,more oliticallywarehavemore onsiderationstthetopof their eads ndavailable or sein answeringurveyuestions.2. Peoplewhohavegreaternterestn an issue hould ave, ll else equal,morethoughtsbout hat ssuereadily ccessiblenmemoryhan ther ersons.3. There hould e strongorrelationsetweenhe deasatthe opofpeople'smindsas they nswerurveytems nd their ecisions n the tems hemselves.4. There hould xist fair mountfover-timenstabilityn people's ttitudeeports.5. Opinions hat resubject orepeatedmeasurementhould ave entralendenciesthat re stable ver ime, ut hould luctuateroundhese entralendencies.6. The attitudeeportsf politicallywarepersonshould xhibit reaterver-timestabilityhan hose f ess-awareersons.7. Peopleshould emore table n their esponsesoclosed-endedolicytems on-cerningoorstepssues-that s, issues o closetoeverydayoncernshatmostpeopleroutinelyaysome ttentionothem.8. Greatermbivalence ught obe associatedwith igherevels fresponsensta-bility.9. Raisingnewconsiderationsn mmediateroximityo a questionhould ffectheanswers iven ymaking ifferentonsiderationsalient.10. Peoplewhoareambivalentn an issue hould e most ffectedymanipulationsthat aisenewconsiderationsn mmediateroximityoa questionbout he ssue.11. Insertinghename fa prominentoliticianrgroupnto questionhould ffectthepublic's esponseso the uestionthe endorsementffect").12. Theraceof an interviewerhould t east ometimesffecthe esponseso ques-tionswhich eor she asks.13. Manipulationshat aise he alience f a referenceroup an affectesponsesoquestionsn which hereferenceroup asa well-knownosition.14. Newsreportsan "prime"ertaindeas,thereby akinghemmore ccessible oruseinformulatingttitudetatementsn relatedubjectsthe primingffect").15. Question rder an"prime"ertaindeas,therebynducingorrelationsith roxi-mate elatedtems.16. Inducingndividualsothinkbout heirdeological rientationn closeproximitytoquestions avingdeological ontentan"prime"deology or se nansweringthose uestions.17. Inducing eople o thinkmore arefullybout n issuebeforetatingnopinionshould nhance he eliabilityf the pinion eport.Not onfirmed.)

long-termmemory nd retrieve hemas required,rather han,as in our model,createattitude tatementsn thespotas they onfrontach new survey uestion.We are sympathetico on-line nformationrocessing ndbelieve that omeform f iteventuallymustbe included n models ofpoliticalattitudes.Yet eventhestrongestdvocates of on-lineprocessingso far have foundno evidence ofsuchprocessing n the formation fpolicyattitudesas againsttheformationfevaluationsofpersons). And this s scarcelysurprising. he issues aboutwhich


32/39

A THEORY OF SURVEY RESPONSE 609citizensmust nswer urvey uestionsre toonumerous,oo multidimensional,and, nmost ases,too obscure ortto be feasible oengage non-line rocess-ing f all relevantnformation.

DiscussionInclosingwe would ike o consider hree road uestions bout urmodel.Is itplausible?What re ts mplicationsor uturepinion esearch?What reitsnormativemplicationsor he oleofmassopinionna democracy?

SubstantivelausibilityftheModelOf thevarious laimswehavemade, he mbivalencexiomalongwith ts

implicationhat ndividuals ormallyo nothavea single, ixed, nd firmtti-tude n issuesbut nstead avemany, otentiallypposing considerations")sperhapshe eastfamiliarndhence he east ntuitive.t also flatlyontradictsthedominantcademic heoriesfpoliticalttitudes. chen1975)assumes hatall respondentsave"true ttitudes"nd maintainshat hese rue ttitudes,l-thoughppearingo vacillate ecause fmeasurementrror,reoverwhelminglystable.Converse1964) likewise ssumes hat, lthough any espondentsavenonattitudes,hosewhodo have ttitudesaveperfectlyixednd table nes.Despite hebreadthf ts ppeal,the onventionaldealoffixed ndstable"true ttitudes"s wellpastdue for trong uestioning.ne needonly onsiderthat, n bothpublicopinion urveysnd in elitedebate,policyquestions retypicallyramedn terms fsummaryudgmentswhetherbortion houldbepermittedrnot;whetherchool hildrenhould e bused opromoteacial n-tegrationrnot) o that hemakingftheseudgmentsequiresn aggregationofone's eelings cross requentlyiverse oncerns. here s absolutelyorea-son to suppose hat personmust eelconsistentlybout ach ofthe lementsthat eorsheaggregatescrosswhenmakinguch ummaryudgments. hus,someonewho upportswoman's ightocontrol er eproductivechedule eednot lso feelcomfortableith bortingetusesScott1989);advocates fgov-ernmentidto theneedyneednotfeelcomfortable ith ig governmentFeld-man ndZaller1992).The heart four rgumentsthat ormost eople,most fthe ime, heresnoneedto reconcile r even orecognizeheirontradictoryeactionso eventsand ssues.Eachrepresentsgenuine eeling, apable fcoexisting ith ppos-ingfeelings nd,dependingnits salience ntheperson'smind, ontrollinge-sponses o survey uestions.Analystsfpublic pinion onghave been awarethat ew itizens re"ideologicallyonsistent"ntheir esponseso differents-sues Converse 964).Ourarguments thatmany itizens reequally nconsis-tent ntheir eactionso differentspects fthe ame ssue,where n"issue" ssimply nybundleof concerns hat pollster sks respondentso aggregateacross nthe ourse fanswering question.Althoughhis ineof rgument ay eemunobjectionable,t seasy oover-


33/39

6to John allerandStanley eldmanlook its mplications. he most mportants that ndividualsypicallyo notdevelop true ttitudes"f the ype hat pinion nalystsoutinelyssume,butpossess series fautonomousnd oftennconsistenteactionso the uestionsaskedby pollsters. r,toput t another ay,most pinions nmost ssueshaveboth centralendencynda variance.Weemphasize, owever,hat othingnourmodel enies hat omepersonsmaydevelopwholly onsistentetsofconsiderations ith espect o some s-sues. ndeed, heres nodoubt hat omepeople, ncluding any oliticalctiv-ists ndothers homighte drivenowardognitiveonsistency,o exactlyhis.Suchpeople rewellcapturedyconventionaltrue ttitude" odels.Butthesepeople an be accommodatedquallywellbyourmodel,which, lthoughold-ing hatmost ndividualsreto some xtentmbivalent,llows ome ndividualstobeunambivalent.hus,ourmodel ccommodatesoth hemajorityfpersonswho re, s Table2 indicates,nconsistentn theireactionsodiversespects fissues ndtheminority ho re table ndconsistent.Empirical otentialftheModel

The value of our modelfor mprovingesearchn publicopinion ies inthree irections. ne involves pening p themicrofoundationsfattitudee-ports oempiricalcrutinyndanalysis. orexample, esearchersften eferoattitudeshat remore r ess"crystallized,"ore r ess deological, r notherwaysheterogeneouscrosspeopleand ssues Rivers1988;Sniderman,rody,andTetlock 991).Butsince he attitude"s always heprimitiveerm fanal-ysis,they ave hadnowayofdirectlyerifyinghese mputations. ithinurmodel,however,crystallizedttitude ighteonethat sfoundo be basedona larger r morehomogeneousetofunderlyingonsiderations;imilarly,nideological ttitude ight e onebasedonabstractrprincipledonsiderations.Theempiricalonsequencesfthesemeasured ifferencesnattitudinal icro-foundationsould hen einvestigated.

Anotherirectionorfutureesearchs inthe rea ofcommunicationndpersuasion.tudies fpolitical ersuasionypicallyssume hat ttitudehangeinvolves conversionxperiencen which necrystallizedttitudetructuree-placesanother. urmodel,however,aisesdifferentossibilities, ostnotably"persuasionyframing."he deahere sthatf nelite rsomeother ersuaderdiscusses n ssue o as tohighlightnesetofconsiderationsatherhannother,thepublicwouldbe expectedorespond y basing tsopiniontatementsntheconsiderationshusmade salient. delman's1977) andBennett's1980)workon the ffectsn massopinion fsymbolicssuemanipulationyelites,Kinderand Sanders's 1990) workon the effectsf differentuestion rames, ndPopkin's1991)observationsnhow andidates se ssues omobilize oterup-port ll affirmhe mportancefmanipulatingassopinion ymanipulatingheconsiderationshatresalient othepublic.


34/39

A THEORY OF SURVEY RESPONSE 6iTraditionalersuasionemains ossiblewithin urconceptionf attitudes,but t assumes somewhatifferentorm.As Zaller 1984, forthcoming)ain-

tains, ttitudehangewithinurframeworkust otbeunderstoods an all-or-nothinghiftna "true ttitude,"ut s an adjustmentnthemixofconsidera-tions elatingo an ssue.Finally,henotion f a fundamentallymbivalentublic anhelpdevelopmore ffectiveinkages etween ublicopinion nd thepolicymakingrocess.Forexample,V. 0. Key 1961) foundnPublicOpinionnd American emoc-racy hat sizablefractionfthe lectorate antedtrongerocial welfare ro-tections hile lsofavoringaxcuts even f tmeans uttingffome mportantthingshat eed o bedone." n a comment uch nthe piritf ourmodel,Keyobservedhat oth ttitudes,houghppearingontradictory,erereal.As hewrote, a simple alculus f self-interestakes imultaneousupport f taxre-duction ndexpansion fsocial welfarectivitiesntirelyonsistent. . . Forthe ystems a whole,however,his ype fopinion ombinations irrationaland createsproblemsn programmaking. . . The balance of forcesdrivespolicymakersacktoward oncealed nd ndirectaxation, hichmayberegres-sive n ts ncidence"168).A similar ointhas beenmadebyFree and Cantril 1967), whointheirstudy he Political eliefs fAmericans ound hat hepublic xhibited com-binationfphilosophicalppositionowelfaretate olicies ndoperationalup-port or irtuallyhefull ange f welfare rograms.his "schizoid" elief at-tern, ree and Cantril rgued,madeit extremely ifficultorgovernmentoengagenrationalolicy-making.Thus,we feel hat conceptionhatmphasizeshat opinion" nanissueisgenerally range f reactionsatherhan single true ttitude" illbemorefruitfulornvestigatinginks etweenmassopinionndthe olitical rocess.NormativemplicationsftheModel

The normativemplicationsf Converse's onattitudeshesis reextremelybleak. n the imitingase ofa publicwithoutttitudes-aclaim hat onversedidnotactuallymake-self-governmentakes ittle ense. As Achen 1975,1,227) put t, Democraticheoryoses ts tartingoint."ncontrast,he mpli-cation f Achen's mpiricalnvestigationas relativelyptimistic: ostmem-bers fthepublichave rue ttitudeshatre lmost erfectlytable. hedisturb-ingly igh evels fresponse hange iscoveredyConverse, chen rgued,redueto thevague uestions fsurveyesearchersatherhan o thevaguemindsofcitizen espondents.Ourposition alls omewhere etweenheConversendAchenpositions.We agreewithConverse hat here s a great eal ofuncertainty,entativeness,and ncomprehensionnthe ypicalmass urveyesponse. heproblem, efur-thergree, s much eeper han ague uestions. ndyet,withAchen,wereject


35/39

612 John aller andStanley eldmanthepremise fConverse's lack-and-whiteodel,which s thatmost esponsefluctuations duetoessentiallyandom uessing y peoplewho haveno mean-ingful pinions.Ourclaim s that venwhen eople xhibit igh evels f responsenstabil-ity, heopinions hey xpressmaystillbe based on real considerations.venwhen hese onsiderationsurn utto be transitory,heopinion tatementsheygeneraterenot, or hat eason, ecessarilyackingnauthenticity.This argumentxtends o the nterpretationf aggregateurvey esults.Thus, f55% of Americans eporthat hey pprove f thewayGeorgeBush sdoinghis ob as president,tshouldnotbe taken s evidence hat majorityfthe ublics unequivocallyupportivefthe resident.ather,t hould e takentomean hat 5% of Americansre on balance positive oward ush's ob per-formance-evenhough heparticular5% whoexpress pprovalwill naturallychange rom ne survey o thenext, ependingnthe ross-cuttingccidents fmany ifferenteople'smemoryearchesPage andShapiro 992).There s, then, o inconsistencyetween ur ssertionsf ndividual-levelambivalencend nstability,n the nehand, ndbelief n themeaningfulnessfaggregate-leveloll results,nthe ther-providedne readspolls s revealinga balanceof onsiderationsatherhan s counts fpeople's true ttitudes."Thisconceptionf an ambivalentublicmay all hortf our dealofwhatpublic pinion ught o be like, s thisdeal sexpressednpolitical ratorynddemocratic ythology.ut fdemocracyspossiblena countryhat oth lori-fies conomic ndividualismnddemands he welfare tate; hat rofesses ocherishquality ndpractices acialdiscrimination;hatnsists n bothhigherlevelsofgovernmentervices nd ower axes; nd that atesCongress utre-elects ongressionalncumbentstextremelyigh ates-then t salso possibleunder urunderstandingfmasspublic pinion.ManuscriptubmittedAugust 991Finalmanuscripteceived December 991

APPENDIXAttitudetems

Jobguarantees. omepeoplethink hat hegovernmentnWashingtonhould ee to itthatevery erson as a job and a goodstandardfliving.Others hink hat overnmenthouldust eteachperson et headon theirwn.Before ellingmehowyoufeel bout his, ouldyou ellmewhat inds fthingsome omindwhenyou hinkbout overnmentakingure hat very erson as a good tandardf iving?Anyothers)Now,whatkinds fthingsometo mindwhenyouthinkbout ettingachperson et headon their wn? Any thers)


36/39

A THEORY OF SURVEY RESPONSE 613Governmentervices. omepeople hinkhegovernmenthould rovide ewer ervices, veninareas uch s healthndeducation,n order o reduce pending. ther eoplefeel t s importantfor he overnmentoprovidemanymore ervicesven f tmeans n ncreasenspending.Before ellingmehowyoufeel bout his, ouldyou ellmewhat inds f hingsome o mindwhenyou hinkboutfewerovernmentervices?Any thers)Now,whatkinds f things ome tomindwhenyou think bout ncreases n governmenter-vices? Any thers)Aid to blacks. Some people feel thegovernmentnWashingtonhouldmake very ffortoimprovehe ocial ndeconomic ositionfblacks.Others eel hat overnmenthould otmake nyspecial ffortohelpblacks ecause heyhould elp hemselves.Before ellingmehowyoufeel bout his, ouldyou ellmewhat inds fthingsome o mindwhenyou hinkbout the ocial ndeconomic ositionfblacks?" Any thers)What omes o mindwhenyou hinkbout ffortso mprovehe ocial and economic osition

ofblacks? Any thers)And what omestomind)whenyou hear hephrase lacks houldhelp themselves?Anyothers)PoliticalAwarenesscale

A 19-pointcale, havingnalphareliabilityf 85,as follows: 635 tov642,v202 uptofourpoints).Tworecognitionests,v242, v244 (one item ach). Fourcomparativeandidate ocationitems: 730 andv731;v749andv750;v81 andv812;v831 andv832; plusv723 a noncomparativelocationest).SocialWelfaredeology cale

The social welfaredeologycaleconsists f 14 tems oncerningndividualismndequality,plustwomeasuresf ideological elf-designation,s follows: 620tov622, v624, v626,v701 tov706,v2176,v2178,v2179,plusv722andvI010.REFERENCES

Achen,Christopher.975. "Mass PoliticalAttitudesndtheSurvey esponse."American oliticalScienceReview 9:1218-31.. 1983."Toward heories f Data."Presentedt the nnual onventionftheAmerican olit-icalScienceAssociation, hicago.Anderson, orman. 974. "Informationntegrationheory: Brief urvey."n Advances nMathe-matical sychology,ol. 2, ed. David Krantztal.Bargh,John, . N. Bond,W. J.Lombardy,nd M. E. Tofa.1986."TheAdditive aturefChronicandTemporary onstruct ccessibility."ournal f Personalitynd Social Psychology 0:869-78.Bennett, ance W. 1980. Public Opinionin American olitics. New York:Harcourt raceJovanovich.Bishop,George. 1990. "Political nvolvementndResponseEffects." ublic OpinionQuarterly53:209-18.Bishop,George,RobertW.Oldendick,ndA. J.Tuchfarber.984. "WhatMustMy nterestnPoli-ticsBe If Just oldYou I Don'tKnow'?"PublicOpinion uarterly8:510-19.Bodenhauser,allenV.,andRobert . Wyer. 987."SocialCognitionndSocialReality."nSocialInformationrocessingndSurveyMethodology,d. Hans J.Hippler, orbertchwarz, ndSeymourudman.NewYork: pringer-Verlag.


37/39

614 John aller and Stanley eldmanCampbell,Angus, hilip . Converse, onaldStokes, ndWarren iller. 960.TheAmerican oter.New York:Wiley.Charters,W.W., Jr., ndTheodoreNewcomb.1958. "SomeAttitudinalffectsfExperimentallyIncreased alienceofGroupMembership."nReadingsn SocialPsychology,d ed., ed. E.Macoby, heodoreNewcomb,ndE. Hartley. ew York:Holt,Rinehart.Converse, hilip . 1964."TheNaturefBelief ystemsn MassPublics." n deologyndDiscon-tent, d. DavidApter. ewYork: reePress.Converse, hilip ., andGregory arkus. 979. "Plus a Change . : The New CPS PanelStudy."Americanolitical cienceReview 3:32-49.Dean, Gillian, nd ThomasMoran.1977."MeasuringMass PoliticalAttitudes:hange nd Unreli-ability."oliticalMethodology:383-401.Edelman,Murray.977. Political anguage.New York:Academic ress.Ericsson, . Anders,ndHerbertimon. 1984.Protocol nalysis. ambridge: IT Press.Erikson, obert. 979. "The SRC PanelData andMass PoliticalAttitudes."ritish ournal fPoliti-cal Science9:16-49.Feldman, tanley. 989."MeasuringssuePreferences:heProblem fResponse nstability."oliti-calAnalysis :25-60.. 1990."Theory, easurement,ndthe urvey esponse." resentedtthe nnualmeetingftheAmerican olitical cienceAssociation, tlanta.Feldman, tanley,nd John aller.1992."Political ulturefAmbivalence:deologicalResponsesto theWelfare tate."AmericanournalfPolitical cience, 6: 268-307.Free, Lloyd, ndHadleyCantril. 967. ThePoliticalBeliefs fAmericans. ewYork: ree Press.Hastie,Reid, ndBernadetteark.1986."TheRelationshipetweenMemoryndJudgmentependson WhetherheJudgmentask Is On-line r MemoryBased." Psychological eview93:258-68.Hastie,Reid,andNancyPennington.989."Notes n theDistinctionetweenMemory-basedndOn-lineJudgments."n On-line ognitiveerson erception,d. John assili.Hillsdale,NJ:Erlbaum.Higgins,E. Tory, nd GaryA. King. 1981. "Accessibilityf Social Constructs:nformation-Processing onsequences f ndividual ndContextual ariability."nPersonality,ognition,andSocial nteraction,d. NancyCantor ndJohn . Kihlstrom.illsdale,NJ:Erlbaum.Hippler, ansJ., ndNorbertchwarz.1987."Response ffectsnSurveys."nSocialInformationProcessing nd SurveyMethodology,d. HansJ.Hippler,Norbertchwarz, ndSeymourSudman.NewYork: pringer-Verlag.Hochschild,Jennifer. 981. What's Fair? Americans'AttitudesowardDistributive ustice.Cambridge: arvard niversityress.Iyengar,hanto.1991. sAnyone esponsible? hicago:UniversityfChicagoPress.Iyengar, hanto, nd DonaldKinder.1987.News ThatMatters.Chicago: UniversityfChicagoPress.Judd, harles, ndCynthiausk. 1984."KnowledgetructuresndEvaluative udgments:ffectsfStructural ariables n Attitudextremity."ournal f Personalitynd Social Psychology8:331-38.Judd, harles, ndMichaelMilburn. 980. "The StructurefAttitudeystemsntheGeneral ublic:A Comparisonfa Structuralquations ystem." mericanociologicalReview 5:627-43.Judd, harles,MichaelMilburn,ndJonKrosnick. 981."Political nvolvementndAttitudetruc-turen theGeneral ublic."AmericanociologicalReview 6:660-69.Kelley, tanley, r. 983. nterpretinglections. rinceton:rincetonniversityress.Key,V. O., Jr. 961.PublicOpinionndAmericanemocracy. ew York:Knopf.Kinder, onald,andLynn anders.1990."Mimickingolitical ebatewith urvey uestions: heCaseof White pinion n Affirmativection or lacks." ocialCognition:73-103.


38/39

A THEORY OF SURVEY RESPONSE 6i5

Krosnick, on. 1988. "Attitudemportancend Attitude hange."Journal f ExperimentalocialPsychology4:240-55.Krosnick, on, nd Howard chuman. 988. "Attitudentensity,mportance,ndCertaintyndSus-ceptibilityo Response ffects."ournalfPersonalityndSocial Psychology4:940-52.Lane, Robert. 962. Political deology. ew York: reePress.Lau, Richard, ndDavid0. Sears.1983."Inducing pparentlyelf-interestedolitical references."American ournal fPolitical cience27:233-52.Lau, Richard, avid0. Sears, ndTomJessor. 990. "FactorArtifactevisited:urvey nstrumentEffectsndPocketbookolitics." olitical ehavior 2:217-42.Lewis-Beck,Michael. 1985. "Pocketbook otingn U.S. National lections: actor Artifact?"American ournal fPolitical cience29:348-56.Lodge, Milton,KathleenMcGraw,ndPatrick troh. 989."An mpression-Drivenodel of Can-didate valuation." merican olitical cienceReview 3:399-420

Loftus, lizabeth ., Stephen . Feinberg,ndJudith . Tanur. 985."Cognitive sychology eetstheNational urvey." mericansychologist0:175-80.Milbum,Michael. 1987. "IdeologicalSelf-schematand Schematically-Inducedttitude onsis-tency." ournal fExperimentalocialPsychology3:383-98.Millar,Murray,nd Abraham esser.1986. "EffectsfAffectivendCognitive ocus on Attitude-Behavior elationship."ournal fPersonalitynd Social Psychology1:270-76.. 1989. "The Effects f Affective-Cognitiveonsistency nd Thought n the Attitude-Behavior elation." ournal fExperimentalocialPsychology5:189-202.Mueller, ohn. 973.War, residents,nd PublicOpinion.New York:Wiley.Page, Benjamin, nd Robert hapiro.1992. TheRationalPublic.Chicago:Universityf ChicagoPress.Popkin, am. 1991. TheReasoning oter. ambridge: ambridge niversityress.Price,Vincent. 991. "PrimingdeologicalLeanings:The Impact f Salience and Commitment."ReportotheNational lection tudies oard fOverseers, nnArbor,Michigan.Raaijmakers, ., ndRichardhiffren.981. "Search fAssociatedMemory." sychological eview88:93-134.Rivers, . Douglas. 1988. "HeterogeneitynModelsofVoter hoice."American ournal fPoliticalScience.Schuman, oward,ndStanley resser. 981.QuestionsndAnswersnAttitudeurveys. ewYork:Academic ress.Schuman, oward,ndJacquelinecott.1987. Problemsn heUse ofSurvey uestionsoMeasure

PublicOpinion." cience 36: 957-59.Scott,Jacqueline. 989. "Conflictingeliefs boutAbortion:egal Approvalnd Moral Doubts."SocialPsychology uarterly2: 319-26.Sears,DavidO., andRichard au. 1983."Inducing pparentlyelf-int

Feldman Zaller Article

Documents

Transcript of Feldman Zaller Article