Dillman, Smyth, Christian, Chap 6, From Questions to Questionaire

7/28/2019 Dillman, Smyth, Christian, Chap 6, From Questions to Questionaire

1/42

Gu i cl int '5.1, ' l :)cu, loy is lsol , tsutrr ,dl( , \ t , r i r ,slurt n, t ,nt t t l.t l lr l .rclusiurGuideline5,14: M.rintai spttcitrlqcttttct,tt lsrt)l,r cttl.,!t,rit,s ltttlis consistent uitlt,ncasurcmci tcntClosed-Ended Nominal

Guideline5.75: Ask respondentso runk only a ezu items tt rttrrrather than a long listGuideline5.76: Auoid biasfrom unequalcornpatisonsGuideline5.77: Randomize responseoptions if there is cotrtr,ttrabout ordereffectsGuiiletine5.78: llse forceil-cioice questions nsteadof check_ultthat-apply questionsGuiileline5.79: Consider using ilifferently shaped ansr,,r,tspaces circlesand squares) o help respontlt,ttt:distinguish betzaeen ingle_and multiple_ n stl tquestiotsClosed-Ended Ordinal

Guideline5,20: Choosean appropriate scale length_in generrrl,limit scales o our ot fioe categoriesGuideline5,21: Choosedirect or constnrct_specificabels o iur_provecognitionGuideline5.22: Proztide cales hat approximate he actual distri_bution of the characteristic n the populationGuidelitre5.23: Proaidebalancedscales herecategoies are rels_oely equaldistancesapflft conceptuallyGuideline5.24: Considerhozooerbally labeling and aiiuatta dis_playing all rcsponse categories may inl.luenccansT0ersGuideline5.25: Carefully eaaluate the useof numeic labels flndtheir impact on meqsulementGuideline5.26: Align response ptions oerticalh1 n one columnor hoizontally in one rozoand strioe for esual.1st ancebetueen categoriesGuideline5.27: Place nonsubstantiaeoptions at the end of thescaleand separate hemfrom substantiaeoptions

CHAPTER 6

From Questions o aQuestionnaireTttlnt ts a huge differencebetweena ist of unorderedsurveyquestionsandlhe questionnaire that delivers them to respondents effectively. The designof a questionnairemust considerhow to motivate someonewho receivestto becomea respondentand complete he survey. t must also avoid manymeasurementproblems, ranging from unintended order effects to unneces-rarily high item nonresponse. his chapterdescribes ow to transition fromI list of questions o a respondent-friendlyquestionnaire hat will maximizercsponseand minimize measurementerror.

. Many of the challengesof transforming questions nto a questionnaire arelllustratedby the now infamous butterfly ballot experience rom the 2000presidentialelection.A dramatic controversyemergedover whether somevoters n Palm BeachCounty,Florida,had mistakenlyvoted or PatBuchananfor president while intending that their punch on the butterfly ballot be forthe Democraticnominee,Al Gore (seeFigure 6.1).C. R. Fox (2000) howedthat about 2,800 unintentional votes were recorded for Buchanan, whereas2,300ncompletepunchesand 11,000 oublepuncheswere recorded hat in-validated ntended votes for Gore. Theseargumentsare supportedby datashowing that the votes recorded in Palm BeachCounty for Pat Buchanan inthat election were proportionally much greater than those recorded in otherFlorida countiesbased on the number of registeredReform Party voters, votesfor Bush, otal votescast, nd votes or Buchanann the 1996 rimary (see ig-ure 6.2).Inaddition,Wand et al. (2001) oncluded rom a detailedpostelectionanalysis hat theseanomalieswere not found among absentee oters n PalrnBeachCounty who did not use he butterfly ballot and that Buchanan's up-port in Palm BeachCounty tended o come lom more Democraticprecincts.15 1


2/42

| ', r l l i rr\ t(-)r r , r r , ' r , , t , ' \ ( - ) r r ' , t t ' r r . r r r \ t t r l

F igure 6. ' l Palm Beach C(nlrly tnrl tot l lybi l l lol

Basedon theseanalyses, here can be l i ttle doubt ihat the desigrrof tlrr ' ' rtterfly ballot produced (a ) many votes for a candidate for which rolt'r",lr 'Inot intend to vote and (b ) many v otes that could not be countcd al .tl l l l r,consequencesof these errors for the outcome of the 2000presidcn fi.r r'1,' t ,'r,make this perhaps he most dramatic and consequential xamplc clt'r ' I 'l I " " rdesign of an electionballot, which is quite similar to a survey qtr('sti{)rrrr,rrr 'lis l ike lvthatanumberofcharacter ist icsofthebal lotcausedthevol i r r l i , r r "r

Figure6. 2 Votes orBuchanann al lFlorida ountiesn 2000presidentiallection, elativeo the number f registered eform Partyvoters.

4000350010002500200015001000

Palm Beach Counl\ +

rr l' r l \ r r ' I r l l l , 1, " 'n

I t t t t t t l l t t t l t t t t l l r , / / () i /r", / /rIrrr i r l ,r ' I 1]

' l l rc r rscol Pttr r l t l , i r rs or tr t , t tk i t t f l l t l ( 's l i ( ) l l l r . l i l fss t ' t t t t tsual 'eopler lo r t t) t vPi r ' , r l lv ' | l (1) t l t l l ( ' tl l ( ' l l l , \ ( ( ' [ ) t wh( l l t l rey v( ' tc,dnd f( ) rmanyr olt.rs thc prt'r ' iotts 'l t 'r ' t iotrwas tike y ;r distant memory Thus, the at-lr,rrti()n )[ v()t( 'rswas l ike y dividc'd between making sure hey Punchedtlrt,b.rl l rt corrcctly and that they registered heir choicecorrectly'I M()st \,ot(,rs knew which candidate they planned to vote for before theywt'rrI into the voting booth, so they probably stopped reading choicesont t' tlrcy found their preferred candidate.r llt'st'arch has shown that peoPle typically start to read in the upper left-Ir.rrrtl corner of the left-hand page and proceed down the page, much astlrt'v would read a book. Thus, George Bush's name was likely seen irst,lrrlkrwed by that of the second listed candidate, Al Gore., lt is unusual tO ist the candidates for one office (or answer categories n,r srrrvt'y) in two different groupings on two different pages. Voters aswr.ll as qucstionnaire respondents are used to having all answer choicesirr a sirrgle,defined group on the same page.r llrt' lines around candidates on each page visually suggest that the can-rlirl,rtes on the first page should be considered as a separate grouPingIlorrr clncl idateson page 2.. ll is ('ven more unusual to have answer choices on two separate Pages"slr,rlr," a column of answer spaces n the middle between them, so that.rraw(,rs must be punched to the ri8hi of candidates listed on the leftgr,rgt.,rndo the left of candidates isted on the right page.r Wlrt'n some people who wanted to vote for Gore saw his name an dIlirrl io figure out how to vote, their eyes were likely drawn to the word" l )('nr()cratic" and the line irnmediately above it, which led directly iol lr l st,trrrrtl ox in the answer column. It was logical for a voter to markllr,rl st,tonci box because the first box in the column was for Bush, the| Ir ,r r bI i t . r r r r r rd idate.. ( )r( (, tlr(' st'eonclholc was punched and voters pulled away their hand,rl n,orrl tl not bt'surprising that some voters noticed the arrow in thel)r,rr(\'r ' .r l icsp.rccand considered th e possibi l i ty thai they may havel'r I ( llr\l th(' wr()ng hole. Ttr emedy the problem, some probably markedllrr,shrrrrl l rolt' l tsicle the Gore "rectangle," hus producing a "doublel r l l l l ( ll "r lt l . rrrrl iLr.ly h.rt somt'onc wil l raise his hand in a poll ing place and,r,,1 or l l)(.,r l l ( 'nti()rrof pol l workers to get a new ballot. Most votersn'orrlrl ik' lv I't 'r.rrrl ,.rrr.tsscrly having to announce o votrng precrnct\\'r,r r.t:, l r,r l l rcv ttt,t l t.rt .'ntati t ' a nt slrke cln he ballot ln addition,,r r . \v o l ( , t r - l r , r l r . t r ( ' l l r r t t l i l t l i l w.rs tPPropr iatt ' o mark two punchesl ,r r ,r tr . , r ., , , l l t r , . t t l ( l ' i ( ' l ) r t i ( l ( ' l l l ia I car l t l i t la tcwere istedIrr l l r r . l ) r ' r r r l r r ,r l r , ' . l , r , i lo t t l t , r . r ' t r ' , t r 'o t ts,l i s lo l st trPt s i tr r l hat doublerrrrr r r r . , , r r orr l r r r r r r r l t' ,1 l l t r " ' ( ' \ l t . l " v l l r " . l r r t l l t r l . r r r .r t t .


3/42

l 5, l l rHont (-)t t ', t tol , , trt . \ (-t t , t . , l r r^tN \lr l

'l l r t , xpt ' r icnec l l l r ( . ) r t l l ( . r l ly r. rh l r r ' , r r r r l t . , t t t i l l t rstr . . r t t 'sl ,utV oI t l rPr( tb lems ss()ci .r tcr li t l r t l t ' r , t . I rPi rr l ] , tl ( 's l i ( |nn. l i r ( ' l |1)n ll ist ) l l l r ( . \ l t { , | l . ,First, espondentsmust und('rstl trd ltow tr: n.gistt'r hcir rt'sPotrst,sivt,n l lr lappropriate technology.For questionnairt's he technology uscd nray rarr11.from optically processed paper questionnaires that may requirc marks kr 1,,.made in a particular way and in a particular location (e.g., il l in thc bubl,L,or place an X in a box) to web surveys where some respondents mny 1,,.unfamiliar with navigating between pages and registering their rcsPonsr-,correctly. People often have to learn the technology of how to record tlr(.,,responses at the same time they are trying to understand the question ,rn,ldecide on a response.

Second, designing a questionnaire involves ordering the questions irr,rlogical way so that the effects of earlier questions on later questions ,u,.diminished. The presidential preference question is particularly complex lrlcause both the presidential and vice-presidential candidates were displavr., Ias one choice and also because there were 10 groups of candidates to chorrs|from, even though people are used to thinking of only two maior party can(iidates. Had this complex question appeared later in the ballot, it is likely ih.rpeople would have already learned the ballot format and, therefore, wotrrrrhave been less ikely to make such a significant errorFinally, respondents answering questionnaires or ballots are respontlirrlito subtle visual clues as to what constitutes a question and how they shorrlrlanswer itbased on how they rypically process written information on a pagr,The butterfly ballot was particularly poor in this regard, making it difficul t t(,lvoters to know where the presidential preference question ended and l.t()u,the answer spaceswere associated with the names of candidates. A detailt.,lanalysis of how visual design played a role in the butterfly ballot has bccnpresented elsewhere (seeDillman, 2007, pp. 463472).In this chapter, we discuss three fundamental aspects of questronn.rrrldesign: the effects of technology, deciding question order, and using visrr.rldesign principles to construct questionnaires. We present guidelines for ordering the questions and visually designing questionnaires that are comnronto both mail and the Web. Becauseof the different technologies used to errnstruct mail and web questioruraires, we then introduce specific guiclt,lirrr,.'for developing each. Finally, we propose means of prctcsting mail arrd r,r,r,lrsurveys/a process hat we believe nvolvcs mort'sirtri lari ti t 's ha n differt,rrrr",for these wo survey modes,and tht' .r inrol wlri t.h s o Proviclca final t lrcrkto prevent problemssuch as thost' h.rl rxr rrl lcrl w,i l l r lrt 'bLrltt 'rflyb.rl lot

H()W I l i ( l lN()l (){ ' \ l \ l i \hl l i A l }| | l r l i l l l lN( lAt l i rst bl rrsh, wr. l r,rrrrl rrr,rrl I t t |rty,t | ] l |(! t . 10 t, t \ ,{. t(n t t(, | l l , t lt l i l lc l t ' l rts ( )r r l tc W.l ' . ' | | | | ' , ' | | ' | | | |r, ,rrrl rr. ,r, ,h'r l l l r. l \ t l l (, , t1 ,t l i l l (, , r. ! , t tr.rr, , t , ,

l l tt| l i t lnnltl t l t l tA* t l) i l l tttnt l5 SIn ,r ,r i l t l l , r t s rc ' i t i t ' r r t r r r t l r , l r l r r r r rl ( ) .( , l rste,rd,pdpcr qucst ionnai res reollr.lr l)ri tl t(,( i rr btxrklt,t ol l .rt l() lrt.lP respe16gp1" ravigate between th eltrtgcs.Usirrg colol q;11 1,.,Wt,[r 5,15 nexpensiveas con;uchng in blackdl(l whik', but n(.,ts() for papcr, where each questionnaire must be printed,Wr.h srrrvt'ys can use answers to early questions to ask follow_up quesuonsdnrl (,v('r 't usk)mize th e question wording (e.g.,,,When yo u began work at'Srrlrrrr,' how Iong was your daily commute?,,), whereas paper questionnairesldnrrot. In addition, branching constructions must be e*plicit u.rd

"asyfor .e_r|lr rtrrlt.n s,hr follow in mail-questionnaires, whereas for-web sr.v.y".espo._rh.nls ne(.d not, as a general rule, even know that branching has tak"a, piu.".(.ilrstructing web questionnaires involves learning howio program itemsI llrnl they appear on the respondent,s screen n the same way they appearrttr lht' designer's screen. Moreover, web programrning will be transmittedIu urllwirre and hardware that may be quiie different aid through electronicllrr('cti()ns that can range from quite limited and slow to incredibly fast, andlhr,sc rl fferences can have substantial effects on the final appearance of thetlufnlronnaire. Developing paper_surveys requires leaming about pnntingt'd1lrtbilitiec,optical scanning and image requirements (wlich vary widelv

lllljlnrr su rvey.organizarions),.postal regulations that constrain whai respon_rh.nlsc,ln receivean d respond to, and processingof the questionnaires nc elhr.y on' returned.()ther seeming differences between mail and the Web no longer exist. Forlt.tnlplc, both technologies now allow variation to be built into question_ndlr|',s ()r specific recipients. Through inteltigent printing, one maii respon_rlrnl nright be asked the following question referring to a!% increase n localldrr,s, wlrereas another is asked the same question but with the 5% changedfrt ll)'11,:Would you be willing to pay '#yo'more in local taxes if a newnlultapurp()sebasketball arena were to be built in Seattle?,,Respondent iden_lllk'irliorr rumbers can then be used to identify which version of a questronPd(h r('spondent completed so one can evaluate the effects of the differenirlroltion w()rding. In addition, the order in which questions are asked or inwhI h rcsrr.nsr' c.rtegoriesprovided can arsobe varied and even randomizedlry lrotll nl(d('rn maii and web survey design technology. Now, even smallI rv(,y ()rgilniz.rtions can obtain these capabilities.. lt is ,rlso inlPortitnt to recognize the substantial differences that now ex_hl lrr lhc t'lcctronic world of surveying. Handheld devices are proliferatingwll lr In'rrx,ndorrs Pecd, aising th e prospect ha t certain surveyswil l be an _twr,r(\i , t)Ul rrol on a trdtl i tional dcskkrp or laptop computei with a largerl ^ 'r ' ' . l ' ,rrl tl i t i .rr, i tarrrc 'r )r ; firt 'sart' t 'mcrging is a suisti tute to rnterac-l lv| wlh strtvcyr lo 1.1, ;',.r,, ' r.inl irrg, haring,arrtl rcspgnding,especial ly nl lx ' wor l ( l ol l r r rs i r r r .ss.,r r rcl r l | . i r , lwl l r srrryqy*ar " aL,, s,rm..t imcs mbed_r l l r l wi l l r i r r . r r r ,r i l r r r ,ss,r1,r . . ,r r , r r l i t , r l r . { , t r , tv in l l () . l i ( .k }na l ink or go to

l l t l W' l r ln I tr rv i r l l , t t r . , Ion,!r . l l r r r , , n,r , l , , t r r r l r r ,r i l r r r .vr ,ys,rrul {rw,,) l ) ( ,s1.


4/42

l r ) { } l rrnrN(-)l l ' , l t ( ,N ' , l ( , , \ (-) l ! . ' l l ( 'NNAll{ l

gcncral tcrms tlratdescribc iri lorirrgcluestionnairt'rrtlsl t tt iott tl t peoplc r trth e move wh o can answer in different placesan d situatiot'ts tsing diffcrcnltechnologies.

The differences that currently separate web from mail survey construc-tion tend to mask equally large similarities in how people respond to visttalstimuli. Both the Web and mail use visual channels to communicate with r('spondents. Nearly a decade of researchhas shown remarkable consistencicsin how visual layouts of questionnaires produce similar answers on PaPerand on the Internet. For example, whether scalar questions are asked in linerr'or nonlinear formats, whether needed instructions about how to respond apprt'a with responsecateSoriesrather than only in the stem of questions, andwlrt,ther tluestions are asked in a check-all format versus a forced choice fornr.rt rtl .rffect rnswers similarly acrossweb and mail surveys (Dillman,2007)'

Moreovet thc obicctives of good questionnaire design remain the samt'across tlrcse two modcs. The first obiective is to reduce nonresPons('Responclent-fricndly questionnaires have been shown to imProve overallresponse rates,but only to a moderate degree (Dillman et al., 1993) More inrportant, some research has shown that making a questionnaire respondclrliriendly is most likely to improve responseamong people who are east ikclyto respond to surveys, thus helping to reduce nonresponse error' For exanrple, in a 1992 est of resPondent-friendly U.S. Decennial Census questiotrnaires, response was improved by only 2.9 percentage points in areas of thoUnited Stateswith high response to the previous Census compared to 7 5'Z' ttareaswith low response (Dillman et al., 1993).Specific design choices,such oswhich question to ask first, can also contribute significantly to the reductiottof nonresponse error by getting people to whom the survey questions do n()lapply to retum their uncompleted questionnaires

In addition to obtaining acceptable response rates, another obiectivc ()lgood questionnaire design is to reduce measurement error' Good questilnrnaire design helps encourage all respondents to read and process questiorrsand their component parts completely and in the prescribed order, antl itminimizes the influence of one question on the measurement of subseqtr(rrtitems. Poor design of individual questions and of the questionnairc carr rrrcrease tem nonresponse by causing questions to be overlooked becausc rrlwhere they are ocated or other design properties. Effective questionnairt' tir'sign, in contrast, can help reduce item nonresponse. In addition, poor cit'silittcan result in some questions or responseoptions being interpreted d ff('r('tlll\than intended. Thus, respondent-friendly questionnaires should hclp gtrirlr'respondents as they comPlete the questionnairc.

The design of th e questionnairc,whiclr is intt'nticrl kr t'trcottr 't14t'atrtltt ot ivate people o rcspontl , r rvokt 's l l thn ' t 'o l lh t 'srr i , t l t ' r l l t ' t t lg t ' t ' l t ' t t tc l l l : 'd isctrsst ' r ' ln ( 'haptt ' r 2. Qtt ls l io tr tr , tt t t l t 's i l i t l ' , t t t t t t l ' t r r l t ' t . 'w ' l r ( ls )v t l l ' l Ii t rg lht ' t1 t t l 's l io t t t t , t i r r 'tP| )r ' , r tt t t l l t 's l i t t l i ' t t l r l sr t t ' t l l \ ' l l l l | | r r l l ' l l l l l l r l r r r" l ",rsr ,otak,r l wi l l r Jt t l r t i (l l r l t i l l l { t t l l r t ' r , t t lvcy (,r l r | t . t { \ l t l ( t ' r l l ry t t t t r l l t rg, l r r '

( )rrl 't i ttr itr ' ) irr ':; l tr:; l\7(lucsti()r1n.rire!rsy o n,lvil l,t l( ' tndcomplctr:. iust is cltcourtrgcdhroughp,ryiirg ttcntion r dt't,ri lancl .rymaking he questionnaireook and seeminrportantand professional. ll of the ssueswe address n this chapter, romthc orderof questionso the visual designofthe questionnaire ndpretesting,drc important to designing a good questionnaire hat encouragesesponseand reducesmeasurement rrot

ORDERING THE QUESTIONSA questionnaire hould be organizedmuch like a conversation,which typ-icallyevolves n accordancewith societalnorms (Schwarz,1995).Mosi con-vt'rsations end to follow a logical order in which people respond to whatother people are saying and contributing to the conversation. f someonejumps to a new iopic immediatelyafter every response ou give, t appearsthat ihey are not listening to or caring about what you said. Consider forn moment receiving he foilowing questionnaire,which was proposed or asclf-administered urvey.We have presented t here n abbreviated orm, butwith the order of the questionsunchanged:

. What was your total family income n20o7?. Do you like to play golf?. What is your opinion on global warming?. Are you married?. How many times have you gone bowling during the past year?. What is your political party preference?. Do you favor or oppose these measures o reduce environmentalpollution?. What is your occupation?r Please escribe our favorite recreational ctivity.r How adequate s your presenthealth care?o Which political party does he best ob of promoting economicgrowth?o How old areyou?. Has your health gottenbetter or worse during the past year?

Inr.rginc he difficulty of trying to respond o thesequestions n the current0r'tlt.rwhere questionsabout personalcharacteristics,olitical issues, ecre-irtirnl, nnd health care are all intermingled and where the questionsare notlltr)up('(l n thc way your knowledBeof the topics s ikely to be organized.(itt//rl ittr6.I: Crotp Ralatcd uestionsha tCouer imilar opics,ndBegin ithQtir 'r l irrrsikrl7 o \ Snlicnto Nenrly l l Respondents( tott|irtg rl l.r lt 't l lrrt 'stionsnlkt's t casier or respondentso answerandlltott 'r ' lost,ly l)l)r()xitrr.lt( '\.ttttcttt,t l onv('rsati()n.wiichingbetween oP-Ir . r t r ' , tns l r , r tlx\) l t l ( , 's ,r ls!vr,rs,rr( ' [ 'ssikcly o l . rcwr ' l l l rought ut, asIr ,w lol ' i ( , l r ( , r rr( ' iLr , l t ' o r , r 'ok. oPol l l r t ' r r rrr l t 's l torrscs.rr rddi t ion,


5/42

l r i l t l rH(|nr- trrr . , r r()N',r) \ (-) l ' , | | ( )NNA||r lconstant lyhanging )picsrack nd orthwithin r;rrr .s l ionrr,r i r t , ,uch s ntlrc originalquestion rder istedpreviously,makcs t appr'.rr hat no effortwas made o order thequestions n a meaningful way (i.c., he questionnaireappearsunprofessional nd thereforeunimportant). Oncerelated questionsare grouped, ii is often best to begin with questions hat are most salientand interesting to the potential respondentsand then move to questionsthat are less salient (Groveset a1.,2006;Heberlien & Baumgartner,1978).Asking the more salientquestions irst will help get respondents o committo the questionnaire.The ollowing exampledemonstrateshe mportanceof orderingquestionsand beginning with salientquestions hat will be of interest o people whowill be asked o respond. One of us was onceasked o provide advice fora particularly difficult survey of licensed ommercialsalmon ishermen hatfocusedmostly on the size of their boats and investment n their equipment.After much discussionwith the survey sponsor, n ntroductory sectionwasadded to the survey that asked about the fishermen'sviews of the futureof salmon fishing, whether they would advise young people to enter thisoccupation,and other issues hat were describedas the "gut" issues acingthe business.The response ate or this study of individuals, many of whomhad not completedhigh school,was well over 50%.Guideline .2:Choosehe FirstQuestion arefullyNo singlequestion s more crucial, especially n web surveys, han the firstone, as it is most likely to determinewhether people will respond to th('survey or choosenot to participate.This is not the place or a question hatis long, boring, difficult to understand, tedious to answer/ or poteniiallyembarrassing.Whereasa mail respondentmay be able o page hrough thcentire survey to get a senseof length and content, the first question in a wetrsurvey (especially ne with a page-by-page esign) must stand alone as aninvitation to continue to respondThe first question should apply to everyoneand be easy to read, conrprehend, and answer. n addition, it should be interesting and reflect thcpurpose of the questionnaire as it has been previously explained to tlrt'respondent.A first question hat is applicable o respondents ommunicak'sto them that theentirequestionnaire s relevant o them and encouragesht'ir'participation. Likewise, a first question hat is interesting will cncorrragr'response y increasing heperceived ewards of completing thc srrrvcy arrrlreducing heperceived osts.n additionkr beingapplicablt ' rrd nt('r( 'slir)l '.,a first question hat s simplehelps o sct he ramcwork hat lrcn'sl ol t lrr 'questions il l be relativey t'asy o answt.r tntl wil l not rer.lt irt ' lol ol t ' l lottfrom potential esponclt 'nls,hrrsn.tl lrcirrg t 'rcr,ivctl osls.Attri l i tt,t l ly,,rf i rst l t r t 's l ionh,r l sclcar ly orrncr ' l t , r Io l t r , r rrPost 'o l l r r ' ( l r ( 's l io| |n, l i r ' r ' . r ' ,i t w.ts xPl , t i t t r r ln (rr l l . r ( l ( , t t r , rsi i \ , ( 'slx, srrr \ ' r 'vt l r ' l t t t l i l r ot tr isk ' t t r. r t t r l t t t l t r , lp trr tnul( ,ruhl ,

( )r 'r lr,riru /rr, ra'slirrrrs 59Ir r lrt,salntolrislrr,r.rrr.rrr,r,rrnplt,discussedn theprevious uideline,heli lst t lucstionaskt.tiwlrt 't lrr,r.ho resp()ndentelt that the benefits f being,r s.rlmon ishermanwerc gctting better,gettingworse,or staying about thesamc.This was a simple closed-ended uestion hat all respondents ouldarrswereasilyand that would be nteresting o the salmon ishermenbeingsurveyed,which would help encouragehem to continuewith the survey.Although it is desirable o choose first question hat applies o everyone,is casy o answer, s interesthg, and connectso the mplementationmateri-itls,specialsituationsmay override the search or a first question hat meetsthese riteria.The most common s when respondentsmust meeteligibilityr(,quirements p front in order to complete he survey. n these nstances,wt' use the first question to determine eligibility and to inform both thosecligible and those not eligible to retum the questionnaire f they are com-pltting it by mail (eligibility criteria are also sometimescommunicated ntlrccover etter-see Chapter 7).Asking people o completea questionaboutcligibility and then return the questionnaire ven f they do not complete hert mainderof the questionss helpful for understandingcoverage, ampling,lnd nonresponsessues.Otherwise, he ineligible personwould be countedrs a nonrespondent. f they are answeringa web questionnaire,hose who

[rc determined o be neligible are politely thankedfor their willingness toporticipateand informedof their ineligibility.Qttilaline6.3:Place ensitizter PotentiallyObjectionableuestions ear heEndnJ h( QuestionnaircIn.lddition to choosing he first questioncarefully, t is also important toplncc sensitiveor potentially obiectionablequestionsnear the end of thequ(.stionnaire fter espondents avehad anopportunity to becomeengagedwitlr thc questionnaire, o establish apport with the suweyor, and havedrtriwt'redhe moresalientand nterestingquestions.Respondents ho havenlrc,rrly esponded o severalquestionsand spent5 to 10minutesansweringlhc tlut'stionnaireare less ikely to quit if asked potentially obiectionablequostions.Moreover, omequestionsmay seem essobjectionablen light of.lu(,sli(tns rlreadyanswered,and placing sensitivequestionsnear the endAvoi(ls nterrupting the flow of the questionnaireas would happen if theywr,rr.,rskt.cl bruptly at the beginningor in the middle. pretestingcan helpucrrlify qtrr,stionshat pcoplemight object o answering, uchas hoseaboutlnrrrrrrr',t 'xrral t 'havior, riminalactivity,medical istoryandsoon .(;xtth'l itn',..1:4r l Qrft,s/irrrr/rorl I '.ttrtt lsn tfu Ordcr lr cEaents ccurredAtt l lrrrt slr 'rrlt l r ls. r1' 11.1111'() )r(l(,rl.csti(nrsrra way thatwill be ogicalIu l l t l t l r . ; r rr t t r I r ' t t t .n l) , t r l i ( t t l , l I \ \ ) l t l ( ' i t ] ( l l ( ' i )s i ( . r ) rcspond o ques-ll o n r lrorr l,vr .nl \| l l r r . r rr | | , r .l t r , , \ , r ,n lsr,r l ; r , r rcr l .ir r cxantpl t , ,f askingrt $' l In, ol {l r. f ' l t ix t ' , , r l r| |rr l I rr,r ' i rrr| i r tr l ( l | | t r. | | l r,rrrl l1oy1111,111lxrsi l iorrs, ile br ,holFlulo wnll llrnruSlrh r d ll l f dn t $rltkxrrwlth rur md nbn


6/42

l { r0 l i r r , r \1 l l: , ( rN:, l | , l ( - )r r , rr rru, l r rrrchronokrgicirlrdor i.c., rom the past o thc pr('s('nt t ltrrr t lrt 'Pr('s('t)lrlthepast).Asking aboutevents n the order hey occurrt'cls ht'lPfulbec,tttst'autobiographicalmemories are often hierarchically inked in a nctwork sothat remembering one event can facilitate accurate ecall of the next cv('rlin the sequenceBelli,1998).n addition, more accurate nd complete ccrllabout eachposition can be facilitated by having respondents nswerwh('r(they worked, n what position, he nature of the position,and how long tht'rwere in that positionbeforeresponding o questionsabout what they likctlmost or why they left the position.Guideline .5:AaoidUnintended uestionOrderEffectsMany surveyors hink of eachof their questionsas standingalone;howevt'r,r('spond('nts ftendraw on surroundingquestions s heyattempt o nterpr('llntl answcr a given question.The effectsof earlierquestionson answers ('latcr tlr.restions rc rcferred o as question rderefects.Altho'tg! the caust'olir tlucstiorr ordcr cffect can vary, the outcome s usually one of two typt's:a cutlntst fftct, whereby he responseso questionsbecomemore differe'nt;or an assiuiltf on affcct,whereby he responses ecomemore similar. Both olthese ypes of effects ecome ncreasingly ikely to occur when the questi(nrsare closer to one anothet both in terms of topic and in terms of physicalproximity on thepageor screen.Figure 6.3summarizes arious causes f assimilationand contrasteffects.As the igure llustrates,questionordereffects anoccurwhen earlyquestionsinfluence the cognitive processingof later questions(i.e.,a cognitiua nt,lorder ffect\ r when earlyquestions nvoke a socialnorm that affects he wavlaterquestionsareanswered i.e.,a normatiae-basedrder /ect).The ollowirrliexamples llustrate how eachof the effectsn Figure6.3can occur n a survt yPrimingResearchersandomly assigned espondents o the National Health lrt('r'view Surveyon Disability to six condition checklistsn an attempt o me.rsr.r'r'the prevalenceof chronic conditions n the populaiion. After the checklislswere administered,all respondentswere askedquestionsabout disabilili('s,including whether hey had a disability and, f so,what caused t. ln n's1'orrs,'to thequestionabout what caused heir disability,nearly 49'Z' f responclt'rrswho had previously beenasked about sensory mpairments reportt'tl srrr'lrconditionsas the causeof their disabilitycomparctl o only 4l'X' ol t ltoslwho had not beenasked boutsenst)rympnirnr('nts. ht'sarlc pntlcn) r'l(foranumberofother ypcs f chrorr i t r rncl i l iorrs.)vt ,r l l l , t 'sporrt l r , r r lslr oreported avingcl isabi l i t i t 's ( 'r ( ' ror( ' ikr ' ly o at l l i l r r rk ' lx ' i l r l is .r l r i l i tyto cont l i t ionsht.yh,rr lh.r .rr rs lr l r l r l r rrrr l) r ' ( ,v iouslyrr l r r , n lc lv i lw t l r , r rl ( ) , r l l ( , r rr . ) t iv( ' t r rrr t l i l iorrr' l i ' , lor , ' r ,l l l | ( l ) l l r r . r ' , r lv t rr , . , l iotr , , {

( ) ,r /r ' r i I , (r r ' ( ) r /r 's l r r r rs l ( )

l loura 6.3 Commoncognitivo- nd normative-basedourcsof questionordertlf.ot8.

I lr t t r f \ r ! , r 's b r' gI ' , r r rxl rnlr !r r l lhr t rslh.n tr !r . i( . .ssrhl c

l{csll(nrdenls crcc'vc

fttl l r i l iorrs rron,ar't 'ssiblc or consideration n the ater questions,a primingiJl r \ | l l r , r t cstr l t r r l n assinr i la t ion.l 'nt tVtpr'tl ler;rl ttrh.nls wr.r'r. rcscnk'tl wi l l l two qucstions: Ho w would you describeyrt t tr r r ,r l l i , r11 ' f"r r r r l l krw rvotr l t lvorrs,ry h ingsart ' th t 'sedays?" Answersht l l r r ,nr ,wr r r lur .s l i { r r ,v,r I i r r l l i l r . , r l ly l t 'Pcrrr l i r rg rr whi t 'h wi rs asked i rst .Wlrr ,r rl r r . r r ,r l r, r1 i i 'lu( .r ,1 i (| $ ' , rs,rsIr \ l i rst ,7 l l ' : i , , l i ( | l r l i l r r ra r iagcwrtsvcryItn l l l , 11 rs11;, ,11111'' I r r ' l l f "r | | " t t in ' l t , r | | | \ , ) , t r r l 52 ' i ' wIr t l ot t lr t s i ty l t t 'y

Conshincy:Respondents nswe. asimilarly to the fiBt in

Nonn: P.esentation f

AvoidingRespondents ttemptselecnngsomc 'lems

Nom: Prcsentation f


7/42

l { r : l l rx0M Ql t. i l t(rN', t( | A Lh t', l t( |NNAtxl

w(,r('v(,ryhappy n rcsP()llse() ht'gent'ral ;utstiotr. ltt l wltt ' l t ht'gt'ttt 'r,t lclucstion tr s sked irst, tnly3tl 'X, nsweredhat hcywe t've y happy. r rsit appears hat when the marriage question was asked first, responclt'ttls'thoughtsabout the happinessof their marriage were carried over into tht'irgeneral udgmentsabouthow thingswere going (Schuman& Presset19lt )In another example, hree groups of undergraduate students took P.rrlin a study in which they were askedabout their general well-being. ()rtt'group was simply askedabout their generalhappinessand life satisfactiorrAnother group wasasked o write abouta ife event hathad made hem ft't'lgoodbeforebeingasked hequestions boutwell-being.The hird group r'r'asasked o write about a life event that had made them feel bad beforebt'irr1iaskedabout heirwell-being.Overall,ParticiPantswho wrote aboutpositiv('life events eportedbeingmore happy and satisfiedwith their ives, where.rsthosewho wrote aboutnegative ife events eported esshappinessand lillsatisfaction Schwarz& Clore,1983). n both of theseexamples,espondentscarricd ovt'r consiclerationsrom an early questionor task nto their answt'rformntion for.r laterquestion, esulting n an assimilationeffect.

A Gallup poll conducted n early SePtember 997contained wo questions:"Do you generallyhink Bill Clinton s honestand rustworthy?"and "Do yotrgenerally hink Al Gore s honestand trustworthy?"Vvhen espondentswcrt'askedabout Bill Clinton first,50% indicated that he was honest and trust'wortht and 60% hen answered hat Al Gore was honest and trustworthyWhen askedaboutGore ffust,68o/"ndicated that he was honest arrd truslworthy, artd57%answered hat BilI Clinton was honest and trustworthy. lrrthe first order (Clinton irst), respondents ad to answerabout Clinton basetlon their own frame of reference, ut in the second hey could use Al Gort'as a standardagainstwhich to judge Bill Clinton. When this was the casc,Clinton benefited *7 percentage oints) from the Gore standard.BecaustClinton and Gore at that time were generallyviewed as a team, t aPPe.trsthat respondents djusted heir answer or Clinton to make t consistentwitlrtheir answer or Gore(Moore,2002). heresult was an assimilationeffect.Results rom an earlierGallup poll (M arch27-29,1995) howhow anchor'ing can also ead o a contrasteffect. n this poll respondentswere asked oindicatewhether a number of characteristics nd qualitiesdescribedNcwtGingrich (then Speakerof the House) and Bob Dole (then SenateMaiorityLeader). 44renGingrichwas mentioned fitst, 41o/o f respondents aid tlr.rthe was honestand trustworthy,but when he was mentionedafter BobDolt',only 33% ndicatedha the washonest nd trustworthy.lavirrg lob )olt 'asa standard gainstwhich o c()mParced to fcwt'r pt'opk' lrirrking iingrit rwa shot l t 's tt t r t l r t lstw()r t l lyht 'oPprxik ' aPPt 'rr t r l. t l ) .L ' whl ' l t lc w'trI isk 't l f i Is l ,6{)' l i ' indi . , t l (\ l l l t , t l l l ( 'w.ts l tott(,s l . t trr l l l rr ' , lw'| l l l ly , l rt t l wl t I t t( i in*r' i r ' l r l rut l rr l l l t r ' \ t , t t t( l , t f t l , t l l t i t l rt wl t t t l t l rc rr ' , t ' , , r ' t t r l ' , t t t ' r l , { ' \ ' ( ' l l l l l rt l r '

( ) r r l ' l l , / , \ / l i ' ( ) t i r , r l i0rrr l ( r .1

It t ,o l ) lc () .1 '1,)orrrr t l )o l t . l rorr t ,str r r t l lustwort l ty (Mtxrre,2002).Whereaslt t l l tr,Prt'viorts 'rarrrPlr'[11rP11',rrtl t.r.lo focus on tlre similari ties between( l irrlorr rrti iort', 'r 'strting irran assimilationeffect,here hey tended o focuslrr llrc rliflt'rt.nct's bt'tween (iingrich and Dole, resulting in a contrast effect.Itr lrolh of thcse examples, the standard formed by the subiect in the first(lr .rli()n was applied to the second question, resulting in an anchoring effect.5rl ' l rurl i t,tlM,rsorr,Carlson, and Tourangeau (1994)asked respondents how they wouldrllsr'r'ibc tlrc economic situation in their communities over the next 5 years'||r(l how thcy felt about the economic situation in their state over the next 5yr,,rls. 'lrcy found that more people (7%-70%) said the state economy wouldXcl lx'ttcr when the state economy was presented before the community econ-onry question. The reason for this difference is that once people answer thelirsl question, they tend to "subtract" out reasonsused to justify their answerlrorrr tlrcir reasoning on the second question. In this particular case, whenllx'r1trt'stion about the community economy came first, new industry wasrr l,r( k)r that strongly influenced respondents to say that their community's('(()nouly would get better. New industry also strongly influenced responsesithrrrt thc state economy when that question came first, but when thc stater, ( nr()myquestioncame after he community question,ne w industry did rr0tpl,ty as argeof a role for the state question (i.e., t was subtractctlout of tl .rct'rrrrsitlcrations).he result was a subtraction effect,whercby inrportirnt con,l i tl t,rationsused o answer an earl ier questionare akenout of consirlera iorrshrr n l.rtcr question.Noru tf ErtenhandednessSlu(l(,nts at Washington State University were asked two questions aboutlhr,r'orrst'quencesof plagiarism. One asked whether a student who had pla-gi,trizcrl should be expelled. The other asked whether a professor who hadPl,rgi rized should be fired. When the professor question was asked first, 34%ln(iic.rh'd on the following question that a plagiarizing student should be ex-lrr'llt,tl. I fowever, when the professor question was asked second, only 217oInrlica trl that tht'student should be expelled (Sangsier,1993).The likely causeol this rliffcrcncc is that once the students had judged someone from outsidellrt.ir gloLrp harshly for plagiarizing (i.e.,by recommending firing), they thenIr,l t l r(,y nrust npply tlrc samc standard to someonewithin their group. Thislrl l r,rtot)r( 'nr)rr)l ,r d rrsting nswcrs to be evenhanded or fair was first noticedIrr r r ' l ,rssitslurlv in l().18 l lyrrran & Sl.rt'atsley,950) n which respondentswr,rr ' rsktrl wlrr.l l rr.r orlrrrrrrrisl r( ' l )orlorsshotrld bc al lowed to report onvi . , i ls o l l r r , l r r i t r r lSt,r l r .s,r r r t l l t r . l l r r .r J.S. tJror l t ' rs hould be al lowed tott ' l \ r t l r r r l is i ls l r r l l t r , $r r i r ' l l l r r i r r r r Wl t l t t t l t t . ; t r t 's l iorrrhotr tcommunistrr , l ,or l ( ,r ' ,r ' l r r l r r l ' o l \ r t t r l , ,o l l r r . l r r r l r . , l i l , r l r , rw,,rs,rskrr li rs l , only i7 '7,

( l (15


8/42

l ( ' l I rr i '\ r (-)r r . | | , , \ . . rr, \ (-)r r ' , I |r,N\ \ I i l

ol {( .sp()nd( 'nts. l i t ly t .s. owt.r , t . t , l r t ' t r h s l r r t 's i iorro l lowt ' t i t l t t ' 1r ls l r , rabottt U.S. eP()rtcrs ep()rti l rt ()lr rips t() lr( 'S()vi(,tUni(nr,73'r1,,r(l !r. ',ConsistencyDil lehay and Jemigan(1970)administered hree questionnaires t'r l ,r r rrtl .pol icies toward criminals to three setsof students.On e wa s strongly l)r.r '. ' ,,toward leniency for criminals, another was strongly biased t()w.rrd h,r slr r ,,for criminals, and the third was constructed to be neutral. All of thc strr llrrt ,were then asked to complete scalesmeasuring their opinions about cn n , ,The results indicated that those who initially completed the questiorrrr.r r ' I'rased toward leniency later displayed more lenient opinions toward crirr rr,r ..than did thosewho initial ly received he harshor neutral questionna rr,. lrrr '.i t appears hat thesestudents, after being swayed by a biased questiorrrr.rrr,into supporting a very lenient stance toward criminals, adjusted thcir .rrrswers to opinion questions about criminals to be more consistent with tlr, rroriginal responses.Aa)idi tts ExtrantanassAs part of an experiment, a group of students were told the study tht'y u', r,taking part in was concemed with language and the structure of spcer'lr. llr,,students were then introduced to the topics of euthanasia and reducetl t r',rrring for doctors through a questionnaire that asked them whether thqy st(x'rlfor or against these controversial practices. Half of the students wt'rt. tlrlrrinformed that they would be participating in a face-to-face nteracti()n $ rtllanother participant about either euthanasia or reduced training for tlot tor..They were told the topic they would be discussing as well as the positi,'rrthe other participant would take. The other half were informcd th.rt tlr,,rwould be listening to a recording of another participant's views on orrr.,,1the two topics, and they were then informed of their particular topit ,rrrrlthe other participant's views. All of the students were then told that rrr,'r,information was needed before beginning and were asked to compl('t(',r :,'.1of four questions on each topic. The results from thesequestions showt'rl tlr,rstudents who expected o have to talk in person with anotht'r plrti( i l ' ,!r labout their topic rated that topic more moderately than thosc who crpcr lr',to only have to l isten to a recording (Cialdini, Levy, Herrnan, & l ivcrrl 'r., .,1973).When confronted with a morc social situir[ion, rcspontlt 'rrts,rvoir|r ',1presenting hemselvesas extremc by nrorlt,r,rting lrt.ir rnswt'rs.

In addition to thesequcstiorrordr.'r ', lhtls, ,r rrly ol rt'st..rrclrr,rs rrrt'r 1. ,that summarizes hc cffr r ls rr l , rski r l i 1 i t , r r r .r ,r ll ur l rn l .r ry i l t ' r r rs, r , lr , r ' ,"How woul , . ly()rr ' , l l ( ' t l l ( 'ovr ,r ' , r l l; r , r l r ty 'o l r l r . r yorrr ' ( r r l l l l tn t lv "' , l ' lort( ) .rn( l l l l ( ' r ' , rski r r l ir l r r r r r tt ntr | l l ' , ' t r r l r I r ,r l t r r l | | t r , r i t t r | l r r , rshl r .r . l | , , | | r r llo .r t ls, r l r r t r ion, r r r l' r' l r r t 'r ' r , , " l : ' tu ' l t r ." l . t \ , o l r r r l l l r , r l l l r l r , r r l r r r r ,r r lr ; t t r ,s l ionr,r tr l . ,r r ' r , 'u tr , , l l r r r r l t I' t r | wl r . tr , r ' rhr .rl r r ' l r rr ' r l r l ll

( , i ' / / l i , / l rl ott t t t tottVi t tut l Sl i " ' l l / l ls

rpl t i l i r rkt t l l , t i t tl r t t 's l i r r t tst t t r l t ig l t r ' r l l t ' t l' tsLt ' t ll l t t t l t t 's l t t 'c i f ic r tmain(l rr .s l ion\Wrl l i t : - 5,r l t i ( '1,r)()5)'' ';,;, it , i ; it. '; '"c.rrch th.rth'r 's tcn conducted n question rdereffectsurr\ l , r t ( 'st l r ( ' r l t 'vcl t rPmcntofwebsurveys'but ' forthemostpart 'onecan1..,..,; ,n.*' cfft'cts ttr be very similar in web surveysas in mail surveys'lkrwr,vcr, h('way thatsurveyorsProgram heirweb-surveys an magnify,,, ,rri,rin'rir.'ordtir effects.For example,presentingatl ot-th: questronson,,,ui ".r,,"n allows the respondent o scroll back and forth

ihroughout the.,,ru,'y, r,lotlng t "asierto identify relationships etween

questions much asi. ,t,"'.ur" with a mail survey). n contrast,using page-by-page onstructionu,t),rr,lt('shc questionsand makesmoving back and forth- hrough them;'l';..';,,;;;;-", if it is altowedat all' L"I his case'onewould expect ewerttttt.slit lntrdereffects, S t l Smoredifficult or therespondento cay th e,l u,,,lt, ,,n,.t *-nry of Previousquestions cross creens lnfact'Tourangeaurt rrl. 2(X)4) howed hat itemsare morehighly correlatedwhen they appearlogr.llr('r )none screen sopPosed o being sPreadacross-severalcreens'irx,th.'r programmingcapabilityof web suweys.that s relevant o ques-ll,,r ,,r.ler effectss the atility to c;ntrol' and vary' the order in which ques-lk,rr,, rrepresentedOne canpresent he samequestionsn the sameorder touit ,l "p",i.r*o -

canvary th; order for randornsubsamples f respondentsVnrvi, 'rqh. ,,rde, an range rom liPPingheorderof tw o questionsha tare;;l;:, i i ;; ";; i"ce orderiffectso randomlv rderingall

f thequestionsal -;ltl;;,;|];;' ;";"r"lly would not recommend he atterwithout good reason)'' s,ltr*nrr'ilqso) detailedhow, in the normal give-and-take f regularcon-vrr",rti,rrrs,peopletend to give answers hat take into account hings theyhnvcalrr',r.lysaid l hus,answers o individual questionsare esscomplete',,i t,,". nLrt,,'krstand alone, than the writers of those questionsprobablyi,,,,',*,"U. Altlrough ordereffectsareprobably argerwith questions hat im-itt,"ii,ti'tt f,,llow oneanothet there ; Iimited evidence hat effects lsooccurwlr,,rt 1,,|.'.ti,,n" rc widely seParatedConsequently't is imPortant to rec-,fi,rfr,"".trfy on th.rt a questionnaire annotbe viewed as a compilation of,'i,r,jri"u'fy

''"A"p"rrtlcrrtquestions hat haveno effectson one another'

Eachd r,|ilion l .rstb(, ,vnu.ttednot only on the basisof its individual contentbutllr,, wirlr rr 'g,rrtlrr ht'lar1;er ontext ha t oftenaddsor subtractsmearung'Wlu,rruu',t i i, i. ip,rtt 'snclucling uestionsik e hosedescribed reviously'ttlldy lx' ill(li( iolls l() c()llsld('rconstructinghalf the questionnaireswith onetrlr.r.rrrtlr, rl with ,rrrothe,or nt a minimum ecognizen reports f findings1lp,Ponr'i lr i l i lyll ( lr l( 'sli(nrrder nfluence n respondent nswers'

( l { l rAl lN(; n ( ()MM()N VISUAL STIMULUSWr,r l l r , ,Lr.r . r lr ( l l , r l ) l ( ' r . llow vist t . t tl t 's igtrt t r r ' layotr t rr f lucnceseoplewltr, t l l rt ,v , | | ' |( ' t , I )| tI r i t t I i lt r sl vl Y (ll ( 's l i (nlh"l tr ' l wt ' Prl rv i t l t ' t l gtt ic lc l ines


9/42

l ( ' t ' |t r nr (-)!; . ,111 11.1,,r, . \ (_)t t , . ||( |NN \ | i l

lo l t l r t . r , isrr ,r l r ( 's( ' l . tn ( rr ol i r r t l iv i r l r ra l lu( ,st i ( )ns.n th is t .h,tptr , lwt,r r rnt inue htr td iscussion ut intr( )ducc t i t i i t ional visualdtsign corrccl .r tso l rr , l ; ,surveyorsunderstandho w rcspondentsperceivecntire qucstionn.rirt,P,r1i,.,,organize.informationon the page,an d navigate he survey questionnairr,.Ensuring that every respondent perceives the survei questions in tlr,.same wa y is consideredessential or obtaining quali ty iurvey data. I lorrevet creatrng a common stimulus for all respondents is far more difficrrlt rrrself-administered surveys because he respo;dents, rather than intervicw(,r.,.control the order in which they answer the questions and whether they Pr'.,cessall of the questions and their component parts. Even though intervrewr.r .,are not present to guide respondents through the questionnai"re,effectivc I.rsual design can help respondents gain entry into the page so they will \.\,.urrto respond to the questionnaire and will understand how the informatio. i,.organized. In addition, visual design can be used to encourage respondent.,to process all of the questions in the desired order and to naiisate betwu,rrsectionsan d individual questions.

How Do ResrollorNrs M,cre StNss op Illpon\4anoN pnesrNrlprN A SURVEYQuEsrrouNalns?When a person is presented with visual information, whether in a newspapt,r,web_site, or survey questionnaire, many separate actions take place vt,r.1.quickly as the eye takes in the information and the brain processes t to m.k(,senseof the page or screen.The ways in which people perceive and attencl tosurvey information are determined by innate tendencies of visual infornr,rtion processing as described in recent work by palmer (1999),Hoffman (2(X).t.and Ware (2004). People do not assign meaning to information all at onl.Instead, they process and give meaning to visual elements and therr pro1.rt.r.iies in multiple steps, although often quiie quickly. The way people proccssvisual information can be divided into three stages: (1) understancting rcbasiclayout of the page, (2) orgaruzing the inforriation on the page, and ( t)focusing on completing the task. During the first two stages, he informatiorron the page is processed on a global level, and in the third stage, thc ftx.trst: 1u..oy"9 to processing parts of the page in more detail. In Figure 6.4, w,.define the three stages and the types ofprocessing that occur ilr th.r,,stug,,.When respondents irst view a paper or web quc.stionn.rrc , th(,y (lui(.kl lscan th e page an d notice visual properties, such as col,,r arrtl sizc, lo rrrrderstand the basicpage ayout. This stage occurs mostly prior. to corrsr,rrusattention at the level of preattentiutprocrssirr,q.,h,,n. .,rr., .l ,r irr,r 1icrrt,r.,r lrrderstanding of the sceneby noticinl l c(,rt.ti rr isu.rl l) .ol)(,t.t(, s1t..1i, rrrr1l,,,r,s ize, hape, ontrast, ncl .srrr t , , r r l . r htrr . r r r t l r r fr . 's i ly, . t r ; l r , r t ,st , ' r r l. r r l .f rom other i tr f( ) r rnal iorr rr l l r t ,P,rgt, . r r , I t I t I i t i ( ' r I , l r rssl ,r l i r . rstt y bol lo t / / / , / r i ) r r 'ssi l / ( ,wl rr . t . r ,orr l \ ,l t r , r ,rsrr ,r l r r ,r tr . l , . r . l l r r r r r rvi t t lo t t t t , t l iprrs tr . l r . i ' r . r I

{ / i1r l r ,r l / r tutunuVi trut lSl i r t t t t l t rs lt 7

Flguro6. 4 Visual esign or)coplsho tguidequestionnaireesign.l hrtt: slagcs J visuslprocctslnglhtu, ldgt h|,t)ut: n hc irst slcp. cspondcnlsuickly scan hepageandpreattentivelyrocesshirsic isualproperlics,uchascolorandsize,o gaina general nderstandingfthe basiclilyoutoflhe page. his stages dominated y bottom-up rocessing.lnlir'ntlltionott:anitalior: n thesecond tep, espondentsegin o organizhe nformation ysogmentinghepagento basic egions, ifferentiatingndividual isualelementsrom he

hackground.ndperceiving rcups r relationshipsmong lements. he stagenvolves othprcattentivendaltentive rocessing,nd espondentsove rom bottom-upo op-downprocess'ng.l (* &mpletion: n the inal step, ttentions on he askof answeringndividual uestionswhere urvcy espondentsocuson a smaller rea f thepageandbeginattentivelyproccssinghecomponentsfeach ndividual uestion. op-do$'n rocessingccurs n thisstagewhere he survey ontext nd espondents'riorknowledgenfluencentetpretation.

Allontion and visualprocessingl't ottcnti|eprocessing: road, apidvisualanalysis f theentire ield ofavailable nformationlhatdetermines hichvisualelements reattendedo n laterstages ccordingo certainpropcrtieshatarenoticed ubconsciouslynd standout" becausebeydeviat rom other

i|lfbrmation n hepage.,4ttottit'( rocessing: onsciousisualprocessinghere le visual ield narrows nd he ocus son r few elementshatenter isualworkingmemoryandaremoreeasily ecalled.Ii,r(d/ r,i.,w:when respondentsre ocused n ndividual urvey uestionsuringattentivepr'occssing,heir ield of view shrinkso about degrees r 8 to 10charactersn width.lh,ltt,,?t-) ro&lsing: V sual nformations quicklyprocessedy the visual ystem lone, ndr)nly hc visual timulustselfinfluenceserception.'11,t,l '', n urssing: Y sual nformations processedased n hecontext f the situation ndthc vrcwcr's ultural nowledge, riorexperiences,ndexpectations.I i$ut /.t:i\tnl t), r,t,tutir)r: hc orSanizationf thevisualscenento what s objectandwhat sb:rlJkgr(nrn(1.ctcrmincd y thecontounofthe visualelements, nddetermining rderofrlrcnliofiduringprocessrng.ith figures ttendedo first.

Irr llu' ittlirnn!iott urvtnizatittnstage, espondents divide the page into basicf ,l{i()lrsn( ( (fr(lirrg kr tht'ir shared visual properties, a process called segmenta-lirll. ( )rrtr. lrt' p,rgr' s rlividcd into regions, the contours and boundaries helpl lrr,rr,sl 'orrrl t,rrll istirrgtrish igure frorn ground and differentiate ndividualvlur,rl r,h,nr.nls l),rl r lr.rrst'd n ftrrthcr visual processing. hen, respondentslrr.girr o pt,rr ,i l r ' plr '()rl ls l )r1)rrg su,rl ' lcments,using the GestaltgroupingIr i l r r1r [ ' r , , rs, l is l r rssrr l ir r ( l r . rptr ,r ' .1 .i rouping information helps speed|| | | l r | r r | ' r , r | | r l l , t | | r l i t t t l t t r rvr" ' or t t l r t t ' l t t ' l ts i t t t t .i r r t 'x i tmpl t ' , rcspondentstt t ig l t l r |gtrr o l ' r ' t r ' t t ' r ' r 'h ' t t t | . l t l ' rrl r t t t t i l , t t ' rr t t l t i ts l h,t l , t t r 'kx,t t t ' t l c l l rsc nrrt t I rur h dr d |r' l ul lr,r l | tl l l | r! l l l l l (| tl r, r rl 8r(rtl l t ' I tin l rttl l r-[relwt'etr

Inl l I 'R( )M L' t rt i oNri tr)ALl||tni (rNNAtRl. )r?tri rl tt f iut\t Vi::t!t lSlir!| ltr l l ' '( l


10/42

sl,lA('wlr(,r( ' lrt.r ' l t 'rrrt 'rrtsll,tt .I.( 'Ptr'.rlt(,t.tt ivt,lyr.111,1,1,q11rr Sl,rgt, ,rrrtlr '.rrly n Stagt'2 art' now netiv('lynttonded o orrtl ,ognilivr,ly r()c(,ss(,(.1.Mcaning s detcrminedby the visualscene, ut lop-rlorrrrrlrr1)tssirrrrlstrbegins, so that the context of the situation and respondents,culturrlknowledge,prior experiences, nd expectations lso nfluence he meanrngassignedo visual nformation.After respondents erceiveand organize he basicvisual elementsof thequestionnaire,heybegin he hird stage,ask ompletiorz.t is during thisstagt,that respondentsshift their focus from the entire page to a much smaller are;r(the foveal view) for more focusedprocessingand that they first begin to readthe text on the page n any detail. During this attentireplocessing,cspan_dentssequentiallyattend o the components f each ndividual question tht.questionstem,any additional instructionsor definitions, he answerspacesand/or response ptions),and these omponents nterworking memoryandare more easily remembered.How respondents erceive he nformation in the question nfluences ht.remaining four stepsof the responseprocessas describedby Tourangeau(1992): omprehending he meaningof the question, ecalling mportant in_formation, forming a udgment, and reporting a response. he taskcomple_tion stage s dominatedby top-down processing,where the survey contexrand the respondents'prior cultural knowledge and experiencesnfluenct.how they nterpret he visual nformation.For example, ymbolssuch ascur-rency symbolscan have different meaningsbasedon cjtural expectatrons,and symbols such as boxes and circleshave different meanings in web thanin mail surveysbecause n the Web,checkboxes allow multiple responsesand circular radio buttonsonly one.In Figure 6.5,we demonstratehow respondentsprocessa questionnairt,by showing imagesof what respondentsperceive in eachof the three stages.During the first stage, espondentsnotice the basichorizontal and verticallayout of inforrnationon the page;distinguishchangesn contrastand color;and notice he dark,medium,and ight gray and white areas n the questiolr,naire page (theStage1 mage n Figure6.5).Respondents erceivemultiplt,

dark gray featuresof different shapesand sizes a circle, ectangle,and fivcsmall squares),hree medium gray rectangles f different sizesat the top olthe page,and white spaces n the ower right and bottornareasof the pagc.We show two imagesof Stage2 n Figure6.5 o differentiateearly Stagt, ,where preattentiveand bottom-up processingstill dominate, ate Stagc 2,wheremore activeattentionbeginsand respondents se op-down prcct,ss.ing. In early Stage , respondents se he boundariesand contrast o s(,snt(.nlthe page nto two regions.Once he page s segmr.nted, (rspond(,nls()(.llson thedarkergrayand whiteelements nd ct thc ightgray rt,ct,t lt,rrtrr lrr.background theStagc a magt').n this ,,trly tagr,,(,sP()n(lcntsrolict,,r.t,rrmorc (' l (tments locl t t .rJ ir r rJ i f f t ' rt '1t ,rr.r ' , ts l l l rr,pi t91,,111,1lr1i l .r, ,1.yrrr11rzr."i ,

r l t , tPr,s, ,rrrr lnr l r ' , rs l .n ,rk.Sl,rgt ,2t l r r ,St,rgt '2bnr. tge),esponclentst tendto lhcscvariotrs .lt 'rrrt.rrls,r(rupirrghcm rnd usingprior knowledgeandlxP('ri( 'nc('s() ssigrr n(\)rrirrgo thcm.Rcspondentsistinguish he headerrcgion with thc ogos dcrrtifying hesponsorand the contact nformation, helil lc of the survey,and the nstructions o respondents.n the question egion,rt'spondentsperceive he sectionheading and instructions, ive questions,(nr(.with four subcomponent arts,and the answerspacesrespondentsmaynlsobriefly notice he "officeuseonly" area n light gray at the bottom of theprrgrr.ln the final task completion stage, espondents ocus their attention onpt'rceivingand comprehending he words in the first questionstem so theyciln hen retrieve he relevant nJormation, ormulate a udgment,and reportI responsen the accompanyinganswer space Stage image n Figure6.5).'l'lren espondentsmove on to focus on processingand answering he next{,lucstron.Once respondents an easily get into the pages, hey can then focus onnnswering the individual questions.The following guidelines help makenure hat respondentso self-administered uestionnairesan understand hebasic ayout of the questionnaire agesand the organizationof informationcontained within them in order to process questions and navigate through

thcm in the intended order.Guidctine .6:Establish onsistencyn theVisualPresentationfQuestionsktcross agesnd Screens),nd UseAlignmentand VerticalSpacingo HeIpRtspondents rgmize he nformation n thePageWc have seenmany questionnairesn which it is not clear to respondentswhcro they should begin and how questionsare organized. n theseques-tionnaires,espondents ave o search he page o figureout how to navigatebctwcenquestions. n addition to presenting he componentsof the ques-tiorr consistentlyand creating clear grouping and subgrouping (discussedin Chapter4, Guidelines4.10-4.12), isplaying questions onsistentlyacross(lur.stionnaire agesand screens an help respondentsas they move fromrlr.rcstiono questionand page o page.'l ht' (lestalt principle of pragnanz, ntroduced in Chapter 4, posits thatvisun inf()rmation hat s regular s easier o process nd remember. n addi-liorr, cpcatcdobservation n cognitive nterviews of respondentso both webilrxl p,rp('rqucstionnaires hows a rhythmic quality in people's esponse at-h.rns. hc rcsp(ndent goessomewhat automatically rom the answerspaceol or)('( lu('sti()n) thc bcginning f thenextappropriate uestion. rranging(lu(,i l i(nrs irrri lar'ly ith {hc prrts in the sameorder or each tem helps e-rpottrlt 'tt lsrc ttot' t ' l l ici lrtt n sctlut,ntially roctssinghe nformation n ther;ttr 'slirrtttt,t itr '.)tttt tcs|onrlt 'tr lstrt ' nkr hc rhythm of rcsponding, evia-lhrrs l orrr l r is rr l r rr , r llk .rr . , r r lo nr isul( l ( , rs l i l t ( l ings.igtrrc6 .5 Stagt '2b)


11/42

atlj

. t;

5iB2Eg-gJE

.E 93EFE;:!. :

::l : iiririil$IEtiiiI iti| lii,

iurigtirllii : : r'diffirtniiiiliil:tils&liiiiuii8.Ed;IDEI

17770

Lit:o4A

7)t:?!l2E'7,1n,t '.r. 1,.3, i

.? {." , . \

stLr.r

; r, ;,i ,;tl I

dlf f l, , . "

;. *II

',, ',l:,,

$w'$!rfFi9r E5. . :AE;i&tl I

'u)0)RYE

oo(!o)'co(!

.gr!q)(!9oeqJIooErotoE|'ttt


12/42

Figurc6t i l x , r r | l l lo l Ir . . .1 it l, t , \ , t t, r! , 1, t , . t , nr . | |

,rnL1 igure6. 6dispJay 'xanrplcsof ir mail an d of a !\,ebquestiorln.rirL,\.lr(,r(Llr.rcstionsrt displaled c()nsistcntl ! on thc p..lge tn d acrossp(rges o h(,11,respondcnts ts heVnar,igatebt.tr\'eclt u(,stions ln(i pagL.s.- .l n adcl i t ion r d ispiavi rrg ut.st ions onsistc:r t l r , ,l igning hc m is.r f .r t , r r ,fttl tool in hclping rtsponclcnts o pfocL.ssnr l org.tnize he qucstions )l r lr(.pagc (l-ir l i .r 't ' l l , olden, & Butler., (X)i i).{hcn pr(,scntinl l nLrlti}rle lucsi i()non th('p.t8c or scrccn, \, c suggcst al igning rltrestiorrsertici tl l \,; so th.t l l ),questioltslents t l bt'gin t tug it citntnton vertici .tlinr-.. h s yerticalal ignnrr,rris consistclttu,i th thc nattrrai re.rrl ing trcier n ntanl languages,n.her-c,pr,,,plt'begJn n thc'trppt'r eftJrdnd colncr of tht page,movc horizorrt.rl l l ,rcr.o,th e pagt'.ancl hc n continuc this proccss ,ertical lv 1or,r,nht pagc. \\t,,r1.,,,sul lgc.st lacing mor.. sp.lccbetl,r 'tcn luc.stionsha n betr,vtc,nhcir.sLrlrtorrlponcnt parts to rcintorce hc Erouping of the subc()mp()nent ., rts , (l s(.U.\( lin Cluidel ine 1.11.{c cannot enphasizL. nough th e effe,ct iverst,o l l , l , r r ispace n th c qucstionrraire or helpi l tg rcsp(ntdcntsorganizc, irr.,l for-nt.rt ,r ron th c pagc anclgror-rp elatcrl ntornrl]tion.Once tht 'qucst io l t stents.rr t ' , t l igrrr .c ln t l tc lc i t s i r l r . r r l l r r ,l ) . lS(. . . r r , . \ \ ,spacescanbeloci t ter la lont. tr rotht,r .cortrnrorrrcr t i r . r l l i l , . tLrt l r t . r i , l l r , . r r , l Io t t thc 'P,rgt ' , . tssl r r ) \ \ ' l ) in l : isUf()5(\ l . r i i r , l l , ) \ l t , r r r r t i, \ \ \ , l , r , t ( r r r , l t r i

t t tg l l tr ', trts\r( r \ l ) i l ( ( , \ () t t .{, \ l ),)tr. ,(. rtr,r lorrr.. rrL, l l , , , t , , L,r rrr, l , I j , .rI r , 1 1 1 ; 1L 1 (||Ir. . . iL,||. . i , . ||I I r,r.r , rrrr|, |, rr tr , , ,, , |, t , ,r rr t r l l ,

\ t, t l t r )r / , r r lr , , l \ t l , t l , l tnLtt l l l, i

r, 1,, ,r r , l rl 1 , , , . , r l r , t r l , l r r r l r , r ,r r , l r r (l i r l r r tJ l t , r1., 003), vhcrt ' th t .r 'l f l . l l ' r , i1 . . . , l r r ' , l | l ( r \ l r ( i I , , 1 r( \ () r ( )ns\\ ' r ' r -paces, et()rL.T lo\ ' lng1, lr l | ( l , l r r r , \ l r ( ) r rl l , , r r l r orrc 1r ls l ion ssl r( ! \ 'npe p,r l le, rs l r man\ ' \ ,cLrur\ ,r .vs,t is helpfr . l lr , , ' { 'I r r ' . r ( r ,1rr t ' ' l ion n thc s,t tr rc laccon thc scrcen o resp()ndents now

rr l r, lo lorLrst l r t ' i r , r t t ( 'n t iononelchpagc.Als() , indcnt ingthc' . l ts\ \ 'crspaces'r r ' . l ) ( )ns( r ' , r i r .L()r iesndenr( 'nt l rhe( luest ion tcrn, s sho\\ ' l t 11 igure6.6,r , , l t , | l l r r ' lp l r r luhfr ] thereisonlYol rcclucst ionpcrpagL.bccauserespol rdents1, , , 't l r.rr . o nrove heir cyes .l l l th(' \\,av across hc scrL'eno percelle th e'l | \\ ( r5l1. la( ) \ .t t r l r . r l l , d isplaving r luest ions onsistcnt ly hroughout thc.qucst ionnai re'r , ,1 r r in ingqu(,st ions.rrrr . lh t ' i l cor lespol td ing.tnstvc.rpaces r categor iesrtr , , r l l1 r t ' lPs esponclentsrganizc hL. nfonrat ion on the page.lnd mainLr r ,r s(.n\a () f order as thcv motr. icross (lucsti(ns. Perhaps even nrorernl,or t.rrt, n \ ( 'rtic.rl r l ignnrentht' lps orcotrragc rc.sponclcntso process he,;r r , t rons n the rr tencleclrc ler ,rnddccrL- i tscshc. ikcl ihtr td t l rat tems wi l l1,,.rrr,rLl rt t t 'nt l l rr issed.r . t t t , l r trt t o. l : LlsL'Ctt lor t tLl Ctt t t 'Ltst o I(lP / i fs l ){)/ /r/r, / / ls{1rr! /r i . :r, 1r( 'I rut l , t turuls oi l l( QttL'st i t)I5utd frf Nrr. 'r,{/ / i rrr l rr l /)nl l t l l l tou,: l t l l tL'

t r ' l . r is .r po 'crful kxrl for l rr ' lping PL\)pl( '()r-tnni , /( i I r i ( l I rrr.rI i ()|r.rr(lun(l ( r. l . rrr, l l l rr,nrr' ,rningofsl -rtc i f icr isu,rlel ( ' i r( 'nis(l i r l$r' l l t l , r l . . r ( ) ( ) " , \ , l l ( . ){)()l )Lrr r l Lolor consistcntl \ , throughout the . l r.rcst iorrr,r i lL .rn l rL, l I rr ' ' l 'orr, lcnl ', l , . .rr l r i r l t nt i f , , , rc l ,rtcd inform.tt ion ancl c.t l t utsf t l t ( ' t , ls l . . rr i . tnsrrcr i lg t l t r.,1rr, . trorr' .(olorc,rl t , t lsoc()rt tr iLrutct()r( 's l r()ndanIn](rI i \ . ] | i ()n.rr1(l t l rt ortral l, r.rr.r l r1,pr' .r l f thc rl trcsf iorrnai fc. l \ I ler' t () l ()f c. l l l -tot bc Ltscrl , .()! t t fnst c. l tl' , \, r i( (l i rtr i rarscale-

\ l rr l l r l l r s l r.rr i rr i ' , rckgrotrnd can hclp rL.spondcnts in sc\ cral \ \ 'a\,s. F irst,rl , r, ,r l r ' \ . r f( r. l i ()n rr \ \ l r ich th t rt 'sponctcnt can focus attcnt ion. Thc' ust 'of,r 11r, 'r. . o ,rsris l Pror-r 'ssing s rl iscussccl n Cuictc. l int.6.E. Sc'cond,.t shadc(tl ' , r, rtrorrrrrl ,r l l ()\ \ \ l ()r-thf Llsooirthi te.rnsl ' t ' rspaces,thebenefi tof whichis, l r , r. , . ,r ' ,1l ' , 1,,rr. I inal l r;blackiontcolorforthetextc.rneasi lVbeperceivecl, ,rr rr,rrrr ' l r ol rrl l r l l r sh,rdrrl b.rckgrouncls sothat respondents c.tn e.ts i ly seerrr, l r, r, i l l rr ' l ( \ l () l l l )(,r1t l r 's t ion ancl i ts cODtpOnent pi trts.

I r l l r l , r . , l r,r, l i . , r) l l ) l l t r ' \ . l t r l gr(\ 'ns nrr,oft( 'n used .ts background cnlorsrr rr\ , \ l , l i , r,( 'nl (rr( 'pl ( ' . ls i l rg nlr! l c.rlnt ing. Oftentimes darkerl r. j , l , ,, 1 l l r, l , , r, : i r, ' rrrr, l ,olor (o r ,r t l . r lkcl grar') c.tn then be use,d fo r

| , r 1 I , , ' or r,1l rr 'r |rIo|| l l , l | |r 'rr rr:L'r l rt t l t t 'srrrvc\.()rs. l jor example,'r r !i l .r !, rt . i ! . l rr, l , , , t I ' lLr, r r r l r l 'r rr,r, , l lol l l1q r.11 1'1111111r1,.rnt1he nlr r, l I j rl Urr, l r r, l , , .rrrl ' , rr , ,1 , 'r . l l i r. tr, | L i rt rr|r lL,r ' t rr t , tkcrl ' r l ' , 1,, , ,r, ' , , ,r, , , , , ,1 rL| i , ,r r l r I r , , , ,1, r , 'r , l . r l r ' r l r\ 1, , . ,( 'r) l )Ut l (), ! | !r i ! l , l ' , , l l , , r rl , ' l l l , , , , , 1,,1r,, i l , r l l r, ,1', ,rr, l rrr l

,. ce lbq2.1r ,: lto* dt,iilbl.ia WSU .r . pt ce to go lo sgloot? Wolld yos say_.

( tt 'tt l i n ( uunttutVi:;ntlSltttttt ltts 175


13/42

l 7. l | t t v (J t tr 't trtN.' l( r A (-l l l " t l ()NNAll l l

Wlrt'nsnrallerwhite nnswersP.lcesreencl()s('t l i t l l i tt ,r ightlycolorcclor shadedarea hey tend to bc Perceivedas a figure, or tlle clements h'ltpeople focus their attention on, and the larger colored area recedes nttrihe background (Dillman, Gertseva,& Mahon-Haft, 2005;Hoffman, 2004)'Consistentlydisplayingall answerspacesn white on a ightly coloredback-ground wili, therefore, help resPondentsquickly and clearly identify whereihey should record their answersand canalsohelp them identify the naviga-tional paih in the early stages f visual processing'Moreover,white answerspaces anhelp respondentsmoreeasily ocus heir mouse,pencil, or pen onthe space, nd n paper surveys hey canencourageespondentso constraintheiimarks within the answer spaces,hereby enhancing legibility and facil-itating optical processing'Lastly,having all answersPaces ppear n whitehelpsrespondents uickly scan heir responseso see f they have completedall of the question(s) n the PageFigures6.5 Stage b) and 6.6showa mail and web survey hat both useda black font color with white answer spacesagainst ightly colored greenbackgrounds although he magesareshown here n grayscale)'Guideline .8:VisuallyGroupRelatednformationn Regionshroughhe UseofContrcst ndEnclosureGrouping information into larger "chunks" helpsrespondentso easilypro-cess nd;rganize the nformation on thepage(Lidwell et al', 2003)Dividingthe page into visually seParate egions through the use of contrast and en-closurehelps respondents to perceive the elementswithin the region asmorerelated to oneanother than to other elementsoutside the region Regions canbe used to group related information, suchasthe title and introduction to thesurvey,or to grouP elatedquestionsnto sections.However, egionsneed obe chosen arefullyso they aid in the completionprocess t is mPortantnotto group questionsor other nformation nto arbitrary regions hat have ittleor no meaning to resPondents.Many questionnaires ontainsectionsof questions hat are more relatedto one i.,othe. than they are to items locatedelsewhere'For example, he2010DecennialCensuswill usequestionnairesn both Englishand SpanishBy using one background color for Spanish and another for English, and byplacing boundariesaround the questionswith the different languages,hfi.gior,i b.co-e utt effectiveguide as o who needs o fill out eachpart of tht'questionnaire.Another effectiveuse of regions is fronr lllt' 2l)-P'lU('U s l)('p'lrtnr('nlof AgricultureAgricultural lt 'sottltt 'M,ttt,t11r'ttt ltt lttrvt'y,'t 2 5-hoLtrongpaPer survey that asks d(' lJ i l (\ l r l t l ( 'hl i (t t tr '' l l rrr l l l l ' l r l l l ( l l )t ' r ' l l i (nls ( 'rr)t l [ r l t r l lrelatccl c l t tcst iorls i l l lo rt ' f ' , iol ls,t t tr l "r 'r l l l t t ' r t" t" 'r l r i ' t l ly i l l l lxrf l ' l l l l i l l l l l is*111yq'y cc,tt tsc sotttr. s|rl i t t ' , , lo tt l t tr 'r 'r l rr l rt ' l i lhrl I t l l t l tv sl )(\ ' i l i ( ly l ){ 's0l l , ln||crs, , trrrl l l rr. r . r , rr r r , , r, l l f r, l l , l l rr, l l l l rr l t rt t l t t t ,t s |), ' t l r. trI t

or l l ( )n( ' t in l( , (c. ! i , . ,( )nr( ., rrr)rrrs rayconrplcte l l of the sect ionshe y(iu) without corrsrrl l irrg rrrrtls artd then leave he remainingsections olrt, cornplt'ted oncc they havc compiled the relevant information). Thefirst page of this questionnaire,shown in Figure 6.5 (Stage2b), has threert'1;ior.rs:he first region contains the title, sponsorship nformation, andH(.ncral nstructions; the second region groups several related questions(,rntl labels that area "Section A"); and the third region groups severalrpaccs or agencyuse only (seeGuideline 6.10 or more information aboutvisually deemphasizing his type of information). The regions are definedtusingenclosure,where a large rectanglewith contoured edgesgroups then,lated nformation. Contoured edgeshelp these egions ift off thepageandlrt,eomehe objectsof visual attention (Hoffrnan,2004). n addition, lightlycolore'd hading s used o demarcate he two regionson which respondentstrt'ed o focus heir attention,whereasno shading s used or the agency-onlyInformation o help this area ecede rom respondents' ttention.llcgions can also play a powerful role in helping respondentsorganizeI n ormation in web surveys, even when only one question is shown per page.l;igure6.6shows wo questionscreens here a consistent age ayout s useddcross creens nd related nformation is grouped into three main regions.'l'hekrp region ncludes he itle and sponsorof the survey, he middle regioneontainshe questionand componentpartsaswell as he navigationbuttons,nnd the bottom region ncludes nformation about how to contact he surveylponsor. These hree regions are visually defined by enclosing he relatedItr,mli n a rectanglewith different backgroundshadingand making changesln thc fonts used or the text.'l'hc op region s repeated rom oneweb page o the next and s ikely tobelgnorcdby respondents fter he first screen r two, similar to the"banners"Incluclcdon many web pages. Enclosing he components of the questionhl .r eommon region helps respondentseasilyprocess hem as a Broup. Inarldition,a colored background s used to encourage espondents o focuslhoir ttention on this areaand on the task of answering he question.Lastly,lht, lrotkrm region usesa white rather than shadedbackground o help de-('t'r,rui('ts visual importance but still provides the contact nformation forNrFrrrclt'ntswho need t. Cognitive nterviews have shown thatrespondentswlro t ornpletetla 25-itemquestionnaire sing this screenayout do not recallhnvingsct'nany information n the survey about how to contact he sponsor.( rlrlr'llrrr'0,1);'rrrs slt ttly ldantify heBeginning f EachQuestion nd/orSrr'1,r,,,All lrolglr l),rp(,fl(,sigrr('rs.rlmostrlwaysuscnumbers r letters o identify$dllr rcw lrrr,slirrrr,cb k,sigtrcrs't,tvt'oftt 'necn t'luctanto do his or woftrdrolts,)trr.s lr,rlw rt,ttP,tgt' v l), l l{( 'rn sl u( iotr s tsed, umbcrs reno tt t l . t , r l r r lr rr 'Lrrowl l1p,l r i r ' l r I r r ,nt iotrs Dr ' , r lr rr i r t tsc' , rr , l tl t tcsl iotrrppcirrs


14/42

lalgl#iages+i+siitEsg*si'11+i?i\tzl+ti1+111t

l7 l4 lrHorrr-) t . , t t , 'N., o, \(-)r r . , loNN l1 r ( ,r1rl i , , ,( l ( rt t t t t t tLtt t t , t t l Sl iuulu:: l7r)


15/42

rother l rnn he black ontuserl i r r l r t ' t t t r r .s l ions,rrr t l rn.hrt , . r l t ,drr ls i r i t .o lth eanswer paces.n addition, gency-onlyrrlorrrrationrr lre lucstionrr,rr.r,in Figure6.5 Stage b) and contact nformation n Figure6.6.rrcpresenl(\lin a gray font againsta white background and are located at the bottorrrof the page o help this information recede rom the respondents'attenti()n.As theseexamples emonstrate, everal ropertiescanbemanipulatedat ()n(,time (e.9., ontrast, ocation,and font style) o helpdeemphasizenformatiolthat none or only a few of the respondentswill need. n addition, layeringmanipulationsby bolding and increasing he size of a word or phrasecnnmake the word or phraseappear more visually promineni, emphasizing tsimportance.For each survey, we recommendsetting rules for when to use vari(tusvisual properties underlining, capital etters, talics,bolding, reverseprjnt,sizeof texL color,etc.)so that information is presented imilarly throughoLrtthe questionnaire.Often a construction rule book" describingwhich properties will be manipulated to display certain types of information can nerl'ensureconsistency n how visual information is presented,particularly f()llongeror more complicatedquestionnaires. or example,when redesigningthe Agricultural ResourceManagementSurveymentionedearlier, urveyorscreateda rule book to guide the consistent nd regularuseof visual nform.rtion, which then could be applied by all thoseworking on the questionnairt,(Dillman et al., 2005).Many of the rules for visual presentationmay be tht,same rom one questionnaire to the next; however, some rules are created for.specific ssues n a suwey and may not apply for other questionnaires, n(ithus constructionor design rule booksare ikely to vary from questionnairt,to questionnaire,Guideline .11:AaoidVisualClutterIt is important to view the questionnaire rom the respondent'sperspecti\'(,and eliminateor reduce he prominenceof any information that is not nt,tessary or completing hesurvey.Weoften seequestionnaire agesdesignt,rlsuch hat too much nformation s ammedonto the pagesand different tcnrson thepagecompete or the respondent's ttention.Somedesignersmay trs,,only an 8-point font so they can fit more questionson the page.Others sr.,,questionnaires san opportunity to promotebrandingand nclude ogosarrttext that prominently identify the sponsor(s),he client, he type of softw,rr|used, the secudty features,and other extraneous nformation on tht,wr.l,pages.Variouslysized ogos and imagescompete or respontlents'att('nlionand often distract hem from completing he taskat lranrl.There reseveral ays o reduce luttcron tht'p,r1ir..ir.rlrrrrr;i IIt 'rrrrrrrIrcrof i tcms on the pagc ncl increasing h( 'nt) l ( ) ln t rr l r l ,111l, ;1,q,, ,, r r r r t . l I rlsP() t tdentsl i n ( 'ntry nk r t r r ,| . rgr ' r r I or l l .urzr . l r r . rr r't t , t t l l I t r ,sr .n l l r lrt l t r ' t t t . l is l r ,r l l i r ' t r l , t r lyn l l t r ) t l , l t l or ' r ,n lo\ ' r . , r r \ , r r ,r l ( ' , r r l r \ ( r r l i , l t t u , , ot

ol l tr,r inlofnr,rl i ()r l l t , r l t t t , tv t l t , tw ,t l t t ' t t l iott awity l t trt l thc task t l f answeringl l t l t l rr,s l iorrs.rtrt l lh,t nr,r nr,r ( ' i l rt tort ' t l i f f icul t to rcspond. Complex group-lngr rrl r l t rcst iotts, sttr ' l r as gri r. ls att t l nratrixes (and part icularly unnecessarylltrn within grids or matrixes),also nterrupt the flow of the questionnaireEnrl t'nrl kr ntroduccmorecomplexity,asPeople ave o connectnformationdr osscolumnsand rows. Also, reducing hevariety of different elementsonlltr. 1r,rge clps to reducecomplexity.For example, ncluding five questionsol llr(.snnre ype may be fine for one Page,but including five questions hatprl lt'syrondentso respond n five different ways and that include differentlylr's o[ t'lements (e.g.,answer sPaces, heck boxes, drop-down menus,oxl ,r slider scale)makes t much more difficult for respondents o Perceivewlr(,rf to begin and how to navigatebetweenquestions.Arrotlrerstrategy or reducing clutter is to selectivelyemPhasizeonly allnritt'ri amount of information. Because isual information is processed nn,ldti(nr o the other information on the page, t is recommended o high-llght or visually emphasize ess han 10% of the total visual scene Lidwell!l 'r1., (X)3), s emphasis s lesseffectiveas the Percentage f highlightedhtkrrnr,rtionncreases. s a greaternumber andvariety of elements nd theirprolx,rti('sare manipulated, t becomesmore difficult for the viewer to dis-llnguish the oncs that deviate rom the other elementson the Page(Ware,l(104),irr example, f red is usedto draw attention o a particular word or;rhrrrst',t will becomemore difficult for the respondent o easilyProcessheIlrdng('irr color as the number of other colors on the page ncreases.Over-tll, rt'ducingvisual clutter improves the visual appeal of the questionnaire,Ittltr,.rsirrg coplc's motivation to respondand helping them to focus on theldll ol ,rrrswcringhequestions.l)nnh'liuro. 2: Minintiza he LIse f Matrixesand TheirComplexityMdlri\(,s r('present ne of the most difficult question ormats to answer orlltfr,r, 'r',rsorrs.rirst, hey requirePeople o match nformation in rows withtltx,i l i(nrsn trrlumns or vice versa), task hat s quitecomplex.Second,lltr lr.rtrrcsl o fill out a matrix and the instructions for how to do so areolh,n li l l i(ull to rurct'rstand,n part because f the comPlexity f the task.'l 'hlt 1, lx,slructrrrc f thematrix eavest up to the espondents o whetherkr t,rviprk,tl l( ' rrr,rlr ix nd fil l in answers rimarily n columnsor rows orr|I||r, (rrrtl) in,rl i()rr[ both. Morcover, ecause f all threeof the previoustll l l I r l l i( 's, lt 'rrrs rrt ' nort' ikely to be missedwhen they are arrangednl dlrl\r,! lr.rr wltr 'rr rrdivitltt,t ll ttestionsrc posed equentiallyn a verticalInyorr l ,' r ,sr t l lt t l ' ,r r r i l i l r t ' r 't t ' t t t tontcspttnst ' .

l {r,r rrl t lsr ' , t tr l t ot t l l t l Att t t ' r ' i r i t t t ( i rt t t trtrt tr i ly Sttrvcy has shtlwn that alw 0 1,,r1',r 'r,r l t t \ l l t , r l , rs l r l rrt t t , t t t tr ' ,1',r ' t t t | l ' r, ,r11',' t ' l , l l i l rt lshiP () ( )ne an()thetr, l l rr l ' l lv , .rrr, l t , t ,r ' lor ,rl l ol l l rr ' l r loIL' l iv i tr l i i rt l l rt ' tcsl rrt l t lc t t l 's l roLtsc-It l, l I ' , , l0$'r ' t t t t ' t t l | | ' \ l ' r

t l t ' , t ' ' t l r ' , t l t r l l i rr"t t(. t i l | t t t t lott t l .s l ){l t ls t ' rt r

IU{} | l tr(} l t r. , ( 'N., t(, , \ (-)t l r ., r }NNA||i l ( trn l iu I t t ( nnuut\ tVtt ' t l 5 l r / , / r / / l /s lu l


16/42

Qt

sol l ' t ( ' i tcms hln dtx 's nsking l ( ) r n,sp()ns( 's() ) ( .nt, t( l ( 'kr .r l l t l t r r .s l i r r r r , ., ,rone person at a time, as s lcl 't i ( 'v('tlry placitrg gLrcsl ionsor t.,rr r rrrlrrrlrr.r ,lihe household n their own vt'rtical column (i.e.,no t a rn.rtrixiC h('st r , . ' l lol iTh e atter method of asking or al l nformation fo r one pcrs()n t J I r)rr '( ,r 1r,,,,the navigational path fo r answering rather than lcaving it up t() h(' 11 .1'r,rldent as to whether to work vertically or horizontally to compl('t(' t r(' n , r

Even with these substantial difficulties in mind, we pnrpost' rrlr, rn11rather than eliminating the use of matrixes because n some survcys llrr. r rptexity of the information asked for can best be communicated irr srrrrl,l,matrixes, and some survey populations are used to thinking abor.rt n 0I rrr,rtion presented in this format (e.g., accountants responding to establislrrrr,.rrlsurveys). However, we have seen matrixes that required the resp()n(l('[t t,,remember as many as seven different specifications (through the use ()l sr l,headings in both columns and rows and additional definitions or instrrrt trorrbooklets) in order to fill in a single cell of the matrix (Dillman, 2000a, p. .l.t trMost people can only remember four to five pieces of information at or, r,and the likelihood of errors being made when so much information nr rst r,connected is enormous.

The Web has had conflicting impacts on the use of matrixes. On thc orrlhand, the fact that many web pages are built using tables has encouragul tlr,.use of relatively simple matrix formats, although they are often excccclirrl,,lrlong. In addition, inappropriate shading and the failure to suppress gritllirrr.'.may make matrixes difficult for respondents to process. For example, rrrr,.survey we saw recently had 19 tems on a single screen,and each tem w,rs tobe rated on a scale ol7 (poor) o 10 (excellent).That's 190radio buttons on or lscreen! Even though their overall structure is quite simple by comparisorr t,'some matrixes, the length of many such rnatrixes is highly burdensomt' .rrr,ltedious for respondents, and connecting information from rows and colrr rr r',is still quite difficult for some. On the other hand, the Web has decrcasul tlrr.use of very complex matrixes with multiple specifications for each colurrrrrand row, The Web is simply not conducive to such matrixes becaust'tlrr.1would require both horizontal and vertical scrolling in order to fu ly conrplr.hend th e questionsbeing asked.Such scrol l ing adds an addition,rl layt'r.r ' lcomplexity that has negative effects on response. Paper surveys, t.sptr.i.rll1those used for business surveys, have exhibited a tendency to usc frrrnr d,rlrhmatrixes n order to reduce he number of pages and costsof mail ing. lrr,rsmuch as many businesssurveys that use matrixes arc now nratl t '.rvai l ,r lr lr.in a printable format or paper altemative, the reducctl tcntt' l rr.y l irr trsirl iexceedinglycomplex matrixes on thc Wt'b is rt't l trcirrg lrci l rrsr.()n l)!rl \ ' las well .

Whcn matr ixt 'scannot bc avoidcd, onst orts . l lor ' , r l rorr l r l rr . r r ,r rh 'oc()ntr() l l r ( ' n trnr l r t , r f l ) i ( ' ( r ,s )l ink) l l r , r t i (nr l r ,r l r r r . , l l I 1 rr rort l t ' t kr Plovi t i t ' , t s i t t lg l r . , r trswr,r ' .r r r r 'orr ,r ,l r r . t r , , r .r r l t , , rn l ,r \r l l , , l l r , r l lr .

l '0l trl l r( t(\l s() rs () (,n(()ur.rl l (.(.,rsy rocessing roth horizontal ly and ver-lhnl ly is t,sst,rrti ,r ltn d ea n irnl.rrovc he quali ty of responses.An examplett Jd ltrrlr i \ rrst'd rr an intl ividual-pcrson, general public survey to encour-l f,t, l l rouglrtful consideration of only two items of information is providedIt t lr lprrl t,( '.7.n morc complete explanation of a similar matrix is providedf l ,wlx.ft ' ( l) i l lman,2008).Examplesof other effectivematrix layouts arepro-vlrlr,rlhy l)i l lnran et al. (2005)andMorrison, Dil lman, and Christian (inpress).

lbulr g.z Exampleol a matrix ormat used etfectively n a household urvey on(x t| trunrly ssues.l0 I'hri( thlnli for a minut sbout thre rclatives who do not liv with you but with whom you(oornrt lcrto mosl freqoently, rnd rosrder hesequ$tions. It may help to list their first nrmelnl t lNh n the inDrovided.

Rclrtiv |v Relative 2w Relative 3Vllr|l n.|||c/lnftlslslol l knt { l )

1) t ' | lh l i rol$l l le l veh l ,cwl r l r )| 'hrhloo?.... . . . . . . . . . , . . . , . . .

l lnor rborl how srl,fiihton/( hrkrkrn dol i r l l lwi '. . . . . . . . . . . . . . . . . . . . . . .

Alrorl hol l okl s hlsrrlal lv{i '. . . . . . . . . . . . . . . . . . . . . . . . .

Apl ro!knnl r ' l ) .ho wlomnrr r l ( { l ( w l lh lhlstr l . l l rr?... . . . . . . . . . . . . . . . . . . . . . .

ror l | rnrnkr t r * l lh lhl rr rhl l r r . whl(h ol t l r(fnll"|rlrs 'lo roo tri(trt0rlol l ! ||?.., . . , , . . , , . . , , . . , . .

h ll h rrl . l l r( l r rrrr ollh r r .nr r o ' I rol r r r l 'na.r , l r rhr r I rn| l l r rr ro| | f

-Mi les

tr l i vcry la ytr l ivcry wcclC l .css hrnon.c a

tr l ' r rs(nr r l i s i ltr I ' , rs l , r lui lD l r nl r r lt r ( f l l |hon.OI xl . , relcphoDc

O lesO t'1,'O l ) ( i ) r ln, ,w

(name)

O YsEN o

-Mi les

O Every dayO Eve ry weekE Less hanoncea

tr Postalmailtr E-mailtr Cel lphoneE Other elephone

t rNoD l)on't know

tr EverydayO EveryweekE Lcss hanoncea

E PostalmailtrCell phoneEOther telephone

tr YesEN oODon't know

tr YstrNo DYstrNo

\r,rrr, I rr. ,rprr rrrrl ' ryr roIrpv o! ( r 'nslnrct ingJucsl ionnaires"pp . l6l -175), byf A l l t l l t rrrrrr.r lnttnrtrt ' t t , t l l lurlhrt l t l , \urvtv l thnhthryt. Ia.D. de Leeuw, . J.l l r ' \ , rrrl l l :\ I ) { | ), . !( l l )8, rt \ \ i ' rL l ' \yrlx ' l ' t Iv l ' t ! . r\

I t l? l r trorrr r r . or ' r ' , o ,r ( - ) t r ,r o l t r r r i r t)trrrlt ltttt:,rU )r.rt{,1,,1{ti ltttrl lNtb t6slirrrrrrrrir, s llJl}


17/42

( ;u l l ) l i l . lNl ,s tr ( ) t( l ) t is t( ;NtN(; MA . n Nt)wl , l i Qt] l .s I t ( )NN A { l STh e guidelines we havc pr('s('rrt(.(l P t() tl l is l .rointon ortlcri lrg thc rlut,stions an d creating a common visul l stinrulus apply k) both mail an d wclrquestionnaires. We have highlighted how some,of thcse guidelines may o(,implemented differently in paper and web surveys; however, the commonprinciples underlying these guidelines are the same for both modes. Becauscof the underlying technological differences in mail and web surveys anrlhow people relate to these different technologies, we now turn to discussingguidelines for questionnaire construction that are specific to either mail or.web surveys.

Tr r -:Mlrl QuesrloNNArREWhen the first edition of this book was published in 197g, he possibility ofusing graphical design features effectively for guiding respondents throughquestionnaires was quite limited. paper questionnaires were typically pro_duced by typewriters, which did not allow for bold versus light type andon which shapes (e.g., check boxes) were difficult to create. Vertical spac_ing, dotted lines to connect text (e.g., opinion items on the left side of fagewith answer choices on the right), and hand-drawn lines were amonq thefew features that could be easily used in most questionnaires. Black-on-whiteprinting was often the only practical choice.When the second edition was published in 2000, computers had madepossible the use of varied shapes, fonts, and graphical displays. However,printing remained predominantly black on white partly becauseof the highercosts associated with color printing. We are now in an era when full-colorprinting has become widely available and less expensive, so even smallerorganizations can use color in their questionnaires. Moreover, the linkineof computers with printers now offers the possibility of intelligent printing,whereby variable inJormation can be inserted into questionnaires while theyare being printed so that each paper questionnaire can, n essence, e aunrqucdocument identified by an assigned identification number. The availabilitvof so many different possibilities for printing and improvements in the tech_nologies for achieving the possibilities now give surveyors choices that wer!.considered impossible less than a decade ago and have increased the use ofmail surveys in combination with other modes. Color can be used effectivelyin both web and rnail surveys, and rotating and completely randomizingquestr(,ns )r rcsponscoptions are also possible n both nrorlt,s.l lowt 'vt ' r , r l , r i l s l r rvt 'ys lso havc certa in imi t , l t i ( )ns l r ,r l l is l i r rgrr ishlristr r tL, l rorrrot l l ' rs. ll p( \)p l ( ' , l r ( , to [r t , t l in ,ctt ,d o ( l i l l r . r , r r l r r l low_rrp l ,r r rstk 'Pr,rr r l in6r | r r r r l r t , i t , r r rswcr ra Pnr l i ( .u l , t r ( lu( ,s l iorr , l l r r . t , r r r r r r l\, 1) t( , \ , i ( l ( \ l

with v( 'rysp( '( i l i ( i r ( 'e l i ( )r ' tso grr i t l t ,ht .nr.r rat ld i t iorr ,t is rrot ossibleolnc()rpor.rt(,lnsw('rs()c,lrl ier lu('stionsnto ater uestionsn order o makelaterclucstionslt 'arcrrnd 'i lsicr ().lnswer s s thecasewith web surveysilnd computer-assistedelcphonenterviews.Neither s t possible o provideilutomatic totals or feedbackwhen answers o questionsare inconsistentwith answers o previous tems (e.g.,when current age s inconsistentwithycar born). Also, later questionscannotbe completelyhidden until earlierquestions reanswered.n addition, t iseasy or paper espondentso perusethe entire questionnaire o get a senseof topics coveredand length,whichmay be more difficult to do on the Web.

Cuideline .13:Determine hether eypunchingr Optical mnging ndScanningWill BeUsed, ndAssessheLimitations hatMay Impose nDcsi n ng and P ocess[nguestonna esKeypunchingi.e.,entering the data from a questionnairento an electronicspreadsheet)has becomean ncreasingly significant cost or thoseusrng paperquestionnaires.t isnow possible o avoidmuchofthat costbyusing softwareto design,scan, and processquestionnairesoptically; however, the use ofopticalscanningplaces ertain imitations on the designof questionnaires.nthepast,optical questionnaires ere not respondent riendly as hey were notvisually pleasing,nor were they easy o completebecause f the limitationsimposedby the technology.For example, he answer spacesneeded o beprinted n red or otherbright inks that contrastedwith the black nk used ortheansweroptionswith which they were grouped,and therewere imitationson which parts of each page could be used for answer categories s wellas what kind of mark neededto be made by the respondent.Significantimprovementshave occurred n optical scanningdesign software and inthe capabilitiesof scanners uring the past decade. n addition, there s not'vidence hat well-designedquestionnaires hat can be optically scannedwill obtain lower response atescompared o other questionnaires. everaloptionsareavailablenow s

Dillman, Smyth, Christian, Chap 6, From Questions to Questionaire

Documents

Transcript of Dillman, Smyth, Christian, Chap 6, From Questions to Questionaire