•

United StatesDepartment ofAgriculture

StatisticalReportingService

StatisticalResearchDivision

SRS Staff ReportNumber 85

June 1985

A Comparison of CAT Iand NONCATI on aNebraska Hog SurveyRichard Coulter

•

A COMPARISON OF CATI AND NONCATI ON A NEBRASKA HOG SURVEY. By RichardCoulter, Statistical Research Division, Statistical Reporting Service, U.S.Department of Agriculture, Washington D.C. 20250. May 1985. SF&SRB StaffReport No. 85.

ABSTRACT This report presents results of a study comparing CAT Iand nonCATl telephone interviewing on the June 1, 1984,Hog Multiple Frame survey in Nebraska. Comparisons aremade in four areas: enumerator productivity, levels ofestimates, number of Generalized Edit program errors,and enumerator evaluations of the two interviewingmethods. The completion rate for CATl was lower thanthat for nonCATl primarily due to factors addressed inlater versions of CATI software. Differences in esti-mates and error rates were insignificant. Present non-CATl interviewers adapt well to CATI. Enumeratorpreference was evenly split between CATl and nonCATI .

........................................... ,." .." ...,..• This report was prepared for limited distribution to '• the research community outside the U.S. Department of •• Agriculture. The views expressed herein are not •• necessarily those of SRS or USDA. •...,..".", ,..,.., , ,.,

ACKNOWLEDGMENTS The author thanks the enumerator and office staff of theNebraska SSO for their enthusiastic assistance. Specialthanks go to Dan Capstick for his contribution to theproject and to this report.

CONTENTS

Page

References .

Eval ua tiona .Conclusions .Recommendations •••.••••.•••••••••

23578

.••.•..•....•••••.••.• 10111616171824

Appendix A: Evaluation Form ••••••Appendix B: Hog Questionnaire ••••

Edit Errors

Summary ••••••••••••••••••••••••••CATI Background ••••••••.•.•••••••Study Design •..•.................Productivity ...•..•.......•......Es tima t es .

SUMMARY Previous research in California indicated that usingCATl r'esulted in fewer General ized Edit errors and sub-sequent updates. no differences in length of interviewor refusal rates. but a difference in estimates for onecattle variable (deaths) (3). This study tested thesefindings on the Hog Multiple Frame survey in Nebraskaand sought interviewer feedback for planning in futureCATr states.

Selected strata on the June 1. 1984. Hog Multiple Framesurvey in Nebraska were randomly divided into two half-samples. CATl was used for telephone interviewing on onehalf-sample and regular "nonCATl" interviewing was usedon the other. From a total sample of 1622. 550 CAT!interviews and 575 nonCATl interviews were completed inthe test strata. The study compared the two methods infour areas: productivity. estimates, Generalized Edit(GE) program errors. and enumerator evaluations admin-istered after the September 1 hog survey.

CATl productivity, measured by the number of completedinterviews (including refusals). was about 12 percentless than nonCAT!. Computer response time and instru-ment design contributed to this difference. Improve-ments in the CATl software were made in those two areasafter the June 1 survey. There were no significantdifferences in refusal rates for the two methods of datacollection.

No significant differences were found in the levels ofestimates for the test strata between CAT! and nonCATI.Differences of up to 18 percent for some variables werenot significant due to the relattvely small sample sizesand small number of hogs in the test strata.

The GE program generated few errors in either sample.Consequently, this measure of relative data quality wasinconclusive.

Evaluation forms revealed that enumerators and respon-dents generally accepted CATI; all enumerators success-fully learned CATl, and about half picked CATl as theirmethod of choice on the evaluation forms.

2

CATI BACKGROUND Since CATI is a relatively new technology, some back-ground information about CATI in general and about theStatistical Reporting Service's use of CATI in particu-lar is presented prior to discussing this study.

Computer Assisted Telephone Interviewing and the acronymCATI refer to the use of a computer and related softwarefor telephone surveys. CATI systems convert survey ques-tionnaires into executable computer programs (instru-ments). Telephone enumerators work with a video displayterminal and speak with a respondent usually through atelephone headset. The terminal's screen displays eachsurvey question and enumerators record answers via theterminal keyboard. Features of CATI systems caninclude:

* Automatic question branching* Pre-programmed on-line edit checks* Ability to move back and forth in the ques-

tionnaire while maintaining the integrity ofquestion branching and edits

* Personalized interviews by insertinginformation from the present or previousinterviews into subsequent questions

* Automatic scheduling of calls and call-backs* Several report modules:

- Status of individual cases- Overall survey Progress- Enumerator evaluation- Various analyses of survey data

• Supervisory interview monitoring at amonitoring station

• Assistance in post-survey coding and editing* Direct data entry

CATI's popularity has recently increased with govern-ment, university, and private research organizations,following increased use of telephone interviewing as analternative to more expensive face-to-face visits and asa means of combating nonresponse in mail surveys.

Government agencies with CATI capability or who are inthe process of acquiring it include:

* The U.S. Department of Agriculture (USDA),Statistical Reporting Service (SRS)

• The U.S. Bureau of the Census* The U.S. Bureau of Labor Statistics* The National Center for Health Statistics

Other agencies, many more university and private surveyorganizations, as well as a number of organizations inother countries are now involved with CATI systems •

House discusses some benefits of CATI that contribute toits popularity (3). In general, CATI benefits can be

3

grouped into two broad, not necessarilycategories:

distinct,

1. Improved data quality through more standardizedinterviewing procedures, flexible questionnaire design,on-line editing, and enumerator monitoring.

2. Efficient survey management through automatic reportfunctions, case scheduling, and eliminating a separatedata entry step.

However, installing CATI includes:

1. costs of procuring and maintaining hardware

2. costs of developing and maintaining software

3. costs of additional training for interviewersand survey managers.

Hardware costs can be shared if the equipment is usedfor functions in addition to CATl. Resources needed fordeveloping software and training enumerators woulddecrease over time in an organization that has a stableinterviewer staff conducting repeated surveys.

SRS signed a research agreement with the University ofCalifornia at Berkeley, Program for Computer AssistedSurvey Methods (CSM), in 1981 to jointly investigate thefeasibility of using CSM's CATl system in SRS surveys.Future references to CATI in this report refer to theUniversity of California at Berkeley CATI system.Shanks et al describe the ongoing activities of CSM (4).

A site was set up in the California State StatisticalOffice (SSO) for the first SRS test of CATIon the Janu-ary 1 Cattle Multiple Frame (MF) survey. Housedescribes the results of this test (3). The CaliforniaSSO now uses CATI operationally on several surveysthe biennial MF Cattle, June Acreage, and Fall A&P,triennial Rice Stocks and Processing Tomatoes, andmonthly Cattle on Feed - as well as for some data entryand list sampling frame maintenance activities. Moresurvey uses are planned. A UNIX-based multi-useroperating system supports these activities.

CATI was first introduced in the Nebraska SSO with atest of the January 1, 1984, Cattle MF survey. A sampleof operators not in the operational sample was inter-viewed using CATL This gave the SSO staff and enumera-tors their first experience with CATI.

A similar test was conducted in Nebraska with the March1 Hog MF survey. The SSO staff gained additionalexperience in using CATI, and research personneldeveloped and tested an initial CATI hog instrument.

4

STUDY DESIGN

CATI was then used on the June 1 and September 1 Hog MFsurveys and planned for use in all subsequent quarters.In addition, the Nebraska SSO is developing instrumentsfor additional CATI surveys and also plans to use CATIfor list sampling frame maintenance activities.

SRS formed a transition team and is preparing to makethe Berkeley CATI system operational in an additionaleight SSOs in 1985.

The purpose of this study was to compare CATI and non-CATI telephone interviewing in three areas: interviewerproductivity, estimates, and data quality. The studyalso sought interviewer feedback for future planning.Research in California indicated no differences inlength of interview time or refusal rate, a differencein estimates for one cattle variable, and a large reduc-tion in Generalized Edit errors and updates (3). Thisreport in conjunction with those findings should provideguidance for future CATI work in SRS. The section oninterviewer feedback will interest CATI survey managersin all field offices.

The June 1. 1984 Hog, survey in Nebraska was a multipleframe survey using both a list and an area frame sample.This report addresses the list sample only. The listsample was selected from the general purpose list framein 11 strata numbered 81-87, 93-95, and 98. Stratadefinitions were based on control data size categories,which ranged from hogs unknown to 5000 or more hogs.

CATI interviewing did not include strata 93 and above(extreme operators). Stratum 98 operations were sentdirectly to the field for personal interviewing. Strata93-95 were excluded because of SSO desires not to changethe method of data collection for this group until CATIwas tested further.

Also, some operations with special characteristics, e.g.prior refusals and overlap with another concurrent sur-vey, were preselected for personal interview withouttelephone contact. The remaining operations were tele-phoned and. if necessary, sent to field interviewers forfollowup.

The initial sample of 1.622 units for list strata 81-87(control data ranging from unknown hogs up to 600 hogs)was randomly divided into two half-samples one forusual telephone interviewing and one for interviewing byCATI. The preselects defined above were deleted fromthese half-samples. NonCATI calling was also done instrata 93 and 94 which had 240 selected units.

5

Interviewers called the evenings of May 24, 25, 29, and30. Limited telephone calling was also done during theday, primarily on scheduled callbacks. A total of 550CATI interviews and 575 nonCATI interviews were com-pleted in strata 81-87. An additional 120 nonCATlinterviews were completed in strata 93 or 94. Remainingcases were sent to field enumerators for followup.

The study employed a staff of 16 enumerators andsupervisor; 13 worked each of the first 3 nights and 12worked the final night. Several factors preventedassigning enumerators in a manner which would bettercontrol any interviewer effects on comparisons in thisstudy. First. only 11 of the non-supervisory enumera-tors were trained to use CATlj thus, 5 did not use CATIat all but were needed to complete the necessary numberof calls. A fairly even mix of CATI and nonCATl callerseach night was desired. This desire. coupled withenumerator availability, resulted in 4 enumerators usingCATI exclusively, and the remaining 7 mixing methodsbetween nights.As a result of the enumerator assignments, differencesbetween enumerators could affect the comparisons betweenCATI and nonCATl made in this report. Enumeratorstrained for CATl did not have any apparent superiorityin interviewing ability. and they were not selected forCATI training on this basis.

The study compared enumerator productivity, levels ofestimates. number of SRS Generalized Edit error mes-sages. and enumerator evaluations administered afterthe September 1 hog survey.

For this analysis, reports 1n each half-sample were ran-domly assigned to one of nine replicates across strata.Replicate totals were used in SAS Proc GUM. For a dis-cussion of this method of analysis including formulassee Hall and Ford (2). Appendix C.

6

PRODUCTIVITY This section compares the relative productivity of CATland nonCATl interviewing as measured by two factors -rate of completed interviews and refusal rates. Table 1shows completion rates, calculated as rate per enumera-tor hour, and weighted refusal rates for the study. Theelapsed time used for calculating completion ratesincludes about 1 hour of training the first evening forall interviewers as well as 20 minute breaks each even-ing.

Completed interviews include refusals and usable (good)reports. Refusal rates were calculated by dividing thenumber of refusals by the number of completed inter-views, by stratum, and then weighting by stratumpopulation totals (3). A univariate analysis of vari-ance was run using replicate assignments (2).

Table 1: Comparison of Completion and Refusal Rates, CAT I andnonCATl, June 1, 1984, Hog MF Survey, Nebraska

Variable CATl : nonCATl : % DlFF 1/:Signif.Level

Completion Rate(per enum-hour)

Refusal Rate

5.3

8.9%

6.0

8.0%

- 11.1 %

+ 11.3 %

2/

.68

1/ % DlFF = 100*(CATl - nonCATl)/nonCATl2/ Not tested for significance because completion

rates were not kept by stratum.

Several factors should be considered before comparingcompletion rates. First, refusal rates were calculatedover all strata including strata 93 and 94 in which noCATl calls were placed. Thus, stratum differences andenumerator differences would influence this comparison.Records of hours worked by stratum were not kept, sothese strata could not be eliminated from the com-parison. The overall effect of these factors is unknownbut it is thought to be small.

Second, CATl enumerators experienced delays in computerresponse time both in searching for cases and in writingcases when finished. While these delays were eliminatedin a subsequent version of the CAT I software, they nega-tively affected CATl completion rates.

1

ESTIMATES

Another improvement was made in the CATI instrumentafter this survey. The introduetory items were stream-lined, thus shortening the overall interview time. Thisshortened version more closely followed the techniquesused by nonCATI interviewers. The unstreamlined versionused in June also negatively affected CATI completionrates.

Given the delays in machine response and the need forinstrument improvement, the completion rates for CATIand nonCATI can be expected to be close, although incor-porating edit checks into the interview tends to makeCATI interviews longer. Further efficiencies in CATIinterviewing are also expected as enumerators becomemore experienced and an automatic scheduler is imple-mented.

Refusal rates were not significantly different. Refusalrates tended to be higher in the larger strata for bothmethods. CATI and nonCATI interviewers had basically thesame resources available for dealing with refusals.Additional explanatory or rebuttal information could bebuilt into the CATI instrument for enumerators to usewhen interviewing reluctant repondents. However, thiswas not done for this survey.

Table 2 compares totals of the seven strata expansionsfor selected hog variables. Two expansions were calcu-lated for each variable. The first (All) used all dataas reported, while the second (Adj) reflects an adjust-ment made for two potential outliers in the CATI sample(one in the unknown hogs stratum and one in the zerohogs stratum). The adjustment removed these two reportsfrom their original strata and set their expansion fac-tors to one. Because positive hogs was a rare event inthe lower strata, it was difficult to define outliers.These two candidates were chosen based on visual inspec-tion of the data (both were reports of around 400 hogsin strata where the next largest value was under 70).Because of this subjectivity, both expansions are shown.

Univariate and multivariate analyses of variance wererun using replicate totals with SAS Proc GLM (2).

8

Table 2: Comparison of Direct Expansions, CATl and nonCATl, June 1 1984, HogSurvey, Nebraska, Totals for Strata 81-87--------------------------------------------------------------------Variable CATl

(000)nonCATl

(000)% Diff : Signif.

11 : Level---------------------------------------------------------------------Total Hogs All 1680 1618 +3.8 % .80

Adj 1593 -1.5 % .98

Sows All 192 167 +15.0 % .35Adj 188 +12.6 % .37

Expect. Farrow. All 96.9 83.9 +15.5 % .41(June - Aug. ) Adj 94.5 +12.6 % .40

Boars All 12.0 13.2 - 9.1 % .64Adj 11 .8 -10.6 % .63

Market Hogs:

under 60 lbs - All 599 628 - 4.6 % .71Adj 553 -11.9 % .54

60-11 9 1bs - All 416 353 +17.8 % .47Adj 387 + 9.6 % .49

120-179 lbs All 240 261 - 8.0 % .53Adj 233 -10.7 % .60

180 lbs & over All 215 191 +12.6 % .65Adj 215 +12.6 % .53

Total - All 1471 1432 +2.7 % .88Adj 1388 -3.1 % .89

Prevo Farrowings All 85.3 89.6 -4.8 % .77Adj 82.9 -7 .5 % .68

Avg. Litter Rate All 7.47 7.62 -2.0 % .57Adj 7.48 -1.8 % .80

Deaths All 46.5 48.0 -3.1 % .79Adj 46.0 -4.2 % .85--------------------------------------------------------------------

Multivariate Tests 2/- All .18- Adj .52--------------------------------------------------------------------1/ % Diff = 100 • (CATl - nonCATl)/nonCATl

21 Wilks' test on individual variables shown above excluding totalhogs and total market hogs.

9

Multivariate anddifferences (atbetween CATI andsome variablespower in theincluded before

univariate tests showed no significantthe 0.10 level) for either expansions

nonCATI. Fairly large differences forwere not significant due to the lack of

tests. Larger sample sizes must bemore powerful tests can be done (3).

EDIT ERRORS

The study revealed a possible shifting in the size ofsome inventory items. The CATI expansion for number ofsows. and thus expected farrowings. was larger than thenonCATI. An overall shift to heavier weight groups formarket hogs also seemed to occur in CATI data. Statist-ical tests did not find these differences significant.but further testing is needed.

After interviewing and a statistician review. both CATIand nonCATI data went through the Generalized Edit (GE)edit program. A comparison of GE errors between CATl andnanCATl in strata 81-87 was inconclusive. As previouslymentioned. only about 19 percent of the cases in thetest strata for CATr and nanCATI had hogs; therefore.total GE errors was quite small. In addition. theamount of manual editing prior to the GE may not havebeen typical of all hog surveys because editors knewthey were in a test situation. Future similar studiesshould capture the data prior to editing by statisti-cians.

The only critical errors (errors requiring an update)for either method dealt with the coding of the enumera-tor code and the quality code boxes. The enumerator codewas invalid in four CATl cases. This was corrected inlater versions of the instrument. The quality code wasinvalid in one nonCATl case.

There were 20 non-critical errors in CATl data and 11 innonCATI data. Only three CATI errors and only one non-CATl error resulted in updates.

Thus. this measurement showed no improvement in thequality of data collected by CATl. However. a similarcomparison made in the Cattle MF survey in Californiarevealed 76 percent fewer updates to the CATI data - 20CATl updates versus 84 nonCATl (3).

10

EVALUATIONS Thirteen enumerators with both CATI and nonCATI experi-ence completed evaluation forms after the September 1survey (see Appendix A). Results from this small groupwith limited CATI experience can not be generalized toall future applications, but they can indicate futureexperiences and can also point to areas in developmentor training needing further attention.

All enumerators were on the existing nonCATI staff.Seven of the 13 enumerators worked as nonCATI phoneenumerators for over 2 years. 5 from 6 months to 2years, and 1 less than 6 months. CATI experience rangedfrom 4 surveys in 1984 (1 to 4 nights each) to only 1survey. Ten enumerators had worked either 2 or 3 CATIsurveys.

Other related skills possessed by these enumeratorsincluded the following. Eleven of the 13 had farm back-grounds, 5 had previous experience working on a computerterminal, and 11 had fair to good typing ability (2 hadno typing skills).

Results from these evaluation forms follow.

11

1. Comparison by Interview Task and Overall

Enumerators were asked to choose which system workedbest for them for several interviewing tasks. Oneenumerator did not complete this section.

They were then asked which method they would use ifgiven a free choice.

The number of enumerator responses in each categoryis shown below.

Little Or NoITEM CATl nonCATI Difference

Word Selection 3 3 6

Question Branching 3 4 5

Changing Answers 1 9 2

Entering NumericAnswers 7 5 0

Entering Notes 5 4 3

Checking for ConsistencyBetween Answers 5 5 2

Probing for AnswersFrom Reluctant Resp. 6 5

Arranging Callbacks 6 2 4

•••••••••••••••••••••••••••••••••••••••••••••••Overall Choice 6 6

•••••••••••••••••••••••••••••••••••••••••••••••

Changing answers was seen as a drawback to CATI, prob-ably because special interviewer commands had to belearned to move around in the instrument and to changean answer.

Enumerators were divided as to how well CATI worked indealing with consistency checks done during the inter-view, probably reflecting an additional enumerator bur-den that CATI imposes. Namely, CATl enumerators had toresolve failed edits which involved complicated datarelationships. Such edits were not a factor for nonCATI

12

interviews where enumerators were expected to know onlythe simpler consistency checks. Resolving these editsimplied a subject-matter knowledge often beyond what isexpected of a nonCATI enumerator. Also, enumeratorsoften had to move around in the instrument to changeanswers when resolving edits.

Thus, while enumerators could do edits on-line. thisimplied learning new CATI skills and also understandingthe subject-matter well enough to resolve complicatededits. The extent to which enumerators can be expectedto have this subject-matter knowledge and what addi-tional training this implies are issues which must befurther addressed.

Probing for answers was also apparently easier with non-CATI possibly because of a general unease with CATI whenthe interview did not flow as smoothly or directly as itappeared on the screen.

Of the six who chose the CATI interviewing method, allworked the prior three CATI surveys indicating thatregularity of use was a preference factor. This conjec-ture was reinforced by enumerators' comments that morefrequent use of CATI - the hog surveys were about 3months apart would help them become more proficientand would eliminate a certain amount of relearning eachtime.

There were no clear differences as to the method pre-ferred between levels of nonCATI experience, typingskills, previous CRT experience, or farm background.

A study by Groves and Mathiowetz found a similar 50 per-cent split in preference for 31 interviewers on aNational Health Interview Survey (1).

13

2. Training

Enumerators were asked to rate the level of diffi-culty and the amount of training provided to learnthe CATl and nonCATl interview methods. The numberresponding by category is shown below.

a. How difficult was it for you to learn?

CATl nonCATIVery EasySomewhat EasySomewhat DifficultVery Difficult

274o

481o

b. How would you rate the amount of training youreceived?

CATl nonCATlToo MuchAbout RightNot Enough

o94

o1 1

2

All enumerators had previous nonCATl experience. sothat learning CATl was an extension of previousskills. New CATl interviewers were given two 4-hourtraining sessions with much of the time devoted toindividual practice at terminals using mock inter-views. More time in these sessions would apparentlyhave been valuable for at least four enumerators.all four of whom preferred nonCATl.

There were no clear differences in the level of dif-ficulty or the amount of training between differentlevels of typing skills. previous CRT experiencenonCATl experience. or farm background.

3. Fatigue Level

Enumerators were asked to rate how they felt afteran evening of calling using either CATl or nonCATI.

CATl nonCATl

Very TiredSomewhat TiredOnly a Little TiredNot Tired At all

1534

2443

CATl caused no apparent additional fatigue.

14

4. Respondents Reaction to CAT!

Enumerators were asked to subjectively judge howoften respondents knew they were using a computerduring a CAT! interview and what their reaction waswhen they did know.

How Often Farmers Knew

Always - 1Usually - 2Sometimes - 7Very Rarely - 3Never - 0

Farmers' Reaction

Liked the Idea 0Did Not Like Idea - 0Did Not Matter - 13

Enumerators perceived no particular aversion to. norany preference for, CAT! by respondents.

5. Other Results

a. Attention must be given to adjusting screen angleand height and possibly to the use of specialscreen filters. Nine of the enumerators woreglasses. seven of whom had bifocals or trifocals.Three of these seven indicated some problemswith reading the terminal screen because of theirglasses. All three preferred nonCATI.

b. Most enumerators preferred an over-the-ear typeheadset over the ear-plug type: eight enumeratorspreferred the over-the-ear type, two preferredthe ear-plug type, and three had no preference.

c. Enumerators expressedterminals with ambergreen displays: fiveferred green. and four

equal preference betweendisplays and those withpreferred amber. four pre-had no preference.

d. Typing skills and previous CRT experience weremost frequently mentioned as skills that were (orwould have been) valuable in learning CATI. How-ever, other results noted above indicate thatthese skills are not critical to either learningCATI or to eventually preferring it.

Typing skills are a valuable asset for a CATlenumerator, particularly in surveys where muchtext needs to be entered. However, SRS surveysare mostly factual with numeric responses. andthus. the usual qualifications for a good nonCATIenumerator should take priority.

15

detected between thenonCATI. However,variables point outcapability is added

CONCLUSIONS

RECOMMENDATIONS

* Interviewer productivity was lower for CATI but it isreasonable to expect that, with the software improve-ments that have been made and with more experience onCATI, productivity should be at least as high as itcurrently is for nonCATI.

* No significant differences werelevel of estimates for CATI anarather large differences for somethe need for further study as CATIto more States.

* Present nonCATI interviewers adapt well to CATl, al-though some would have benefited from more training.More frequent use of CATI would help interviewersbecome more proficient in its use.

* Survey managers should be aware of enumerator prefer-ences in types of headsets, terminals, and of specialproblems with vision. Enumerators with no experiencein typing or use of a keyboard will need to practice,but this alone should not prevent them from becominggood CATI enumerators for SRS surveys.

* Respondents had no discernible aversion to using CATI

* Conduct more research as more surveys and States areadded to the CATI program.

* Continue to monitor CATI productivity.

* Compare estimates for other surveys with larger sam-ples. Further test the potential differences in hogitems discussed in this study.

* Compare data quality using data prior to any manualreview.

* Trainers of CATI interviewers must strive to provideadequate subject-matter training for interviewers tounderstand and resolve complex edits. Possibleimprovements in instrument design should continue tobe investigated.

16

REFERENCES(1) Groves, Robert M. and Mathiowetz, Nancy A. "Computer

Assisted Telephone Interviewing: Effects on Interviewersand Respondents." Public Opinion Quarterly. Spring 1984

(2) Hall, Kathy Burstrom and Ford, Barry. The Effects of DataCollection Methods. U.S. Department of Agriculture.Statistical Reporting Service. October 1979.

(3) House, Carol C. Computer Assisted TelephoneInterviewing on the Cattle Multiple Frame Survey.U.S. Department of Agriculture, Statistical ReportingService. October 1984.

(4) Shanks. J. Merrill, McIntosh, Richard A•• and Morton, BetsyT. "COMPUTER-ASSISTED SURVEYS: Recent Activities of theComputer-Assisted Surveys Methods Program (CSM)." Presentedat the 1984 Annual Meeting of the American SociologicalAssociation and at the 1984 Annual Meetings of the AmericanPolitical Science Association.

17

APPENDIX A

INTERVIEWER EVALUATION OF CATI & NON-CATINebraska - Fall 1984

Instructions: Please help us with our evaluation of CATI and"regular" non-CATI telephone interviewing byanswering the following questions as honestly andas thoughtfully as possible. Your answers areconfidential and you need not sign your name.

Your responses will help future CATI enumerators I

For each item, check appropriate response(s) andthen add any comments that you wish.

Thank You I

1. How much CATI experience have you had?(check all that apply)

a. January Cattle practice surveyb. March Hog practice surveyc. June Hog Surveyd. September Hog Survey

2. How much regular (non-CATI) telephoning experiencedo you have ?

a. Less than 6 monthsb. 6 months - 1 Yearc. 1 - 2 Yearsd. More than 2 Years

3. Do you have a farm background?(Raised or worked on a farm or gained a fairly extensive

knowledge of farm activities from others who are farmers)

a. Yesb. No

_. How difficult would you say it was for you to learn to interviewusing CATI ?

a. Very Difficultb. Somewhat Difficultc. Somewhat Easyd. Very Easy

COMMENTS:

-18-

5. How difficult would you say it was for you to learn to interviewusing regular (non-CATI) procedures?

a. Very Difficultb. Somewhat Difficultc. Somewhat Easyd. Very Easy

COMMENTS:

~. How would you rate the amount of training that you received whenlearning to use CATI 1

a. Too Much Trainingb. About The Right Amountc. Not Enough Training

COMMENTS:

7. How would you rate the amount of training that you received whenlearning Wregular- (non-CATI) telephone interviewing 1

a. Too Much Trainingb. About The Right Amountc. Not Enough Training

COMMENTS:

8. Did you have any experience working with a computer terminalprior to learning CATI 1

a. Yesb. No

COMMENTS:

9. Please rate your typing skills prior to learning CATI.a. Never Learned To Typeb. Poorc. Faird. Good

COMMENTS:

-19-

10. Do you think experience with typing, computer terminals. or anyother skills JOU may have had were (or would have been) helprulin learning CATI ?

a. Nob. Yes Please Explain

11. Arter an evening or interviewing using CATI. describehow you reel.

a. Very Tiredb. Somewhat Tiredc. Only a Little Tiredd. Not Tired At All

COMMENTS:

12. Arter an evening or interviewing using Wregular- (non-CATI)procedures, describe how you teel.

a. Very Tiredb. Somewhat Tiredc. Only a Little Tiredd. Not Tired At All

COMMENTS:

13. When using CATI how orten do you think rarmers knew that youwere using a computer to help with the interview?

a. Al waysb. Usuallyc. Sometimesd. Very Rarelye. Never

COMMENTS:

-20-

14. When farmers did know that you were using CATI. how do you thinkthey usually felt about using a computer in this way?

a. Liked the Ideab. Did Not Like the Idea.c. Did Not Seem to Matter to Them

COMMENTS:

15. Please rate the following interview tasks as to whether CATI ornon-CATI worked best for you. (check one for each item)

a. Word selectionwhen asking questions ...........

Little orCATI non-CAT! no difference

b. Question branching(which question to ask next) ....

c. Changing answers to previousq aest10ns •••.•••••••••••••••••••

d. Entering numeric answers ••••••••

e. Entering notes ••••••••••••••••••

f. Checking for consistency betweenanswers (edit checks) •••••••••••

g. Probing for answers fromreluctant respondents •••••••••••

h. Arranging callbacks(day, time, person etc.)

COMMENTS:........

-21-

16. It you were given a tree choice to use either CATI or -regular-non-CATI procedures tor interviewing, which would you choose?

a. CA TIb. non-CATlc. No Preterence

COMMENTS:

17. Please respond to the tollowing CATl-related items.a. What type ot headset do you preter ?

Clip on with earplugOver the head with -ear-mutts-No Preterence

Why ?

b. What type ot terminal do you preter 1

Brown/beige with amber print (rubout key)Grey with green print (clear/delete key)No Preterence

Why ?

c. Do you wear glasses while interviewing?No [Go to Q. 18]Yes

c1. Do you use bitocals 1

NoYes

c2. Did your glasses cause any problems using CATI ?NoYes Explain

-22-

18. Please comment on any other advantages or disadvantages that you thinkCATI bas. Include any suggestions that you might have on how to.ake CATI better or easier tor you to uee.

-23--

re~~ng~BOIrdStatistical ReportinvService

U.S. DePlrtmentof AQric:uIQlrw

Washinvton. D.C.20250

APPENDIX BHOG AND PIG SURVEY

JUNE 1, 1984Form ApproYedO.M.B. Number OS3!HlOOSExplrltion Clta 1/31117

C.E. 11.ooaa

1-18-MF

Strltum 10 Trlct

0 0 o 1---- -----.---- --SurYlY Rnpon •• 2 Tel. Optional S. Unit H 1· Hu

813 110 3 Int. 111 120 U9 H~I7 TR 2· No8 IR H~II Inlc. 3· Don't

1 Know

Pleas. ma/(e cO"Ktions in name, address, and Zip Code, If neCflssery.

Mr. , I am . _

from , We are now conducting the June 1 Hog and Pig Survey and yourname was selected in a sample of farmers in this State. Response to this survey is voluntary and not required byJaw. However, your cooperation Is very important to Insure timely and accurate estimates. Your report is con-fidential and used only In combination with reports from other producers to arrive at State estimates.Than~ you for your cooperation.

1. Is your operation known by any name other than 7 (Read above name to respondent)

DYES ---------.- Enter name

1_-2. Are there now any hogs or pigs regardless of ownership,

on the land you now operate?

YES o NO --- ..•~. 2a. Have there been any HOGS or PIGS on the landyou now operate since March 1, 19847

DYES· Skip to Item 9, page 2.

o NO • Skip to Item 18, page 3.

(Please continue on page 2) -2!t-

I-8-MF

HOG AN 0 PIG INVENTORY

3. How many are: .......•••

Now I want to ask you about the hogs and pigs on the land you operate, regardless of ownership.Include hogs and pigs purchased and still on hand.First I would like to ask about HOGS AND PIGS FOR BREEDING.

a. Sows, gills, and young gilts bred and to 1301be bred? ....••...••.....•.•.....•..•..••••••. 1- _

Of the SOWS and GilTS (reported In Item 3a) howmany are expected to farrow:

(1) From now through June, July 1331and August 1984? ••••••••••• ",-, _

(2) During September, October 1332and November 19841 ••••.... ",-, _

c. Sows and boars no longer used for breeding? ....•.

Now let's talk about the HOGS and PIGS for MARKET and HOME USE In each of thefollowing four weight groups. (Exclude breeding hogs already reported in Item 3.)

a. Under 60 pounds?(Include pigs not yet weaned) ..••.•..•.••••..••.

5. Add Items 3a through 4d: Then the total hogs and pig. now 1300 Ion the land you operate Is •••••••••••••••••••••.•.• 1 _Is that correct?

1311

1312

1313

1314

!3021303

b. Boars and young males for breeding?

b. 60 - 119 pounds? •••••••••••••••••••••••••••••

c. 120 - 179 pounds? ••••••••••••••••.•••.••••••.

d. 180 pounds and over? .•.•••••••••••..•..•..••••(Exclude hogs no longer used for breeding)

How many are •...•.••.•.<t.

!YES Conllnue

PREVIOUS THREE MONTHS FARROWINGS

o NO Correct answers In 3, 4, and 5.

9.~~~I ~n~n~:~1~:4~~~t~~~T;/~~~.~~~~. ~~~I~~. ~~r.c~: •••••••••••.••••••••••••.• 1...

326_

10. How many PIGS from these (Item 9) litters :

•• Now on hand •••••••••••••••••••••••••••••••••••••••••••••••••••••••••

b. Already sold .••••••••••••••••••••••••••••••••••••••••••••••••••••••••

-2~

I-eMF

PURCHASES

11. How many HOGS and PIGS PURCHASED _Inee December 1, 1983 1317are now on hand? (Include feeder pigs purchased) ..••.•........•..••••••.••••.••• ~

) It '''m 11 Ia zero, Iklp to Item 13.]

12. How many FEEDER PIGS were purchased during May 19847 .....••.•.•••.••••• " •••

•. What was the average PRICE PER HEAD ..••.•..•••............ Dollars and Cent_

b. What was the average WEIGHT PER HEAD .•.••••..•.•..•....•.•••••••.• Pound_

341

342.--

DEATHS AFTER WEANING

13. ~~~ :nadn~~~~~;D. :.I~.~ ~~~. ~.~~~~. ~.~~.~ ~~~~.~~r~~~.~a.r~~: ••.•••••.••.••••.. F _OPERATION DESCRIPTION OF LAND

Addltlonaiinformation is needed on your operation to assist in detecting possible duplication in reporting.

18. Which of the following best describes your farming or ranching operation?(Check only one unless you, the Individual or operation listed on the face page, havemore than one operating arrangement.)

o = 1 Individually operated land.

o = 2·7 Partnership: Partners Jointly operate land and share In the decision making.

o = B Hired manager of land owned by someone else.

o = 9 Do not now operate land for agricultural purposes.(Out-of·buslness. landlord, retired, etc.),

Speclfy _

Completl! Items 19 and 20 only If Partnership Is checked.Please make any correct!ons If operation description Information Is entered.

19. Does this partnership or Joint arrangement have a name other than thatlisted on the face page?

DYES --------(Enter name, then continue on page 4)

o NO • Continue on page 4.

-26 ....

I

1-_I

I-8M F

19a. Who are the persons In this partnership or Joint land arrangement with you?(Exclude Londlord- Tenant, cash rent or share crop arrangements.)

a. Name Phone t25

(First) (M) (Last)

b. Address(Rt. or St.) (City) (State) (Zip)

a. Name Phone 1926

(First) (M) (Last)

b. Address(Rt. or St.) (City) (State) (Zip)

a. Name Phone 1927

(First) (M) (Last)

b. Address(Rt. or St.) (City) (State) (Zip)

20. How many hogs end pigs are now on this partnership or Joint land? •.••.••.... Numberl-------

a. How many of these hogs and pigs were Included in Item 5, page 2? •........ Number _. _

22. SSO OPTION: The results of this survey will be released June 21, 1984Would you like to receive a copy?

DYES = 1

o NO

COMMENTSPlease commt!TIt on any unusual death loss. average gains, or farrowing problems aff«ting your answrrs.

Any comments on problems or factors affecting hog production in your area will be appr«iated.

I

Number

That completes the survey. Another hog survey will be conducted In about three months and we may need to con-tact you again. Thank you for your help.

Reported by Telephone Number _

(Are. Code)

Enumerltor Dlte

~7- . I: