Validity of a Questionnaire for AssessingOccupational Activities

Lesley Campbell, BSc, Brian Pannett, MSc, Peter Egger, PhD, Cyrus Cooper, MA, DM, MRCP,David Coggon, PhD, DM, FRCP*

The objective of this article is to assess the validity of data on occupational activities obtainedby a questionnaire. 152 employees from 16 jobs were interviewed independently aboutactivities in their work, using a structured questionnaire. For 11 of the jobs, two furtherworkers were observed during a typical shift, and their activities recorded. The agreement ofreporting between subjects doing the same job and with observation was assessed. There wasgood agreement between subjects and with observation for most of the activities examinedwhen the activities were classified dichotomously. However, frequency of heavy lifting wasreported inconsistently. Questionnaires are able to distinguish major differences in physicalactivity that occur between jobs in the general population, but do not allow more detailedquantification of activities.Am. J. Ind. Med. 31:422–426, 1997.r 1997 Wiley-Liss, Inc.

KEYWORDS: physical activity; questionnaire; lifting; reliability; validity

INTRODUCTION

Musculoskeletal disorders are a common cause ofdisability and time lost from work [Kelsey and Hochberg,1988; Hodgson et al., 1993], and in many cases are thoughtto be precipitated or aggravated by physical activities carriedout in the workplace [Hodgson et al., 1993]. Investigatingthe contribution of occupational activities to musculoskel-etal disease epidemiologically requires reliable assessmentof exposure to such activities. Various methods are avail-able, including the use of recording devices to monitorposture [Baty et al., 1986] and intra-abdominal pressure[Davis and Stubbs, 1977]; filming people at work [van derBeek et al., 1992]; direct observation [Karhu et al., 1981];use of questionnaires [Hagberg, 1992]; and inference fromjob title [Hagberg, 1992]. The last two techniques have theadvantage that they can be applied retrospectively, as isnecessary in case-control studies. On the other hand, the

information which they generate will generally be lessaccurate than that obtained by direct measurement orobservation.

We here report a study in which we have examined therepeatability with which occupational activities are reportedby different people doing the same job, and comparedreporting of activities with findings from direct observation.

MATERIALS AND METHODS

With help from management at nine employers in theSouthampton area, we identified a total of 16 jobs in whichat least seven employees worked to a similar long-termpattern. The jobs varied in their physical demands andincluded some that were carried out mainly by women.

From each job we randomly selected seven to 17workers (Table I) and interviewed them independently aboutthe physical activities in their work, using a structuredquestionnaire. The ages of the 158 men and 16 womenchosen ranged from 21 to 73 years, with a mean of 38 years.All had worked in their job for at least one month (averagenine years). The interviews were conducted by twomembersof the study team (LC and BP), and care was taken to ensurethat subjects who had completed interviews did not revealthe content to colleagues who were yet to be seen. Thequestionnaire asked whether or not an average working day

MRC Environmental Epidemiology Unit, University of Southampton, Southamp-ton General Hospital, Southampton, UK.*Correspondence to: Dr. Coggon, MRC Environmental Epidemiology Unit,

University of Southampton, Southampton General Hospital, Southampton SO166YD UK.

Accepted for publication 3 July 1996.

AMERICAN JOURNAL OF INDUSTRIAL MEDICINE 31:422–426 (1997)

r 1997 Wiley-Liss, Inc.

entailed any of the seven activities listed in Table II, and alsothe frequency with which the subject lifted or carried loadsin different weight ranges. A copy of the relevant questionsis reproduced at the end of this article. Subjects whoreported lifting weights in excess of 10 kg were asked toidentify the relevant loads and give an estimate of theirweights. Where possible, the estimated weight was subse-quently checked by direct measurement or from employers’records.

For 11 of the 16 jobs, we were also able to observe twofurther workers (not interviewees) during a typical shift

(identified as typical by discussion with management). Werecorded the times for which different activities were carriedout and the loads lifted, and used a pedometer to assess thedistance walked.

In our analysis we assessed the level of agreement inreporting of activities between different people doing thesame job, and compared reported activities with thoseactually observed.

RESULTS

Activities Other Than Lifting

The frequency with which different activities werereported by interviewees varied widely. Thus, in 13 of the16 jobs a majority of people reported standing or walkingfor two or more hours in an average working day, and in 11jobs a majority of people said that an average working dayentailed walking more than two miles. At the otherextreme, there were no jobs in which a majority of peoplereported kneeling for longer than one hour per day andthere was only one where the majority described squattingfor more than an hour.

For the most part there was good agreement withinjobs in the reporting of activities (Table II), although majordiscrepancies (fewer than 75% of subjects giving the modalresponse) occurred for kneeling (two jobs), squatting (fourjobs), driving (four jobs), walking (three jobs) and stairclimbing (three jobs).

In the 11 jobs in which workers were observed over thecourse of a shift, reported activities agreed closely withthose observed (Table III). The largest inconsistencies inreporting occurred for driving (one job) and walking (threejobs), when only one of the two observed workers was seento carry out the task.

Lifting

Table IV shows the maximum weights which peoplereported lifting in different jobs and the heaviest lifting thatwas observed in jobs during a single shift. The questionnaireappeared to discriminate clearly between jobs that entailedlifting weights in excess of 10 kg and those that did not.When lifting occurred, however, there were discrepancies inthe maximum loads reported within jobs, and in three jobsanswers were spread across four weight bands. Thesediscrepancies occurred mainly because subjects from thesame job identified different lifting tasks. Their estimates ofthe weights of reported loads were for the most partaccurate.

Table V shows reported frequencies of lifting weights inexcess of 25 kg in different jobs. Among people whoindicated that they handled loads of this magnitude, therewas often substantial disagreement on how often such liftingwas required.

TABLE I. Numbers of Subjects Interviewed and Observed in Each Job

Job

Number of subjects

Interviewed Observed

1. Cartographers 17 2

2. Bus drivers 11 2

3. Bus cleaners 7 2

4. Storekeepers (stationery and light equipment) 7 2

5. Carpet loom operators 10 2

6. Milk deliverymen 10 2

7. Refuse collectors 10 2

8. Delivery drivers (frozen food) 9 2

9. Dispatch workers (bakery) 10 2

10. Delivery drivers (bakery) 10 2

11. Storekeepers (gas meters, heavy tools, and equip-

ment) 9 2

12. Tool assemblers (making up orders of wood-

working and metalworking tools) 10 0

13. Packers (packing woodworking and metal working

tools for delivery) 10 0

14. Clerical workers (tele sales) 8 0

15. Storemen (frozen food) 7 0

16. Production line operators (bakery) 7 0

TABLE II. Levels of Agreement in Report of Activities Between PeopleCarrying Out Similar Jobs

Activity

Proportion (%) of subjects in

job giving modal response

Mean Range

Sit .2 hours 95 78–100

Stand or walk .2 hours 97 89–100

Kneel .1 hour 89 70–100

Squat .1 hour 81 56–100

Drive .4 hours 89 56–100

Walk .2 miles 89 50–100

Climb .30 flights of stairs 89 60–100

423Validity of a Questionnaire

DISCUSSION

This study has shown mostly good agreement (. 90%)between subjects in the reporting of a range of physicalactivities at work when classified dichotomously. Reportedactivities also agreed well with findings on direct observa-tion. In particular, people appeared able to distinguishreliably whether an average working day entailed liftingweights in excess of 10 kg, although there was less

agreement in the reporting of lifting above higher weightthresholds. Frequency of heavy lifting was reported inconsis-tently.

The validation of reported occupational activities is amethodological challenge. One approach is to questionindividuals and then observe them over a limited periodwhile at work. However, if the job varies from day to day theobservation period may be unrepresentative, leading tospurious discrepancies between reported and observed activi-ties. An alternative is to observe people over one or more

TABLE III. Number of Subjects Reporting Activities According to Whether the Activity Was Observed in Others Doing the Same Jobs

Activity

Activity not observed in job Activity observed in job Observations of activity inconsistent*

No. of

jobs

No. of subjects

interviewed

No. of subjects

reporting activity

No. of

jobs

No. of subjects

interviewed

No. of subjects

reporting activity

No. of

jobs

No. of subjects

interviewed

No. of subjects

reporting activity

Sit .2 hours 6 53 2 5 57 48 0 0 0

Stand or walk .2 hours 2 28 2 9 82 80 0 0 0

Kneel .1 hour 11 110 11 0 0 0 0 0 0

Squat .1 hour 11 110 22 0 0 0 0 0 0

Drive .4 hours 9 90 14 1 11 9 1 9 5

Walk .2 miles 1 17 0 7 63 58 3 30 13

Climb .30 flights of

stairs 10 103 16 1 7 6 0 0 0

*The activity was seen for only one of the two workers observed throughout a shift.

TABLE IV. Heaviest Lifting Observed and Reported in Different Jobs

Job

Heaviest lifting

observed (kg)

Number of people reporting

different maximum weights lifted

F10 kg 10–24 kg 25–49 kg 50–99 kg H100 kg

1 ,10 17 — — — —

2 ,10 11 — — — —

3 ,10 6 1 — — —

4 10–24 — 2 3 2 —

5 10–242 — 1 6 2 1

6 25–492 — — 5 4 1

7 25–49 — 1 1 4 4

8 25–49 — 1 8 — —

9 25–49 — 1 7 2 —

10 25–492 — 5 4 1 —

11 50–992 — — 4 4 1

12 1 — 1 3 4 2

13 1 — — 5 5 —

14 1 8 — — — —

15 1 — 1 6 — —

163 1 — — 5 1 —

1No subjects were observed in these jobs.2Only one of the two subjects observed lifted a weight in this category.3One subject did not answer this question.

TABLE V. Reported Frequency of Lifting Weights $25 kgin Different Jobs

Job

Number of people reporting different frequencies of lifting

Never F1/week 1–10/week G10/week

1 17 — — —

2 11 — — —

3 7 — — —

4 2 — 1 4

5 1 3 4 2

6 — 1 6 3

7 1 — 5 4

8 1 — — 8

9 1 2 2 5

10 5 — 4 1

11 — 2 3 4

12 1 2 6 1

13 — 1 5 4

14 8 — — —

15 1 — 2 4

16* — — — 6

*One subject did not answer this question.

424 Campbell et al.

shifts and then ask them retrospectively about their activitiesduring the observation period. With this method, however,there is a possibility that reporting is more accurate thannormal because people’s awareness has been raised by thepresence of observers.

The design of our investigation was different, but againhad several limitations. For example, the fact that twopeople from the same job independently reported similaractivities at work does not necessarily imply that theirinformation is accurate. For this reason, where possible wealso compared reports with findings from direct observationof colleagues in the same occupation. While direct observa-tion may not be completely accurate, we would expect anyerrors to be small. In a separate exercise, our two observerssimultaneously monitored each of six workers over afour-hour period, and were found to agree closely in theirassessment of activities. More important is the possibilitythat despite our efforts to study typical working shifts, someof the shifts observed were not in fact representative oflong-term patterns of work. This may explain the discrepan-cies between observations on driving and walking in severalof the jobs, and it is notable that it was for these activitiesand jobs that levels of disagreement in reporting betweensubjects were highest.

Another potential weakness was the limited range ofjobs studied. There were no occupations in which a majorityof people reported kneeling for more than an hour in anaverage working day; and squatting for more than an hourand climbing more than 30 flights of stairs per day were onlyconsistently reported in one and two jobs, respectively. Forthese activities, therefore, conclusions must be circumspect.However, for the other activities there was good heterogene-ity between jobs, and the findings can be interpreted withgreater confidence.

Several previous studies have also addressed the reliabil-ity of self-reported occupational activities [Baty et al., 1986;Hagberg, 1992; Kilbom et al., 1984; Rossignol and Baetz,1987; Woodcock Webb, 1988; Ahlborg, 1990; Kuorinka andKilbom, 1990; Burdorf and Laan, 1991; Wiktorin et al.,1993; Burdorf, 1995]. In the main these have focused onposture, although some have also looked at manual han-dling. For the most part the accuracy of reporting has beenacceptable when activities were classified dichotomously[Rossignol and Baetz, 1987; Ahlborg, 1990; Wiktorin et al.,1993], but poor when more detail was sought about theirfrequency and duration [Baty et al., 1986; Kilbom et al.,1984; Rossignol and Baetz, 1987; Woodcock Webb, 1988;Burdorf and Laan, 1991; Wiktorin et al., 1993]. Our findingsare consistent with this pattern.

It follows that questionnaires are unlikely to be of muchvalue for fine quantification of physical activities, particu-

larly when there is a need to discriminate between peopledoing similar work. However, they do distinguish the majordifferences that occur across the range of jobs in the generalpopulation. Whether they do this reliably in retrospect (e.g.,for lifetime occupational histories in case-control studies)has yet to be established.

REFERENCES

Ahlborg GA (1990): Validity of exposure data obtained by questionnaire.Scand JWork Environ Health 16:284–288.

Baty D, Buckle PW, Stubbs DA (1986): Posture recording by directobservation, questionnaire assessment and instrumentation: A comparisonbased on a recent field study. In Corlett EN, Wilson J, Manenica I (eds):‘‘The Ergonomics of Working Postures.’’ London: Taylor and Francis, pp283–292.

Burdorf A (1995): Reducing random measurement error in assessingpostural load on the back in epidemiologic surveys. Scand J Work EnvironHealth 21:15–23.

Burdorf A, Laan J (1991): Comparison of methods for the assessment ofpostural load on the back. Scand JWork Environ Health 17:425–429.

Davis PR, Stubbs DA (1977): Safe levels of manual forces for young males.Appl Ergon 8:141–150; 8:219–228; 9:33–37.

Hagberg M (1992): Exposure variables in ergonomic epidemiology. Am JInd Med 21:91–100.

Hodgson JT, Jones JR, Elliott RC, Osman J (1993): ‘‘Self-reportedwork-related illness. (Research Paper 33).’’ Sudbury: HSE Books.

Karhu O, Harkonen R, Sorvali P, Va¨psalainen P (1981): Observing workingpostures in industry: Examples of OWAS application. Appl Ergon 8:13–17.

Kelsey JL, Hochberg MC (1988): Epidemiology of chronic musculoskeletaldisorders. Annu Rev Public Health 9:379–401.

Kilbom Å, Liew M, Lagerlof E, Broberg E (1984): Ergonomisk studie avmuskuloskeletala sjukdomar anma¨lda som arbetsskador. Arbete och Ha¨lsa45:7–41.

Kuorinka I, Kilbom Å (1990): Sja¨lvrapportering av arbetssta¨llningar ocharbetsmetoder samt deras relation till muskuloskeletala besva¨r [Self-reported working postures and working methods and their relation tomusculoskeletal complaints]. Stockholm: National Institute for Occupa-tional Health [Underso¨kningsrapport 10].

Rossignol M, Baetz J (1987): Task-related risk factors for spinal injury:Validation of a self-administered questionnaire on hospital employees.Ergonomics 30:1531–1540.

van der Beek A, van Gaalen LC, Frings-Dresen HW (1992): Workingpostures and activities of lorry drivers: A reliability study of on-site observation and recording on a pocket computer. Appl Ergon 23:331–336.

Wiktorin C, Karlqvist L, Winkel J, Stockholm MUSIC 1 study group(1993): Validity of self-reported exposures to work postures and manualmaterials handling. Scand JWork Environ Health 19:208–214.

WoodcockWebb K (1988): Self-reports in ergonomics:Agreement betweenworkers and ergonomist. In Aghazadeh F (ed.) ‘‘Trends in Ergonomics/Human Factors V.’’ North Holland: Elsevier, pp 387–394.

425Validity of a Questionnaire

APPENDIX

Questionnaire About Occupational Activities

1) In an average working day do you

i) Sit for more than two hours in total? No h Yes hii) Stand or walk for more than two hours in total? No h Yes hiii) Kneel for more than one hour in total? No h Yes hiv) Squat for more than one hour in total? No h Yes hv) Drive for more than four hours in total? No h Yes hvi) Walk more than two miles in total? No h Yes hvii) Climb more than 30 flights of stairs? No h Yes h

2) In the course of your work, how often on average do you lift or carry weights of 20 lbs (10 kg) or more?

Never h Less than once a week h 1 to 10 times per week h More than 10 times per week h3) In the course of your work, how often on average do you lift or carry weights of 56 lbs (25 kg) or more?

Never h Less than once a week h 1 to 10 times per week h More than 10 times per week h4) In the course of your work, how often on average do you lift or carry weights of 1 cwt (112 lbs, 50 kg) or more?

Never h Less than once a week h 1 to 10 times per week h More than 10 times per week h5) In the course of your work, how often on average do you lift or carry weights of 2 cwt (224 lbs, 100 kg) or more?

Never h Less than once a week h 1 to 10 times per week h More than 10 times per week h

426 Campbell et al.