1

1

Using traditional control systems Using traditional control systems to reveal changes when to reveal changes when introducing new data collection introducing new data collection instruments instruments

Sindre Børke (sbe@ssb.no)

Senior Adviser

Division for Data Collection Methods, Statistics Norway

2

Information model(survey definitions)

Web-solution

Webquestionnaire

Paper questionnaire

Surveydatabase

Data from webquestionnaires

Data from paperquestionnaires

Editing Process

Paper contra web Questionnaires

3

Outsourced editing gives better data

• Household census: average 6,2 and 3,4 errors

• Hotel statistics: average 1.7 and 0.7 errors

• Hotels, prize per room: 41% contra 36% with error

4

Without controls, more errors in web?

• Not in household census 2001

• Hotels, number of rooms let out: 12% contra 18% error

• (PPI: 1.27% contra 1.40% outside the defined range)

• Road goods transport: Different patterns to be explained

5

Behind – or on - the screen

• Following the respondent’s filling-in process, and the warnings presented during this process

6

Errors in incoming web and paper version of the hotel statistics questionnaire

7

Errors detected in the web and paper version of the hotel statistics questionnaire

25.2

18.1

16.7

3.0

One error message

More error messages

One error message

More error messages

8

Information model(survey definitions)

Supporting software

Basic software

Individualadaptions

Enterprisedatabase

Web-solution

Webquestionnaire

Paper questionnaire

Surveydatabase

Data from Enterprise system

Data from webquestionnaires

Data from paperquestionnaires

Editing Process

Transfer of electronic data

9

Transfer of electronic data

• Allready in wide use for accounts and salary data in Norway.

• Presents challenges in development and upkeeping– Supporting software from the software vendors– Individual adaptions at the respondent’s

• Looking into one case with traditional controls, finding better results than from paper questionnaires

• Requires different control/editing, due to different set of possible errors and different consequences when errors occurs

10

In short:

• Few surveys seems to have data to observe possible changes in quality

• Those data we find seem to tell that there are changes

• Possible changes in data that are not part of a control can not be revealed with the instrument we use in this paper

• This field must be further investigated, and give feed-back/-forward to the editing procedure and to questionnaire developing