Errors in Statistical Survey

Errors in Statistical Surveys (Sampling, Non-sampling, Derivation and Effects)

Presented Okpe ThankGod Damion

THANKGOD COMPUTER INSTITUTE

12-Oct-13ThankGod Computer Institute (TCI)1

Outline of Presentation

Introduction

Types of Errors in Statistical Survey

Stages of Non Sampling Errors

Non Response Errors

Response Errors

Control of Errors in Statistical Survey


INTRODUCTION The quality of survey results is a measure of

the following three items:How relevant is the result of the survey to the

objective.How accurate it is to the objective of the

survey.The timeliness of the result.


Types of Errors There are two major types of errors in

statistical surveys which can affect the accuracy of the survey data:

Sampling ErrorsNon-sampling Errors

We shall treat these errors separately


Sampling Errors The errors are due to the fact that data are

collected only for a sample of the target population with consequence that the estimate derived may differ from values that would have been obtained from complete census

The effect of this type of error can be reduced or minimized by increasing the sample size or improvement in the sample design efficiency.


Sampling Errors Contd.The magnitude of sampling errors depends on

the sample design and the estimation procedure used

These two can be reduced by increasing Sample size Improving sample design

By using: stratification or multi-stage designvarying selection probabilities improved estimation procedures


Non- Sampling Errors These are errors which are not due to

sampling In order words they are the residual errors

i.e. all other types of errors which are not resulting from sampling and affecting the quality of data collection

Some of these non-sampling errors could seriously affect the quality of data collected 12-Oct-13ThankGod Computer Institute (TCI)7

Non- Sampling Errors Contd.Non-sampling errors are of many

sources and have many methods of controlling them

The non-sampling errors are:Coverage errors Response errors Processing errors


Non- Sampling Errors Contd.Coverage errors

Are the errors that occur due to the difference between what is actually covered from what ought to have been covered which can either be over coverage or under-coverage as the case may be

Response errors Occur due to the difference in the answers actually

recorded for a question and what ought to be the correct answers or answer.

Processing errorsAre errors that set-in due to editing, coding,

punching/data keying, etc.


Stages of Non – Sampling Errors Non-sampling errors can be classified into

three stages Survey design and Planning Stage Data collection Stage Data processing and Analysis Stage

We shall now consider these stages one by one and identify the probable errors that can come up at any of the stage


Stages of Non – Sampling Errors 10Survey Design/Planning

Under this stage the types of error that can occur are either coverage, non-response and response errorsCoverage Errors

The objective of the sample survey is to make inferences about a desired target population from the observation of units confined to a sample

The selection of the units is done by a randomized procedure in which all units of the target population are put which we call


Stages of Non – Sampling Errors Sampling Frame Errors A situation where any of the unit in the frame is not

covered, results in Non-coverage which in turn give rise to coverage error

This may include a situation where some units of observation either directly or implicitly in the operational sampling frame are excluded

Also it may be a case of over-coverage in which case some units appear more than once in the frame in which case we say the sampling frame is defective

Coverage errors may occur as a result of selection of EAs, Wrong Geographic Codes, overlapping EA boundaries, Listing Exercise, Selection of the Ultimate Sample Units, Incorrect Application of Sampling Procedures, Incorrect application of rules of Association, etc 12-Oct-13ThankGod Computer Institute (TCI)12

Stages of Non – Sampling Errors

Estimation of Coverage Errors Let us look at how we can estimate the

magnitude of the effect of coverage errors on our survey results in other to determine the quality of our data

Though it is not easy to estimate the quantity of non-coverage error and it is also expensive but we shall mention a few methods to estimate it


Stages of Non – Sampling Errors Some of the methods include re-interview e.g.

conducting a post enumeration check on sub-sample of the survey records with some independent source, analytically, we may use data from other sources such as prior census, or vital records, external migration, etc which are secondary data to develop values for the total population and compare with corresponding survey figures

We may also compare with aggregates from administrative records.


Non-response Errors

Non-response Errors This is a case where one can not obtain data or

information from a selected unit of observation It is either total i.e unit non-response or partial

i.e item non-response To measure effect of non-response, it can be by

any of the following approaches: Measuring the response rate in case of unit non-

response or Item response rate in the case of item non-response.


Non-response Errors

The measures may give an indication of response bias and pointer to specific problems which may call for urgent solution either by reversal or otherwise.

In most cases the respondents will return the instrument back to the interviewer.


Non-response Errors Non response could be due to

Failure to gain access to sample units and may be as a result of non-accessibility to the EA.

Failure to contact the respondent (a case of proxy respondent)

Failure to gain the cooperation of the respondent which may be complete or partial situation

Response burden as result of the length of the questionnaire, e.g. enough time, memory lapse, lack of documentation or keeping of diary, interviewer level of education, experience, age etc.


Non-response Errors 17Improving Response RateIn other to ensure a good quality data,

efforts must be made as a matter of policy to maximize response in surveys

To achieve this, response rates had to be improved by either of the following or combination of all


Non-response Errors 18Contacting respondentsImproving Sampling FrameReduce time lag between listing and

conduction of the interview.Make many call-backs as situation may warrantGaining respondents co-operationIntensive training of the interviewersClose supervisionCareful choice of interviewers (and should be

well motivated)


How to Deal with Non-response Errors Ways of compensating for non-response

include the following: Intensive follow-up during the Data Collection process of sub

sample of the non-respondents Collection of limited information from neighbours of the

households that were away. Substitution method may be used i.e substituting similar units

or elements but must be made within the homogenous group if it is to be efficient.

This method requires that the substitution be made in the field so that the selected substitutes could be interviewed directly

This method is not encouraged and it requires skill staff


How to Deal with Non-response Errors 20

Estimation based methods e.g. use of adjustment factors

Imputation (Replacing missing information) from useable data from other sources

It is mostly used when treating item non-response. Some of the imputation methods includes deductive imputation, mean value impartation, registration method etc

Deductive imputation is made when the missing response in a questionnaire can be deduced with certainty based on other information on the same records e.g. a questionnaire on fertility with response for number of birth but fails to put the sex of the respondent on the same record, we could easily deduce that the sex is female


ThankGod Computer Institute (TCI)

Response Errors Sources of response errors can be traced to the followings

Interviewer inadequacy Inability of respondent to provide the desired information

Interviewer’s inadequacy can be as a result of Failure to put the questions clearly Influencing respondents to answer incorrectly Mis-recording correct responses

While respondent’s inability to provide the information may be as a result of not able to provide the desired information for some reasons which may include


Response Errors 22Limit impose by their knowledge, e.g. age,

size of holdings

Inability to recall or report facts correctly at the time of interview

Deliberate mis-information or withholding information due to dignity, tribal sentiment or prestige


Response Error Response error depends largely on:

Design of the Survey Operation Nature and Complexity of its ContentSystems of concepts and definitionsDesign and layout of questionnaireWording of the questions in the questionnaireAdequacy of the trainingMonitoring and supervision programmes put in

place during the data collection process


Control of errors in statistical survey Generally, errors in survey can be controlled

through either of the followings or combination of the followings:Adequate planning and preparatory work

including development of survey instruments

Adequate training of field personnel

Effective monitoring and supervision of field work during the data collection phase


Control of errors in statistical survey contd.Skim-checking of work at various stages

of the data collectionAlso, on the spot check i.e. spot check

of data collected should be done during the data collection phase

Post enumeration survey (PES) for the evaluation of survey results should be done and finally documentation of errors by sources, type and magnitude should be done.


End of PresentationTHANKS.


Errors in Statistical Survey

Documents

Transcript of Errors in Statistical Survey