An Editing Procedure for Low Pay Data

Salah Merad, Mike Hidiroglou and Fiona Crawford

Office for National Statistics, UK

Survey Methods Division

Outline

• Background

• Problem

• Solution

Background

• Annual Survey of Hours and Earnings– Statistics produced include estimates of average pay

and distribution of hourly pay around the National Minimum Wage (NMW) in domains of interest

• Basic hourly pay obtained in two ways– Directly: Stated hourly rate (available in 45% of

records)– Indirectly: Derived basic hourly rate

Derived basic weekly pay/Average weekly hours

Problem• Selective editing is applied to the whole data set

– Targets estimates of averages and totals overall and in important domains

• Picks up large errors

• Need to target estimates of the number of employees below the NMW

– Small errors can be important– Additional editing

• Validation costs high: reduce editing costs whilst resulting estimates of the number of employees below the NMW are nearly unbiased

Solution: Outline of editing strategy• Stated hourly rate available: preliminary edit followed

by main low pay edits– Preliminary edit based on difference between Stated and

Derived• Threshold determined so that resulting bias is small• Threshold value depends on position of Stated and Derived in

relation to NMW

– Main low pay edits: compare current and previous Derived, and use other relevant information

• Stated hourly rate not available: main low pay edits

• Large number of failed records: manually edit a random sample, and impute remainder using data from edited records