Slide 1

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY LIFECYCLE Vicki Crompton and Mike Sivyer Atlantic DLI Training Slide 2 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM The financial perspective. Core statistical programs dictated by statutes, regulatory instruments and contractual obligations account for 93% of STC budget $510 million in base budget supplemented by $130 million in supplementary revenues ($45 million in cost-recovery surveys) Social programs account for 65% of base budget, 75% of supplementary revenues Slide 3 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM Weekly1 Monthly70 Quarterly52 Tri-annual2 Semi-annual8 Annual194 Biennial22 Triennial2 Quinquennial6 Occasional28 TOTAL385 ACTIVE SURVEYS Social Surveys: 93 Economic Surveys:292 The statistical perspective Slide 4 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM The rationale. Produce information to support statutory and regulatory purposes Produce information on the economic and socio- demographic structure of Canada Produce information needed to support informed policy formulation, decision-making, research, and discussion Produce information required to meet emerging issues and new challenges Slide 5 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM Maintaining relevance. National Statistics Council Network of subject-specific professional Advisory Committees Federal Provincial Consultative Council Stakeholder consultations Slide 6 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM The deciding factors. Relevance Public benefit Cost Response burden Public sensitivies Slide 7 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 MAJOR COMPONENTS OF A SURVEY Slide 8 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 What the heck is a survey? Slide 9 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 WHAT IS A SURVEY? The collection of information From all or some units of a population Employing well-defined concepts, methods and procedures The compilation of such information into useful summary form Slide 10 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CENSUSSAMPLE TYPES OF SURVEYS Includes all units of the population Population = total set of units to which survey results apply Includes some units of the population Sample = a representative part of the entire population used to reflect the entire population Slide 11 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 In Statistics Canada we also categorize by: Business vs Household vs Institution Voluntary vs Mandatory Interviewer Administered vs Self-completed Administrative Records vs Direct Collection Cross-classified vs Longitudinal Ad Hoc vs On-going/Core NOTE: These are NOT mutually exclusive! TYPES OF SURVEYS Slide 12 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 EXAMPLES OF SURVEYS WITHIN STC Regular (on-going/core) Surveys -Labour Force Survey - Retail Commodity Survey - Census of Population - Unified Enterprise Survey - Survey of Household Spending - Annual Survey of Manufactures - General Social Survey - Survey of Steel Inventory - Etc. Slide 13 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 EXAMPLES OF SURVEYS WITHIN STC Special (ad hoc/irregular) Surveys - Youth Smoking Survey - Survey of Graduates Who Moved to the USA - Child Care Survey - Household Internet Use Survey - Survey of Approaches to Educational Planning - Adult Education and Training Survey - Survey of Importance of Wildlife to Canadians - International Adult Literacy and Skills Survey Slide 14 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 EXAMPLES OF SURVEYS WITHIN STC Longitudinal Surveys - Survey of Labour and Income Dynamics - National Population Health Survey - National Longitudinal Survey of Children and Youth - Workplace and Employee Survey Slide 15 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 User Consultation and Content Determination Questionnaire Development Data Collection Data Processing Data Quality Measurement Dissemination Slide 16 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY OBJECTIVES Reasons for collecting the data Elaboration of information requirements How will the information/data be used? Identification of Population/Target Group What is the population to be studied? Are there any sub populations to be identified? Slide 17 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY OBJECTIVES Identify Units of Analysis Individual Household Family - (Census or Economic) Business - Enterprise - Company - Establishment Slide 18 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY OBJECTIVES Analysis Plan - Purpose/uses of the data - Statement of outputs Level of Geography Required for Analysis - National vs provincial - Urban/rural - Cities/economic regions Level of accuracy required Slide 19 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY DESIGN ISSUES Feasibility of collecting data required Type of survey required (census/sample) Resources required Training Costs Slide 20 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY DESIGN ISSUES Method of Collection Paper and Pencil Interview (PAPI) Computer Assisted Interviewing (CAI) By interviewer By respondent (incl. electronic transmission) Proxy vs Non-proxy Personal vs Telephone Interview vs Self-completed Special Testing Applications Slide 21 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SAMPLE PLAN Target/survey population Sampling units Population must be divisible into distinctive non- overlapping units called sampling units Each member of the population should belong to just one sampling unit Size of sample required Sample selection methods Sample Frame Slide 22 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SAMPLE FRAME Means to access survey population A list containing the survey population For Example: A printed telephone book A computer database of registered motor vehicles Business Register Address Register Should contain all the relevant sampling units and all necessary and relevant information Slide 23 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SAMPLE FRAME Sample of other surveys Labour Force Survey sample (supplements) Census of Population (post-censal) Canadian Community Health Survey Geographic Frames Lists of EAs or CMAs from census files Slide 24 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT First perform a literature search Other sources which asked similar questions Other surveys in STC Surveys from other countries Define/develop concepts and definitions Similar to other STC surveys Specific to this survey Easy to understand, be accepted and used by interviewers and respondents. Slide 25 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 COMPONENTS OF A SURVEY Slide 26 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT Wording of questions Easily understood by interviewer and respondent Easy to read - sounds natural friendly Appropriate level of language Flows and skip patterns Easy to follow Logical to interviewer and respondent Capture all relevant information from respondents Slide 27 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT Questionnaire layout (print version) Easy to follow? Need additional instructions for Interviewer? CAI Application Ability to program Appropriate questions introduced at correct time User-friendly (for when completed by respondent) Slide 28 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT Determine which type of question is best for each question asked: Closed question - Identify answer categories to include Open-ended question (write-in) - Coding of write-in responses Slide 29 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 Slide 30 Slide 31 TESTING ACTIVITIES An important part of all major components Test wording of questions (in both official languages) Are concepts and definitions understood? Level of language Sequence of questions (flows/skips) Length of interview Slide 32 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 TESTING ACTIVITIES Focus groups Informal with colleagues Field procedures testing Pilot testing CAI Applications and other related programs Slide 33 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 TESTING ACTIVITIES EXTREMELY IMPORTANT SOMETIMES SACRIFICED DUE TO TIME AND COST CONSTRAINTS Slide 34 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DATA COLLECTION/FIELD ACTIVITIES Interviewer training Sample Selection and send to Interviewers Send questionnaires /CAI application to Interviewers Necessary equipment in place (laptops, phones, reference materials, etc.) Field collection/ Interviewing Data capture & transmission Respondent relations Slide 35 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES STC Policy requires that we inform respondent of: Purpose of survey Confidentiality assurances Volunteer/mandatory aspects of survey Any projected linking of data Any projected sharing of data Normally start by sending a letter to respondents informing them of survey Slide 36 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES Also have an introduction at the start of the interview process on the questionnaire Slide 37 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES For business surveys there is a special program called the Key Provider Manager (KPM). The top 300 large complex enterprises in Canada account for one third of all economic activity. This can result in these enterprises being in many surveys leading to respondent burden and possible refusal to participate. Slide 38 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES Duties of KPM are: To co-ordinate and facilitate flow of information between these enterprises and STC; To negotiate reporting arrangements with enterprise; To explain concepts, definitions reporting requirements; To do follow-up; Etc. Slide 39 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA There are a series of steps that include: Editing Imputation Coding Creation of derived variables Weighting calculations Preparation of metadata Final file creation Slide 40 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Why edit the data? Data must be logical and consistent. Errors that may have been introduced into the data must be fixed. Errors can include: Interviewer errors Capture errors Respondent errors Slide 41 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA What is included in editing? Inter-question relationships Paths and flows Coding of write-in responses Imputation for non-response or response error Slide 42 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Imputation activities can include: Replacing response error with true value Imputation of non-response Imputation of missing values based on other responses from same record Imputation of missing values with responses from a similar record Replace values for an entire section or record with responses from similar record Remove record from file Slide 43 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Imputation of Data from Other Records on File Slide 44 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Imputation of Data from Other Records on File Slide 45 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 Slide 46 PROCESSING THE DATA During the editing process there is also the creation of Derived Variables. This can be the combination of responses of one question into a more useable format: e.g. Individual income into groups or ranges. Or the combination of two or more variables into a new variable: e. g. Usually smoke cigarettes with Usual number of cigarettes smoked to create Type of smoker: heavy, moderate, light. Slide 47 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA It is during the editing stage that decisions and actions may be taken to suppress data This is done because of: Confidentiality risk Sensitivity of data Quality/reliability issues Slide 48 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA One of the final activities of the processing stage is the calculation of the weights (sometime called the sample weight) The weight variable indicates the number of units in the population that are represented by each unit in the sample The sample weight corresponds to the inverse of the probability of selection of the unit in the sample Slide 49 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA For example: In a survey of Ontario farms lets assume that: Farm population = 153,000 (farms) Survey sample required = 9,000 Probability of selection = 9,000 / 153,000 = 1/17 Sample weight = 17 However. Slide 50 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Most samples are based upon a complex sample design, with stratification, multiple stages of selection, and unequal probabilities of selection of respondents Weighting MUST take this into account Weighting also has to take into account any non- response Slide 51 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA After all the editing and weighting activities are completed the end result is what is called theMaster File. This file..: contains all (unsuppressed) data can use this file to produce tabulations for analysis For most business surveys this is the only file produced. This is because population is very small and the risk of disclosure is very high Slide 52 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Many household (social) surveys produce a Public Use Microdata File. This file: Contains suppressed data for public utilization May contain fewer variables than master file Must be approved by the Microdata Release Committee Slide 53 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA The Microdata Release Committee reviews all of the procedures and activities done by the Survey Team related to Confidentiality Issues. This includes: Removal of all identifying information (name, address, etc.); Removal of any variables which allow linking of records across files; Suppression; Creation of derived variables; Etc. Slide 54 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA The survey team also produces all of the metadata required for the dissemination of the data files This includes: Data quality information Methodological information Questionnaires/questions asked in CAI applications User guides for microdata files Record layouts/codebooks Slide 55 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION Must register all products and services Announcement in the Daily Release - availability of data Products released Highlights of the release (optional) Integrated Metadata Base (IMDB) is updated STC Online Catalogue is updated CANSIM updated Slide 56 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION PRODUCTS: Statcan.ca (free) Publications ($ and free) Data Tables/ electronic files ($ and free) Canadian Statistics tables (free) Public Use Microdata Files ($$) CANSIM, Trade data, etc ($$) Custom Tabulations ($$) Census, Input/Output, etc Slide 57 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION CHANNELS: Advisory Services All publications added to STC library collection Catalogued in BiblioCat Depository Services Program Full depository libraries 52, Selectives - 800 libraries Legal Deposit of publications to LAC Slide 58 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION CHANNELS: Data Liberation Initiative 67 subscribing Canadian universities and colleges All standard electronic products Research Data Centres 14 across the country Access to Master files strict controls Secondary distributors Slide 59 STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION OTHERS: Secondary Analysis - Other government department(s) - Experts in field - Academics - Etc.