Lorna-Lee De Bruyn Lima

Back to Basics:

Relooking at sampling frames in the aftermath of AMPS within South Africa

Plus 94 Research (Pty) Ltd

Introduction / Overview

Census

Sampling Frames

Media & Behavioural Currency Studies

Big Data

Survey Evolution

Behavioural Prompted Sampling Frames

ConclusionCO

NTE

NTS

2

SAMPLE FRAMES

PAMRO Conference 2018

SAMPLING FRAME SAMPLE

ELEMENT

RESPONSESTRATA

SUBJECTENUMERATIONUnit/Element: Subject on which information is obtained.

Population: A complete group of entities/elements.

Sampling Frame: A list of elements from which a sample may be drawn / group of people who you use as a basis to contact to conduct a survey.

Sampling: Process of obtaining information about an entire population by examining part of it. Data is collected from a representative part of the universe through sampling.

Sample: Part of a larger population / people you contact for the survey and who complete the survey.

POPULATION / UNIVERSE

PAMRO Conference 2018 3

Spatial denominated

Characteristic / digital denominated

Imp

ort

ance

leve

l

Age of data

high

lowold current

4

Spat

ial

de

no

min

ated

Ch

aracteristic / d

igital d

en

om

inated

Sampling frames are generally insufficient and imperfect.

Are we having to make do?

We don’t always have sufficient information.

When last was a census conducted?

6

WHEN LAST WAS A CENSUS CONDUCTED?

Census conducted in 1999 and then again

in 2009 (results published in 2010), with

the next census scheduled for August

2019.

Source: https://www.knbs.or.ke/overview-

of-census-2009/ and https://www.the-

star.co.ke/news/2018/02/05/government-

preparing-for-national-census-in-august-

2019_c1709414

Census conducted in 2001 and 2011 and the next census is scheduled for 2021, with a Community Survey in 2016.

Source: http://www.statssa.gov.za

Census conducted in 1991 (results released March 1992) and then again in 2006, showing a growth of more than 50%. The next census was due in 2016.

Source: http://www.nigerianstat.gov.ng/ , http://population.gov.ng/about-us/departments/census/ and http://nigeria.opendataforafrica.org/xspplpb/nigeria-census

PAMRO Conference 2018 7

SAMPLING FRAMESAs it is not always feasible to study the entire population, you can study a

sample of the population. Before you can do that, you need to establish a

sampling frame (a list of all units of the population of interest) and this will

depend on the type of study. It is the means by which to select the target

population which will be surveyed.

8

• It should be complete, up to date and accurate (which is not always achievable)

• Include all individuals in the target population

• Exclude all individuals not in the target population

• Includes accurate information that can be used to contact selected individuals (selectable)

• Other general factors that you would want to make sure you have:

• A unique identifier for each member (e.g. from 1 to 1000)

• Check to make sure there are no duplicates in the frame

• A logical organisation to the list (e.g. alphabetical order)

• Up-to-date information (e.g. periodically checked for address changes)

Spat

ial

de

no

min

ated

Ch

aracteristic / d

igital d

en

om

inated

BASIC RULES LEADING TO A SAMPLING FRAME

9

TYPES OF SAMPLING FRAMES

PAMRO Conference 2018

List Sampling Frames• Made up of a list of the target population• It can be made up of almost anything01

02

03Multiple Sampling Frames• Combining a list frame with a general area population frame• Caution: easier for duplicates, list must be current, might need to travel

Area Sampling Frames• Boundaries should be well defined• Population figures should be available for it

• Should have geographical cover and can be mapped

10

• Non-sampling bias (frame is obsolete, inaccurate or incomplete).

• Over-coverage (population that should have been excluded) – can be amended by removing the populationthat should have been excluded prior to the survey being conducted, post survey, but involves wastefulexpenditure; or

• Under-coverage (missing population) – which could mean bias, which could affect the quality of the surveyresults as it could have sections of the population that had no chance of being selected.

• A flawed sampling frame can be fixed by updating it, or relooking and redesigning the sampling frame. It isbest to ensure that an accurate sampling frame is used at the beginning.

• Study limitations.

CONSEQUENCES OF POOR SAMPLING FRAMES

PAMRO Conference 2018

Sample Population

relationship

Image: www.nationsonline.org Image: reviewonline.co.za

11

SURVEYING EVOLUTION

Spat

ial

den

om

inat

edC

haracteristic / d

igital d

eno

min

ated

https://www.questionpro.com/blog

TraditionalPen to Paper

CAPI (Computer-Assisted Personal Interview)

Social Media

Telephonic

Face-to-face

CATI (Computer-Assisted Telephonic Interview)

CAWI (Computer-Aided Web Interview)

Self-complete

Stratification

Panel

Behaviour prompted sampling frames

Sampling frame

Samplestratification Data

weighting Extrapolation/Population Conclusion

12

• Reality and importance of brick and mortar methods hence back to basics.

• Originally things were done pen and paper. Still the case for Africa?

• Current and historical practices (household register to panels).

• We are moving more online and digital.

• Virtual population?

• Social media taking over all aspects of our lives; and role of sampling frames.

• Sampling frames and social media.

• Municipal data / Big data.

• Weighting often goes back to census. Is weighting needed for online data collection? Skewed against poor?

TRADITIONAL SAMPLING FRAMES

PAMRO Conference 2018 13

Study / Survey Sampling Frame Who How What When Where Why Websites

AMPS

All Media and

Products Survey

Population estimates

as provided by IHS

South African

Audience Research

Foundation

Multi-stage, area stratified, probability

sample is taken to represent the

population of South Africa

Last publication in

2016

AMPS universe size is updated each year

in accordance with the changes in the

South African population

http://www.saarf.c

o.za/

MAPS

Replacement of

AMPS

Enumerated Area (EA)

Stats SA 2011 census

Market Research

Foundation

Products and

brands research http://mrfsa.org.za/

TGI

Target Group

Index

Enumerated Area (EA) Ask Afrika

Survey delivery

software developed

by Kantar Media’s

Target Group Index

(TGI)

Global products and

brands surveyTGI is not a measurement currency itself, but it’s the only current

products and brands survey that complements all the new

measurement currencies

http://www.askafri

ka.co.za/

ES

Establishment

Survey

Master sample frame

is delivered by

Integrated Household

Survey (IHS)

bi-annually (stratified

multi-stage sample

design)

Collaborative

partnership:

Publisher Research

Council (PRC),

Broadcast Research

Council (BRC)

Annual sample of

25 000 households

(12 500 per wave)

amongst people

15 years or older

(small areas at sub-

sample level – per

area type per

province)

ES will survey

media usage across

all media types by

all major

demographics (excl

udes brands)

ES data is released

twice a year

Covers seven media categories: Viewing

(TV), Listening (Radio), Reading

(Newspapers, Magazines), Digital, Cinema

and Out of home

www.prc.za.com ;

http://brcsa.org.za/

; www.amf.org.za

Brand Mapp Wealthy consumers whyfiveOnline methods

only

Annual survey of wealthy consumers, 270

measures/questions, 820 brands, 450

media brands, 130 categories

13% of top-end South Africans who pay

40% of the taxes

http://www.whyfiv

e.co.za/brandmapp

MEDIA AND BEHAVIOURAL CURRENCY STUDIES

14

Spat

ial

de

no

min

ated

Ch

aracteristic / d

igital d

en

om

inated

Spat

ial

de

no

min

ated

Ch

aracteristic / d

igital d

en

om

inated

Ronald D. Fricker, Jr Sampling Methods for Web and E-mail Surveys Fielding: Online Research Methods (Handbook) Pages 195–217 (http://faculty.nps.edu/rdfricke/docs/5123-Fielding-Ch11.pdf)

ONLINE TRADITIONALvs.

15

BEHAVIOUR PROMPTED SAMPLING FRAMES

PAMRO Conference 2018

A sampling frame can be made up of all those who have used the site to make a purchase. Additionally, incentives can be added for those to leave reviews.

Image: thehustle.co

16

BIG Data. . .breaking the rules?

• Cambridge Analytica and Facebook

• Sampling frames and digital online populations

• News channels reported up to 87 million Facebook users’ data compromised

www.miltons.law.za http://newscycle.com

www.theguardian.com/news

www.vanityfair.com/news

www.cnbc.com

www.theguardian.com/news

Image: https://112.international17

CONCLUSION

Spatial is still king but we are making do…

• Sampling frames are important requirements in conducting sample-based research to the extent that their absence,incompleteness or inaccuracy may mean that samples drawn therefrom cannot be extrapolated to the total populationthat they purport to represent.

• With the proliferation of different media currency and behavioural surveys in many African countries, and their use ofvarying sampling frames, it is important to take a closer look at the validity of sampling frames used and some of thereasoning behind them.

• However, as we are now living in a digital age, we are seeing a move away from spatially designated samples towardsdigitally oriented samples that are relevant in the information age, and we are also seeing a growth in samples that arebehaviourally prompted, as well as a growth in big data which often flexes its muscles by virtue of its size.

• Therefore our timeline moves from spatial to digital data age.

• How relevant are brick and mortar sampling frames today given their inherent incompleteness and outdatedness?

• Market research companies may/will eventually move from spatial denominations to digital denominations, which willimpact the way sampling frames are done.

CONCLUSION

PAMRO Conference 2018 19

Spat

ial

den

om

inat

ed

Ch

aracteristic / d

igital d

eno

min

ated

Thank YouPlus 94 Research (Pty) Ltd

Hyde West Building

2 Albury Road

Dunkeld West

Johannesburg, RSA

Tel: 011 327 2020

www.plus94.co.za

lorna@plus94.co.za