ICAP METHODOLOGY WEBINAR

DECEMBER 7, 2011

Upcoming Methodology Webinars

• January 26: Using routinely-collected data to estimate patient retention in care and loss to follow-up (Matt Lamb)

• February date TBD: Overview of ICAP Geographic Information System Resources (Charon Gwynn, Yingfeng Wu, and Mark Becker)

• Future Methodology Webinar ideas? Email wr2205@columbia.edu

METHODS FOR ESTIMATING THE SIZE

OF HARD-TO-REACH POPULATIONS

ANNA DERYABINA, ICAP IN CENTRAL ASIA

Background (1)

• All countries affected by the HIV epidemic need information about their epidemic in order to combat its spread.

• Information about trends in the spread of HIV —whether and by how much the rates are increasing or decreasing and which populations are affected—can help countries monitor the epidemic and provide information to improve planning and evaluation of prevention activities

Background (2)

Low-level

• relatively little HIV is measured in any group

Concentrated

• HIV is over 5% percent in any sub-population at higher risk of infection

Generalized

• HIV prevalence is over 1% in general population

UNAIDS

Background (3)• In countries with a generalized epidemic,

national estimates of HIV prevalence are based on data generated by surveillance systems that focus on pregnant women who attend a selected number of sentinel antenatal clinics, and in an increasing number of countries on nationally representative serosurvey.

Background (4)

• In countries with a low level or concentrated epidemic national estimates of HIV prevalence are primarily based on surveillance data collected from most at-risk populations (MARPs) and estimates of the size of populations at high and low risk.

Most at-risk populations

• People who inject drugs (PWIDs)• Sex workers• Men who have sex with men (MSM)• Sexual partners of sex workers

Why do we need to estimate the size of hard-to-reach populations?• Surveillance• Advocacy• Response planning• Efficient resource allocation• Intervention planning• Monitoring and evaluation

Relevance to ICAP’s work

• ICAP is increasingly working with MARPs (Rwanda, Zambia, Central Asia, Mali, South Africa)

• ICAP is providing Technical Assistance to countries in Strategic Information

Why special methods?

• Most-at-risk populations (MARPs) are difficult to count, because they are stigmatized, and harder to reach.

• Usual epidemiological methods such as national household or schools surveys do not work

• Existing statistics based on case registration is not reliable

Process for estimating the size of hidden populations

Preparation Selection of method and

data collection

Data analysis and

dissemination

Key MethodsCollected from MARPs•Census and enumeration•Capture-Recapture•Multiplier methods•Nomination

Collected from the general population •Population surveys•Network scale-up•Consensus/Delphi

Methods that rely on data collected from MARPs

Census and enumeration - method• Census methods count every individual in a

population • Enumeration methods count only sub-set of

individuals selected from a defined sampling frame, and then need to multiply number according to size & structure of sample frame• Counting of people to get a full count in select

venues• Done in short time period to avoid double

counting• Use reliable community guides to gain access to

the community

Census and enumeration - example

• Census: Map every venue where FSW wait for clients in town X and count every FSW at each venue

• Enumeration: – Map and count all venues in town X– Visit to get average # FSW in a sample of sites– Multiply total # venues by average # FSW per

venue = estimated number of FSWs in town X

Census and enumeration

Pros•Mathematically straightforward

Cons• Only reaches visible portion of population• Difficult to avoid duplication at multiple sites• Expensive, logistically difficult and time-

consuming

Capture/recapture - method

• Was first used in 1662 to estimate the population of London, then for size estimation of animal populations in the wild.

• Conduct a “capture” and tag everyone before releasing

• Conduct independent recapture and calculate proportion who were already tagged

• “Tagging” can be done– Unique object – Services received • either by two different services in two different

institutions • same service at the same institution, but at

different times

• Ideally sampling should be as random as possible, services should be independent.

Capture/recapture – method (2)

Capture/recapture - formula

(NC1 * NC2) ÷ (total number in both captures)

NC1 – number of people in the first captureNC2 – number of people in the second capture

Capture/Recapture – example 1• Opiate users in city X–Opiate users receiving services at NGO Y

(n=2000)–Positive urine testing for opiates of all

persons held in police stations in city X (n=500)–Overlap (n=171)– (2000 X500)/171 = 5,850 opiate users

Capture/Recapture – example 2

• Sex workers in city X– women treated for STIs in 2010 (n= 300)– women treated for STIs in 2011 (n= 280)–Overlap (n= 40)– (300X280)/40 = 2100 female sex workers

Capture-recapture (4)Pros

•Mathematically simple

Cons

• Difficult not to violate assumptions:• Closed population • Independent samples• Positive correlation will lead to underestimated size• Negative correlation will lead to overestimated size

• Hard to identify people accurately between samples• Equal selection probabilities• Requires large sample size to be more reliable

Multiplier - method• Mathematically this estimate is equivalent to

capture-recapture calculations, but the sources of data and interpretation is different.

• Relies on two sources of information and their overlap:– First source: count or listing from program

data (program registers)– Second source: representatives from the

population being measured (survey data)

Multiplier - formula

# of key populations registered with a service_____________________________________ % of population reported being registered

with a service

Multiplier method - example

• A survey among IDUs shows that 12% of them are registered with the drug treatment programs

• Drug treatment registers include 150 IDUs

• Estimate of number of IDUs in the city is 150 ÷ 0.12 = 1,250

Multiplier

Pros•Use of existing data sources•Mathematically simple

Cons• Need high-quality service statistics• Two data sources must be independent• Can go wrong if no clear consistent definitions between

data sources: population, time reference period, etc. • Hard to avoid selection bias• Miss people who are not accessible through either data

source

Methods that rely on data collected from the general

population

Population survey - method

• Surveys of general population/subsets of general population

• Questionnaire is administered to residents of a sample of households drawn from a sample frame that is representative at a national or regional level

Population survey - formula

• Numerator: assess risk behaviors such as male-male sex, injecting drug use, proportion of men who have commercial sex

• Denominator: comes from the general population figures (e.g. from census) or the number of people included into the survey

Population survey - example

# men surveyed reporting having male sexual partner during the last X months

___________________________________Total number of men included into survey

reporting having sex during the last X months

Population survey

Pros

•Results are robust and can be extrapolated•Relatively easy to implement and analyze

Cons• Expensive• Will give minimum estimates for stigmatized behaviors

Network scale-up - method

• “New method” to estimate size of MARPs• Asks about behaviors of the respondent’s

acquaintances• Requires respondents to determine personal

network size using – Summation– Known Population Method

Network scale-up - formula((M1+M2+M3+…Mn)/(C1+C2+C3+…Cn))X T

M – number of acquaintances who practice certain behaviors

C – personal network size (number of acquaintances)

T – total population

Step 1: Identify network size• Summation – ask people to estimate their

personal network size, usually by mutually exclusive categories:– Friends– Relatives– co-workers

Network scale-up - example

Network scale-up – example (2)Step 1: Identify network size (cont.)• Known population method –ask respondents about

the number of people the know in specific population for which true value is known.

• For example, there birth registers or census data shows there are 300,000 adults named Anna in the country with 15 million people. Your respondent knows 5 women named Anna.

• The estimated personal network size will then be 5 divided by 300,000 and multiplied by 15 mln = 250.

Network scale-up - example

Step 2 – estimation• Respondent A knows 300 people (c1)• Two of those inject drugs (m1)• Respondent B knows 200 people (c2)• Three of those inject drugs (m2)• Total population of the country is 15 mln (T)• Estimated number of people who inject drugs

is equal to ((m1+m2)/(c1+c2))X T = 150,000

Network scale-upPros

• Does not require members of hidden populations • Can be incorporated into existing surveys• Can create size estimations for multiple populations

Cons• Average network size is difficult to estimate• Respondents may not be aware of risky behaviors of their

acquaintances or hesitant to acknowledge they know people with risky behaviors

Delphi method

• Asks key informants to agree (reach consensus) on number of people practicing certain behaviors in a population

• Can be done at all levels from local to national• Should be triangulated with other methods

Estimating the size of sex worker populations

• Census methods have been shown to be useful for brothel-based sex workers. Use enumeration for situations where there are large numbers of venues and the sex workers do not move quickly between locations.

• Use capture-recapture to estimate the size of street-based sex worker populations when it is not possible to create a list of venues or conduct a census.

• Multiplier methods can be useful for local estimates (challenge: finding lists from administrative sources to provide a multiplier for a national estimate).

Estimating the population size of persons who inject drugs

• Capture-recapture methods should be considered in settings where program data sources are of reasonably good quality and where injection drug use is not punishable by imprisonment or death.

• Multiplier methods can be useful where treatment service records are of good quality. In this case, since multipliers vary by place, the national estimate should aggregate as many local area estimates as are available.

Estimating the population size of men who have sex with men

• If men are open about having sex with other men, a census conducted at gathering locations could be useful.

• If no program data sources are available, a cost effective option is to include same-sex behavior on existing general population surveys. However these estimates are likely to be underreported, especially in settings where such behaviors are highly stigmatized

• Capture–recapture was also successfully piloted among MSM (use of unique object)

Analyzing data

• Using multiple sources and several methods is the way to go (Triangulate to Validate)

• Present range (min. – max.) of estimates to reflect level of uncertainty inherent in the size estimation process

• Documenting the process (to be useful size estimates need to be updated over time)

CONCLUSION…

“Torture numbers and they’ll confess

to anything”

–Gregg Easterbrook. “Statistics may be defined as "a body

of methods for making wise decisions

in the face of uncertainty.”

–W.A. Wallis