Incidence Estimates

Nanette Benbow, Past-ChairHIV Workgroup

Council of State and Territorial Epidemiologists (CSTE)

2009 NASTAD Annual MeetingMay 3, 2009

Outline

History of Incidence Estimation Description of Incidence Surveillance Incidence Estimation Uses of the Incidence Estimate

History of Incidence Estimation

1998 CDC publishes a description of the STARHS assay, a laboratory test that can distinguish recent HIV infections from long-standing ones on a population basis. (Janssen RS, Satten GA, Stramer SL, et al. New testing strategy to detect early HIV-1 infection for use in incidence estimates and for clinical and prevention purposes. JAMA 1998; 280:42-48.)

2001 (Feb) CDC holds large consultation with statisticians and surveillance experts to discuss the feasibility of using the STARHS assay to estimate HIV incidence. Conclusion: CDC should establish an HIV surveillance system based on STARHS rather than continuing with serosurveys.

2001 (Fall) CDC proposes the statistical methods that form the basis of the new incidence system.

2001 (Oct) CDC convenes a small consultation with statisticians and surveillance experts to discuss the statistical methodology. Conclusion: The statistical theory was solid but behavioral data on HIV testing was needed in order to generate probabilities.

2001 (Oct) CDC funds 5 areas to pilot the feasibility of employing these techniques to collect data.

2002 (Dec) CDC funds an additional 19 areas to refine and further develop implementation protocols.

2004 (Jan) CDC funds 33 areas to conduct incidence surveillance. These areas were to use one of three protocols in light of regulatory requirements regarding informed consent as applied to the STARHS testing (detuned assay).

2005 (Mar) FDA designates the BED assay as a surveillance, not a diagnostic, test. This designation sets the stage for population-based incidence surveillance. Additional level of consent to employ STARHS testing is no longer needed and all areas could implement one protocol.

2005 CDC funds one additional area to conduct incidence surveillance.

HIV Incidence Methodology Timeline

HIV Incidence Methodology Timeline

2006 (Mar) Scientists submit methods paper to Statistics in Medicine.

2006 (June) CDC holds a consultation on the validity of the statistical method (sample survey approach). Outcome: The consultants proposed a simplified approach and the back-calculation method.

2006 (June) - 2007 (Feb)

CDC modifies its statistical methods as recommended at the consultation.

2007 (Feb) CDC convenes a consultation to present 2005 estimates and methods to peers in order to receive feedback and meet the requirement for CDC Information Quality Peer Review for Influential Scientific Information.

2007 (Feb) - 2007 (June)

CDC finalizes its statistical methods by combining the modified sample survey approach and the simplified approach into the stratified extrapolation approach and refined the back-calculation approach.

2007 (June-Oct) CDC scientific review and clearance.

2007 (Oct) Methods manuscript accepted by Statistics in Medicine.

2007 (Oct) - 2008 (June)

CDC submits HIV incidence manuscript for scientific review, receives and addresses comments, including up-dates with 2006 data.

2008 (June) Incidence manuscript accepted by JAMA.

2008 (Aug) CDC’s methods paper is published in Statistics in Medicine (Karon KM, Song R, Kaplan E, Brookmeyer R, Hall HI. Estimating HIV incidence in the United States from HIV/AIDS surveillance data and biomarker HIV test results. Stat Med. 2008).

2008 (Aug) CDC’s incidence paper published in JAMA (Hall HI, Song R, Rhodes P, et al. Estimation of HIV incidence in the United States. JAMA 2008).

CDC Incidence Estimate

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

180,000

Old Estimate (avg. of studies)

New Estimate (back-calculation)

55,40040,000

56,300

New estimate (incidence surveillance)

Description of Incidence Surveillance

Case-based Surveillance Data Information from HIV/AIDS case reports

All diagnosed cases Personal characteristics Status at diagnosis

Information from other sources Laboratory results (BED, CD4) Questionnaires (HIV test, ART) Individually linked

How does the HIV incidence reporting system relate to routine HIV reporting?

HIV incidence reporting is an extension of the population-based HIV reporting system

It uses the existing reporting infrastructure to collect the information necessary to estimate HIV incidence from all newly diagnosed HIV cases that are reported

Adds additional information to complement standard HIV reporting

Questions?

Incidence Estimation

Explanation of Probabilities The three probabilities

Calculating the Estimate Stratification, 200x40x10 Rule Multiple Imputation

Calculating the Estimate—Important Variables and How they Interact

Using simple survey sampling methodology, the following is needed to estimate the size of a specific population based on a random subset sample: Observable Sample (R) Probability of being in the sample (P)

Estimation of HIV Incidence

Sampling Frame

Sample Selected

Estimated Probability

Sample Weight

Population Size

N R P Wt = 1/P N = R/P = R*Wt

Sample Selected = R

All persons who were diagnosed in the selected

period of time and classified as BED “recent”

represent the sample selected.

NSampling Frame = N

All persons who became infected with HIV

in the selected period of time including

those not diagnosed.

N

Sampling Frame

R

SampleSelected

RHIV/AIDS

Diagnosed

Estimating P – The probability that a new infection is classifiedas a BED recent

Does everyone in the sample have the same probability of being selected? No, it changes depending on the following:

Infected person was tested within 1 year after infection Person diagnosed with HIV had a BED test result* BED result for a person tested within 1 year after

infection was “recent”

*Because some of the people sampled do not have a BED test result, a BED result is “filled” in using a statistical technique called “multiple imputation”

Estimating P (cont.)

P1= Probability of being tested within 1 year after infection (changes depending on whether or not a person tests frequently or not)

P2= Probability that a person diagnosed with HIV had a BED test result

P3= Probability of having a BED test “recent” if the test is within one year after infection

P= P1* P2* P3

“P” is calculated for each demographic/risk subgroup (Strata) to adjust for difference in

testing patterns between these different groups

67 StrataBlack, non-

Hispanic• Male

– 13-29– 30-39– 40-49– >=50– MSM– IDU– MSM/IDU– Heterosex.– Total

• Female– 13-29– 30-39– 40-49– >=50– IDU– Heterosex.– Total

• Total

Other, non-Hispanic

• Male– 13-29– 30-39– 40-49– >=50– MSM– IDU– MSM/IDU– Heterosex.– Total

• Female– 13-29– 30-39– 40-49– >=50– IDU– Heterosex.– Total

• Total

Hispanic

• Male– 13-29– 30-39– 40-49– >=50– MSM– IDU– MSM/IDU– Heterosex.– Total

• Female– 13-29– 30-39– 40-49– >=50– IDU– Heterosex.– Total

• Total

MaleFemaleTotal

White, non-Hispanic

• Male– 13-29– 30-39– 40-49– >=50– MSM– IDU– MSM/IDU– Heterosex.– Total

• Female– 13-29– 30-39– 40-49– >=50– IDU– Heterosex.– Total

• Total

Within each group, incidence is estimated by the

number of BED-recent specimens divided by the

probability of being classified as recent:

N = R/P

The total incidence in the population is the

sum of incidences of all strata:

r

iiNI

1

SamplingFrame

SampleSelected

EstimatedProbability

Sample Weight

Population Size

N R P Wt = 1/P N = R/P = R*Wt

Final Incidence Estimate

Questions?

Uses of the Incidence Estimate Information

Use of Incidence Estimate (1) Using this method, CDC estimates that

56,300 adolescents and adults were newly infected with HIV in 2006 in the US (95% confidence interval [CI], 48,200-64,500) Because the Incidence number is a statistical

estimate (commonly referred to as a “point estimate”) you need to also consider the confidence interval and interpret the estimate as follows: If you were to take 100 samples to estimate the

number of new HIV infections in 2006, 95% of the samples will produce an HIV incidence estimate between - [48,000 - 64,500] The accuracy of point estimates is highly dependant on the

number of observations used to make the estimate. Small sample sizes produce wider confidence intervals (CI).

Use of Incidence Estimate (2) When comparing two point estimates with their

respective CIs, you can say that the two numbers are statistically significantly different if the two CIs overlap (i.e. have values in common). If they do not overlap, you need to perform a statistical test to determine if the numbers are significantly different

Example – 2006 US Incidence Estimate: Males 41,400 95% CI [35,100 – 46,600] Females 15,000 95% CI [12,600 – 17,300]

CIs do not overlap, thus, the incidence rate for men is significantly higher than for females

White 19,600 95% CI [16,400 – 22,800] Black 24,900 95% CI [21,100 – 28,700]

CIs overlap, thus, you cannot conlcude that there is statistically significant difference between the incidence for Whites and Blacks

Uses of data obtained from the Incidence Estimation Process:Data from North Point State

2006 HIV IncidenceN = 1,368

C=Recently Infected Cases = 639

2006 HIV DiagnosesD = 1,446

Estimated number of people newly infected with HIV in 2006 who are diagnosed that year

Estimated number of people newly infected in 2006 but not diagnosed in 2006:- not tested (hence, unaware of their status)- tested anonymously

N – C = 729

Estimated number of people infected in previous years but not diagnosed until 2006

D–C = 807

Interpretation (1) Out of every 100 people newly infected with

HIV in 2006, what percentage are unaware of their status in that year? 53%

Out of every 100 people newly infected with HIV in 2006, what percentage are diagnosed in that year? 47%

Out of every 100 HIV diagnoses in 2006, what percentage are recent infections? 56%

How can you use these quantities to guide, target or evaluate prevention efforts?

Uses of data obtained from the Incidence Estimation Process:Data from North Point State (2)

2006 HIV IncidenceN = 223

C=Recently Infected Cases = 117

2006 HIV DiagnosesD = 312

Estimated number of people newly infected with HIV in 2006 who are diagnosed that year

Estimated number of people newly infected in 2006 but not diagnosed in 2006:- not tested (hence, unaware of their status)- tested anonymously

N – C = 106

Estimated number of people infected in previous years but not diagnosed until 2006

D–C =312

2006 HIV IncidenceN = 844

C=Recently Infected Cases = 379

2006 HIV DiagnosesD = 312

Estimated number of people newly infected with HIV in 2006 who are diagnosed that year

Estimated number of people newly infected in 2006 but not diagnosed in 2006:- not tested (hence, unaware of their status)- tested anonymously

N – C = 465

Estimated number of people infected in previous years but not diagnosed until 2006

D–C = 358

Whites

Blacks

What can you say about the differences/similarities in new

infections in these two populations?

Interpretation (2) Out of every 100 people estimated to be

newly infected with HIV in 2006, what percentage are unaware of their status in that year? Whites: 48% Blacks: 55%

Out of every 100 people estimated to be newly infected with HIV in 2006, what percentage are diagnosed in that year? Whites: 52% Blacks: 45%

Out of every 100 HIV diagnoses in 2006, what percentage are estimated to be recent infections? Whites: 38% Blacks: 51%

Uses of data obtained from the Incidence Estimation Process:Data from North Point State (3)

Transmission Rates For every 100 people living with HIV, the number of

HIV infections transmitted to HIV-seronegative partners in a year 23,500 people living with HIV/AIDS

1,368/23,500 x 100 = 5.8 = 6 For every 100 persons living with HIV in North State,

there are six HIV transmission per year And…94% of person living with HIV did not transmit the virus

Having an incidence estimate allows us to calculate transmission rates which can be used as a way to measure the speed of spread

of HIV infection

.

Summary This is a new surveillance system. Because it is

so new, we probably should not assume that the data are 100% accurate until the surveillance system is mature (this usually takes about 4 years)

By combining diagnoses, prevalence and incidence data we are able to get a more accurate and timely picture of the epidemic and its changes over time that will helpful to guide and evaluate prevention efforts

More time is still needed to assess how these data can be used locally over time